摘要
该文针对Hadoop自带的MapReduce调度器的不足,提出了具有优先级和权重的公平调度算法。算法可以清楚地区分出用户服务的等级,同时又保证一定程度的公平性,保证不至于被少量高优先级用户占有系统绝大部分资源。同时可以更好的实现数据的本地性,提高系统的整体效率和并行度,从而减小相应时间,避免节点闲置,浪费系统资源。
This paper proposes a degin and implementation priority based Weighted Fair Queue of based on MapReduce clusters,according to drawbacks of Hadoop's MapReduce scheduler.It can clearly distinguish the level of customer service,while ensuring a degree of fairness and high priority was not subject to a small majority of users sharing system resources.It can achieve better data locality,improve the performance of the system and parallel access,and setting the appropriate for users and weight to ensure special tasks,according to the mathematical model.
出处
《电脑知识与技术》
2011年第3X期2129-2132,共4页
Computer Knowledge and Technology
