期刊文献+
任意字段
题名或关键词
题名
关键词
文摘
作者
第一作者
机构
刊名
分类号
参考文献
作者简介
基金资助
栏目信息

一种改进的DRF算法对BESIII集群资源管理的优化 被引量：4

The Optimization of BESIII Cluster Resource Management by Using the Improved DRF Algorithm
下载PDF
导出
摘要 根据BESIII集群的实际需求,在测试和研究了不同类型的高能物理数据处理作业不同配置机器上的运行效果的基础上,提出了一种改进的DRF资源分配算法,加入了机器性能评级和作业类型匹配两个因素作为资源调度的依据,对算法进行实现并使用真实数据进行了实验测试。测试结果表明:新算法能够更加合理地分配资源,有效提高系统资源利用率,缩短作业运行时间。 This paper presents an improved DRF resource allocation algorithm for BESIII cluster , as a result of the tests and research on different type of high energy physics jobs running on different machines .The machine performance rating and job type matching are added to the new algorithm as two factors for resource allocation . The new algorithm is implemented and tested with real data .The results show that the new algorithm can allo-cate resources more effectively , which improves the resource utilization and reduces the run time of jobs .
作者 霍菁 石京燕 孙功星 阚博文
机构地区 中国科学院大学 中国科学院高能物理研究所
出处 《核电子学与探测技术》 CAS CSCD 北大核心 2014年第10期1153-1158,共6页 Nuclear Electronics & Detection Technology
基金 国家自然科学基金(11375223 11375221) 国家自然科学基金A3前瞻计划(61161140454)
关键词 集群资源管理 DRF资源分配算法 Mesos TORQUE HADOOP cluster resource management DRF allocation algorithm Mesos Torque Hadoop
分类号 TP319 [自动化与计算机技术—计算机软件与理论]
  • 相关文献

参考文献12

  • 1Staples G. TORQUE resource manager[ C ]. Proceed- ings of the 2006 ACM/IEEE conference on Supercom- puting. ACM, 2006: 8.
  • 2Maui. [ EB/OL]. (2014 - 02 - 15 ). http://www. adaptivecomputing, corn/products/open - source/ maul/.
  • 3Apache Hadoop. [ EB/OL]. ( 2005 ). http ://hadoop. apache, org/.
  • 4Dean J, Ghemawat S. MapReduce: simplified data processing on large clusters[ C]. Proc. of the 6th con- ference on Symposium on Operating Systems Design & Implementation. San Francisco. CA. USA, ACM Press, 2004.
  • 5臧冬松,霍菁,梁栋,孙功星.基于MapReduce的高能物理数据分析系统[J].计算机工程,2014,40(2):1-5. 被引量：9
  • 6Hindman B, Konwinski A, Zaharia M, et al. Mesos : A platform for fine - grained resource sharing in the data center[ C ]. Proceedings of the 8th USENIX con- ference on Networked systems design and implementa-tion. 2011 22 -22.
  • 7Ghodsi A, Zaharia M, Hindman B, et al. Dominant resource fairness: fair allocation of multiple resource types [ C ]. USENIX NSDI. (2011 ).
  • 8Parkes D C, Procaccia A D, Shah N. Beyond domi- nant resource fairness : extensions, limitations, and indivisihilities[ C]. Proceedings of the 13th ACM Con- ference on Electronic Commerce. ACM, 2012 : 808 - 825.
  • 9Zaharia M, Borthakur D, Sen Sarma ], et al. Delay scheduling: a simple technique for achieving localityand fairness in cluster scheduling [ C ]. Proceedings of the 5th European conference on Computer systems. ACM, 2010:265-278.
  • 10Foley D K. Resource allocation and the public sector [ J ]. YALE ECON ESSAYS, VOL 7, NO 1, PP 45 - 98, SPRING 1967. 7 FIG, 13 REF., 1967.

二级参考文献30

  • 1Ghemawat S, GobioffH. The Google File System[C]//Proc. of the 19th ACM Symposium on Operating Systems Principles. New York, USA: ACM Press, 2003: 29-43.
  • 2Dean J, Ghemawat S. MapReduce: Simplified Data Processing on Large Clusters[C]//Proc. of the 6th Symposium on Operating Systems Design & Implementation. San Francisco, USA: ACM Press, 2004: 107-113.
  • 3Chang F, Dean J, Ghemawat S, et al. Bigtable: A Distributed Storage System for Structured Data[J]. ACM Transactions on Computer Systems, 2008, 26(2): 205-218.
  • 4Apache. HADOOP[EB/OL]. (2012-05-01). http://HADOOP. apache.org.
  • 5Bradley D, Dasu S, Maier W, et al. A Highly Distributed, Petascale Migration from dCache to HDFS[C]//Proc. of HEPiX Fall 2011 Workshop. Vancouver, USA: [s. n.], 2011: 1- 24.
  • 6Riahi H, Donvito G, Fanb L. Using HADOOP File System and MapReduce in a Small/Medium Grid Site[J]. Journal of Physics: Conference Series, 2012, 396(4): 50-55.
  • 7Glaser F, Neukirchen H. Analysing High-energy Physics Data Using the MapReduce Paradigm in a Cloud Computing Environment[EB/OL]. (2012-05-11). https://notendur.hi.is/- helmut/publications/VHI-01-2012.pdf.
  • 8The ROOT Team. ROOT[EB/OL]. ,(2010-04-12). http://root. cern.ch.
  • 9Antcheva I, Ballintijn M, Bellenot, et al. ROOT A C++ Framework for Petabyte Data Storage, Statistical Analysis and Visualization[J]. Computer Physics Communications, 2009, 180(12): 1384-1385.
  • 10Apache Hadoop HDFS Architecture Guide[EB/OL]. (2012-03- 22). http://hadoop.apache.org/docs/r 1.0.4/hdfs_design.html.

共引文献10

同被引文献16

引证文献4

二级引证文献19

核电子学与探测技术
2014年 第10期

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部