期刊文献+

基于HBase的并行BFS方法 被引量:4

HBase Based Parallel BFS Method
下载PDF
导出
摘要 NoSQL数据库作为下一代巨型数据的存储模式,在科学计算和商业计算领域均发挥着重要作用,受到当前学术界和企业界的广泛关注。提出一种新的基于NoSQL数据库HBase的并行求取最短路径树的方法。首先利用Watts-Strogatz模型完成对巨型网络的数学建模,这种建模方式使得网络模型具有一定的聚类效果;其次利用HBase最近发布的Coprocessor简化和改进并行BFS方法,提高其计算效率。此外,还设计并实施了大量实验,得出了巨型网络的最短路径树,验证了该算法的正确性和有效性;同时对比其它路径算法,验证了该算法的高效性。 As the next generation of storage model of giant data, NoSQL database plays an important role both in the fields of scientific computing and commercial computing, and has gained wide attention in academia and business com- munity. We presented a new parallel method based on HBase to gain the shortest path tree. Firstly,Watts-strogatz mode was used to complete the mathematical modeling of giant network, therefore the network would have some cluster effects. Secondly, we made a simplification and improvement to the parallel breath-first search method, in order to im- prove its calculation efficiency. In addition, we designed and implemented a large number of experiments. According to the experiment resuhs,we obtained the giant network shortest path tree, and verified the correctness and validity of the algorithm. Meanwhile,Contrast to the other path algorithm, we verified the efficiency of the algorithm.
出处 《计算机科学》 CSCD 北大核心 2013年第3期228-231,共4页 Computer Science
基金 国家自然科学基金项目(61202163 61240035) 山西省自然科学基金(2012011015-1) 山西省科技攻关项目(20120313032-3)资助
关键词 HBASE 协处理器 并行广度优先算法 MAPREDUCE NOSQL数据库 HBase, Ccoprocessor, Parallel BFS, Mapreduce, NoSQL database
  • 相关文献

参考文献11

  • 1Lakshman A,Malik P.Cassandra:A Decentralized Structured Storage System[C] //SIGOPS.2010.
  • 2HBase Homepage[OL].http://hbase.apache.org.
  • 3Chang F,et al.Bigtable:A Distributed Storage System for Structured Data[C] //OSDI.2006.
  • 4许春聪,黄小猛,吴诺,孙宁伟,杨广文.分布式文件系统存储介质评测与分析[J].计算机学报,2010,33(10):1873-1880. 被引量:9
  • 5Lin J,Dyer C.Data-Intensive Text Processing with MapReduce,ser.Synthesis Lectures on Human Language Technologies[M].Morgan &.Claypool Publishers,2010.
  • 6Dean J,Ghemawat S.Mapreduce:simplified data processing on large clusters[J].Commun.ACM,2008,51(1):107-113.
  • 7周国亮,陈红.基于图形处理器的并行方体计算[J].软件学报,2010,33(10):1788-1798.
  • 8Dean J.Large-scale distributed systems at google:Current systems and future directions[C] //The 3rd ACM SIGOPS International Workshop on Large Scale Distributed Systems and Middleware.2009.
  • 9Dijkstra E W.A note on two problems in connexion with graphs[C] //Numerische Mathematik.1959:269-271.
  • 10Cormen T H,Leiserson C E,Rivest R L.Introduction to Algorithms[M].Cambridge,Massachusetts:MIT Press,1990.

二级参考文献12

  • 1Ghemawat S, Gobioff H, Leung S T. The google file system//Proceedings of the 19th ACM Symposium on Operating Systems Principles. Sagamore, 2003:29-43.
  • 2Caulfield A M, Grupp L M, Swanson S. Gordon: Using flash memory to build fast, power-efficient clusters for dataintensive applications//Proceedings of the International Con ference on Architectural Support for Programming Languages and Operating Systems. Wangington, 2008:217-228.
  • 3Andersen D G, Franklin J, Kaminsky M. FAWN: A fast ar ray of wimpy nodes//Proceedings of the 22nd ACM Symposi um on Operating Systems Principles. Big Sky, 2009:1-14.
  • 4Outerhout J, Agrawal P, Erickson D et al. The case for RAMClouds: Scalable high-performance storage entirely in DRAM. Operating Systems Review, 2009, 43(4) 92-105.
  • 5Schmidt K, Ou Y, Harder T. The promise of solid state disks: Increasing efficiency and reducing cost of DBMS processing//Proceedings of the Canadian Conference on Comput er Science & Software Engineering. Montreal, 2009. 35-41.
  • 6Polte M, Simsa J, Gibson G. Comparing performance of solid state devices and mechanical disks//Proceedings of the 3rd Petascale Data Storage Workshop Held in Conjunction with Supercomputing. Pittsburgh, 2008:1-7.
  • 7Narayanan D, Thereska Eno, Donnelly A et al. Migrating server storage to SSDs: Analysis of tradeoffs//Proceedings of the 4th ACM European Conference on Computer Systems (EuroSys'09). Nuremberg, 2009. 145-158.
  • 8Anderson E, Spence S, Swaminathan R et al. Quickly finding near-optimal storage designs. ACM Transactions on Computer System, 2005, 23(4) : 337- 374.
  • 9Strunk J, Thereska E, Faloutsos C, Ganger G. Using utility to provision storage systems//Proceedings of the USENIX Conference on File and Storage Technologies. San Jose, 2008, 313- 328.
  • 10Kryder M H, Kim C S. After hard drives-What comes next? IEEE Transactions on Magnetics, 2009, 45 (10): 3406-3413.

共引文献8

同被引文献23

  • 1郝占刚,王正欧.基于遗传算法和k-medoids算法的聚类新算法[J].现代图书情报技术,2006(5):44-46. 被引量:5
  • 2TomWhite.Hadoop权威指南[M].周敏奇,王晓玲,译.北京:清华大学出版社,2011.
  • 3张吉赞,李洪波,王峰.Ad Hoc网络的加权可靠路由策略[J].计算机工程与应用,2007,43(35):140-145. 被引量:2
  • 4Zhang Qiaoping,Couloigner I.A new and efficient K-medoid algorithm for spatial[C]//Computational Science and its Applications-ICCSA,2005:181-189.
  • 5Park Hae-Sang,Jun Chi-Hyuck.A simple and fast algorithm for K-medoids clustering[J].Expert Systems with Applications,2009,36(2):3336-3341.
  • 6Alper Z G.K-harmonic means data clustering with simulated[J].Applied Mathematics and Computation,2007,184:199-209.
  • 7Pei Ying,Xu Jungang,Cen Zhiwang,et al.IKMC:An improved K-medoids clustering method for near-duplicated records detection[C]//International Conference on Computational Intelligence and Software Engineering,2009:1-4.
  • 8Cardot H,Cénac P,Monnez J M.A fast and recursive algorithm for clustering large datasets with k-medians[J].Computational Statistics and Data Analysis,2012,56:1434-1449.
  • 9Qiao Shaoyu,Geng Xinyu,Wu Min.An improved method for K_medoids algorithm[C]//International Conference on Business Computing and Global Informatization,2011:440-444.
  • 10孙胜,王元珍.基于核的自适应K-Medoid聚类[J].计算机工程与设计,2009,30(3):674-675. 被引量:14

引证文献4

二级引证文献8

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部