摘要
针对如何将海量数据安全、高效和均匀地分布在集群节点上的数据存储问题,提出一种简洁、高效、支持权重和数据冗余机制的对象分布算法。描述对象和节点一对多的映射关系,将集群中的存储节点及相应节点的权重映射到二维数组中,采用跳跃Hash实现对象副本的快速定位;在定位过程中,该算法具有节点选择冲突检测处理机制,能够避免同一对象的不同副本映射到相同存储节点这一问题。实验结果表明,该算法具有数据分布均匀、迁移量小的优势。
Aiming at the problem of storing the massive data on the cluster nodes securely,efficiently and evenly,a simple and efficient object distribution algorithm which supported the weight and data redundancy mechanism was put forward.The mapping relationship between the object and the node was built,and the weight of the storage node and the corresponding node in the cluster were mapped to the two-dimensional array.The quick positioning of the object copy was realized by jump Hash.In the process of positioning,the selection conflict detection mechanism was utilized to avoid the same object with different copies mapping into the same storage node.Experimental results show that the proposed algorithm has the advantages of uniform data distribution and small amount of migration.
作者
谢兵兵
王勇
叶苗
俸皓
XIE Bing-bing;WANG Yong;YE Miao;FENG Hao(School of Computer Science and Information Security,Guilin University of Electronic Technology,Guilin 541004,China;Guangxi Colleges and Universities Key Laboratory of Cloud Computing and Complex Systems,Guilin University of Electronic Technology,Guilin 541004,China;Guangxi Cooperative Innovation Center of Cloud Computing and Big Data,Guilin University of Electronic Technology,Guilin 541004,China;School of Information Science and Engineering,Guilin University of Technology,Guilin 541004,China)
出处
《计算机工程与设计》
北大核心
2019年第1期131-135,178,共6页
Computer Engineering and Design
基金
国家自然科学基金项目(61662018
61661015)
广西自然科学基金项目(2016GXNSFAA380153
2015GXNSFAA139294)
广西云计算与大数据协同创新中心基金项目(YDQ17001)
桂林理工大学科研启动基金项目(GUTQDJJ20172000019)
关键词
海量数据
数据冗余
二维数组
对象分布
跳跃哈希
massive data
data redundancy
two-dimensional array
object distribution
jump Hash
