摘要
采用目前方法对大数据进行存储时,无法有效分配数据中存在的冗余数据,导致方法存在带宽占用率高、负载均衡度低、节点剩余能量低和数据存储量低的问题。提出基于群体智能算法的大数据分布式存储方法,利用一致性树分布存储方法完成大数据的分布存储,通过群体智能算法选择存储节点,对存储节点和大数据进行映射处理,将数据映射到对应的存储节点中,并采用遗传算法对大数据中存在的冗余数据进行分配,完成大数据的分布式存储。实验结果表明,所提方法的带宽占用率低、负载均衡度高、节点剩余能量高、数据存储量高。
Currently,some methods fail to effectively allocate redundant data.Therefore,a method of distributed storage for big data based on a swarm intelligence algorithm was proposed.This method used the consistency tree dis-tributed storage to complete the distributed storage of big data,and selected storage nodes by the swarm intelligence algorithm.And then,the storage nodes and big data were mapped to storage nodes correspondingly.Moreover,a genet-ic algorithm was adopted to allocate the redundant data existing in the big data.Finally,the distributed storage was completed.Experimental results show that the proposed method can achieve low bandwidth occupancy,high load bal-ance,high node residual energy and high data storage capacity.
作者
胡媛媛
江春然
甘杜芬
HU Yuan-yuan;JIANG Chun-ran;GAN Du-fen(Department of Artificial Intlligence and Big Data,Hebei Polytechnic Institute,Shijiazhuang Hebei 050000,China;School of Computer Engineering,Guilin University of Electronic Technology,Beihai Guangxi 536000,China)
出处
《计算机仿真》
北大核心
2023年第11期447-451,共5页
Computer Simulation
基金
2020年河北省人力资源社会保障研究课题(JRS-2020-1030)。
关键词
群体智能算法
一致性树分布存储
数据映射
大数据分布式存储
Swarm intelligence algorithm
Consistency tree distributed storage
Data mapping
Big data distributed storage
