摘要
针对如何安全有效地实现大数据存储,提出一种大数据存储系统架构及数据安全放置机制,实现海量数据安全有效的存储。设计的大数据存储系统通过文件组合策略改进了HDFS小文件的读写过程,在HDFS文件系统上构建大数据存储平台。提出了数据存储的安全感知数据放置机制,在满足安全性能要求的同时实现高检索性能,将数据放置问题表示为数学模型,最小化数据的总检索时间,最后使用图着色的贪婪算法给出数据放置机制解决方案。实验结果表明:在具有随机网络拓扑的云存储系统上,设计的存储系统及数据安全放置机制在检索时间、数据存储总量和拒绝次数性能方面优于其他方法,且在不同存储节点数量和安全级别下,该方法都具有最少的检索时间。
The safe and effective implementation of big data storage is the requirement of more and more big data storage systems.The existing big data storage systems are lacking in this aspect.To solve this problem,a large data storage system architecture and data security placement mechanism is proposed to achieve safe and effective storage of massive data.In this paper,the proposed big data storage system improves the reading and writing process of small HDFS files through file combination strategy,and a big data storage platform on the HDFS file system is built.The security-aware data placement mechanism of data storage is proposed,which satisfies the security requirements and achieves high retrieval performance.Firstly,the data placement problem is expressed as a mathematical model,which can minimize the total retrieval time of the data and then the data is divided and distributed in the storage node.Finally,the greedy algorithm of graph coloring is used to implement the data placement mechanism solution.The experimental results show that on cloud storage systems with random network topology,the proposed storage system and data security placement mechanism is superior to other methods in terms of retrieval time,total data storage and rejection times,and the placement mechanism has the minimum retrieval time under different storage nodes and security levels.
作者
刘海宁
李德山
LIU Haining;LI Deshan(School of Economics and Trade Management,Tianjin Sino-German University of Applied Sciences,Tianjin 300350,China;School of Economics and Management,Southwest University of Science and Technology,Mianyang 621010,China)
出处
《重庆理工大学学报（自然科学）》
CAS
北大核心
2019年第8期170-177,共8页
Journal of Chongqing University of Technology：Natural Science
基金
天津市科技计划项目技术创新引导专项科技特派员项目“大数据背景下基于动态博弈的数据存储优化模型研究”(18JCTPJC51600)
天津市教委科研计划项目“线上线下融合模式下快递包装回收路径研究”(2018SK154)
