摘要
针对高性能MapReduce集群速率不高、自适应控制能力不好的问题,提出基于GPU加速的高性能MapReduce集群设计方法。构建高性能MapReduce集群控制的网络结构模型,采用并行计算框架方法构建高性能MapReduce集群云平台,结合GPU加速方法进行数据同态管理以及并行计算的优化控制,在多层次多粒度并行计算框架下实现GPU的整体存储容量扩展融合处理;设计GPU加速并行计算的自适应算法,构建优化的GPU计算交替执行模式,通过多核CPU的Hadoop并行计算体系实现高性能MapReduce集群设计和自适应参数融合调节,采用大数据挖掘和空间状态特征匹配的方法,提高高性能MapReduce集群控制和动态响应的能力。仿真结果表明,采用该方法进行设计的高性能MapReduce集群输出稳定性较高、计算速率较好,且提高了GPU动态响应和并行计算能力。
Aiming at the problems of low speed and poor adaptive control ability of high-performance MapReduce cluster,this paper proposes a design method of high performance MapReduce cluster based on GPU acceleration.The network structure model of high performance MapReduce cluster control is constructed,and the parallel computing framework method is used to build the high performance MapReduce cluster cloud platform,the data homomorphism management and parallel computing optimization control are carried out based on the GPU acceleration method,and the overall storage capacity expansion and fusion processing of the GPU is realized under the multi-level and multi-granularity parallel computing framework.The adaptive algorithm of GPU acceleration parallel computing is designed and the optimized GPU computing alternate execution mode is built,and the high performance graphs cluster design fusion and adaptive parameter adjustment are achieved through the multi-core CPU Hadoop parallel computing system,and the ability of high performance graphs cluster control and dynamic response is improved by means of bid data mining and spatial state feature matching.The simulation results show that the high performance MapReduce cluster designed by this method has higher output stability,better computing speed,and helps to improve the GPU dynamic response and parallel computing capability.
作者
赵少东
ZHAO Shaodong(Shenzhen Power Supply Bureau Co.,Ltd.,Shenzhen 518033,Guangdong,China)
出处
《电网与清洁能源》
北大核心
2021年第4期90-94,共5页
Power System and Clean Energy
基金
南方电网公司科技项目(090000KK52170003)。
