摘要
区块链是分布式的数据存储系统,共识算法为区块链实现安全存储数据提供支撑和保障。Kafka作为共识算法中的一种,其高吞吐速率、低时延的特点受到青睐。但使用Kafka算法的系统接受大量交易时,易产生数据倾斜,即分布式系统的多节点结构中,大量数据集中在少数节点,导致系统资源被占用、性能下降。为解决上述问题,本文提出基于时间序列模型长短期记忆网络(LSTM)的智能优化方法。通过学习过往生产者接收到的交易量,预测下一时刻面临的交易量,动态调整生产者节点数量,减少数据集中在少数节点的情况。实验结果显示,本文方法可以将Kafka系统时延降低2~3倍,吞吐速率提升2~3倍,与优化前相比系统效率提升52.62%,比2种传统优化方法分别提升近3%和40%,能耗仅小幅提升,系统使用情况保持更加合理。
Blockchain is a distributed data storage system.Consensus algorithm can support and guarantee blockchain to achieve safe data storage.Kafka,as one of the consensus algorithms,is favored for its high throughput rate and low delay.However,when a system using the Kafka algorithm accepts a large number of transactions,it is prone to data skew,that is,in the multi-node structure of a distributed system,a large amount of data is concentrated on a few nodes,which leads to the occupation of system resources and performance degradation.To solve these problems,an intelligent optimization method based on long short term memory(LSTM)is proposed.By learning the trading volume received by producers in the past,the trading volume faced at the next moment can be predicted,and the number of producer nodes can be dynamically adjusted to reduce the data concentration in a few nodes.The experimental results show that the proposed method can reduce the delay of Kafka system by 2-3 times,increase the throughput rate by 2-3 times,and improve the system efficiency by 52.62%compared with the original Kafka system,which is nearly 3%and 40%higher than the two traditional optimization methods,respectively.The energy consumption is only slightly improved,and the system usage remains more reasonable.
作者
周宇泽
司鹏搏
张延华
李萌
杨睿哲
ZHOU Yuze;SI Pengbo;ZHANG Yanhua;LI Meng;YANG Ruizhe(Faculty of Information Technology,Beijing University of Technology,Beijing 100124)
出处
《高技术通讯》
CAS
2023年第10期1047-1059,共13页
Chinese High Technology Letters
基金
国家自然科学基金(61901011)
北京市自然科学基金(L211002)
北京市教育委员会科技计划一般项目(KM202110005021)资助。
