摘要
针对深度信念网(Deep belief network,DBN)预训练耗时长的问题,提出了一种基于自适应学习率的DBN(Adaptive learning rate DBN,ALRDBN).ALRDBN将自适应学习率引入到对比差度(Contrastive divergence,CD)算法中,通过自动调整学习步长来提高CD算法的收敛速度.然后设计基于自适应学习率的权值训练方法,通过网络性能分析给出学习率变化系数的范围.最后,通过一系列的实验对所设计的ALRDBN进行测试,仿真实验结果表明,ALRDBN的收敛速度得到了提高且预测精度也有所改善.
A deep belief network with adaptive learning rate （ALRDBN） is proposed to solve the time-consuming problem in the pre-training period of DBN. The ALRDBN introduces the idea of adaptive learning rate into contrastive divergence （CD） algorithm and accelerates its convergence by a self-adjusting learning rate. The training method of weights in this case is designed, in which the adjusting scope of the coefficient in learning rate is determined by performance analysis. Finally, a series of experiments are carried out to test the performance of ALRDBN, and the corresponding results show that the convergence rate is accelerated significantly and the accuracy of prediction is improved as well.
出处
《自动化学报》
EI
CSCD
北大核心
2017年第8期1339-1349,共11页
Acta Automatica Sinica
基金
国家自然科学基金(61533002
61473034)
国家杰出青年科学基金(61225016)
内涵发展-引进人才科研启动费资助~~
关键词
深度信念网
自适应学习率
对比差度
收敛速度
性能分析
Deep belief network, adaptive learning rate, contrastive divergence, convergence rate, performance analysis
