摘要
由于卷积神经网络中多层感知器使用梯度下降算法进行训练,存在收敛速度慢和易于陷入局部极小的问题。针对此问题,提出一种利用信息交互计算最优初始化权重的方法改善网络结构,该方法可有效减少训练时间并可避免陷入局部极小。利用数学理论推导出Re LU函数最优初始化权值的公式,利用该方法改进2-channel网络结构,直接代入数据可求出最优初始权值。通过3个数据集的多次训练和测试,灰度图像的平均匹配准确率提升了1. 0%左右,FPR95平均值也由5. 23降至4. 65,初始化权重的设置可避免神经元进入硬饱和区,同时网络还具有效果稳定、收敛速度快的优点。
Multilayer perceptrons of convolutional neural networks use the gradient descent algorithm for training,there are often problems of slow convergence and small localization. In order to solve the problem,we propose a method to calculate the optimal initialization weight by using information interaction to improve the proposed network structure,which can effectively reduce the training time and avoid the local minimum problem. Firstly we use mathematical theory to derive the formula for the optimal initialization weight of the ReLU function. We target 2-channel network structure using this method,directly substituting data to find the optimal initial weight.Through multiple training and testing of the three data sets,better results are obtained. Average matching accuracy for grayscale images is increased by about 1. 5%. For the FPR95,the average value also dropped from5. 23 to 4. 65. Initialization weight setting prevents neurons from entering the hard saturated region. And it has the advantages of higher precision,stable effect and fast convergence.
作者
徐阳
张忠伟
刘明
XU Yang;ZHANG Zhongwei;LIU Ming(School of Electrical Information and Engineering,Northeast Petroleum University,Daqing 163318,China;Pipeline Qinhuangdao Oil and Gas Branch,China National Petroleum Corporation,Qinhuangdao 066000,China)
出处
《吉林大学学报（信息科学版）》
CAS
2019年第1期107-112,共6页
Journal of Jilin University（Information Science Edition）
基金
国家大学生创新计划基金资助项目(201810220004)
关键词
神经网络
权重
信息交互
neural network
weight
information interaction
