摘要
提出了一种基于权值拟熵的“剪枝算法”与权值敏感度相结合的新方法,在“剪枝算法”中将权值拟熵作为惩罚项加入目标函数中,使多层前向神经网络在学习过程中自动约束权值分布,并以权值敏感度作为简化标准,避免了单纯依赖权值大小剪枝的随机性.同时,又针对剪枝算法在优化多输入多输出网络过程中计算量大、效率不高的问题,提出了一种在级联—相关(cascade-correlation,CC)算法的基础上从适当的网络结构开始对网络进行构建的快速“构造算法”.仿真结果表明这种快速构造算法在收敛速度、运行效率乃至泛化性能上都更胜一筹.*
Based on pseudo-entropy of weights, a new method is proposed to integrate pruning algorithm with sensitivity weights. The pruning algorithm introduces the pseudo-entropy of weights as a penalty term into the normal objective function, and the distribution of weights is automatically constrained by a muhilayer feed-forward neural network during the training process. The weight sensitivity is served as the simplification criteria of pruning to avoid the pruning randomicity caused by only using the weights. Meanwhile, for the problems of heavy computation hurden and low efficiency of pruning algorithm in optimizing the multi-input and multi-output networks, a fast constructive algorithm is put forward, which is based on the Cascade-Correlation （CC） algorithm and constructs the new neural network from a proper network structure. The simulation results show that this fast constructive algorithm is a better choice in terms of convergence rate, computational efficiency and even generalization performance.
出处
《信息与控制》
CSCD
北大核心
2006年第6期700-704,710,共6页
Information and Control
基金
国家自然科学基金资助项目(60574051)
关键词
神经网络结构优化
剪枝算法
权值拟熵
权值敏感度
快速构造算法
泛化性能
neural network structure optimization
pruning algorithm
pseudo-entropy of weights
weight sensitivity
fast constructive algorithm
generalization performance
