摘要
针对铁轨缺陷检测神经网络模型参数多、计算量大的问题,提出了一种基于知识蒸馏的轨道检测轻量化模型及其训练方法,该网络模型由六层卷积层和三层全连接层构成,将训练好的DenseNet模型作为教师网络,采用知识蒸馏的方法指导训练,使得轻量级模型的训练更加简单,也保证了准确性。在模型的训练阶段加入最小化锐度SAM优化算法,提高了模型的泛化能力,然后将VggNet、ResNet、DenseNet等模型当作对比实验,评价模型优劣。经过知识蒸馏训练的自定义轻量级模型在铁轨检测数据集中的平均准确率为97.3%,且模型参数大小仅为0.738 M,均优于其他网络模型,可部署在众多移动终端中。
Aiming at the problems of many parameters and large calculation of the neural network model for rail defect detection,a lightweight model for rail detection based on knowledge distillation and its training method are proposed.The network model is composed of six layers of convolution layer and three layers of full connection layer.The trained DenseNet model is used as the teacher network to guide the training with the method of knowledge distillation,which makes the training of lightweight model simpler and ensures its accuracy.In the training phase of the model,SAM optimization algorithm with minimum sharpness is added to greatly improve the generalization ability of the model.Then VggNet,ResNet,DenseNet and other models are used as comparative experiments to evaluate the model.The average accuracy of the customized lightweight model trained by knowledge distillation in the rail detection data set is 97.3%,and the model parameter size is only 7.38 M,which is superior to other network models and can be deployed in many mobile terminals.
作者
汤文亮
曾建杨
何文晶
TANG Wenliang;ZENG Jianyang;HE Wenjing(School of Information Engineering,East China Jiaotong University,Nanchang 330013,China)
出处
《重庆理工大学学报（自然科学）》
北大核心
2023年第9期173-179,共7页
Journal of Chongqing University of Technology：Natural Science
基金
国家自然科学基金项目(52062016)
江西省重点研发计划项目(20203BBE53034)
江西省教育厅科技重点项目(GJJ190296)。
关键词
深度学习
知识蒸馏
图像增强
图像识别
优化算法
deep learning
knowledge distillation
image enhancement
image recognition
optimization algorithm
