摘要
为加快卷积神经网络的训练,该研究提出一种受区域分解方法启发的新型学习策略。将该方法应用于残差网络(ResNet)进行图像分类时,使用ResNet32可获得最佳结果。进一步地,将ResNet32分成4个子网络,其中每个子网具有0.47 M参数,此为原始ResNet32的1/16,从而简化了学习过程。此外,由于可以并行训练子网络,因此在使用CIFAR-10数据集进行分类任务时,计算时间可以从8.53 h(通过常规学习策略)减少到5.65 h,分类准确性从92.82%提高到94.09%。CIFAR-100和Food-101数据集也实现了类似的改进。实验结果显示,所提出的学习策略可以大大减少计算时间,并提高分类的准确性。这表明所提出的策略可以潜在地应用于训练带有大量参数的卷积神经网络。
We propose a novel learning strategy inspired by domain decomposition methods to accelerate the training of convolutional neural network(CNN).The proposed method is applied to residual networks(ResNet)for image classification tasks.The best result is achieved with ResNet32.In this case,we split ResNet32 into 4 sub-networks.Each sub-network has 0.47 M parameters which is 1/16 of the original ResNet32,thereby facilitating the learning process.Moreover,because the sub-networks can be trained in parallel,the computational time can therefore be reduced to 5.65 h from 8.53 h(by the conventional learning strategy)in the classification task with the CIFAR-10 dataset.We also find that the accuracy of the classification is improved to 94.09%from 92.82%.Similar improvements are also achieved with the CIFAR-100 and Food-101 datasets.In conclusion,the proposed learning strategy can reduce the computational time substantially with improved accuracy in classification.The results suggest that the proposed strategy can potentially be applied to train CNN with a large amount of parameters.
作者
张卫
古林燕
刘嘉
ZHANG Wei;GU Linyan;LIU Jia(Shenzhen Institutes of Advanced Technology,Chinese Academy of Sciences,Shenzhen 518055,China;Shenzhen College of Advanced Technology,University of Chinese Academy of Sciences,Shenzhen 518055,China;Shenzhen Key Laboratory for Exascale Engineering and Scientific Computing,Shenzhen 518055,China)
出处
《集成技术》
2020年第6期48-58,共11页
Journal of Integration Technology
基金
国家自然科学基金项目(81661168015,81871447)
深圳市科学技术创新委员会项目(ZDSYS201703031711426)。
关键词
学习策略
区域分解
并行训练
加速
learning strategy
domain decomposition
parallel training
speed up
