摘要
近年来,深度神经网络在各种具有挑战性的任务上取得了巨大的成功,不断刷新人们对人工智能的认识。但是,深度神经网络模型的参数量巨大、计算成本、存储成本过高,难以部署到资源受限的边缘计算设备中。因此,人们开始从网络的架构和参数量两个角度尝试对网络进行轻量化设计,同时保证神经网络性能可接受。本文从网络参数轻量化的角度出发,首先简要回顾深度神经网络发展历史和工作原理;其次,介绍当前主流的3类参数轻量化方法:参数量化、张量分解以及参数共享;然后,从思想描述、适用层级、训练方式等维度对比分析方法优劣;最后,对神经网络轻量化的未来发展方向进行展望。
In recent years,deep neural networks have achieved great success on a variety of challenging tasks,constantly refreshing people’s understanding of artificial intelligence.However,the deep neural network model is difficult to deploy to resource-limited edge computing devices because of its huge parameters,high computing and storage costs.Therefore,people begin to try to lightweight network design from the perspective of network architecture and parameter number,while ensuring the performance of neural network is acceptable.In this paper,the development history and working principle of deep neural networks are reviewed briefly from the angle of network parameter lightweight.Secondly,three kinds of current mainstream parameter lightweight methods are introduced:parameter quantization,tensor decomposition and parameter sharing.Then the advantages and disadvantages of the method are compared and analyzed from the dimensions of thought description,application level and training mode.Finally,the future development direction of neural network lightweight is prospected.
作者
林冲
闫文君
纪纲
于斌
王莹
LIN Chong;YAN Wen-jun;JI Gang;YU Bin;WANG Ying(Unit 91206 of PLA,Qingdao 266109,China;Institute of Information Fusion,Naval Aviation University,Yantai 264001,China)
出处
《中国电子科学研究院学报》
2024年第4期350-363,379,共15页
Journal of China Academy of Electronics and Information Technology
基金
国家自然科学基金面上项目(62371645)
泰山学者工程专项基金(ts201511020)
山东省高等学校青年创新团队发展计划(2022kj084)
海军航空大学青年基金(H3202209003)。
关键词
深度神经网络
人工智能
边缘计算设备
神经网络参数轻量化
deep neural networks
artificial intelligence
edge computing devices
neural networks parameters lightweight
