结合双注意力模块和ShuffleNet模型的微表情识别

Micro⁃expression recognition based on dual attention module and ShuffleNet model

下载PDF

导出

摘要针对微表情运动的局限性和识别效果不理想的问题,提出了一种结合双注意力模块和ShuffleNet模型的微表情识别方法。该方法将提取的峰值帧的水平和垂直光流图,以通道叠加的方式连接送进所设计的网络进行训练。利用高效且轻量化的ShuffleNet模型堆叠的卷积神经网络(Convolutional neural network,CNN),极大地降低了训练的参数量,在ShuffleNet网络中加入可自适应特征细化的双注意力模块,使得网络在通道和空间维度寻找微表情运动的有用特征信息。在通道注意力模块中,使用一维卷积融合全局池化后的一维通道特征来保持相邻通道的相关性;在空间注意力模块中,采用较小的3×3和5×5卷积核提取不同的空间信息并融合。实验结果表明,在微表情识别方面,相比于基准方法的三个正交平面的局部二值模式(Local binary patterns from three orthogonal planes,LBP-TOP),未加权F1值(Unweighted F1-score,UF1)和未加权平均召回率(Unweighted average recall,UAR)分别提高了0.1445和0.1556,识别性能有很大的提升。 Aiming at the limitation of micro⁃expression movement and the problem of unsatisfactory recognition effect,the micro⁃expression recognition method combining dual attention module and ShuffleNet model is proposed.In this method,the horizontal and vertical optical flow graphs of the extracted apex frames are connected and sent to the designed network for training in the way of channel stacking.The Convolutional neural network(CNN)stacked by the efficient and lightweight ShuffleNet model greatly reduces the amount of training parameters.The dual attention module with adaptive feature refinement is added to the ShuffleNet network,which enables the network to find useful feature information in micro⁃expression movements in channel and spatial dimensions.In the channel attention module,1D convolution is used to fuse the globally pooled 1D channel features to maintain the correlation of adjacent channels.In the spatial attention module,smaller convolution kernels of3×3 and5×5 are used to extract different spatial information and integration.The experimental results show that compared with Local binary patterns from three orthogonal planes(LBP⁃TOP)of the benchmark method,Unweighted F1⁃score(UF1)and Unweighted average recall(UAR)are improved by 0.1445 and 0.1556 respectively in micro⁃expression recognition,and the recognition performance is greatly improved.

作者李飞汪国强尤美明 LI Fei;WANG Guoqiang;YOU Meiming(College of Electronic Engineering,Heilongjiang University,Harbin 150080,China)

机构地区黑龙江大学电子工程学院

出处《黑龙江大学自然科学学报》 CAS 2023年第4期468-478,共11页 Journal of Natural Science of Heilongjiang University

基金国家自然科学基金(51607059) 黑龙江省自然科学基金(QC2017059)。

关键词微表情识别深度学习光流法双注意力模块 ShuffleNet模型卷积神经网络 micro⁃expression recognition deep learning optical flow mothod dual attention module ShuffleNet model convolutional neural network

分类号 TP391 [自动化与计算机技术—计算机应用技术]