摘要
情绪识别系统旨在提供一种人与机器之间沟通情感的通道,为提高情绪识别准确率,开发了一款基于面部和脑电两个模态决策融合的情绪识别系统,实现4分类情绪识别。针对面部模态使用OpenCV对面部图像进行人脸定位、截取、直方图均衡化等预处理,在Keras平台构建VGG16、DenseNet121和本文设计的卷积神经网络(CNN)模型。针对脑电模态使用FP1通道进行快速傅里叶变换后分为5个频段,再使用傅里叶反变换为时域信号,使用标准差特征提取,最后使用K近邻算法实现情绪识别。决策融合方面提出一种基于加权平均的决策融合算法。系统方面基于树莓派平台开发,搭载TGAM芯片、蓝牙、摄像头和触控屏。使用Fer2013表情数据集对面部模态分类算法测试最高准确率86%。使用DEAP情感脑电数据集验证脑电模态算法取得最高准确率77%。最后设计实验验证多模态融合系统最高取得准确率为96%。
The emotion recognition system aims to provide a channel for emotional communication between humans and machines.To improve the accuracy of emotion recognition,this thesis develops an emotion recognition system based on the fusion of facial and EEG two modal decision-making,achieving four classification emotion recognition.For facial mode,OpenCV is used to preprocess facial images such as face location,interception,histogram equalization,etc.,and VGG16,DenseNet121 and CNN models designed in this paper are built on Keras platform.Using FP1 channel for fast Fourier transform of EEG modality,it is divided into five frequency bands.Then,Fourier inverse transform is used as a time-domain signal,standard deviation feature extraction is used,and finally,K-nearest neighbor algorithm is used for emotion recognition.In terms of decision fusion,a weighted average based decision fusion algorithm is proposed.The system is developed based on the Raspberry Pi platform,equipped with TGAM chip,Bluetooth,camera,and touch screen.The highest accuracy rate of facial modality classification algorithm testing using the Fer2013 facial expression dataset is 86%.Using the DEAP emotional EEG dataset to validate the EEG modality algorithm achieved the highest accuracy of 77%.Finally,experimental design was conducted to verify that the multimodal fusion system achieved a maximum accuracy of 96%.
作者
韩凌
李栋
王增霖
张佳
赵歆卓
Han Ling;Li Dong;Wang Zenglin;Zhang Jia;Zhao Xinzhuo(Shenyang University of Technology,School of Electrical Engineering,Shenyang 110870,China)
出处
《国外电子测量技术》
北大核心
2023年第10期190-195,共6页
Foreign Electronic Measurement Technology
基金
国家自然科学基金(62101357)项目资助。
关键词
情绪识别
人机交互
脑电信号
面部表情
多模态融合
深度学习
emotional recognition
human-computer interaction
EEG signals
facial expressions
multimodal fusion
deep learning
