摘要
情感在感知、决策、逻辑推理和社交等一系列智能活动中起到核心作用,是实现人机交互和机器智能的重要元素。近年来,随着多媒体数据爆发式增长及人工智能的快速发展,情感计算与理解引发了广泛关注。情感计算与理解旨在赋予计算机系统识别、理解、表达和适应人的情感的能力来建立和谐人机环境,并使计算机具有更高、更全面的智能。根据输入信号的不同,情感计算与理解包含不同的研究方向。本文全面回顾了多模态情感识别、孤独症情感识别、情感图像内容分析以及面部表情识别等不同情感计算与理解方向在过去几十年的研究进展并对未来的发展趋势进行展望。对于每个研究方向,首先介绍了研究背景、问题定义和研究意义;其次从不同角度分别介绍了国际和国内研究现状,包括情感数据标注、特征提取、学习算法、部分代表性方法的性能比较和分析以及代表性研究团队等;然后对国内外研究进行了系统比较,分析了国内研究的优势和不足;最后讨论了目前研究存在的问题及未来的发展趋势与展望,例如考虑个体情感表达差异问题和用户隐私问题等。
Humans are emotional creatures.Emotion plays a key role in various intelligent actions,including perception,decision-making,logical reasoning,and social interaction.Emotion is an important and dispensable component in the realization of human-computer interaction and machine intelligence.Recently,with the explosive growth of multimedia data and the rapid development of artificial intelligence,affective computing and understanding has attracted much research attention.It aims to establish a harmonious human-computer environment by giving the computing machines the ability to recognize,understand,express,and adapt to human emotions,and to make computers have higher and more comprehensive intelligence.Based on the input signals,such as speech,text,image,action and gait,and physiological signals,affective computing and understanding can be divided into multiple research topics.In this paper,we will comprehensively review the development of four important topics in affective computing and understanding,including multi-modal emotion recognition,autism emotion recognition,affective image content analysis,and facial expression recognition.For each topic,we first introduce the research background,problem definition,and research significance.Specifically,we introduce how such topics were proposed,what the corresponding tasks do,and why it is important in different applications.Second,we introduce the international and domestic research on emotion data annotation,feature extraction,learning algorithms,performance comparison and analysis of some representative methods,and famous research teams.Emotion data annotation is conducted to evaluate the performances of affective computing and understanding algorithms.We briefly summarize how categorical and dimensional emotion representation models in psychology are used to construct datasets and the comparisons between these datasets.Feature extraction aims to extract discriminative features to represent emotions.We summarize both hand-crafted features in the early years and deep features in the deep learning era.Learning algorithms aim to learn a mapping between extracted features and emotions.We also summarize and compare both traditional and deep models.For a better understanding of how existing methods work,we report the emotion recognition results of some representative and influential methods on multiple datasets and give some detailed analysis.To better track the latest research for beginners,we briefly introduce some famous research teams with their research focus and main contributions.After that,we systematically compare the international and domestic research,and analyze the advantages and disadvantages of domestic research,which would motivate and boost the future research for domestic researchers and engineers.Finally,we discuss some challenges and promising research directions in the future for each topic,such as 1)image content and context understanding,viewer contextual and prior knowledge modeling,group emotion clustering,viewer and image interaction,and efficient learning for affective image content analysis;2)data collection and annotation,real-time facial expression analysis,hybrid expression recognition,personalized emotion expression,and user privacy.Since emotion is an abstract,subjective,and complex high-level semantic concept,there are still some limitations of existing methods,and many challenges still remain unsolved.Such promising future research directions would help to reach the emotional intelligence for a better human-computer interaction.
作者
姚鸿勋
邓伟洪
刘洪海
洪晓鹏
王甦菁
杨巨峰
赵思成
Yao Hongxun;Deng Weihong;Liu Honghai;Hong Xiaopeng;Wang Sujing;Yang Jufeng;Zhao Sicheng(Harbin Institute of Technology,Harbin 150006,China;Beijing University of Posts and Telecommunications,Beijing 100876,China;Institute of Psychology,Chinese Academy of Sciences,Beijing 100083,China;Nankai University,Tianjin 300071,China;Columbia University,New York 10032,USA)
出处
《中国图象图形学报》
CSCD
北大核心
2022年第6期2008-2035,共28页
Journal of Image and Graphics
基金
国家自然科学基金项目(61831022)。
关键词
情感计算
情感识别
孤独症
图像识别
表情识别
affective computing
emotion recognition
autism
image recognition
expression recognition
