摘要
传统情感分析方法无法有效处理社交平台中的大量多模态图文数据,暴露出多模态特征融合效果不佳的问题。为此,结合注意力机制与前馈神经网络建立基于对偶注意力机制融合多模态的情感分析模型。该模型利用预训练模型提取文本与图像特征,采用跨模态特征融合模块强化属于多个模态的公有特征,采用单模态自注意力模块提取单个模态私有特征中的有效信息,最终拼接融合多模态特征,实现对多模态数据的高效表征。在推特图文数据集上进行验证实验,通过与多种方法进行比较以及对内部各模态进行消融实验,证实所提模型具有较好的情感分类效果。
Traditional sentiment analysis methods are unable to effectively handle a large amount of multimodal graphic and textual data on so-cial platforms,exposing the problem of poor performance in multimodal feature fusion.To this end,a multimodal sentiment analysis model based on dual attention mechanism fusion is established by combining attention mechanism and feedforward neural network.This model utiliz-es pre trained models to extract text and image features,strengthens public features belonging to multiple modalities using a cross modal fea-ture fusion module,extracts effective information from private features belonging to a single modality using a single modal self attention mod-ule,and finally concatenates and fuses multimodal features to achieve efficient representation of multimodal data.Validation experiments were conducted on the Twitter image and text dataset,comparing with various methods and conducting ablation experiments on internal modalities,confirming that the proposed model has good sentiment classification performance.
作者
李劲哲
吴宇贤
蔡珺恺
王成济
蒋兴鹏
LI Jinzhe;WU Yuxian;CAI Junkai;WANG Chengji;JIANG Xingpeng(School of Computer Science,Central China Normal University,Wuhan 430079,China)
出处
《软件导刊》
2024年第4期178-185,共8页
Software Guide
基金
国家语委“十四五”科研规划研究基地项目(重点项目)(ZDI145-56)
中央高校基本科研业务费资助项目(CCNU23XJ001)
中国博士后科学基金面上项目(2023M741305)。
关键词
多模态情感分析
多头注意力
特征融合
对偶融合
multimodal sentiment analysis
multi-head attention
feature fusion
dual fusion
