摘要
不同领域的情感文本表达方式不一样,通常需要为各个领域训练相应的情感分析模型。针对无法用一个模型进行高效多领域情感分析的问题,提出了基于提示微调(prompt tuning)的多领域文本情感分析方法MSAPT。借助hard prompt,指示情感文本的所属领域和待选的情感标签,调动不同领域情感分析相关的知识,再为情感分析预训练一个统一的“通才模型”,在下游的各领域文本学习中,保持模型冻结,通过prompt tuning使模型学习到下游各领域情感文本的特征。MSAPT仅需保存一个模型和一些参数量远远小于模型的prompt,实现了多领域情感分析。在多个属于不同领域的情感文本数据集上进行实验,结果表明仅进行prompt tuning时,MSAPT效果优于模型微调(model tuning)的。最后,分别对适应特定领域的prompt tuning、hard prompt、soft prompt的长度和中间训练数据集的大小进行消融实验,从证明其对情感分析效果的影响。
The expression of sentiment texts in different domains are different,so it is usually necessary to train the corresponding sentiment analysis model for each domain.In order to solve the problem that one model cannot be used for multi-domain sentiment analysis,this paper proposes a multi-domain text sentiment analysis method based on prompt tuning,called MSAPT.With the help of hard prompts,indicating the domain of the emotional text and the selected emotional labels,the model is prompted to draw on its knowledge of different domain sentiment analysis.Then,a unified"generalized model"is pretrained for sentimental analysis.In downstream learning of various domain texts,the model is frozen and prompt tuning is used to make the model learn the characteristics of emotional text in each downstream domain.MSAPT only requires saving a model and some prompts with far fewer parameters than the model for multi-domain sentiment analysis.Experiments were conducted using multiple datasets of emotional text in different fields,and the results show that MSAPT outperforms model fine-tuning when only prompted tuning is applied.Finally,the length of prompt tuning,hard prompt adapted to specific domains,soft prompt and the size of intermediate training dataset are ablated respectively,to prove their impact on the effectiveness of sentiment analysis.
作者
赵文辉
吴晓鸰
凌捷
HOON Heo
ZHAO Wen-hui;WU Xiao-ling;LING Jie;HOON Heo(School of Computer Science and Technology,Guangdong University of Technology,Guangzhou 510006,China;Samsung Electro-mechanics,Suwon 16674,Korea)
出处
《计算机工程与科学》
CSCD
北大核心
2024年第1期179-190,共12页
Computer Engineering & Science
基金
广东省国际科技合作领域项目(2019A050513010)
工业装备质量大数据工业和信息化部重点实验室开放课题(2021-1EQBD-02)。
关键词
多领域情感分析
提示微调
预训练语言模型
T5
multi-domain sentiment analysis
prompt tuning
pre-trained language model
