摘要
随着互联网信息的发展,如何有效地表示不同语言所含的信息已成为自然语言处理(Natural Language Processing,NLP)领域的一项重要任务.然而,很多传统的机器学习模型依赖在高资源语言中进行训练,无法迁移到低资源语言中使用.为了解决这一问题,结合迁移学习和深度学习模型,提出一种多语言双向编码器表征量(Multi-lingual Bidirectional Encoder Representations from Transformers,M-BERT)的迁移学习方法.该方法利用M-BERT作为特征提取器,在源语言领域和目标语言领域之间进行特征转换,减小不同语言领域之间的差异,从而提高目标任务在不同领域之间的泛化能力.首先,在构建BERT模型的基础上,通过数据收集处理、训练设置、参数估计和模型训练等预训练操作完成M-BERT模型的构建,并在目标任务上进行微调.然后,利用迁移学习实现M-BERT模型在跨语言文本分析方面的应用.最后,在从英语到法语和德语的跨语言迁移实验中,证明了本文模型具有较高的性能质量和较小的计算量,并在联合训练方案中达到了96.2%的准确率.研究结果表明,该文模型实现了跨语言数据迁移,且验证了其在跨语言NLP领域的有效性和创新性.
With the development of internet-based information,effectively representing the information contained in different languages has become an important task in the field of Natural Language Processing(NLP).However,many traditional machine learning models rely on training in high-resource languages and cannot be used in low-resource languages.To address this issue,this paper proposes a migration learning method called Multi-lingual Bidirectional Encoder Representations from Transformers(M-BERT)that combines migration learning with deep learning models.This method utilizes M-BERT as a feature extractor to transform features between the source language domain and the target language domain,thereby reducing the differences between different language domains and improving the generalization ability of the target task across domains.First,the BERT model was constructed.Then,the construction of the M-BERT model was completed through pre-training operations such as data collection and processing,training setup,parameter estimation,and model training.Fine-tuning was performed on the target task.Finally,migration learning was employed to apply the M-BERT model in cross-lingual text analysis.The cross-lingual migration experiments from English to French and German demonstrated that the model proposed in this paper exhibited high performance quality and required minimal computational effort,achieved an accuracy of 96.2%in the joint training scheme.The research results indicate that this model achieved cross-lingual data migration,validated its effectiveness and innovation in the field of cross-lingual NLP.
作者
王坤
盛鸿宇
WANG Kun;SHENG Hongyu(Sichuan Vocational College of Information Technology,Guangyuan Sichuan 628017,China;College of Robotics,Beijing Union University,Beijing 100101,China)
出处
《西南大学学报（自然科学版）》
CAS
CSCD
北大核心
2024年第4期153-163,共11页
Journal of Southwest University(Natural Science Edition)
基金
国家自然科学基金项目(12104289).
