摘要
自然语言处理(NLP)研究从早期基于规则的方法转向基于特征分析的机器学习,再转向无需事先进行特征抽取的深度学习,发展很快,但因其需要的文本数据量越来越大,模型训练所需的标注工作量巨大,对算力的要求也越来越高,而难以被广泛应用。基于Transformer的预训练语言模型(T-PTLM)提供了一个新的研究和应用路径:通过大规模无标注文本数据广泛学习语言现象,使模型具有很强的通用性,然后将模型进行迁移和微调,在NLP的许多具体任务应用中均取得了很好的效果。
The research on natural language processing(NLP) has developed rapidly from the early rule-based methods to the machine learning based on the feature analysis, and then to the deep learning without prior feature extraction. However, due to the increasing amount of text data required, the labeling work required for training the models is huge, and the requirements for computing power are getting higher and higher, difficult to be widely used. The transformer-based pre-trained language models(T-PTLM) provide a new research and application path: the extensive learning language phenomena through large-scale unlabeled text data makes the model highly versatile, then transferring and fine-tuning this model in many NLP applications has achieved great results.
作者
易顺明
许礼捷
周洪斌
Yi Shunming;Xu Lijie;Zhou Hongbin(Shazhou Professional Institute of Technology,Zhangjiagang 215600,Jiangsu,China)
出处
《沙洲职业工学院学报》
2022年第3期1-6,共6页
Journal of Shazhou Professional Institute of Technology
基金
2022年江苏高校“青蓝工程”优秀教学团队培养项目(苏教师函[2022]29号)。
