摘要
处理机器阅读理解任务时,识别其中没有答案的问题是自然语言处理领域的一个新的挑战。该文提出U-Net模型来处理这个问题,该模型包括3个主要成分:答案预测模块、无答案判别模块和答案验证模块。该模型用一个U节点将问题和文章拼接为一个连续的文本序列,该U节点同时编码问题和文章的信息,在判断问题是否有答案时起到重要作用,同时对于精简U-Net的结构也有重要作用。与基于预训练的BERT不同,U-Net的U节点的信息获取方式更多样,并且不需要巨大的计算资源就能有效地完成机器阅读理解任务。在SQuAD 2.0中,U-Net的单模型F_(1)得分72.6、EM得分69.3,U-Net的集成模型F_(1)得分74.9、EM得分71.4,均为公开的非基于大规模预训练语言模型的模型结果的第一名。
Machine reading comprehension with unanswerable questions is a challenging task.In this paper,we propose a unified model,called U-Net,with three important components:answer pointer,no-answer pointer,and answer verifier.We introduce a universal node which processes the question and its context passage as a single contiguous sequence of tokens.The universal node encodes the fused information from both the question and passage,and plays an important role to predict whether the question is answerable and also greatly improves the conciseness of the U-Net.Different from the models based on pre-trained BERT,universal node fuses information from passage and question in a variety of ways and avoids the huge computation.The single U-Net model achieves the F_(1)score of 72.6 and the EM score of 69.3 on SQuAD 2.0,and the ensemble version,74.9 and 71.4,respectively.Both version of U-Net models rank top among the models without a large scale pre-trained language model.
作者
孙付
李林阳
邱锡鹏
刘扬
黄萱菁
SUN Fu;LI Linyang;QIU Xipeng;LIU Yang;HUANG Xuanjing(School of Computer Science,Fudan University,Shanghai 201210,China;Liulishuo Silicon Valley AI Lab,San Francisco 94104,USA)
出处
《中文信息学报》
CSCD
北大核心
2021年第2期99-106,共8页
Journal of Chinese Information Processing
基金
国家自然科学基金(61672162)。
