多文档文摘作为自然语言处理领域的重要技术之一,能从不同角度辅助用户实现高效的信息获取.由于文档集合内的内容往往来自不同的信息源,文本之间通常存在丰富而复杂的语义关系.常用的基于词的文档表示法,难以为文摘的语义分析过程提供...多文档文摘作为自然语言处理领域的重要技术之一,能从不同角度辅助用户实现高效的信息获取.由于文档集合内的内容往往来自不同的信息源,文本之间通常存在丰富而复杂的语义关系.常用的基于词的文档表示法,难以为文摘的语义分析过程提供充足而准确的数据信息.为此,我们提出使用维基百科——当今世界最大的在线概念语料库——为多文档文摘的提取提供语义支持.一方面,我们通过提取文档中的维基概念,生成准确一致的句子表示形式.另一方面,在计算句子特征时,我们利用维基词条的首段指导机器文摘的提取.我们首先通过计算概念在维基中的全局相关性和当前文档集内的局部相关性,获取概念的权重.然后在维基概念表示的基础上,为文档中的句子提取多种基于维基的特征,并最后用于文摘生成.在实验中,我们依次用各个维基特征独立生成文摘,并使用ROUGE(Recall-Oriented Understudy for Gisting Evaluation,面向召回率的要点评估)指标评价文摘质量.通过比较,实验验证了维基词条首段能较好的提升文摘质量.展开更多
A concept-based approach is expected to resolve the word sense ambiguities in information retrieval and apply the semantic importance of the concepts, instead of the term frequency, to representing the contents of a d...A concept-based approach is expected to resolve the word sense ambiguities in information retrieval and apply the semantic importance of the concepts, instead of the term frequency, to representing the contents of a document. Consequently, a formalized document framework is proposed. The document framework is used to express the meaning of a document with the concepts which are expressed by high semantic importance. The framework consists of two parts： the ＂domain＂ information and the ＂situation ＆ background＂ information of a document. A document-extracting algorithm and a two-stage smoothing method are also proposed. The quantification of the similarity between the query and the document framework depends on the smoothing method. The experiments on the TREC6 collection demonstrate the feasibility and effectiveness of the proposed approach in information retrieval tasks. The average recall level precision of the model using the proposed approach is about 10% higher than that of traditional ones.展开更多
文摘多文档文摘作为自然语言处理领域的重要技术之一,能从不同角度辅助用户实现高效的信息获取.由于文档集合内的内容往往来自不同的信息源,文本之间通常存在丰富而复杂的语义关系.常用的基于词的文档表示法,难以为文摘的语义分析过程提供充足而准确的数据信息.为此,我们提出使用维基百科——当今世界最大的在线概念语料库——为多文档文摘的提取提供语义支持.一方面,我们通过提取文档中的维基概念,生成准确一致的句子表示形式.另一方面,在计算句子特征时,我们利用维基词条的首段指导机器文摘的提取.我们首先通过计算概念在维基中的全局相关性和当前文档集内的局部相关性,获取概念的权重.然后在维基概念表示的基础上,为文档中的句子提取多种基于维基的特征,并最后用于文摘生成.在实验中,我们依次用各个维基特征独立生成文摘,并使用ROUGE(Recall-Oriented Understudy for Gisting Evaluation,面向召回率的要点评估)指标评价文摘质量.通过比较,实验验证了维基词条首段能较好的提升文摘质量.
基金The National Basic Research Program of China(973Program)(No.2004CB318104),the Knowledge Innovation Pro-gram of Chinese Academy of Sciences (No.13CX04).
文摘A concept-based approach is expected to resolve the word sense ambiguities in information retrieval and apply the semantic importance of the concepts, instead of the term frequency, to representing the contents of a document. Consequently, a formalized document framework is proposed. The document framework is used to express the meaning of a document with the concepts which are expressed by high semantic importance. The framework consists of two parts： the ＂domain＂ information and the ＂situation ＆ background＂ information of a document. A document-extracting algorithm and a two-stage smoothing method are also proposed. The quantification of the similarity between the query and the document framework depends on the smoothing method. The experiments on the TREC6 collection demonstrate the feasibility and effectiveness of the proposed approach in information retrieval tasks. The average recall level precision of the model using the proposed approach is about 10% higher than that of traditional ones.
