摘要
组合型歧义切分字段一直是汉语自动分词研究中的一个难点。该文将之视为与WordSenseDisambiguation(WSD)相等价的问题。文章借鉴了WSD研究中广泛使用的向量空间法,选取了20个典型的组合型歧义进行了详尽讨论。提出了根据它们的分布“分而治之”的策略,继而根据实验确定了与特征矩阵相关联的上下文窗口大小、窗口位置区分、权值估计等要素,并且针对数据稀疏问题,利用词的语义代码信息对特征矩阵进行了降维处理,取得了较好的效果。笔者相信,这个模型对组合型歧义切分字段的排歧具有一般性。
Combinatorial ambiguity is a vital issue in Chinese word segmentation.We regard it as an equivalence of the problem of word sense disambiguation(WSD)in language computing.In sight of the vector space model commonly used in WSD and based on detailed observations on 20 typical combinatorial ambiguities,this paper at first presents the strategy of treating these ambiguities separately according to their distribution,then determines by experiments the key factors regarding feature matrix(the size of the context window,the sensitivity of locations in the window as well as weighting of feature words),and lastly makes use of semantic codes of words so as to reduce the dimension of the feature matrix.Preliminary results show that the proposed scheme is satisfactory in performance and may serve as a general solution for processing combinatorial ambiguities.
出处
《计算机工程与应用》
CSCD
北大核心
2001年第19期87-89,106,共4页
Computer Engineering and Applications
基金
国家重点基础研究发展规划项目资助课题(课题编号:G1998030507)
