摘要
中文分词应用中一个很重要的问题就是缺乏词的统一性定义。不同的分词标准会导致不同的分词结果,不同的应用也需要不同的分词结果。而针对不同的分词标准开发多个中文分词系统是不现实的,因此针对多种不同的分词标准,如何利用现有的分词系统进行灵活有效的输出就显得非常重要。本文提出了一种新的基于转换的学习方法,对分词结果进行后处理,可以针对不同的分词标准进行灵活有效的输出。不同于以往的用于分词的转换学习方法,该方法有效利用了一些语言学信息,把词类和词內结构信息引入规则模板和转换规则中。为了验证该方法,我们在4个标准测试集上进行了分词评测,取得了令人满意的效果。
A well known problem for Chinese word segrnentation（CWS）is that we can not have a unique definition of words. Different standards may result in different word segmentation outputs. It is unrealizable to develop different CWS systems according to different applications or standards, so it is significantly important to flexibly adapt segmentation outputs towards different standards or applications using existing CWS system. The paper presents a linguistically enriched transformation-based learning approach for performing CWS adaptation as a postprocessor. Different from other transform-based learning used in CWS, the approach utilizes some linguistics information, and introduces word class and word internal structure to rule templates and transformations. The performance of the approach is evaluated on four different test sets, which represent four different standards. It turns out to be comparable to several state-ofthe-art approaches which perform Chinese word segmentation based on single standard.
出处
《计算机科学》
CSCD
北大核心
2006年第3期160-164,共5页
Computer Science
关键词
中文分词
规则模板
词类
词内结构
基于转换的学习(TBL)
Chinese word segmentation, Rule template, Word class, Word internal structure, Transformation-based Learning（TBL）
