Research on Chinese Word Segmentation Based on Data-Driven Approach

Chinese automatic segmentation is one of the most difficult problems in computer Chinese information disposal. Introduces a data-driven Chinese word segmentation, develops a word segmentation system based on this approach. Closed tests conducted on PKU-ICL-PD-Corpus perform well. It consists of a new words recognizer, a base segmentation algorithm, and procedures for combining single characters, suffixes, and checking segmentation consistencies.