斯坦福中文分词 java,斯坦福的中文分词开源项目

This software is a Java implementation of the CRF-based Chinese Word Segmenter described in:

Huihsin Tseng, Pichuan Chang, Galen Andrew, Daniel Jurafsky

and Christopher Manning. "A Conditional Random Field Word Segmenter."

In Fourth SIGHAN Workshop on Chinese Language Processing. 2005.The

system requires Java 1.5+ to be installed. For the current setting in

the scripts files, 1G of memory is required. But with smaller files,

you can change the option java -mx1g to smaller numbers. Two models with two different segmentation standards are included -- Chinese Penn Treebank standard and Beijing University standard.

The segmenter is licensed under the GNU GPL. (Note that this is the full GPL - which allows its use for research purposes or other free software projects but does not allow its incorporation into any type of commercial software, even in part or in translation; see GPL FAQ.) Source is included. The package includes components for command-line invocation and a Java API.

The download is a 14 MB gzipped tar file (mainly consisting of

included model files). If you unpack the tar file, you should have

everything needed. Simple scripts are included to invoke the segmenter.

Please send any questions or feedback, or extensions and bugfixes to: java-nlp-support@lists.stanford.edu.

其他的项目包括：

Supported software distributions

This code is being developed, and we try to answer questions and fix bugs on a best-effort basis.

Java

implementations of probabilistic natural language parsers, both highly

optimized PCFG and dependency parsers, and a lexicalized PCFG parser.

Including: Parser FAQ and Online parser demo.A Java implementation of a maximum-entropy (CMM) part-of-speech (POS) tagger.A

Java implementation of a Conditional Random Field sequence model,

together with well-engineered features for Named Entity Recognition.A Java implementation of a CRF-based Chinese Word SegmenterA

Java implementation of conditional loglinear model classification

(a.k.a. maximum entropy or multiclass logistic regression models)A

Java implementation of a Tgrep2-style utility for matching patterns in

trees, and a tree-transformation utility built on top of this matching

language.