Stanford NER CRF FAQ

原网页http://nlp.stanford.edu/software/crf-faq.html

• 1.我能够训练自己的NER模型么？
• 2.我可以使用尽量少的内存来训练NER模型么？
• 3.我如何从多个文件中训练一个模型？
• 4.在程序中使用CRFClassifier，API是什么?
• 5.我可以设置斯坦福NER系统，允许单个jar部署，而不是必须从分离的多个文件加载NER模型？
• 6.对于我们的web 5.0系统，我能够运行Stanford NER作为一个server/service/servlet？
• 7.有哪些选项可用于格式化分类器的输出？
• 8.为什么我无法重现您的CoNLL 2003系统的结果？ 或者：我如何使用CoNLL 2003在英语上获得更好的性能？
• 9.如何让德语NER工作正常？ 我对字符编码理解不深。
• 10.什么是分类器的渐进内存增长？
• 11.现有的模型可以扩展么？
• 12.你可以发布用于构建公开发布的模型的数据么？
• 13.NER结果是确定性的吗？ 为什么同一数据的结果会发生变化？
• 14.如何NER标记已分词的文本？
• 15.什么选项可用于文本输入处理？
• 16.NER是否使用词性标签？
• 17.我如何在Stanford NER的帮助下使用gazette？
• 18.斯坦福NER可以与Solr集成么？

这里仅仅先只对第一个大致翻译
你应该至少浏览CRFClassifier 和 NERFeatureFactory类的javaDoc。
训练数据应该是使用tab键分隔的列，至少要有两列，单词一列，单词的类别标签一列。
新建jane-austen-emma-ch1.txt文件（将下面内容粘贴进去），或者直接下载从http://nlp.stanford.edu/software/ner-example/jane-austen-emma-ch1.txt

CHAPTER I

Emma Woodhouse handsome clever and rich with a comfortable home
and happy disposition seemed to unite some of the best blessings
of existence and had lived nearly twenty one years in the world
with very little to distress or vex her 

She was the youngest of the two daughters of a most affectionate
indulgent father and had in consequence of her sister's marriage
been mistress of his house from a very early period  Her mother
had died too long ago for her to have more than an indistinct
remembrance of her caresses and her place had been supplied
by an excellent woman as governess who had fallen little short
of a mother in affection 

Sixteen years had Miss Taylor been in Mr Woodhouse's family
less as a governess than a friend very fond of both daughters
but particularly of Emma  Between them it was more the intimacy
of sisters  Even before Miss Taylor had ceased to hold the nominal
office of governess the mildness of her temper had hardly allowed
her to impose any restraint and the shadow of authority being
now long passed away they had been living together as friend and
friend very mutually attached and Emma doing just what she liked
highly esteeming Miss Taylor's judgment but directed chiefly by
her own 

The real evils indeed of Emma's situation were the power of having
rather too much her own way and a disposition to think a little
too well of herself these were the disadvantages which threatened
alloy to her many enjoyments  The danger however was at present
so unperceived that they did not by any means rank as misfortunes
with her 

Sorrow came  a gentle sorrow  but not at all in the shape of any
disagreeable consciousness  Miss Taylor married  It was Miss
Taylor's loss which first brought grief  It was on the wedding day
of this beloved friend that Emma first sat in mournful thought
of any continuance  The wedding over and the bride people gone
her father and herself were left to dine together with no prospect
of a third to cheer a long evening  Her father composed himself
to sleep after dinner as usual and she had then only to sit
and think of what she had lost 

The event had every promise of happiness for her friend  Mr Weston
was a man of unexceptionable character easy fortune suitable age
and pleasant manners and there was some satisfaction in considering
with what self denying generous friendship she had always wished
and promoted the match but it was a black morning's work for her 
The want of Miss Taylor would be felt every hour of every day 
She recalled her past kindness  the kindness the affection of sixteen
years  how she had taught and how she had played with her from five
years old  how she had devoted all her powers to attach and amuse
her in health  and how nursed her through the various illnesses
of childhood  A large debt of gratitude was owing here but the
intercourse of the last seven years the equal footing and perfect
unreserve which had soon followed Isabella's marriage on their
being left to each other was yet a dearer tenderer recollection 
She had been a friend and companion such as few possessed intelligent
well informed useful gentle knowing all the ways of the family
interested in all its concerns and peculiarly interested in herself
in every pleasure every scheme of hers  one to whom she could speak
every thought as it arose and who had such an affection for her
as could never find fault 

How was she to bear the change  It was true that her friend was
going only half a mile from them but Emma was aware that great must
be the difference between a Mrs Weston only half a mile from them
and a Miss Taylor in the house and with all her advantages
natural and domestic she was now in great danger of suffering
from intellectual solitude  She dearly loved her father but he
was no companion for her  He could not meet her in conversation
rational or playful 

The evil of the actual disparity in their ages and Mr Woodhouse had
not married early was much increased by his constitution and habits
for having been a valetudinarian all his life without activity
of mind or body he was a much older man in ways than in years
and though everywhere beloved for the friendliness of his heart
and his amiable temper his talents could not have recommended him
at any time 

Her sister though comparatively but little removed by matrimony
being settled in London only sixteen miles off was much beyond
her daily reach and many a long October and November evening must
be struggled through at Hartfield before Christmas brought the next
visit from Isabella and her husband and their little children
to fill the house and give her pleasant society again 

Highbury the large and populous village almost amounting to a town
to which Hartfield in spite of its separate lawn and shrubberies
and name did really belong afforded her no equals  The Woodhouses
were first in consequence there  All looked up to them  She had
many acquaintance in the place for her father was universally civil
but not one among them who could be accepted in lieu of Miss
Taylor for even half a day  It was a melancholy change and Emma
could not but sigh over it and wish for impossible things
till her father awoke and made it necessary to be cheerful 
His spirits required support  He was a nervous man easily depressed
fond of every body that he was used to and hating to part with them
hating change of every kind  Matrimony as the origin of change
was always disagreeable and he was by no means yet reconciled
to his own daughter's marrying nor could ever speak of her but
with compassion though it had been entirely a match of affection
when he was now obliged to part with Miss Taylor too and from
his habits of gentle selfishness and of being never able to
suppose that other people could feel differently from himself
he was very much disposed to think Miss Taylor had done as sad
a thing for herself as for them and would have been a great deal
happier if she had spent all the rest of her life at Hartfield 
Emma smiled and chatted as cheerfully as she could to keep him
from such thoughts but when tea came it was impossible for him
not to say exactly as he had said at dinner

Poor Miss Taylor  I wish she were here again  What a pity it
is that Mr Weston ever thought of her

I cannot agree with you papa you know I cannot  Mr Weston is such
a good humoured pleasant excellent man that he thoroughly deserves
a good wife  and you would not have had Miss Taylor live with us
for ever and bear all my odd humours when she might have a house of her own

A house of her own  But where is the advantage of a house of her own
This is three times as large  And you have never any odd humours
my dear

How often we shall be going to see them and they coming to see
us  We shall be always meeting We must begin we must go and pay
wedding visit very soon

My dear how am I to get so far Randalls is such a distance 
I could not walk half so far

No papa nobody thought of your walking  We must go in the carriage
to be sure

The carriage But James will not like to put the horses to for
such a little way  and where are the poor horses to be while we
are paying our visit

They are to be put into Mr Weston's stable papa  You know we
have settled all that already  We talked it all over with Mr Weston
last night  And as for James you may be very sure he will always like
going to Randalls because of his daughter's being housemaid there 
I only doubt whether he will ever take us anywhere else  That was
your doing papa  You got Hannah that good place  Nobody thought
of Hannah till you mentioned her  James is so obliged to you

I am very glad I did think of her  It was very lucky for I would
not have had poor James think himself slighted upon any account
and I am sure she will make a very good servant she is a civil
pretty spoken girl I have a great opinion of her  Whenever I see her
she always curtseys and asks me how I do in a very pretty manner
and when you have had her here to do needlework I observe she
always turns the lock of the door the right way and never bangs it 
I am sure she will be an excellent servant and it will be a great
comfort to poor Miss Taylor to have somebody about her that she is
used to see  Whenever James goes over to see his daughter you know
she will be hearing of us  He will be able to tell her how we
all are

Emma spared no exertions to maintain this happier flow of ideas
and hoped by the help of backgammon to get her father tolerably
through the evening and be attacked by no regrets but her own 
The backgammon table was placed but a visitor immediately afterwards
walked in and made it unnecessary 

Mr Knightley a sensible man about seven or eight and thirty was not
only a very old and intimate friend of the family but particularly
connected with it as the elder brother of Isabella's husband 
He lived about a mile from Highbury was a frequent visitor
and always welcome and at this time more welcome than usual
as coming directly from their mutual connexions in London  He had
returned to a late dinner after some days absence and now walked
up to Hartfield to say that all were well in Brunswick Square 
It was a happy circumstance and animated Mr Woodhouse for some time 
Mr Knightley had a cheerful manner which always did him good
and his many inquiries after poor Isabella and her children were
answered most satisfactorily  When this was over Mr Woodhouse
gratefully observed It is very kind of you Mr Knightley to come
out at this late hour to call upon us  I am afraid you must have
had a shocking walk

Not at all sir  It is a beautiful moonlight night and so mild
that I must draw back from your great fire

But you must have found it very damp and dirty  I wish you may
not catch cold

Dirty sir Look at my shoes  Not a speck on them

Well that is quite surprising for we have had a vast deal
of rain here  It rained dreadfully hard for half an hour
while we were at breakfast  I wanted them to put off the wedding

By the bye  I have not wished you joy  Being pretty well aware
of what sort of joy you must both be feeling I have been in no hurry
with my congratulations but I hope it all went off tolerably well 
How did you all behave Who cried most

Ah poor Miss Taylor Tis a sad business

Poor Mr and Miss Woodhouse if you please but I cannot possibly
say poor Miss Taylor I have a great regard for you and Emma
but when it comes to the question of dependence or independence  At
any rate it must be better to have only one to please than two

Especially when one of those two is such a fanciful troublesome creature
said Emma playfully  That is what you have in your head
I know  and what you would certainly say if my father were not by

I believe it is very true my dear indeed said Mr Woodhouse
with a sigh  I am afraid I am sometimes very fanciful and troublesome

My dearest papa You do not think I could mean you or suppose
Mr Knightley to mean you  What a horrible idea Oh no I meant
only myself  Mr Knightley loves to find fault with me you know  
in a joke  it is all a joke  We always say what we like to one another

Mr Knightley in fact was one of the few people who could see
faults in Emma Woodhouse and the only one who ever told her of them
and though this was not particularly agreeable to Emma herself
she knew it would be so much less so to her father that she would
not have him really suspect such a circumstance as her not being
thought perfect by every body 

Emma knows I never flatter her said Mr Knightley but I
meant no reflection on any body  Miss Taylor has been used
to have two persons to please she will now have but one 
The chances are that she must be a gainer

Well said Emma willing to let it pass  you want to hear
about the wedding and I shall be happy to tell you for we all
behaved charmingly  Every body was punctual every body in their
best looks not a tear and hardly a long face to be seen  Oh no
we all felt that we were going to be only half a mile apart
and were sure of meeting every day

Dear Emma bears every thing so well said her father 
But Mr Knightley she is really very sorry to lose poor Miss Taylor
and I am sure she will miss her more than she thinks for

Emma turned away her head divided between tears and smiles 
It is impossible that Emma should not miss such a companion
said Mr Knightley  We should not like her so well as we do sir
if we could suppose it but she knows how much the marriage is to
Miss Taylor's advantage she knows how very acceptable it must be
at Miss Taylor's time of life to be settled in a home of her own
and how important to her to be secure of a comfortable provision
and therefore cannot allow herself to feel so much pain as pleasure 
Every friend of Miss Taylor must be glad to have her so happily
married

And you have forgotten one matter of joy to me said Emma
and a very considerable one  that I made the match myself 
I made the match you know four years ago and to have it take place
and be proved in the right when so many people said Mr Weston would
never marry again may comfort me for any thing

Mr Knightley shook his head at her  Her father fondly replied
Ah my dear I wish you would not make matches and foretell things
for whatever you say always comes to pass  Pray do not make any
more matches

I promise you to make none for myself papa but I must indeed
for other people  It is the greatest amusement in the world And
after such success you know  Every body said that Mr Weston would
never marry again  Oh dear no Mr Weston who had been a widower
so long and who seemed so perfectly comfortable without a wife
so constantly occupied either in his business in town or among his
friends here always acceptable wherever he went always cheerful  
Mr Weston need not spend a single evening in the year alone if he did
not like it  Oh no Mr Weston certainly would never marry again 
Some people even talked of a promise to his wife on her deathbed
and others of the son and the uncle not letting him  All manner
of solemn nonsense was talked on the subject but I believed none
of it 

Ever since the day  about four years ago  that Miss Taylor and I
met with him in Broadway Lane when because it began to drizzle
he darted away with so much gallantry and borrowed two umbrellas
for us from Farmer Mitchell's I made up my mind on the subject 
I planned the match from that hour and when such success has blessed
me in this instance dear papa you cannot think that I shall leave
off match making

I do not understand what you mean by success said Mr Knightley 
Success supposes endeavour  Your time has been properly and
delicately spent if you have been endeavouring for the last four
years to bring about this marriage  A worthy employment for a young
lady's mind But if which I rather imagine your making the match
as you call it means only your planning it your saying to yourself
one idle day I think it would be a very good thing for Miss Taylor
if Mr Weston were to marry her and saying it again to yourself
every now and then afterwards why do you talk of success Where
is your merit What are you proud of You made a lucky guess
and that is all that can be said

And have you never known the pleasure and triumph of a lucky guess  
I pity you  I thought you cleverer  for depend upon it a lucky
guess is never merely luck  There is always some talent in it 
And as to my poor word success which you quarrel with I do not
know that I am so entirely without any claim to it  You have drawn
two pretty pictures but I think there may be a third  a something
between the do nothing and the do all If I had not promoted Mr Weston's
visits here and given many little encouragements and smoothed
many little matters it might not have come to any thing after all 
I think you must know Hartfield enough to comprehend that

A straightforward open hearted man like Weston and a rational
unaffected woman like Miss Taylor may be safely left to manage their
own concerns  You are more likely to have done harm to yourself
than good to them by interference

Emma never thinks of herself if she can do good to others
rejoined Mr Woodhouse understanding but in part  But my dear
pray do not make any more matches they are silly things and break up
one's family circle grievously

Only one more papa only for Mr Elton  Poor Mr Elton You
like Mr Elton papa  I must look about for a wife for him 
There is nobody in Highbury who deserves him  and he has been
here a whole year and has fitted up his house so comfortably
that it would be a shame to have him single any longer  and I thought
when he was joining their hands to day he looked so very much as if
he would like to have the same kind office done for him I think
very well of Mr Elton and this is the only way I have of doing
him a service

Mr Elton is a very pretty young man to be sure and a very
good young man and I have a great regard for him  But if you
want to shew him any attention my dear ask him to come
and dine with us some day  That will be a much better thing 
I dare say Mr Knightley will be so kind as to meet him

With a great deal of pleasure sir at any time said Mr Knightley
laughing and I agree with you entirely that it will be a much
better thing  Invite him to dinner Emma and help him to the best
of the fish and the chicken but leave him to chuse his own wife 
Depend upon it a man of six or seven and twenty can take care
of himself

然后，You can convert it to one token per line with our tokenizer (included in the box) with the following command:

java -cp stanford-ner.jar edu.stanford.nlp.process.PTBTokenizer jane-austen-emma-ch1.txt > jane-austen-emma-ch1.tok

运行完成此命令之后，你可以看到生成了一个jane-austen-emma-ch1.tok文件，此文件的每一行均只有一个单词。
然后需要对训练数据进行实体标记，有多种工具可以使用。可以使用文本文档工具进行手工标记。由于内容太多，可以使用程序对每一行加上制表符（tab）和0，perl的代码如下；然后，再使用文本文档编辑器，对特殊的行进行手工修改标记。
运行下面命令产生jane-austen-emma-ch1.tsv文件（每一行均加上’\t0’字符串）

perl -ne 'chomp; print "$_\tO\n"' jane-austen-emma-ch1.tok > jane-austen-emma-ch1.tsv

如果你不想做上述步骤，可以从这个链接下载tsv文件http://nlp.stanford.edu/software/ner-example/jane-austen-emma-ch1.tsv
我们仅仅标注了一个实体类型——用“PERS”表示人名；但是，你可以在这个数据中加入第二种实体类型，如地理位置“LOC”。你应该确定每行仅仅只包括内容字段和制表符（tab），空格也不能有，额外的制表符（tab）也不能有，否则训练文件解析器可能会出错。一个空行分隔两个“document”，一个“document”只能是一个句子，或者更大一点儿的单元，如段落；这是CRF推断的单位。“document”不能太大，否则你将浪费许多内存，并且还可能碰到很多问题。你仍然可以使用一些测试数据去判断系统是否可以良好运行，这里有jane-austen-emma-ch2.txt（http://nlp.stanford.edu/software/ner-example/jane-austen-emma-ch2.txt）和jane-austen-emma-ch2.tsv（http://nlp.stanford.edu/software/ner-example/jane-austen-emma-ch2.tsv）

新建austen.prop属性文件

#location of the training file
trainFile = jane-austen-emma-ch1.tsv
#location where you would like to save (serialize to) your
#classifier; adding .gz at the end automatically gzips the file,
#making it faster and smaller
serializeTo = ner-model.ser.gz

#structure of your training file; this tells the classifier
#that the word is in column 0 and the correct answer is in
#column 1
map = word=0,answer=1

#these are the features we'd like to train with
#some are discussed below, the rest can be
#understood by looking at NERFeatureFactory
useClassFeature=true
useWord=true
useNGrams=true
#no ngrams will be included that do not contain either the
#beginning or end of the word
noMidNGrams=true
useDisjunctive=true
maxNGramLeng=6
usePrev=true
useNext=true
useSequences=true
usePrevSequences=true
maxLeft=1
#the next 4 deal with word shape features
useTypeSeqs=true
useTypeSeqs2=true
useTypeySequences=true
wordShape=chris2useLC

然后用下面的命令训练一个分类器

java -cp stanford-ner.jar edu.stanford.nlp.ie.crf.CRFClassifier -prop austen.prop

一旦完成上述命令。将产生一个NER模型文件ner-model.ser.gz，判断它是否有效，使用下面命令来测试

java -cp stanford-ner.jar edu.stanford.nlp.ie.crf.CRFClassifier -loadClassifier ner-model.ser.gz -testFile jane-austen-emma-ch2.tsv

最后得到的结果如下

可以看到第一列是作为输入项的单词列，第二列是正确标注的结果，第三类是由Stanford NER猜的结果。你能够发现此分类器发现了大多数的人名实体，但不是全部的。主要是因为训练数据比较少，而且这只是一个相对基本的特征集。代码最后评估出实体精确分类的性能，用recall（召回率）表示，用符号F1代表。最后的结果F1是85.95%.（不知道为什么我的是77.11%）


所以训练自己的NER模型步骤是：
1)训练数据的源数据
2)指定了你想用的属性的属性文件（这个是可选的，但是有最好）
3)测试文件
对于训练数据的源数据，需要保证每一个单词占据一行，并且附有正确答案（单词与答案之间使用制表符分隔开）。如果要明确指定单词的更多特征，可以将这些特征添加到文件的新列中，然后在属性文件的map 行中设置适当结构。例如，如果你的数据中添加了第三列表示新的特征，则可以写成“map = word = 0，answer = 1，mySpecialFeature = 2”。

Right now, most arbitrarily named features (like mySpecialFeature) will not work without making modifications to the source code. To see which features can already be attached to a CoreLabel, look at edu.stanford.nlp.ling.AnnotationLookup. There is a table which creates a mapping between key and annotation type. For example, if you search in this file for LEMMA_KEY, you will see that lemma produces a LemmaAnnotation. If you have added a new annotation, you can add its type to this table, or you can use one of the known names that already work, like tag, lemma, chunk, web.

If you modify AnnotationLookup, you need to read the data from the column, translate it to the desired object type, and attach it to the CoreLabel using a CoreAnnotation. Quite a few CoreAnnotations are provided in the class appropriately called CoreAnnotations. If the particular one you are looking for is not present, you can add a new subclass by using one of the existing CoreAnnotations as an example.

If the feature you attached to the CoreLabel is not already used as a feature in NERFeatureFactory, you will need to add code that extracts the feature from the CoreLabel and adds it to the feature set. Bear in mind that features must have unique names, or they will conflict with existing features, which is why we add markers such as “-GENIA”, “-PGENIA”, and “-NGENIA” to our features. As long as you choose a unique marker, the feature itself can be any string followed by its marker and will not conflict with any existing features. Processing is done using a bag of features model, with all of the features mixed together, which is why it is important to not have any name conflicts.

Once you’ve annotated your data, you make a properties file with the features you want. You can use the example properties file, and refer to the NERFeatureFactory for more possible features. Finally, you can test on your annotated test data as shown above or annotate more text using the -textFile command rather than -testFile.

备用：在上文中提到了使用perl语言，对文本文档的每一行后面添加一个tab键与0，不仅仅可以使用编程语言perl，python等来完成这样的步骤，其实还有一个更简单的方法，使用sublime text，或者notepad++等这样的文本编辑软件，也可以达到目的。
    给每行首部位置加上内容，用sublime text打开文本文档文件之后，菜单find—replace，打开替换功能，先勾选替换框下部的模式为正则表达式(Regular Expression)，在“查找目标”(Find what)处输入^，在“替换为”(Replace with)处输入想要添加的内容，最后点击按钮“replace all”即可。
    给每行尾部位置加上内容，一样的步骤，在“查找目标”(Find what)处输入$，后面一样。通过这种文本编辑软件来进行处理，速度更快！