我试图用spaCy创建一个新的实体分类'物种'与物种名称列表,例如他可以找到here。
我在this spaCy tutorial(Github代码here)中找到了一个培训新实体类型的教程。但是,问题是,我不想为每个物种名称手动创建一个句子,因为这将非常耗时。
我创建了以下培训数据,如下所示:TRAIN_DATA = [('Bombina',{'entities':[(0,6,'SPECIES')]}),
('Dermaptera',{'entities':[(0,9,'SPECIES')]}),
....
]
我创建训练集的方式是:我不提供完整的句子和匹配实体的位置,而只提供每个物种的名称,并且开始和结束索引是通过编程生成的:[( 0, 6, 'SPECIES' )]
[( 0, 9, 'SPECIES' )]
下面是我用来训练模型的训练代码。(从上面的超链接复制的代码)nlp = spacy.blank('en') # create blank Language class
# Add entity recognizer to model if it's not in the pipeline
# nlp.create_pipe works for built-ins that are registered with spaCy
if 'ner' not in nlp.pipe_names:
ner = nlp.create_pipe('ner')
nlp.add_pipe(ner)
# otherwise, get it, so we can add labels to it
else:
ner = nlp.get_pipe('ner')
ner.add_label(LABEL) # add new entity label to entity recognizer
if model is None:
optimizer = nlp.begin_training()
else:
# Note that 'begin_training' initializes the models, so it'll zero out
# existing entity types.
optimizer = nlp.entity.create_optimizer()
# get names of other pipes to disable them during training
other_pipes = [pipe for pipe in nlp.pipe_names if pipe != 'ner']
with nlp.disable_pipes(*other_pipes): # only train NER
for itn in range(n_iter):
random.shuffle(TRAIN_DATA)
losses = {}
for text, annotations in TRAIN_DATA:
nlp.update([text], [annotations], sgd=optimizer, drop=0.35, losses=losses)
print(losses)
我是新来的NLP和spaCy请让我知道我做得是否正确。为什么我的尝试在训练中失败了(当我运行它时,它会抛出一个错误)。
[更新]
我只想向训练模型提供关键字的原因是,理想情况下,我希望模型首先学习那些关键字,一旦它识别出包含关键字的上下文,它将学习相关的上下文,从而增强当前模型。
乍一看,它更像regex表达式。但随着越来越多的数据输入,该模型将不断学习,最终能够识别出原来训练集中不存在的新物种名称。
谢谢,
凯蒂