NER判别实战--demo

NER任务简介

命名体识别（Name Entity Recognition）是自然语言处理(Nature Language Processing)领域中比较重要的一个任务，几乎百分之50的和文本处理有关的项目中都会涉及到命名体识别。

NER是一个序列标注任务，和分词，词性标注的任务属同一类。任务的输入是一串序列，输出也是一串序列。例子如下：
输入：[北，京，天，气，真，不，错]
输出：[1,2,0,0,0,0,0]
其中1表示位置实体的开头（B_LOC），2表示位置实体的中间（I_LOC），通过 B_LOC和I_LOC我们就可以锁定北京这个位置实体。

NER算法简介

NER算法从上古时期的HMM，到CRF，再到现在的特别火爆的深度学习CNN+BiLSTM+CRF（论文地址），算法的准确率可谓是节节高升，现在的比较先进CNN+BiLSTM+CRF算法已经可以达到97%以上的准确率了。
本人自己构建了标注数据，基于Keras框架就CNN算法，写了一个demo，为了提升预测准确率以及模型泛化能力，模型还叠加了TimeDistributed layer 和Bidirectional layer。预测准确率还不错，给大家分享一下。

构建数据

构建标记函数

任务是识别name, age 和 city；我们分别用[]，(), {}, 确定命名实体的位置；由于BIO标记过于复杂且本文数据集较为简单，本文使用BI标记。

lst = 'O	B-Name	I-Name	B-AGE	I-AGE	B-City	I-City'
print(lst.split('\t'))

# ['O', 'B-NAME', 'I-NAME', 'B-AGE', 'I-AGE', 'B-CITY', 'I-CITY']
# Name: []
# Age: ()
# City: {}

label_tag = {
    'NAME': '[]',
    'AGE': '()',
    'CITY': '{}'
}

label_tag

print：

['O', 'B-Name', 'I-Name', 'B-AGE', 'I-AGE', 'B-City', 'I-City']
{'NAME': '[]', 'AGE': '()', 'CITY': '{}'}

下面我们将每个英文字符标记，B表示命名体的开头，I表示中间位置，O表示不重要信息。

txt = "Hi, I'm [Bob] and I from {SZ}, I'm (8) years old."

def annotator(text):
    data = []
    outside = 'O'
    tag = outside
    flag = False
    for i, c in enumerate(text):  #enumerate对数据标序号
        for k, v in label_tag.items():
            start, end = v     #v包含[],{},(),每个分别有两个
            if c == start:
                tag = k
                flag = True
                break
            elif c == end:
                tag = outside
                break
        else:
            if flag:
                _tag = 'B-' + tag
            elif tag != outside:
                _tag = 'I-' + tag
            else:
                _tag = tag
            data.append((c, _tag))
            flag = False
    
    return ['\t'.join(line) for line in data]

annotator(txt)

输出如下：

['H\tO',
 'i\tO',
 ',\tO',
 ' \tO',
 'I\tO',
 "'\tO",
 'm\tO',
 ' \tO',
 'B\tB-NAME',
 'o\tI-NAME',
 'b\tI-NAME',
 ' \tO',
 'a\tO',
 'n\tO',
 'd\tO',
 ' \tO',
 'I\tO',
 ' \tO',
 'f\tO',
 'r\tO',
 'o\tO',
 'm\tO',
 ' \tO',
 'S\tB-CITY',
 'Z\tI-CITY',
 ',\tO',
 ...

Creating data set

import re
import random
import string

names = [''.join(random.sample(string.ascii_lowercase, random.randint(3, 5))).capitalize() for i in range(1000)]
cities = [''.join(random.sample(string.ascii_lowercase, random.randint(5, 10))).upper() for i in range(1000)]
age = [random.randint(20, 100) for i in range(1000)]

text = '''
Hi, I'm [Bob] and I from {SZ}, I'm (18) years old.
Hello, I am [RPI], I'm (20) years old and I come from {GZ}.
I born in {YuLin}. I'm (24), just call me [ATA].
I don't want to tell you my name, but I from {ShangHai}.
Hey My name is [CZW], I live in {HangZhou}, what's your name?
I don't wanna tell you my name, but I from {BeiJing}.
Oh, I'm [Linda] and I from {NanNing}, I'm (25) years old.
Hi, I'm [Tian] and I from {YUI}, I'm (19) years old
Hey, I'm [Lily] and I from {PEI}, I'm (19) years old
'''
train_text = []
sentences = text.strip().split('\n')
for i in range(500):
    for row in  sentences:
        strs = row
        strs = re.sub(r'\[.*?\]', f'[{random.choice(names)}]', strs)
        strs = re.sub(r'\{.*?\}', '{%s}'%(random.choice(cities)), strs)
        strs = re.sub(r'\(.*?\)', '(%s)'%(random.choice(age)), strs)
        train_text.append(strs)

random.shuffle(train_text)

char_lst = [annotator(row)+['END'] for row in train_text]
with open('train.txt','w+') as fp:
    fp.write('\n'.join(sum(char_lst, [])))

构建数据集方法也可参考我的上一篇blog，用正则匹配生成固定格式的随机文本python

文本处理

对text进行处理，统一x，y的vector长度，构造适合输入model的array

# 对x,y值统一长度为sentence_max_length
sentence_max_length = 100
# voc为text里面的所有可能取值，本训练集为大小写字母和数字
voc = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ,.'? "
label = ['O', 'B-NAME', 'I-NAME', 'B-AGE', 'I-AGE', 'B-CITY', 'I-CITY']

###### 补齐为0
def voc_idx(lst):
    _voc = list(voc)
    res = [_voc.index(i)+1 if i in _voc else 0 for i in lst]
    if len(res) > sentence_max_length:
        raise Exception('exceed sentence_max_length ')
        
    return res + [0]*(sentence_max_length - len(res))

def lable_idx(lst):
    res = [label.index(i) for i in lst]
    if len(res) > sentence_max_length:
        raise Exception('exceed sentence_max_length ')
        
    return res + [0]*(sentence_max_length - len(res))


with open('train.txt') as fp:
    text = [row.strip().split('\n') for row in fp.read().split('END')]
    
train_data = []
lable_data = []

for row  in text:
    if len(row) <= 1:
        continue        
    train_lst, lable_lst = zip(*[i.split('\t') for i in row])  #解压
    train_data.append(voc_idx(train_lst))    #补齐到100个
    lable_data.append(lable_idx(lable_lst))  #补齐到100个

## 构造二元组
import numpy as np
x, y = np.array(train_data), np.array(lable_data)  #x为25000*100矩阵
y = y.reshape((y.shape[0], y.shape[1], 1))  #reshape重组array

Create model layers

载入相关package

import pickle
import numpy as np
from keras.preprocessing.sequence import pad_sequences
from keras import Sequential
from keras_contrib.layers import CRF
import pickle
from keras.layers import Embedding,Bidirectional,LSTM,TimeDistributed,Dense

Create model

model = Sequential()
# len(self.vocab)
model.add(Embedding(len(voc)+1, 200))  
# model.add(Bidirectional(GRU(256)))
# add Bidirectional GRU
model.add(Bidirectional(GRU(256, return_sequences=True)))
model.add(Bidirectional(GRU(256, return_sequences=True)))
# add timedistributed
model.add(TimeDistributed(Dense(len(label))))
crf = CRF(len(label), sparse_target=True)
model.add(crf)
model.summary()

Model strcture

Model: "sequential_9"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_9 (Embedding)      (None, None, 200)         13600     
_________________________________________________________________
bidirectional_17 (Bidirectio (None, None, 512)         701952    
_________________________________________________________________
bidirectional_18 (Bidirectio (None, None, 512)         1181184   
_________________________________________________________________
time_distributed_1 (TimeDist (None, None, 7)           3591      
_________________________________________________________________
crf_7 (CRF)                  (None, None, 7)           119       
=================================================================
Total params: 1,900,446
Trainable params: 1,900,446
Non-trainable params: 0
_________________________________________________________________

Trian model

model.compile('adam', loss=crf.loss_function, metrics=[crf.accuracy])
# model.fit(data,label,batch_size=16,epochs=EPOCHS)
# y = y.reshape((y.shape[0], y.shape[1], 1))
model.fit(x, y, batch_size=32, epochs=10)

模型参数较多，train的过程比较长，可以减少epochs缩短时间

Model predict

## d为x, voc为y

d = np.array([45, 67, 13, 11, 24, 67, 24, 25, 30, 67, 30, 15, 22, 22, 67, 35, 25,
       31, 67, 23, 35, 67, 24, 11, 23, 15, 63, 67, 12, 31, 30, 67, 45, 67,
       16, 28, 25, 23, 67, 58, 37, 40, 38, 39, 41, 64,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0])

voc = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ,.'? "

result_label = [np.argmax(i) for i in model.predict(x[5])]
print(result_label)

最后根据model predict的序号判断预测是否准确，例如，若是0，则是O；6为city。