BERT-BiLSTM-CRF命名实体识别应用

最新推荐文章于 2024-09-15 07:14:08 发布
置顶 最新推荐文章于 2024-09-15 07:14:08 发布
阅读量1w 收藏 97
点赞数 10
分类专栏： 徐奕的专栏 机器学习 python 文章标签： 机器学习 tensorflow 深度学习 自然语言处理 python
© 2023 • OmegaXYZ.com-版权所有，转载请注明出处-原创代码采用 Apache 2.0 授权，原创文章采用 BY-NC-SA 4.0 授权
本文链接：https://blog.csdn.net/xyisv/article/details/106191763
版权
徐奕的专栏 同时被 3 个专栏收录
228 篇文章 38 订阅
订阅专栏
机器学习
89 篇文章 27 订阅
订阅专栏
python
78 篇文章 11 订阅
订阅专栏

引言

本文将采用BERT+BiLSTM+CRF模型进行命名实体识别（Named Entity Recognition 简称NER），即实体识别。命名实体识别，是指识别文本中具有特定意义的实体，主要包括人名、地名、机构名、专有名词等。

  • BERT（Bidirectional Encoder Representation from
    Transformers），即双向Transformer的Encoder。模型的创新点在预训练方法上，即用了Masked LM和Next
    Sentence Prediction两种方法分别捕捉词语和句子级别的表示。

  • BiLSTM是Bi-directional Long Short-Term Memory的缩写，是由前向LSTM与后向LSTM组合而成。

  • CRF为条件随机场，可以用于构造在给定一组输入随机变量的条件下,另一组输出随机变量的条件概率分布模型。

环境

采用的Python包为：Kashgari，此包封装了NLP传统和前沿模型，可以快速调用，快速部署模型。

  • Python: 3.6
  • TensorFlow: 1.15
  • Kashgari: 1.x

其中Kashgari1.x版本必须使用TensorFlow一代。

BERT中文预训练数据

谷歌提前训练好的数据，其中中文模型可以从https://storage.googleapis.com/bert_models/2018_11_03/chinese_L-12_H-768_A-12.zip下载。

更多预训练模型参考：https://github.com/ymcui/Chinese-BERT-wwm

自带数据训练评价

数据为中国日报的NER语料库，代码自动下载。

训练集、测试集和验证集的存储格式：

train_x: [[char_seq1],[char_seq2],[char_seq3],..... ]
train_y:[[label_seq1],[label_seq2],[label_seq3],..... ]
其中 char_seq1:["我"，"爱"，"中"，"国"]
对应的的label_seq1:["O"，"O"，"B_LOC"，"I_LOC"]

代码：

from kashgari.corpus import ChineseDailyNerCorpus
from kashgari.embeddings import BERTEmbedding
from kashgari.tasks.labeling import BiLSTM_CRF_Model
import kashgari


train_x, train_y = ChineseDailyNerCorpus.load_data('train')
test_x, test_y = ChineseDailyNerCorpus.load_data('test')
valid_x, valid_y = ChineseDailyNerCorpus.load_data('valid')


embedding = BERTEmbedding("chinese", sequence_length=10, task=kashgari.LABELING)
model = BiLSTM_CRF_Model(embedding)
model.fit(train_x, train_y, x_validate=valid_x, y_validate=valid_y, epochs=1, batch_size=100)
model.evaluate(test_x, test_y)

# model.save('save')
# loaded_model = kashgari.utils.load_model('xxx')

最后注释的为模型保存和调用代码。

实例

此本分将用自己的数据来进行命名实体识别。train_x和y存储格式和上面相同。

可用的标注格式

BIO标注模式: (B-begin，I-inside，O-outside)

BIOES标注模式: (B-begin，I-inside，O-outside，E-end，S-single)

  • B，即Begin，表示开始
  • I，即Intermediate，表示中间
  • E，即End，表示结尾
  • S，即Single，表示单个字符
  • O，即Other，表示其他，用于标记无关字符

代码

访问 https://www.omegaxyz.com/2020/05/18/bert-bilstm-crf/查看。

结果

_____________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
Input-Token (InputLayer)        [(None, 20)]         0                                            
__________________________________________________________________________________________________
Input-Segment (InputLayer)      [(None, 20)]         0                                            
__________________________________________________________________________________________________
Embedding-Token (TokenEmbedding [(None, 20, 768), (2 16226304    Input-Token[0][0]                
__________________________________________________________________________________________________
Embedding-Segment (Embedding)   (None, 20, 768)      1536        Input-Segment[0][0]              
__________________________________________________________________________________________________
Embedding-Token-Segment (Add)   (None, 20, 768)      0           Embedding-Token[0][0]            
                                                                 Embedding-Segment[0][0]          
__________________________________________________________________________________________________
Embedding-Position (PositionEmb (None, 20, 768)      15360       Embedding-Token-Segment[0][0]    
__________________________________________________________________________________________________
Embedding-Dropout (Dropout)     (None, 20, 768)      0           Embedding-Position[0][0]         
__________________________________________________________________________________________________
Embedding-Norm (LayerNormalizat (None, 20, 768)      1536        Embedding-Dropout[0][0]          
__________________________________________________________________________________________________
Encoder-1-MultiHeadSelfAttentio (None, 20, 768)      2362368     Embedding-Norm[0][0]             
__________________________________________________________________________________________________
Encoder-1-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-1-MultiHeadSelfAttention[
__________________________________________________________________________________________________
Encoder-1-MultiHeadSelfAttentio (None, 20, 768)      0           Embedding-Norm[0][0]             
                                                                 Encoder-1-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-1-MultiHeadSelfAttentio (None, 20, 768)      1536        Encoder-1-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-1-FeedForward (FeedForw (None, 20, 768)      4722432     Encoder-1-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-1-FeedForward-Dropout ( (None, 20, 768)      0           Encoder-1-FeedForward[0][0]      
__________________________________________________________________________________________________
Encoder-1-FeedForward-Add (Add) (None, 20, 768)      0           Encoder-1-MultiHeadSelfAttention-
                                                                 Encoder-1-FeedForward-Dropout[0][
__________________________________________________________________________________________________
Encoder-1-FeedForward-Norm (Lay (None, 20, 768)      1536        Encoder-1-FeedForward-Add[0][0]  
__________________________________________________________________________________________________
Encoder-2-MultiHeadSelfAttentio (None, 20, 768)      2362368     Encoder-1-FeedForward-Norm[0][0] 
__________________________________________________________________________________________________
Encoder-2-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-2-MultiHeadSelfAttention[
__________________________________________________________________________________________________
Encoder-2-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-1-FeedForward-Norm[0][0] 
                                                                 Encoder-2-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-2-MultiHeadSelfAttentio (None, 20, 768)      1536        Encoder-2-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-2-FeedForward (FeedForw (None, 20, 768)      4722432     Encoder-2-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-2-FeedForward-Dropout ( (None, 20, 768)      0           Encoder-2-FeedForward[0][0]      
__________________________________________________________________________________________________
Encoder-2-FeedForward-Add (Add) (None, 20, 768)      0           Encoder-2-MultiHeadSelfAttention-
                                                                 Encoder-2-FeedForward-Dropout[0][
__________________________________________________________________________________________________
Encoder-2-FeedForward-Norm (Lay (None, 20, 768)      1536        Encoder-2-FeedForward-Add[0][0]  
__________________________________________________________________________________________________
Encoder-3-MultiHeadSelfAttentio (None, 20, 768)      2362368     Encoder-2-FeedForward-Norm[0][0] 
__________________________________________________________________________________________________
Encoder-3-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-3-MultiHeadSelfAttention[
__________________________________________________________________________________________________
Encoder-3-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-2-FeedForward-Norm[0][0] 
                                                                 Encoder-3-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-3-MultiHeadSelfAttentio (None, 20, 768)      1536        Encoder-3-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-3-FeedForward (FeedForw (None, 20, 768)      4722432     Encoder-3-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-3-FeedForward-Dropout ( (None, 20, 768)      0           Encoder-3-FeedForward[0][0]      
__________________________________________________________________________________________________
Encoder-3-FeedForward-Add (Add) (None, 20, 768)      0           Encoder-3-MultiHeadSelfAttention-
                                                                 Encoder-3-FeedForward-Dropout[0][
__________________________________________________________________________________________________
Encoder-3-FeedForward-Norm (Lay (None, 20, 768)      1536        Encoder-3-FeedForward-Add[0][0]  
__________________________________________________________________________________________________
Encoder-4-MultiHeadSelfAttentio (None, 20, 768)      2362368     Encoder-3-FeedForward-Norm[0][0] 
__________________________________________________________________________________________________
Encoder-4-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-4-MultiHeadSelfAttention[
__________________________________________________________________________________________________
Encoder-4-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-3-FeedForward-Norm[0][0] 
                                                                 Encoder-4-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-4-MultiHeadSelfAttentio (None, 20, 768)      1536        Encoder-4-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-4-FeedForward (FeedForw (None, 20, 768)      4722432     Encoder-4-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-4-FeedForward-Dropout ( (None, 20, 768)      0           Encoder-4-FeedForward[0][0]      
__________________________________________________________________________________________________
Encoder-4-FeedForward-Add (Add) (None, 20, 768)      0           Encoder-4-MultiHeadSelfAttention-
                                                                 Encoder-4-FeedForward-Dropout[0][
__________________________________________________________________________________________________
Encoder-4-FeedForward-Norm (Lay (None, 20, 768)      1536        Encoder-4-FeedForward-Add[0][0]  
__________________________________________________________________________________________________
Encoder-5-MultiHeadSelfAttentio (None, 20, 768)      2362368     Encoder-4-FeedForward-Norm[0][0] 
__________________________________________________________________________________________________
Encoder-5-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-5-MultiHeadSelfAttention[
__________________________________________________________________________________________________
Encoder-5-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-4-FeedForward-Norm[0][0] 
                                                                 Encoder-5-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-5-MultiHeadSelfAttentio (None, 20, 768)      1536        Encoder-5-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-5-FeedForward (FeedForw (None, 20, 768)      4722432     Encoder-5-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-5-FeedForward-Dropout ( (None, 20, 768)      0           Encoder-5-FeedForward[0][0]      
__________________________________________________________________________________________________
Encoder-5-FeedForward-Add (Add) (None, 20, 768)      0           Encoder-5-MultiHeadSelfAttention-
                                                                 Encoder-5-FeedForward-Dropout[0][
__________________________________________________________________________________________________
Encoder-5-FeedForward-Norm (Lay (None, 20, 768)      1536        Encoder-5-FeedForward-Add[0][0]  
__________________________________________________________________________________________________
Encoder-6-MultiHeadSelfAttentio (None, 20, 768)      2362368     Encoder-5-FeedForward-Norm[0][0] 
__________________________________________________________________________________________________
Encoder-6-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-6-MultiHeadSelfAttention[
__________________________________________________________________________________________________
Encoder-6-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-5-FeedForward-Norm[0][0] 
                                                                 Encoder-6-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-6-MultiHeadSelfAttentio (None, 20, 768)      1536        Encoder-6-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-6-FeedForward (FeedForw (None, 20, 768)      4722432     Encoder-6-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-6-FeedForward-Dropout ( (None, 20, 768)      0           Encoder-6-FeedForward[0][0]      
__________________________________________________________________________________________________
Encoder-6-FeedForward-Add (Add) (None, 20, 768)      0           Encoder-6-MultiHeadSelfAttention-
                                                                 Encoder-6-FeedForward-Dropout[0][
__________________________________________________________________________________________________
Encoder-6-FeedForward-Norm (Lay (None, 20, 768)      1536        Encoder-6-FeedForward-Add[0][0]  
__________________________________________________________________________________________________
Encoder-7-MultiHeadSelfAttentio (None, 20, 768)      2362368     Encoder-6-FeedForward-Norm[0][0] 
__________________________________________________________________________________________________
Encoder-7-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-7-MultiHeadSelfAttention[
__________________________________________________________________________________________________
Encoder-7-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-6-FeedForward-Norm[0][0] 
                                                                 Encoder-7-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-7-MultiHeadSelfAttentio (None, 20, 768)      1536        Encoder-7-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-7-FeedForward (FeedForw (None, 20, 768)      4722432     Encoder-7-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-7-FeedForward-Dropout ( (None, 20, 768)      0           Encoder-7-FeedForward[0][0]      
__________________________________________________________________________________________________
Encoder-7-FeedForward-Add (Add) (None, 20, 768)      0           Encoder-7-MultiHeadSelfAttention-
                                                                 Encoder-7-FeedForward-Dropout[0][
__________________________________________________________________________________________________
Encoder-7-FeedForward-Norm (Lay (None, 20, 768)      1536        Encoder-7-FeedForward-Add[0][0]  
__________________________________________________________________________________________________
Encoder-8-MultiHeadSelfAttentio (None, 20, 768)      2362368     Encoder-7-FeedForward-Norm[0][0] 
__________________________________________________________________________________________________
Encoder-8-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-8-MultiHeadSelfAttention[
__________________________________________________________________________________________________
Encoder-8-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-7-FeedForward-Norm[0][0] 
                                                                 Encoder-8-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-8-MultiHeadSelfAttentio (None, 20, 768)      1536        Encoder-8-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-8-FeedForward (FeedForw (None, 20, 768)      4722432     Encoder-8-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-8-FeedForward-Dropout ( (None, 20, 768)      0           Encoder-8-FeedForward[0][0]      
__________________________________________________________________________________________________
Encoder-8-FeedForward-Add (Add) (None, 20, 768)      0           Encoder-8-MultiHeadSelfAttention-
                                                                 Encoder-8-FeedForward-Dropout[0][
__________________________________________________________________________________________________
Encoder-8-FeedForward-Norm (Lay (None, 20, 768)      1536        Encoder-8-FeedForward-Add[0][0]  
__________________________________________________________________________________________________
Encoder-9-MultiHeadSelfAttentio (None, 20, 768)      2362368     Encoder-8-FeedForward-Norm[0][0] 
__________________________________________________________________________________________________
Encoder-9-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-9-MultiHeadSelfAttention[
__________________________________________________________________________________________________
Encoder-9-MultiHeadSelfAttentio (None, 20, 768)      0           Encoder-8-FeedForward-Norm[0][0] 
                                                                 Encoder-9-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-9-MultiHeadSelfAttentio (None, 20, 768)      1536        Encoder-9-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-9-FeedForward (FeedForw (None, 20, 768)      4722432     Encoder-9-MultiHeadSelfAttention-
__________________________________________________________________________________________________
Encoder-9-FeedForward-Dropout ( (None, 20, 768)      0           Encoder-9-FeedForward[0][0]      
__________________________________________________________________________________________________
Encoder-9-FeedForward-Add (Add) (None, 20, 768)      0           Encoder-9-MultiHeadSelfAttention-
                                                                 Encoder-9-FeedForward-Dropout[0][
__________________________________________________________________________________________________
Encoder-9-FeedForward-Norm (Lay (None, 20, 768)      1536        Encoder-9-FeedForward-Add[0][0]  
__________________________________________________________________________________________________
Encoder-10-MultiHeadSelfAttenti (None, 20, 768)      2362368     Encoder-9-FeedForward-Norm[0][0] 
__________________________________________________________________________________________________
Encoder-10-MultiHeadSelfAttenti (None, 20, 768)      0           Encoder-10-MultiHeadSelfAttention
__________________________________________________________________________________________________
Encoder-10-MultiHeadSelfAttenti (None, 20, 768)      0           Encoder-9-FeedForward-Norm[0][0] 
                                                                 Encoder-10-MultiHeadSelfAttention
__________________________________________________________________________________________________
Encoder-10-MultiHeadSelfAttenti (None, 20, 768)      1536        Encoder-10-MultiHeadSelfAttention
__________________________________________________________________________________________________
Encoder-10-FeedForward (FeedFor (None, 20, 768)      4722432     Encoder-10-MultiHeadSelfAttention
__________________________________________________________________________________________________
Encoder-10-FeedForward-Dropout  (None, 20, 768)      0           Encoder-10-FeedForward[0][0]     
__________________________________________________________________________________________________
Encoder-10-FeedForward-Add (Add (None, 20, 768)      0           Encoder-10-MultiHeadSelfAttention
                                                                 Encoder-10-FeedForward-Dropout[0]
__________________________________________________________________________________________________
Encoder-10-FeedForward-Norm (La (None, 20, 768)      1536        Encoder-10-FeedForward-Add[0][0] 
__________________________________________________________________________________________________
Encoder-11-MultiHeadSelfAttenti (None, 20, 768)      2362368     Encoder-10-FeedForward-Norm[0][0]
__________________________________________________________________________________________________
Encoder-11-MultiHeadSelfAttenti (None, 20, 768)      0           Encoder-11-MultiHeadSelfAttention
__________________________________________________________________________________________________
Encoder-11-MultiHeadSelfAttenti (None, 20, 768)      0           Encoder-10-FeedForward-Norm[0][0]
                                                                 Encoder-11-MultiHeadSelfAttention
__________________________________________________________________________________________________
Encoder-11-MultiHeadSelfAttenti (None, 20, 768)      1536        Encoder-11-MultiHeadSelfAttention
__________________________________________________________________________________________________
Encoder-11-FeedForward (FeedFor (None, 20, 768)      4722432     Encoder-11-MultiHeadSelfAttention
__________________________________________________________________________________________________
Encoder-11-FeedForward-Dropout  (None, 20, 768)      0           Encoder-11-FeedForward[0][0]     
__________________________________________________________________________________________________
Encoder-11-FeedForward-Add (Add (None, 20, 768)      0           Encoder-11-MultiHeadSelfAttention
                                                                 Encoder-11-FeedForward-Dropout[0]
__________________________________________________________________________________________________
Encoder-11-FeedForward-Norm (La (None, 20, 768)      1536        Encoder-11-FeedForward-Add[0][0] 
__________________________________________________________________________________________________
Encoder-12-MultiHeadSelfAttenti (None, 20, 768)      2362368     Encoder-11-FeedForward-Norm[0][0]
__________________________________________________________________________________________________
Encoder-12-MultiHeadSelfAttenti (None, 20, 768)      0           Encoder-12-MultiHeadSelfAttention
__________________________________________________________________________________________________
Encoder-12-MultiHeadSelfAttenti (None, 20, 768)      0           Encoder-11-FeedForward-Norm[0][0]
                                                                 Encoder-12-MultiHeadSelfAttention
__________________________________________________________________________________________________
Encoder-12-MultiHeadSelfAttenti (None, 20, 768)      1536        Encoder-12-MultiHeadSelfAttention
__________________________________________________________________________________________________
Encoder-12-FeedForward (FeedFor (None, 20, 768)      4722432     Encoder-12-MultiHeadSelfAttention
__________________________________________________________________________________________________
Encoder-12-FeedForward-Dropout  (None, 20, 768)      0           Encoder-12-FeedForward[0][0]     
__________________________________________________________________________________________________
Encoder-12-FeedForward-Add (Add (None, 20, 768)      0           Encoder-12-MultiHeadSelfAttention
                                                                 Encoder-12-FeedForward-Dropout[0]
__________________________________________________________________________________________________
Encoder-12-FeedForward-Norm (La (None, 20, 768)      1536        Encoder-12-FeedForward-Add[0][0] 
__________________________________________________________________________________________________
Encoder-Output (Concatenate)    (None, 20, 3072)     0           Encoder-9-FeedForward-Norm[0][0] 
                                                                 Encoder-10-FeedForward-Norm[0][0]
                                                                 Encoder-11-FeedForward-Norm[0][0]
                                                                 Encoder-12-FeedForward-Norm[0][0]
__________________________________________________________________________________________________
non_masking_layer (NonMaskingLa (None, 20, 3072)     0           Encoder-Output[0][0]             
__________________________________________________________________________________________________
layer_blstm (Bidirectional)     (None, 20, 256)      3277824     non_masking_layer[0][0]          
__________________________________________________________________________________________________
layer_dense (Dense)             (None, 20, 64)       16448       layer_blstm[0][0]                
__________________________________________________________________________________________________
layer_crf_dense (Dense)         (None, 20, 10)       650         layer_dense[0][0]                
__________________________________________________________________________________________________
layer_crf (CRF)                 (None, 20, 10)       100         layer_crf_dense[0][0]            
==================================================================================================
Total params: 104,594,222
Trainable params: 3,295,022
Non-trainable params: 101,299,200
__________________________________________________________________________________________________
Epoch 1/10

1/1 [==============================] - 6s 6s/step - loss: 52.2269 - accuracy: 0.1250
Epoch 2/10

1/1 [==============================] - 1s 687ms/step - loss: 22.6029 - accuracy: 0.6750
Epoch 3/10

1/1 [==============================] - 1s 754ms/step - loss: 12.7078 - accuracy: 0.8500
Epoch 4/10

1/1 [==============================] - 1s 767ms/step - loss: 12.8406 - accuracy: 0.8250
Epoch 5/10

1/1 [==============================] - 1s 717ms/step - loss: 10.0257 - accuracy: 0.8750
Epoch 6/10

1/1 [==============================] - 1s 638ms/step - loss: 7.3283 - accuracy: 0.8750
Epoch 7/10

1/1 [==============================] - 1s 738ms/step - loss: 4.4533 - accuracy: 0.9500
Epoch 8/10

1/1 [==============================] - 1s 734ms/step - loss: 5.0040 - accuracy: 0.9750
Epoch 9/10

1/1 [==============================] - 1s 698ms/step - loss: 2.6457 - accuracy: 1.0000
Epoch 10/10

1/1 [==============================] - 1s 743ms/step - loss: 2.0256 - accuracy: 1.0000
[{'text': '吴 恩 达 在 北 京 大 学 。', 'text_raw': ['吴', '恩', '达', '在', '北', '京', '大', '学', '。'], 'labels': [{'entity': 'B_PER', 'start': 0, 'end': 0, 'value': '吴'}, {'entity': 'I_PER', 'start': 1, 'end': 2, 'value': '恩 达'}, {'entity': 'I_AFF', 'start': 4, 'end': 7, 'value': '北 京 大 学'}]}]

输出展示了BERT的12层Transformer结构，以及它的参数量。最后是NER的结果。

参考资料

https://kashgari-zh.bmio.net/
https://www.jianshu.com/p/1d6689851622
https://blog.csdn.net/ctwy291314/article/details/102819221

更多内容访问 omegaxyz.com
网站所有代码采用Apache 2.0授权
网站文章采用知识共享许可协议BY-NC-SA4.0授权
© 2020 • OmegaXYZ-版权所有 转载请注明出处

Reacubeth
关注
专栏目录
[Python人工智能] 四十二.命名实体识别 (3)基于Bert+BiLSTM-CRF的中文实体识别万字详解（异常解决中）
杨秀璋的专栏
03-05 928
从本专栏开始，作者正式研究Python深度学习、神经网络及人工智能相关知识。前文讲解如何实现中文命名实体识别研究，构建BiGRU-CRF模型实现。这篇文章将继续以中文语料为主，介绍融合Bert的实体识别研究，使用bert4keras和kears包来构建Bert+BiLSTM-CRF模型。然而，该代码最终结果有些问题，目前还在解决中，但现阶段方法先作为在线笔记分享出来。基础性文章，希望对您有帮助，如果存在错误或不足之处，还请海涵。且看且珍惜！
基于BERT-BiLSTM-CRF模型的中文实体识别
05-06
命名实体识别是自然语言处理的一项关键技术. 基于深度学习的方法已被广泛应用到中文实体识别研究中. 大多数深度学习模型的预处理主要注重词和字符的特征抽取, 却忽略词上下文的语义信息, 使其无法表征一词多义, 因而实体识别性能有待进一步提高. 为解决该问题, 本文提出了一种基于BERT-BiLSTM-CRF模型的研究方法. 首先通过BERT模型预处理生成基于上下文信息的词向量, 其次将训练出来的词向量输入BiLSTM-CRF模型做进一步训练处理. 实验结果表明, 该模型在MSRA语料和人民日报语料库上都达到相当不错的结果, F1值分别为94.65%和95.67%.
探索深度学习序列标注新高度：Bert-BiLSTM-CRF-pytorch
最新发布
gitblog_01162的博客
09-15 750
探索深度学习序列标注新高度：Bert-BiLSTM-CRF-pytorch Bert-BiLSTM-CRF-pytorch 使用谷歌预训练bert做字嵌入的BiLSTM-CRF序列标注模型 项目地址: https://gitcod...
学习笔记——ERT-BiLSTM-CRF命名实体识别模型的训练是不是比HanLP的使用要难
JLY19970726的博客
08-05 536
在比较BERT-BiLSTM-CRF命名实体识别模型的训练与HanLP的使用难度时，我们需要从多个维度进行考量，包括模型复杂度、数据准备、训练过程、以及最终的部署和应用。
探索先进自然语言处理：BERT-BiLSTM-CRF模型
gitblog_00097的博客
06-08 499
探索先进自然语言处理：BERT-BiLSTM-CRF模型 项目地址:https://gitcode.com/hertz-pj/BERT-BiLSTM-CRF-NER-pytorch 项目介绍 BERT-BiLSTM-CRF 是一个专为中文实体识别（NER）设计的强大模型，它将BERT的预训练能力与双向长短时记忆网络（BiLSTM）和条件随机场（CRF）相结合，以实现卓越的性能。该项目提供了一个简单...
探索BERT-BiLSTM-CRF-NER：深度学习命名实体识别的新里程
gitblog_00027的博客
03-21 894
探索BERT-BiLSTM-CRF-NER：深度学习命名实体识别的新里程 项目地址:https://gitcode.com/macanv/BERT-BiLSTM-CRF-NER 在这个数字化时代，自然语言处理（NLP）成为了人工智能领域的一个关键分支。而在这个领域中，命名实体识别（NER）是一项基础且重要的任务，用于在文本中定位和分类具有特定意义的实体，如人名、地名或组织名。BERT-BiLST...
探索BERT-BiLSTM-CRF：NLP任务的强大工具
gitblog_00062的博客
04-24 712
探索BERT-BiLSTM-CRF：NLP任务的强大工具 项目地址:https://gitcode.com/AidenHuen/BERT-BiLSTM-CRF BERT-BiLSTM-CRF是一个在自然语言处理（NLP）领域广泛应用的深度学习模型。该项目巧妙地结合了三种强大的技术：BERT、双向长短时记忆网络（BiLSTM）和条件随机场（CRF），以实现更准确的序列标注任务，如命名实体识别（NER...
NLPer必会：bert+bilstm+CRF进行中文命名实体识别（NER）
Clairezcy的博客
02-27 8327
1、命名实体识别由于有标注标签的存在，转化为多分类问题。 标注标签本次训练集中为‘BIO’方式标注 命名实体标注三种：人物，地点，组织机构 标签共7个类别：B-PER,I-PER,B-LOC,I-LOC,B-ORG,I-ORG,O 2、对于文本向量表示，如使用预训练模型BERT的向量： 安装腾讯肖涵博士开发的Bert Server(bert-serving-server)和Client(bert-...
BERT+BiLSTM+CRF命名实体识别模型训练及深度学习网络的构建
11-14 1万+
开头先附上 BERT的论文，还有中文翻译版哦 ，笑纳！ 原版： BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding BERT中文翻译PDF版.pdf 1、下载BERT及中文模型 BERT 源码地址：https://github.com/google-res...
Bert+Bilstm+Crf 命名实体识别(NER) Keras实战
weixin_42598761的博客
03-01 1万+
0.引言 最近做了一个命名实体识别(NER)的任务，发现了bert的强大，不愧是谷歌花了大精力训练的模型。其实网上相关tensorflow，pytorch的代码很多，但是用Keras的感觉没有一个是简单明了的，所以我在写这个网络的时候遇到不少麻烦，作为总结，把自己的经验分享给大家，尽量做到简单明了可实现，代码粗陋，适合新手学习，也欢迎大佬多给些宝贵意见。 1.前期准备 keras中要用bert和c...
BERT+CRF实现中文命名实体识别（TensorFlow）
热门推荐
Q_M_X_D_D_的博客
06-26 1万+
本篇解读了基于BERT+CRF做中文NER这篇文章中的代码，在该篇作者的GitHub上可以下载源码：源代码。这段代码对BERT模型的实现较为简洁，删掉了谷歌源代码中我们可能用不到的部分，保留了核心部分。对于那些想要快速上手BERT的同学是非常好的学习机会，在看懂这个之后再去学习谷歌的源代码会更加轻松。 BERT-CRF模型 之前有写过BERT模型和CRF模型的详解，建议往下看之前一定要了解这两个模型的原理：结合原理和代码来理解bert模型、结合原理与代码理解BiLSTM-CRF模型（pytorch）。B
BERT-BiLSTM-CRF-NER-master.zip
08-09
BERT只是一个预训练的语言模型，在各大任务上都刷新了榜单。我们本次实验的任务也是一个序列标注问题，简而言之，就是是基于BERT预训练模型，在中文NER（Named Entity Recognition，命名实体识别）任务上进行fine-tune。 Fine-tune是什么意思，中文译为微调。在transfer learning中，对事先训练好的特征抽取网络，直接拿来用在下游任务上。固定其特征抽取层的网络参数，只在原有的网络上增加少量神经元，做最后的分类任务，而且只更新分类参数。
BERT-BiLSTM-CRF-master.zip
07-17
命名实体识别代码，解压即可用 # BERT-BiLSTM-CRF BERT-BiLSTM-CRF的Keras版实现 ## BERT配置 1. 首先需要下载Pre-trained的BERT模型，本文用的是Google开源的中文BERT模型： - https://storage.googleapis.com/bert_models/2018_11_03/chinese_L-12_H-768_A-12.zip 2. 安装BERT客户端和服务器 pip install bert-serving-server pip install bert-serving-client，源项目如下： - https://github.com/hanxiao/bert-as-service 3. 打开服务器，在BERT根目录下，打开终端，输入命令： - bert-serving-start -pooling_strategy NONE -max_seq_len 144 -mask_cls_sep -model_dir chinese_L-12_H-768_A-12/ -num_worker 1
【BERT系列】——命名实体识别
01-07
本文是BERT实战的第二篇，使用BERT进行命名实体识别（序列标注类任务）。 1. 准备 1.1 环境 python 3.7； pytorch 1.3； transformers 2.3 （安装教程）； 1.2 数据 数据链接（链接：https://pan.baidu.com/s/1spwmV3_07U0HA9mlde2wMg 提取码：reic）； 2. 实战 2.1 训练代码 lr = 5e-5 max_length = 256 batch_size = 8 epoches = 20 cuda = True # cuda = False max_grad_norm = 1 warmup_s
基于pytorch的bert-bilstm-crf中文命名实体识别
05-20
目前只测试了bert_crf模型，其余的可根据需要自行调整。 问题汇总 ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. 解决方法：pip ...
基于Pytorch的BERT-IDCNN-BILSTM-CRF中文实体识别实现
05-20
基于Pytorch的BERT-IDCNN-BILSTM-CRF中文实体识别实现 模型训练（可选） 下载pytorch_model.bin到data/bert 下载训练集和测试集到data/ 检查配置constants.py 执行train.py，命令为 python train.py 中文命名实体...
Pytorch实现基于BERT+ BiLSTM+CRF的命名实体识别项目源码（课程设计）.zip
11-15
Pytorch实现基于BERT+ BiLSTM+CRF的命名实体识别项目源码（课程设计）.zip 这是95分以上高分必过课程设计项目，下载即用无需修改，确保可以运行。也可作为期末大作业。 Pytorch实现基于BERT+ BiLSTM+CRF的命名实体...
基于BERT-BILSTM-CRF进行中文命名实体识别python源码+项目使用说明+数据+模型.zip
10-25
基于BERT-BILSTM-CRF进行中文命名实体识别python源码+项目使用说明+数据+模型.zip 依赖 ```python scikit-learn==1.1.3 scipy==1.10.1 seqeval==1.2.2 transformers==4.27.4 pytorch-crf==0.7.2 ``` # 目录结构 ...
基于pytorch的bert-bilstm-crf中文命名实体识别项目源码+文档说明.zip
11-15
基于pytorch的bert_bilstm_crf中文命名实体识别项目源码+文档说明.zip 这是95分以上高分必过课程设计项目，下载即用无需修改，确保可以运行。也可作为期末大作业。 基于pytorch的bert_bilstm_crf中文命名实体识别...
BERT-BiLSTM-CRF命名实体识别原理详解
04-29
BERT-BiLSTM-CRF是一种常用的命名实体识别（Named Entity Recognition, NER）模型，下面简单介绍一下其原理。 BERT（Bidirectional Encoder Representations from Transformers）是一种预训练语言模型，可以学习到大量的语言知识并将其应用于各种自然语言处理任务中。在BERT-BiLSTM-CRF模型中，首先使用BERT模型对输入的文本进行编码，得到每个词的上下文向量表示。这些向量表示可以提供更丰富的语义信息，有助于提高NER任务的准确性。 接下来，使用BiLSTM（Bidirectional Long Short-Term Memory）来对上下文向量进行建模，以捕获更多的语义特征。BiLSTM是一种循环神经网络（Recurrent Neural Network, RNN），它可以同时从前向和后向对序列进行处理，从而对文本中的每个位置进行建模。这有助于处理上下文中的依赖关系，提高NER任务的性能。 最后，使用CRF（Conditional Random Field）对识别的实体进行标注。CRF是一种基于概率的序列标注模型，可以考虑不同实体之间的关系，从而更准确地进行实体识别。在BERT-BiLSTM-CRF模型中，CRF会利用BiLSTM提取的特征来为每个单词分配标签，以获得最终的实体识别结果。 综上所述，BERT-BiLSTM-CRF模型通过利用BERT模型的上下文向量表示、BiLSTM的序列建模和CRF的标注模型，可以在NER任务中实现更准确的实体识别。
评论 2
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值