计算Bert+LSTM+CRF实现的ner中的参数量有多少

已于 2022-06-10 18:12:21 修改
阅读量509 收藏 1
点赞数
分类专栏： 机器学习 算法 文章标签： lstm bert tensorflow
于 2022-06-10 17:51:28 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/bian_h_f612701198412/article/details/125225917
版权

1、加载模型，查看日志
INFO:tensorflow:** Trainable Variables ****
INFO:tensorflow: name = bert/embeddings/word_embeddings:0, shape = (21128, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/embeddings/token_type_embeddings:0, shape = (2, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/embeddings/position_embeddings:0, shape = (512, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/embeddings/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/embeddings/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/self/query/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/self/query/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/self/key/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/self/key/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/self/value/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/self/value/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/output/dense/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/intermediate/dense/kernel:0, shape = (768, 3072), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/intermediate/dense/bias:0, shape = (3072,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/output/dense/kernel:0, shape = (3072, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/attention/self/query/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/attention/self/query/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/attention/self/key/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/attention/self/key/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/attention/self/value/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/attention/self/value/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/attention/output/dense/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/attention/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/attention/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/attention/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/intermediate/dense/kernel:0, shape = (768, 3072), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/intermediate/dense/bias:0, shape = (3072,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/output/dense/kernel:0, shape = (3072, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_1/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/attention/self/query/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/attention/self/query/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/attention/self/key/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/attention/self/key/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/attention/self/value/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/attention/self/value/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/attention/output/dense/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/attention/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/attention/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/attention/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/intermediate/dense/kernel:0, shape = (768, 3072), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/intermediate/dense/bias:0, shape = (3072,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/output/dense/kernel:0, shape = (3072, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_2/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/attention/self/query/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/attention/self/query/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/attention/self/key/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/attention/self/key/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/attention/self/value/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/attention/self/value/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/attention/output/dense/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/attention/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/attention/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/attention/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/intermediate/dense/kernel:0, shape = (768, 3072), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/intermediate/dense/bias:0, shape = (3072,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/output/dense/kernel:0, shape = (3072, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_3/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/attention/self/query/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/attention/self/query/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/attention/self/key/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/attention/self/key/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/attention/self/value/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/attention/self/value/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/attention/output/dense/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/attention/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/attention/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/attention/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/intermediate/dense/kernel:0, shape = (768, 3072), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/intermediate/dense/bias:0, shape = (3072,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/output/dense/kernel:0, shape = (3072, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_4/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/attention/self/query/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/attention/self/query/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/attention/self/key/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/attention/self/key/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/attention/self/value/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/attention/self/value/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/attention/output/dense/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/attention/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/attention/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/attention/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/intermediate/dense/kernel:0, shape = (768, 3072), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/intermediate/dense/bias:0, shape = (3072,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/output/dense/kernel:0, shape = (3072, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_5/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/attention/self/query/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/attention/self/query/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/attention/self/key/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/attention/self/key/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/attention/self/value/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/attention/self/value/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/attention/output/dense/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/attention/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/attention/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/attention/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/intermediate/dense/kernel:0, shape = (768, 3072), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/intermediate/dense/bias:0, shape = (3072,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/output/dense/kernel:0, shape = (3072, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_6/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/attention/self/query/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/attention/self/query/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/attention/self/key/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/attention/self/key/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/attention/self/value/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/attention/self/value/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/attention/output/dense/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/attention/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/attention/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/attention/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/intermediate/dense/kernel:0, shape = (768, 3072), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/intermediate/dense/bias:0, shape = (3072,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/output/dense/kernel:0, shape = (3072, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_7/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/attention/self/query/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/attention/self/query/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/attention/self/key/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/attention/self/key/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/attention/self/value/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/attention/self/value/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/attention/output/dense/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/attention/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/attention/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/attention/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/intermediate/dense/kernel:0, shape = (768, 3072), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/intermediate/dense/bias:0, shape = (3072,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/output/dense/kernel:0, shape = (3072, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_8/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/attention/self/query/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/attention/self/query/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/attention/self/key/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/attention/self/key/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/attention/self/value/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/attention/self/value/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/attention/output/dense/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/attention/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/attention/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/attention/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/intermediate/dense/kernel:0, shape = (768, 3072), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/intermediate/dense/bias:0, shape = (3072,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/output/dense/kernel:0, shape = (3072, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_9/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/attention/self/query/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/attention/self/query/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/attention/self/key/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/attention/self/key/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/attention/self/value/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/attention/self/value/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/attention/output/dense/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/attention/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/attention/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/attention/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/intermediate/dense/kernel:0, shape = (768, 3072), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/intermediate/dense/bias:0, shape = (3072,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/output/dense/kernel:0, shape = (3072, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_10/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/attention/self/query/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/attention/self/query/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/attention/self/key/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/attention/self/key/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/attention/self/value/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/attention/self/value/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/attention/output/dense/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/attention/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/attention/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/attention/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/intermediate/dense/kernel:0, shape = (768, 3072), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/intermediate/dense/bias:0, shape = (3072,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/output/dense/kernel:0, shape = (3072, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_11/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/pooler/dense/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/pooler/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = rnn_layer/bidirectional_rnn/fw/basic_lstm_cell/kernel:0, shape = (896, 512)
INFO:tensorflow: name = rnn_layer/bidirectional_rnn/fw/basic_lstm_cell/bias:0, shape = (512,)
INFO:tensorflow: name = rnn_layer/bidirectional_rnn/bw/basic_lstm_cell/kernel:0, shape = (896, 512)
INFO:tensorflow: name = rnn_layer/bidirectional_rnn/bw/basic_lstm_cell/bias:0, shape = (512,)
INFO:tensorflow: name = project/hidden/W:0, shape = (256, 128)
INFO:tensorflow: name = project/hidden/b:0, shape = (128,)
INFO:tensorflow: name = project/logits/W:0, shape = (128, 11)
INFO:tensorflow: name = project/logits/b:0, shape = (11,)
INFO:tensorflow: name = crf_loss/transitions:0, shape = (11, 11)* *

从日志中可以看到bert模型：
（1）、初始化这三个的参数word_embeddings token_type_embeddings position_embeddings
INFO:tensorflow: name = bert/embeddings/word_embeddings:0, shape = (21128, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/embeddings/token_type_embeddings:0, shape = (2, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/embeddings/position_embeddings:0, shape = (512, 768), INIT_FROM_CKPT
（2）、连续经过12层的encoder：
INFO:tensorflow: name = bert/embeddings/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/embeddings/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/self/query/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/self/query/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/self/key/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/self/key/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/self/value/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/self/value/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/output/dense/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/output/LayerNorm/beta:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/attention/output/LayerNorm/gamma:0, shape = (768,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/intermediate/dense/kernel:0, shape = (768, 3072), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/intermediate/dense/bias:0, shape = (3072,), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/output/dense/kernel:0, shape = (3072, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/encoder/layer_0/output/dense/bias:0, shape = (768,), INIT_FROM_CKPT
（3）、获取最后一层encoder的输出：
INFO:tensorflow: name = bert/pooler/dense/kernel:0, shape = (768, 768), INIT_FROM_CKPT
INFO:tensorflow: name = bert/pooler/dense/bias:0, shape = (768,), INIT_FROM_CKPT
（4）、通过双向LSTM，获取表征输出。
INFO:tensorflow: name = rnn_layer/bidirectional_rnn/fw/basic_lstm_cell/kernel:0, shape = (896, 512)
INFO:tensorflow: name = rnn_layer/bidirectional_rnn/fw/basic_lstm_cell/bias:0, shape = (512,)
INFO:tensorflow: name = rnn_layer/bidirectional_rnn/bw/basic_lstm_cell/kernel:0, shape = (896, 512)
INFO:tensorflow: name = rnn_layer/bidirectional_rnn/bw/basic_lstm_cell/bias:0, shape = (512,)
（5）、经过两次全连接，256–>128–>11，最终生成11个分类的表征向量
INFO:tensorflow: name = project/hidden/W:0, shape = (256, 128)
INFO:tensorflow: name = project/hidden/b:0, shape = (128,)
INFO:tensorflow: name = project/logits/W:0, shape = (128, 11)
INFO:tensorflow: name = project/logits/b:0, shape = (11,)
（6）、通过CRF层，获取最终的损失函数。
INFO:tensorflow: name = crf_loss/transitions:0, shape = (11, 11)

2、计算参数量：由于第二列很多都是空，利用Excel将这个列中的数值置为1：
公式为：=IF(H1=“”,1,H1)在这里插入图片描述
3、对最后一列进行求和1,0322,0612
可是官方宣称bert参数量是1.1亿，为什么这里少了这么多呢？
答：因为官方vocab数量是30522，而不是咱们平时用到的bert中的词表是21128。明白了吧。
4、那么通过bert实现的ner以后的效果怎么样呢？
在测试集上面的准确率召回率：

			eval_f = 0.95210534
			eval_precision = 0.960108
			eval_recall = 0.944235
			global_step = 195600
			loss = 4.2875824
	看到了这个效果以后，我还是挺震撼的， 我想象中的效果应该是80%左右，没有想到bert这么强悍。让我重新对认识transformer的能力了。
	而在训练集上面，直接100%
			eval_f = 1.0
			eval_precision = 1.0
			eval_recall = 1.0
			global_step = 195600
			loss = 0.00012448594
北門大官人
关注
  • 0
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 2
    评论
专栏目录
bert+lstm+crf ner实体识别
weixin_42357472的博客
08-14 5541
参考： https://github.com/BrikerMan/Kashgari https://eliyar.biz/nlp_chinese_text_classification_in_15mins/ bert预训练中文模型下载：百度云下载地址：https://pan.baidu.com/s/16JJp6NMCjmIT8PqP9OfN4g 包含这些内容的整体下载下来 !pip install kashgari !pip install -i https://pypi.tuna.tsinghua.ed
基于BERT-BiLSTM-CRF模型的中文实体识别
05-06
命名实体识别是自然语言处理的一项关键技术. 基于深度学习的方法已被广泛应用到中文实体识别研究中. 大多数深度学习模型的预处理主要注重词和字符的特征抽取, 却忽略词上下文的语义信息, 使其无法表征一词多义, 因而实体识别性能有待进一步提高. 为解决该问题, 本文提出了一种基于BERT-BiLSTM-CRF模型的研究方法. 首先通过BERT模型预处理生成基于上下文信息的词向量, 其次将训练出来的词向量输入BiLSTM-CRF模型做进一步训练处理. 实验结果表明, 该模型在MSRA语料和人民日报语料库上都达到相当不错的结果, F1值分别为94.65%和95.67%.
BERT-BiLSTM-CRF-NER:NER任务的Tensorflow解决方案将BiLSTM-CRF模型与Google BERT微调和私有服务器服务结合使用
05-09
BERT-BiLSTM-CRF-NER Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning 使用谷歌的BERT模型在BLSTM-CRF模型上进行预训练用于中文命名实体识别的Tensorflow代码' 中文文档请查看 如果对您有帮助，麻烦点个star,谢谢~~ Welcome to star this repository! The Chinese training data($PATH/NERdata/) come from: The CoNLL-2003 data($PATH/NERdata/ori/) come from: The evaluation codes come from: Try to implement NER work based on google'
大数据TensorFlow深度学习——基于BERT+LSTM+CRF深度学习识别模型医疗知识图谱问答可视化系统(完整系统源码+PPT+详细开发文档+论文+源码解析)
qq_63981678的博客
02-01 1018
基于BERT+LSTM+CRF深度学习识别模型医疗知识图谱问答可视化系统通过构建医疗领域的知识图谱来实现计算机的深度学习，并且能够实现自动问答的功能。本次的内容研究主要是通过以Python技术来对医疗相关内容进行数据的爬取，通过爬取足量的数据来进行知识图谱的的搭建，基于Python语言通过echarts、Neo4j来实现知识图谱的可视化。通过智慧问答的方式构建出以BERT+LSTM+CRF的深度学习识别模型，从而完成对医疗问句主体的识别，构建出数据集以及实现文本的训练。通过Django来进行web网页的开发
TensorFlow|基于双向LSTM+条件随机场(BiLSTM-CRF)的中文分词模型
shillyshally的博客
12-01 1万+
  更新内容（2018-12-2）： 已开源在GIthub上： 地址：https://github.com/shillyshallysxy/Learning_NLP 前言 中文分词在中文信息处理中是最为基础的一步，无论机器翻译还是信息检索还是其他相关应用，如果涉及中文，都离不开中文分词，因此中文分词具有极高的地位。 中文分词是自然语言处理的分支，是指用计算机对中文进行处理。和大部分西方...
命名实体识别NER & 如何使用BERT实现
满腹的小不甘
11-14 1万+
目录 1、NER 简介 2. NER方法 2.1 传统机器学习方法：HMM和CRF 2.2 LSTM+CRF：BiLSTM-CRF 2.3 CNN+CRF：IDCNN-CRF 2.4BERT+（LSTM）+CRF：BERT实现 （1）获取BERT预训练模型 （2）修改 数据预处理代码：DataProcessor （3）构建模型：create_model （4） 模...
【深度学习、工程实践】基于BERT-LSTM-CRF的命名实体识别(PyTorch 实现)
Dong_ZH的博客
12-28 9060
命名实体识别(NER)是自然语言处理的基础任务，处于工程性任务的上游，很多的下游任务依赖于命名实体识别的效果，所以命名实体识别也是自然语言处理中非常重要的一环。命名实体识别的任务非常简单，给出一段文本，识别出文本中的人名、机构名称、歌曲名称、专辑名称、地点名称等名词（或者称之为实体）
BERT中进行NER为什么没有使用CRF，我们使用DL进行序列标注问题的时候CRF是必备么？
weixin_36378508的博客
04-14 395
Q1: 为什么LSTM加上CRF后，序列标注效果会有稳定提升？ 序列标注中CRF存在的意义究竟是什么？其实从模型的损失函数可以很直接的看出添加CRF前后的差异。 单纯使用神经网络做序列标注时，使用的损失函数通常为对每一个时间步的交叉熵损失取平均， 此时模型对应的学习目标为 而加入CRF后，模型的学习目标变成了 其中， X 和 Y分别为训练集的输入文本序列和标签序列， theta 表示LSTM的模型结构和参数，也就是模型拟合能力的体现。Ytrans是从训练集所有标签序列统计得到的转移概率（尽管是训练过程
transformers实践:基于BERT训练自己的NER模型
和木的专栏
08-27 3702
文章目录transformers实践:基于BERT训练自己的NER模型数据集处理训练过程模型的调用和使用附：两个数据集说明：附：参考 transformers实践:基于BERT训练自己的NER模型 基于训练好的BERT进行迁移NER的原理如下： 官方样例集成的很好，直接运行run_ner.py即可，下面对几个步骤(数据预处理、运行参数、模型调用)做下补充说明 数据集处理 run_ner.p中train_file要求的格式样例，如 https://github.com/huggingface/transfo
Bert+LSTM+CRF命名实体识别pytorch代码详解
qq_48034566的博客
03-28 1万+
Bert+LSTM+CRF命名实体识别 从0开始解析源代码。 理解原代码的逻辑，具体了解为什么使用预训练的bert，bert有什么作用，网络的搭建是怎么样的，训练过程是怎么训练的，输出是什么 调试运行源代码 NER目标 NER是named entity recognized的简写，对人名、地名、机构名、日期时间、专有名词等进行识别。 结果输出标注方法 采用细粒度标注，就是对于每一个词都给一个标签，其中连续的词可能是一个标签，与原始数据集的结构不同，需要对数据进行处理，转化成对应的细粒度标注形
Python-用谷歌BERT模型在BLSTMCRF模型上进行预训练用于中文命名实体识别的Tensorflow代码
08-10
Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning
基于 pytorch 实现 bert-bilstm-crf-ner 命名实体识别 完整代码+数据 可直接运行
03-17
基于 pytorch 实现 bert_bilstm_crf_ner 命名实体识别 完整代码+数据 可直接运行
BERT+BiLSTM+CRF NER简单实战
09-09
BERT+BiLSTM+CRF NER简单实战
python毕业设计&课设-基于BERT+BiLSTM+CRF实现中文命名实体识别（源码+文档）.zip
最新发布
04-16
BERT+BiLSTM+CRF是一种用于中文命名实体识别（Named Entity Recognition，简称NER）的模型，结合了BERT模型、双向长短时记忆网络（Bidirectional LSTM）和条件随机场（CRF）。 BERT是一种预训练的深度双向变换器...
自然语言处理实体抽取算法基于pytorch框架bert+bilstm+crf
10-24
BERT+BiLSTM+CRF是一种用于命名实体识别（Named Entity Recognition, NER）的深度学习模型。其中，BERT（Bidirectional Encoder Representations from Transformers）是一种预训练的语言模型，用于捕捉上下文信息；...
25行代码实现完整的RSA算法
热门推荐
专栏
02-24 5万+
25行代码实现完整的RSA算法   网络上很多关于RSA算法的原理介绍，但是翻来翻去就是没有一个靠谱的算法实现，即使有代码介绍，也都是直接调用JDK或者Python代码包中的API实现，或者即使有代码也都写得特别烂。无形中让人感觉RSA加密算法竟然这么高深，然后就看不下去了。还有我发现对于“大整数的幂次乘方取模”竟然采用直接计算的幂次的值，再取模，类似于(2 ^ 1024) ^ (2 ^ 102...
25行代码实现完整的RSA算法Java版
专栏
03-29 1万+
25行代码实现完整的RSA算法Java版   我的上一篇博客《25行代码实现完整的RSA算法》自从上个月发表了以后，很多程序员给我打电话或者发短信说，终于看到了一篇能把RSA算法的代码写明白的，他们问我说能不能把代码写成Java版的，我说Java的会看着很费劲，Python代码的直观性在数字计算方面有很大的优势。   但是架不住他们非要我写，我一拍肩膀说，好吧，我答应你们的请求。花了一晚上的时...
25行代码实现完整的RSA算法（Python3.X版本）
专栏
02-13 1万+
25行代码实现完整的RSA算法（Python3.X版本）   python2.7版本的请点击这里25行代码实现完整的RSA算法   网络上很多关于RSA算法的原理介绍，但是翻来翻去就是没有一个靠谱、让人信服的算法代码实现，即使有代码介绍，也都是直接调用JDK或者Python代码包中的API实现，也有可能并没有把核心放在原理的实现上，而是字符串转数字啦、或者数字转字符串啦、或者即使有代码也都写得特别...
BERT-BILSTM-CRF的中文NER模型
04-28
具体来说，在BERT-BILSTM-CRF模型中，BERT模型用于将中文文本编码成上下文相关的向量序列，然后将这个向量序列输入到双向LSTM网络中，以提取更高层次、更抽象的特征表示。最后，CRF层将这些特征序列转换为标签序列，...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 2
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值