/home/panda/miniconda3/envs/py37/bin/python /home/panda/Desktop/PythonProject/spert-master-xu/spert.py train --config configs/example_train.conf
Config:
{‘label’: ‘conll04_train’, ‘model_type’: ‘spert’, ‘model_path’: ‘bert-base-cased/pytorch-model.bin’, ‘tokenizer_path’: ‘bert-base-cased/bert-base-vocab.txt’, ‘config_path’: ‘bert-base-cased/bert_config.json’, ‘train_path’: ‘data/conll04/conll04_train.json’, ‘valid_path’: ‘data/conll04/conll04_dev.json’, ‘types_path’: ‘data/conll04/conll04_types.json’, ‘train_batch_size’: ‘2’, ‘eval_batch_size’: ‘1’, ‘neg_entity_count’: ‘100’, ‘neg_relation_count’: ‘100’, ‘epochs’: ‘300’, ‘lr’: ‘5e-5’, ‘lr_warmup’: ‘0.1’, ‘weight_decay’: ‘0.01’, ‘max_grad_norm’: ‘1.0’, ‘rel_filter_threshold’: ‘0.4’, ‘size_embedding’: ‘50’, ‘prop_drop’: ‘0.3’, ‘max_span_size’: ‘10’, ‘store_predictions’: ‘true’, ‘store_examples’: ‘true’, ‘sampling_processes’: ‘4’, ‘sampling_limit’: ‘100’, ‘max_pairs’: ‘1000’, ‘final_eval’: ‘true’, ‘log_path’: ‘data/log’, ‘save_path’: ‘data/model’}
Repeat 1 times
Iteration 0
Calling BertTokenizer.from_pretrained() with the path to a single file or url is deprecated
2021-08-22 06:09:05,562 [MainThread ] [INFO ] Datasets: data/conll04/conll04_train.json, data/conll04/conll04_dev.json
2021-08-22 06:09:05,562 [MainThread ] [INFO ] Model type: spert
Parse dataset ‘train’: 100%|██████████| 922/922 [00:01<00:00, 524.62it/s]
Parse dataset ‘valid’: 100%|██████████| 231/231 [00:00<00:00, 482.52it/s]
2021-08-22 06:09:07,813 [MainThread ] [INFO ] Relation type count: 6
2021-08-22 06:09:07,813 [MainThread ] [INFO ] Entity type count: 5
2021-08-22 06:09:07,813 [MainThread ] [INFO ] Entities:
2021-08-22 06:09:07,813 [MainThread ] [INFO ] No Entity=0
2021-08-22 06:09:07,813 [MainThread ] [INFO ] Location=1
2021-08-22 06:09:07,813 [MainThread ] [INFO ] Organization=2
2021-08-22 06:09:07,814 [MainThread ] [INFO ] People=3
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Other=4
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Relations:
2021-08-22 06:09:07,814 [MainThread ] [INFO ] No Relation=0
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Work for=1
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Kill=2
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Organization based in=3
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Live in=4
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Located in=5
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Dataset: train
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Document count: 922
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Relation count: 1283
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Entity count: 3377
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Dataset: valid
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Document count: 231
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Relation count: 343
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Entity count: 893
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Context size: 150
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Updates per epoch: 461
2021-08-22 06:09:07,814 [MainThread ] [INFO ] Updates total: 138300
Some weights of the model checkpoint at bert-base-cased/pytorch-model.bin were not used when initializing SpERT: [‘cls.predictions.bias’, ‘cls.predictions.transform.dense.weight’, ‘cls.predictions.transform.dense.bias’, ‘cls.predictions.decoder.weight’, ‘cls.seq_relationship.weight’, ‘cls.seq_relationship.bias’, ‘cls.predictions.transform.LayerNorm.weight’, ‘cls.predictions.transform.LayerNorm.bias’]
- This IS expected if you are initializing SpERT from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing SpERT from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of SpERT were not initialized from the model checkpoint at bert-base-cased/pytorch-model.bin and are newly initialized: [‘rel_classifier.weight’, ‘rel_classifier.bias’, ‘entity_classifier.weight’, ‘entity_classifier.bias’, ‘size_embeddings.weight’, ‘W0.weight’, ‘W0.bias’, ‘W1.weight’, ‘W1.bias’, ‘W2.weight’, ‘W2.bias’, ‘W3.weight’, ‘W3.bias’, ‘W4.weight’, ‘W4.bias’, ‘W5.weight’, ‘W5.bias’, ‘W6.weight’, ‘W6.bias’, ‘W7.weight’, ‘W7.bias’, ‘W_gmlp.weight’, ‘W_gmlp.bias’, ‘mlp_encoder.weight’, ‘mlp_encoder.bias’, ‘multi_att.linear_q.weight’, ‘multi_att.linear_k.weight’, ‘multi_att.linear_v.weight’]
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Train epoch 0: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 06:09:12,299 [MainThread ] [INFO ] Train epoch: 0
Train epoch 0: 100%|██████████| 461/461 [03:05<00:00, 2.49it/s]
Evaluate epoch 1: 0%| | 0/231 [00:00<?, ?it/s]2021-08-22 06:12:17,761 [MainThread ] [INFO ] Evaluate: valid
Evaluate epoch 1: 100%|██████████| 231/231 [01:07<00:00, 3.41it/s]
Evaluation
— Entities (named entity recognition (NER)) —
An entity is considered correct if the entity type and span is predicted correctly
type precision recall f1-score support
Org 0.00 0.00 0.00 170.0
Other 0.00 0.00 0.00 118.0
Peop 0.00 0.00 0.00 283.0
Loc 0.00 0.00 0.00 322.0
micro 0.00 0.00 0.00 893.0
macro 0.00 0.00 0.00 893.0
— Relations —
Without named entity classification (NEC)
A relation is considered correct if the relation type and the spans of the two related entities are predicted correctly (entity type is not considered)
type precision recall f1-score support
LocIn 0.00 0.00 0.00 65.0
Live 0.00 0.00 0.00 91.0
Work 0.00 0.00 0.00 69.0
OrgBI 0.00 0.00 0.00 76.0
Kill 0.00 0.00 0.00 42.0
micro 0.00 0.00 0.00 343.0
macro 0.00 0.00 0.00 343.0
With named entity classification (NEC)
A relation is considered correct if the relation type and the two related entities are predicted correctly (in span and entity type)
type precision recall f1-score support
LocIn 0.00 0.00 0.00 65.0
Live 0.00 0.00 0.00 91.0
Work 0.00 0.00 0.00 69.0
OrgBI 0.00 0.00 0.00 76.0
Kill 0.00 0.00 0.00 42.0
micro 0.00 0.00 0.00 343.0
macro 0.00 0.00 0.00 343.0
2021-08-22 06:13:26,743 [MainThread ] [INFO ] Train epoch: 1
Train epoch 1: 100%|██████████| 461/461 [03:07<00:00, 2.46it/s]
2021-08-22 06:16:33,955 [MainThread ] [INFO ] Train epoch: 2
Train epoch 2: 100%|██████████| 461/461 [03:11<00:00, 2.41it/s]
Train epoch 3: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 06:19:45,415 [MainThread ] [INFO ] Train epoch: 3
Train epoch 3: 100%|██████████| 461/461 [03:13<00:00, 2.38it/s]
Train epoch 4: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 06:22:58,997 [MainThread ] [INFO ] Train epoch: 4
Train epoch 4: 100%|██████████| 461/461 [03:05<00:00, 2.48it/s]
2021-08-22 06:26:04,921 [MainThread ] [INFO ] Train epoch: 5
Train epoch 5: 100%|██████████| 461/461 [03:07<00:00, 2.45it/s]
Train epoch 6: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 06:29:12,767 [MainThread ] [INFO ] Train epoch: 6
Train epoch 6: 100%|██████████| 461/461 [03:15<00:00, 2.36it/s]
Train epoch 7: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 06:32:27,822 [MainThread ] [INFO ] Train epoch: 7
Train epoch 7: 100%|██████████| 461/461 [03:08<00:00, 2.44it/s]
2021-08-22 06:35:36,429 [MainThread ] [INFO ] Train epoch: 8
Train epoch 8: 100%|██████████| 461/461 [03:14<00:00, 2.38it/s]
Train epoch 9: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 06:38:50,479 [MainThread ] [INFO ] Train epoch: 9
Train epoch 9: 100%|██████████| 461/461 [03:08<00:00, 2.44it/s]
Train epoch 10: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 06:41:59,257 [MainThread ] [INFO ] Train epoch: 10
Train epoch 10: 100%|██████████| 461/461 [03:11<00:00, 2.41it/s]
2021-08-22 06:45:10,902 [MainThread ] [INFO ] Train epoch: 11
Train epoch 11: 100%|██████████| 461/461 [03:13<00:00, 2.39it/s]
Train epoch 12: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 06:48:24,077 [MainThread ] [INFO ] Train epoch: 12
Train epoch 12: 100%|██████████| 461/461 [03:15<00:00, 2.36it/s]
Train epoch 13: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 06:51:39,384 [MainThread ] [INFO ] Train epoch: 13
Train epoch 13: 100%|██████████| 461/461 [03:18<00:00, 2.32it/s]
Train epoch 14: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 06:54:58,072 [MainThread ] [INFO ] Train epoch: 14
Train epoch 14: 100%|██████████| 461/461 [03:11<00:00, 2.40it/s]
2021-08-22 06:58:10,049 [MainThread ] [INFO ] Train epoch: 15
Train epoch 15: 100%|██████████| 461/461 [03:14<00:00, 2.38it/s]
2021-08-22 07:01:24,127 [MainThread ] [INFO ] Train epoch: 16
Train epoch 16: 100%|██████████| 461/461 [03:17<00:00, 2.33it/s]
Train epoch 17: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 07:04:41,951 [MainThread ] [INFO ] Train epoch: 17
Train epoch 17: 100%|██████████| 461/461 [03:13<00:00, 2.39it/s]
Train epoch 18: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 07:07:55,191 [MainThread ] [INFO ] Train epoch: 18
Train epoch 18: 100%|██████████| 461/461 [03:16<00:00, 2.34it/s]
Train epoch 19: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 07:11:12,065 [MainThread ] [INFO ] Train epoch: 19
Train epoch 19: 100%|██████████| 461/461 [03:08<00:00, 2.44it/s]
Train epoch 20: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 07:14:20,983 [MainThread ] [INFO ] Train epoch: 20
Train epoch 20: 100%|██████████| 461/461 [03:18<00:00, 2.32it/s]
2021-08-22 07:17:39,297 [MainThread ] [INFO ] Evaluate: valid
Evaluate epoch 21: 100%|██████████| 231/231 [01:08<00:00, 3.37it/s]
Evaluation
— Entities (named entity recognition (NER)) —
An entity is considered correct if the entity type and span is predicted correctly
type precision recall f1-score support
Org 48.64 73.53 58.55 170
Other 57.94 61.86 59.84 118
Peop 81.79 90.46 85.91 283
Loc 61.49 92.24 73.79 322
micro 63.70 84.10 72.49 893
macro 62.46 79.52 69.52 893
— Relations —
Without named entity classification (NEC)
A relation is considered correct if the relation type and the spans of the two related entities are predicted correctly (entity type is not considered)
type precision recall f1-score support
LocIn 29.31 78.46 42.68 65
Live 25.87 81.32 39.26 91
Work 22.91 75.36 35.14 69
OrgBI 17.42 40.79 24.41 76
Kill 59.18 69.05 63.74 42
micro 25.93 69.10 37.71 343
macro 30.94 69.00 41.04 343
With named entity classification (NEC)
A relation is considered correct if the relation type and the two related entities are predicted correctly (in span and entity type)
type precision recall f1-score support
LocIn 29.31 78.46 42.68 65
Live 25.87 81.32 39.26 91
Work 22.47 73.91 34.46 69
OrgBI 17.42 40.79 24.41 76
Kill 59.18 69.05 63.74 42
micro 25.82 68.80 37.55 343
macro 30.85 68.71 40.91 343
2021-08-22 07:18:49,313 [MainThread ] [INFO ] Train epoch: 21
Train epoch 21: 100%|██████████| 461/461 [03:11<00:00, 2.41it/s]
Train epoch 22: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 07:22:00,689 [MainThread ] [INFO ] Train epoch: 22
Train epoch 22: 100%|██████████| 461/461 [03:18<00:00, 2.32it/s]
2021-08-22 07:25:19,599 [MainThread ] [INFO ] Train epoch: 23
Train epoch 23: 100%|██████████| 461/461 [03:17<00:00, 2.34it/s]
Train epoch 24: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 07:28:36,735 [MainThread ] [INFO ] Train epoch: 24
Train epoch 24: 100%|██████████| 461/461 [03:07<00:00, 2.45it/s]
Train epoch 25: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 07:31:44,691 [MainThread ] [INFO ] Train epoch: 25
Train epoch 25: 100%|██████████| 461/461 [03:11<00:00, 2.41it/s]
Train epoch 26: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 07:34:56,338 [MainThread ] [INFO ] Train epoch: 26
Train epoch 26: 100%|██████████| 461/461 [03:12<00:00, 2.39it/s]
2021-08-22 07:38:09,090 [MainThread ] [INFO ] Train epoch: 27
Train epoch 27: 100%|██████████| 461/461 [03:13<00:00, 2.38it/s]
Train epoch 28: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 07:41:22,404 [MainThread ] [INFO ] Train epoch: 28
Train epoch 28: 100%|██████████| 461/461 [03:11<00:00, 2.40it/s]
Train epoch 29: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 07:44:34,378 [MainThread ] [INFO ] Train epoch: 29
Train epoch 29: 100%|██████████| 461/461 [03:12<00:00, 2.39it/s]
2021-08-22 07:47:47,195 [MainThread ] [INFO ] Train epoch: 30
Train epoch 30: 100%|██████████| 461/461 [03:09<00:00, 2.44it/s]
Train epoch 31: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 07:50:56,509 [MainThread ] [INFO ] Train epoch: 31
Train epoch 31: 100%|██████████| 461/461 [03:16<00:00, 2.34it/s]
2021-08-22 07:54:13,479 [MainThread ] [INFO ] Train epoch: 32
Train epoch 32: 100%|██████████| 461/461 [03:17<00:00, 2.33it/s]
Train epoch 33: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 07:57:31,343 [MainThread ] [INFO ] Train epoch: 33
Train epoch 33: 100%|██████████| 461/461 [03:19<00:00, 2.31it/s]
Train epoch 34: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 08:00:51,162 [MainThread ] [INFO ] Train epoch: 34
Train epoch 34: 100%|██████████| 461/461 [03:08<00:00, 2.44it/s]
Train epoch 35: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 08:04:00,137 [MainThread ] [INFO ] Train epoch: 35
Train epoch 35: 100%|██████████| 461/461 [03:15<00:00, 2.36it/s]
Train epoch 36: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 08:07:15,188 [MainThread ] [INFO ] Train epoch: 36
Train epoch 36: 100%|██████████| 461/461 [03:09<00:00, 2.43it/s]
Train epoch 37: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 08:10:24,540 [MainThread ] [INFO ] Train epoch: 37
Train epoch 37: 100%|██████████| 461/461 [03:13<00:00, 2.38it/s]
Train epoch 38: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 08:13:38,159 [MainThread ] [INFO ] Train epoch: 38
Train epoch 38: 100%|██████████| 461/461 [03:13<00:00, 2.38it/s]
Train epoch 39: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 08:16:51,570 [MainThread ] [INFO ] Train epoch: 39
Train epoch 39: 100%|██████████| 461/461 [03:17<00:00, 2.34it/s]
Train epoch 40: 0%| | 0/461 [00:00<?, ?it/s]2021-08-22 08:20:08,715 [MainThread ] [INFO ] Train epoch: 40
Train epoch 40: 100%|██████████| 461/461 [03:13<00:00, 2.38it/s]
Evaluate epoch 41: 0%| | 0/231 [00:00<?, ?it/s]2021-08-22 08:23:22,695 [MainThread ] [INFO ] Evaluate: valid
Evaluate epoch 41: 100%|██████████| 231/231 [01:07<00:00, 3.42it/s]
Evaluation
— Entities (named entity recognition (NER)) —
An entity is considered correct if the entity type and span is predicted correctly
type precision recall f1-score support
Org 67.74 61.76 64.62 170
Other 66.02 57.63 61.54 118
Peop 86.82 90.81 88.77 283
Loc 78.83 87.89 83.11 322
micro 78.09 79.84 78.96 893
macro 74.85 74.52 74.51 893
— Relations —
Without named entity classification (NEC)
A relation is considered correct if the relation type and the spans of the two related entities are predicted correct