albert_pytorch
This repository contains a PyTorch implementation of the albert model from the paper
by Zhenzhong Lan. Mingda Chen....
Dependencies
pytorch=1.10
cuda=9.0
cudnn=7.5
scikit-learn
sentencepiece
Download Pre-trained Models of English
Official download links: google albert
Adapt to this version,download pytorch model (google drive):
v1
v2
Fine-tuning
1. Place config.json and 30k-clean.model into the prev_trained_model/albert_base_v2 directory. example:
├── prev_trained_model
| └── albert_base_v2
| | └── pytorch_model.bin
| | └── config.json
| | └── 30k-clean.model
2.convert albert tf checkpoint to pytorch
python convert_albert_tf_checkpoint_to_pytorch.py \
--tf_checkpoint_path=./prev_trained_model/albe