Finetune简介
XTuner
微调训练
xtuner train ${CONFIG_NAME_OR_PATH}
部署与测试,将 HuggingFace adapter 合并到大语言模型:
xtuner convert merge ./internlm-chat-7b ./hf ./merged --max-shard-size 2GB
# xtuner convert merge \
# ${NAME_OR_PATH_TO_LLM} \
# ${NAME_OR_PATH_TO_ADAPTER} \
# ${SAVE_PATH} \
# --max-shard-size 2GB
训练第一轮
第二轮epoch 2
第三轮epoch train3
跑完训练后,当前路径长这样:
|-- internlm-chat-7b
|-- internlm_chat_7b_qlora_oasst1_e3_copy.py
|-- openassistant-guanaco
| |-- openassistant_best_replies_eval.jsonl
| `-- openassistant_best_replies_train.jsonl
`-- work_dirs
`-- internlm_chat_7b_qlora_oasst1_e3_copy
|-- 20231101_152923
| |-- 20231101_152923.log
| `-- vis_data
| |-- 20231101_152923.json
| |-- config.py
| `-- scalars.json
|-- epoch_1.pth
|-- epoch_2.pth
|-- epoch_3.pth
|-- internlm_chat_7b_qlora_oasst1_e3_copy.py
`-- last_checkpoint