p-tuning v2 微调 chatGLM实战

环境安装

!pip install rouge_chinese nltk jieba datasets

Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Requirement already satisfied: rouge_chinese in ./miniconda3/lib/python3.10/site-packages (1.0.3)

…

chatGLM框架部署b

!git clone https://gitee.com/952202/ChatGLM2-6B.git
#git clone https://github.com/THUDM/ChatGLM2-6B.git

fatal: destination path 'ChatGLM2-6B' already exists and is not an empty directory.

• 注意，这里要求 transformers==4.30.2 但这个版本无法识别qwen2 模型，如果确认是qwen1版本，则不需要进行下面的修改

• 先安装 pip install transformers==4.37.0 或者把 requirements.txt 中的transformers版本改成这 4.37.0

• 执行vim $CONDA_PREFIX/lib/python3.10/site-packages/transformers/modeling_utils.py

safe_serialization 改成 False

def save_pretrained(
        self,
        save_directory: Union[str, os.PathLike],
        is_main_process: bool = True,
        state_dict: Optional[dict] = None,
        save_function: Callable = torch.save,
        push_to_hub: bool = False,
        max_shard_size: Union[int, str] = "5GB",
        safe_serialization: bool = True,
        variant: Optional[str] = None,
        token: Optional[Union[str, bool]] = None,
        save_peft_format: bool = True,
        **kwargs,
    ):
    ...

%cd ChatGLM2-6B
!pip install -r requirements.txt

[Errno 2] No such file or directory: 'ChatGLM2-6B'
/data/ChatGLM2-6B


/data/miniconda3/lib/python3.10/site-packages/IPython/core/magics/osm.py:393: UserWarning: using bookmarks requires you to install the `pickleshare` library.
  bkms = self.shell.db.get('bookmarks', {})


Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Requirement already satisfied: protobuf in /data/miniconda3/lib/python3.10/site-packages (from -r requirements.txt (line 1)) (3.20.3)
Collecting transformers==4.30.2 (from -r requirements.txt (line 2))
  Downloading https://mirrors.aliyun.com/pypi/packages/5b/0b/e45d26ccd28568013523e04f325432ea88a442b4e3020b757cf4361f0120/transformers-4.30.2-py3-none-any.whl (7.2 MB)
[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m7.2/7.2 MB[0m [31m682.4 kB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m
[?25hRequirement already satisfied: cpm_kernels in /data/miniconda3/lib/python3.10/site-packages (from -r requirements.txt (line 3)) (1.0.11)
Requirement already satisfied: torch>=2.0 in /data/miniconda3/lib/python3.10/site-packages (from -r requirements.txt (line 4)) (2.3.0)
Requirement already satisfied: gradio in /data/miniconda3/lib/python3.10/site-packages (from -r requirements.txt (line 5)) (4.26.0)
Requirement already satisfied: mdtex2html in /data/miniconda3/lib/python3.10/site-packages (from -r requirements.txt (line 6)) (1.3.0)
Requirement already satisfied: sentencepiece in /data/miniconda3/lib/python3.10/site-packages (from -r requirements.txt (line 7)) (0.2.0)
......
[0mSuccessfully installed tokenizers-0.13.3 transformers-4.30.2

处理训练数据集

import json

instruction="请对句子进行评价分类(正向评价,负向评价,中性评价): "
# 读取JSON文件
with open('/data/train_data/训练集示例.json', 'r', encoding='utf-8') as f:
    data = json.load(f)

# 遍历每个条目，在content的值前面加上指定的字符串
for item in data:
    item['content'] = instruction + item['content']

# 将更新后的数据写回JSON文件
with open('/data/train_data/formatted_train_data.json', 'w', encoding='utf-8') as f:
    json.dump(data, f, ensure_ascii=False, indent=4)

模型训练

%cd ptuning
!pwd

/data/ChatGLM2-6B/ptuning
/data/ChatGLM2-6B/ptuning

# 禁用 wandb
import os
os.environ["WANDB_DISABLED"] = "true"

• –max_steps 1000 训练步数，要根据训练集的记录数评估下,如果需要epochs=3 应该是训练集的总记录数*3
• –save_steps 500 这个步数要能被max_steps整除
• –model_name_or_path 模型路径
• –output_dir 训练输出路径，推理时要用
• –learning_rate 其它模型我习惯设到 1e-4 ，但glm可能不太一样，缺省设的2e-2
• –prompt_column content 这是训练集中的字段名
• –response_colum summary 这是训练集中的字段名

!CUDA_VISIBLE_DEVICES=1 torchrun --standalone --nnodes=1 --nproc-per-node=1 main.py \
    --do_train \
    --train_file /data/train_data/formatted_train_data.json \
    --preprocessing_num_workers 10 \
    --prompt_column content \
    --response_column summary \
    --overwrite_cache \
    --model_name_or_path /models/chatglm2-6b \
    --output_dir output/adgen-chatglm2-6b-pt-128-2e-2 \
    --overwrite_output_dir \
    --max_source_length 64 \
    --max_target_length 128 \
    --per_device_train_batch_size 1 \
    --per_device_eval_batch_size 1 \
    --gradient_accumulation_steps 16 \
    --predict_with_generate \
    --max_steps 1000 \
    --logging_steps 500 \
    --save_steps 100 \
    --learning_rate 2e-2 \
    --pre_seq_len 128\
    --quantization_bit 4

Using the `WANDB_DISABLED` environment variable is deprecated and will be removed in v5. Use the --report_to flag to control the integrations used for logging result (for instance --report_to none).
08/02/2024 05:23:17 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1distributed training: True, 16-bits training: False
08/02/2024 05:23:17 - INFO - __main__ - Training/evaluation parameters Seq2SeqTrainingArguments(
_n_gpu=1,
adafactor=False,
adam_beta1=0.9,
adam_beta2=0.999,
adam_epsilon=1e-08,
auto_find_batch_size=False,
bf16=False,
bf16_full_eval=False,
data_seed=None,
dataloader_drop_last=False,
dataloader_num_workers=0,
dataloader_pin_memory=True,
ddp_backend=None,
ddp_bucket_cap_mb=None,
ddp_find_unused_parameters=None,
ddp_timeout=1800,
debug=[],
deepspeed=None,
disable_tqdm=False,
do_eval=False,
do_predict=False,
do_train=True,
eval_accumulation_steps=None,
eval_delay=0,
eval_steps=None,
evaluation_strategy=no,
fp16=False,
fp16_backend=auto,
fp16_full_eval=False,
fp16_opt_level=O1,
fsdp=[],
fsdp_config={'fsdp_min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False},
fsdp_min_num_params=0,
fsdp_transformer_layer_cls_to_wrap=None,
full_determinism=False,
generation_config=None,
generation_max_length=None,
generation_num_beams=None,
gradient_accumulation_steps=16,
gradient_checkpointing=False,
greater_is_better=None,
group_by_length=False,
half_precision_backend=auto,
hub_model_id=None,
hub_private_repo=False,
hub_strategy=every_save,
hub_token=<HUB_TOKEN>,
ignore_data_skip=False,
include_inputs_for_metrics=False,
jit_mode_eval=False,
label_names=None,
label_smoothing_factor=0.0,
learning_rate=0.02,
length_column_name=length,
load_best_model_at_end=False,
local_rank=0,
log_level=passive,
log_level_replica=warning,
log_on_each_node=True,
logging_dir=output/adgen-chatglm2-6b-pt-128-2e-2/runs/Aug02_05-23-17_gpuserver01,
logging_first_step=False,
logging_nan_inf_filter=True,
logging_steps=500,
logging_strategy=steps,
lr_scheduler_type=linear,
max_grad_norm=1.0,
max_steps=1000,
metric_for_best_model=None,
mp_parameters=,
no_cuda=False,
num_train_epochs=3.0,
optim=adamw_hf,
optim_args=None,
output_dir=output/adgen-chatglm2-6b-pt-128-2e-2,
overwrite_output_dir=True,
past_index=-1,
per_device_eval_batch_size=1,
per_device_train_batch_size=1,
predict_with_generate=True,
prediction_loss_only=False,
push_to_hub=False,
push_to_hub_model_id=None,
push_to_hub_organization=None,
push_to_hub_token=<PUSH_TO_HUB_TOKEN>,
ray_scope=last,
remove_unused_columns=True,
report_to=['tensorboard'],
resume_from_checkpoint=None,
run_name=output/adgen-chatglm2-6b-pt-128-2e-2,
save_on_each_node=False,
save_safetensors=False,
save_steps=100,
save_strategy=steps,
save_total_limit=None,
seed=42,
sharded_ddp=[],
skip_memory_metrics=True,
sortish_sampler=False,
tf32=None,
torch_compile=False,
torch_compile_backend=None,
torch_compile_mode=None,
torchdynamo=None,
tpu_metrics_debug=False,
tpu_num_cores=None,
use_ipex=False,
use_legacy_prediction_loop=False,
use_mps_device=False,
warmup_ratio=0.0,
warmup_steps=0,
weight_decay=0.0,
xpu_backend=None,
)

num_proc must be <= 2. Reducing num_proc to 2 for dataset of size 2.
08/02/2024 05:25:17 - WARNING - datasets.arrow_dataset - num_proc must be <= 2. Reducing num_proc to 2 for dataset of size 2.
Running tokenizer on train dataset (num_proc=2): 100%|█| 2/2 [00:00<00:00,  4.74
input_ids [64790, 64792, 790, 30951, 517, 30910, 30939, 30996, 13, 13, 54761, 31211, 55073, 54570, 39099, 31636, 32581, 33328, 30946, 46163, 32581, 30932, 55176, 54759, 32581, 30932, 51102, 32581, 1528, 34346, 54999, 54573, 54542, 33847, 54530, 54574, 54652, 34204, 31642, 34213, 31123, 33969, 42524, 31123, 31838, 31876, 32884, 31155, 13, 13, 55437, 31211, 30910, 46163, 32581, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
inputs [Round 1]

问：请对句子进行评价分类(正向评价,负向评价,中性评价): 一百多和三十的也看不出什么区别，包装精美，质量应该不错。

答： 正向评价
label_ids [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 30910, 46163, 32581, 2, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100]
labels 正向评价
[INFO|trainer.py:577] 2024-08-02 05:25:19,287 >> max_steps is given, it will override any value given in num_train_epochs
/data/miniconda3/lib/python3.10/site-packages/transformers/optimization.py:411: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning
  warnings.warn(
[INFO|trainer.py:1786] 2024-08-02 05:25:19,912 >> ***** Running training *****
[INFO|trainer.py:1787] 2024-08-02 05:25:19,912 >>   Num examples = 2
[INFO|trainer.py:1788] 2024-08-02 05:25:19,912 >>   Num Epochs = 1,000
[INFO|trainer.py:1789] 2024-08-02 05:25:19,912 >>   Instantaneous batch size per device = 1
[INFO|trainer.py:1790] 2024-08-02 05:25:19,912 >>   Total train batch size (w. parallel, distributed & accumulation) = 16
[INFO|trainer.py:1791] 2024-08-02 05:25:19,912 >>   Gradient Accumulation steps = 16
[INFO|trainer.py:1792] 2024-08-02 05:25:19,912 >>   Total optimization steps = 1,000
[INFO|trainer.py:1793] 2024-08-02 05:25:19,913 >>   Number of trainable parameters = 1,835,008
  0%|                                                  | 0/1000 [00:00<?, ?it/s]08/02/2024 05:25:20 - WARNING - transformers_modules.chatglm2-6b.modeling_chatglm - `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...
/data/miniconda3/lib/python3.10/site-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.
  warnings.warn(
 10%|████                                    | 100/1000 [00:37<05:33,  2.70it/s]Saving PrefixEncoder
{'loss': 0.0001, 'learning_rate': 0.0, 'epoch': 1000.0}                         
100%|███████████████████████████████████████| 1000/1000 [06:01<00:00,  2.82it/s]Saving PrefixEncoder
[INFO|configuration_utils.py:458] 2024-08-02 05:31:21,906 >> Configuration saved in output/adgen-chatglm2-6b-pt-128-2e-2/checkpoint-1000/config.json
[INFO|configuration_utils.py:364] 2024-08-02 05:31:21,907 >> Configuration saved in output/adgen-chatglm2-6b-pt-128-2e-2/checkpoint-1000/generation_config.json
[INFO|modeling_utils.py:1853] 2024-08-02 05:31:21,915 >> Model weights saved in output/adgen-chatglm2-6b-pt-128-2e-2/checkpoint-1000/pytorch_model.bin
[INFO|tokenization_utils_base.py:2194] 2024-08-02 05:31:21,916 >> tokenizer config file saved in output/adgen-chatglm2-6b-pt-128-2e-2/checkpoint-1000/tokenizer_config.json
[INFO|tokenization_utils_base.py:2201] 2024-08-02 05:31:21,916 >> Special tokens file saved in output/adgen-chatglm2-6b-pt-128-2e-2/checkpoint-1000/special_tokens_map.json
[INFO|trainer.py:2053] 2024-08-02 05:31:21,955 >> 

Training completed. Do not forget to share your model on huggingface.co/models =)


{'train_runtime': 362.0417, 'train_samples_per_second': 44.194, 'train_steps_per_second': 2.762, 'train_loss': 0.015981700539588927, 'epoch': 1000.0}
100%|███████████████████████████████████████| 1000/1000 [06:02<00:00,  2.76it/s]
***** train metrics *****
  epoch                    =     1000.0
  train_loss               =      0.016
  train_runtime            = 0:06:02.04
  train_samples            =          2
  train_samples_per_second =     44.194
  train_steps_per_second   =      2.762

模型推理

from transformers import AutoConfig, AutoModel, AutoTokenizer
import torch

# 载入Tokenizer
tokenizer = AutoTokenizer.from_pretrained("/models/chatglm2-6b", trust_remote_code=True)

config = AutoConfig.from_pretrained("/models/chatglm2-6b", trust_remote_code=True, pre_seq_len=128)
model = AutoModel.from_pretrained("/models/chatglm2-6b", config=config, trust_remote_code=True).half().cuda()

Loading checkpoint shards:   0%|          | 0/7 [00:00<?, ?it/s]


Some weights of ChatGLMForConditionalGeneration were not initialized from the model checkpoint at /models/chatglm2-6b and are newly initialized: ['transformer.prefix_encoder.embedding.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

训练的checkpoint，应该是 output_dir 路径下的 最后一个checkpoin-？？？ 路径

prefix_state_dict = torch.load(os.path.join("output/adgen-chatglm2-6b-pt-128-2e-2/checkpoint-1000", "pytorch_model.bin"))
new_prefix_state_dict = {}
for k, v in prefix_state_dict.items():
    if k.startswith("transformer.prefix_encoder."):
        new_prefix_state_dict[k[len("transformer.prefix_encoder."):]] = v
model.transformer.prefix_encoder.load_state_dict(new_prefix_state_dict)

<All keys matched successfully>

model = model.eval()

response, history = model.chat(tokenizer, "你好", history=[])

print(response)

你好，有什麼我可以幫忙的嗎？

import json

# 读取JSON文件
with open('/data/train_data/测试集示例.json', 'r', encoding='utf-8') as f:
    data = json.load(f)

# 遍历每个条目，调用LLM模型会话，更新summary
for item in data:
    content = instruction + item['content']
    response, history = model.chat(tokenizer, content, history=[])
    item['summary'] = response

# 将更新后的数据写回JSON文件
with open('/data/train_data/output.json', 'w', encoding='utf-8') as f:
    json.dump(data, f, ensure_ascii=False, indent=4)