1. 模型权重转换报错 hf2mcore_1.5_v2.py
报错为:
/mnt/cpfs/kexin/dlc_code/qwen1.5/PAI-Megatron-Patch/toolkits/model_checkpoints_convertor/qwen/hf2mcore_1.5_v2.py
正确文件替换如下,更改了477行,删除了 args.hidden_size 这个维度,在tp>1时也支持转换:
elif 'linear_qkv.bias' in k and 'norm' not in k:
# raw
viewed = v.view(args.num_query_groups, -1, head_dim, args.hidden_size)
# changed
viewed = v.view(args.num_query_groups, -1, head_dim)
替换为:
import os
import re
import json
import torch
import transformers
import