工作:
把shortgpt的llama代码改成了Qwen的剪枝。
具体方法:
用wikitext数据,计算每层的影响力分数即BI分数(1-层前后隐层状态余弦相似度),剪掉影响力低的P%的层数。
剪枝脚本:bash short_qwen_xin.sh
推理脚本:bash short_qwen_xin_test.sh
bash short_qwen_xin.sh
short_qwen_xin.sh
# 模型路径
model_names="/cpfs/074bqrkckm2dg5dq9nc/shared/AI-QIHUAN/OpenModels/Qwen2/Qwen2-0.5B-Instruct /cpfs/074bqrkckm2dg5dq9nc/shared/AI-QIHUAN/OpenModels/Qwen2/Qwen2-1.5B-Instruct"
# 遍历模型
# 遍历剪枝层数从1到20
for model_name in ${model_names};do
echo "[^.^] 模型名称为:${model_name}"
for prune_layers in {1..20};do
echo