在导入大模型时,有时会报错:'torch.dtype' object has no attribute 'element_size'。
可以修改报错文件 /miniconda3/lib/python3.8/site-packages/transformers/modeling_utils.py中的信息,只需注释5行代码,并新增9行代码即可顺利运行,修改如下:
for param in total_parameters:
if param.requires_grad or not only_trainable:
# For 4bit models, we need to multiply the number of parameters by 2 as half of the parameters are
# used for the 4bit quantization (uint8 tensors are stored)
if is_loaded_in_4bit and isinstance(param, bnb.nn.Params4bit):
# quant_storage = self.hf_quantizer.quantization_config.bnb_4bit_quant_storage
# # For compatibility with older PT version - see: https://github.com/huggingface/peft/pull/1635
# nb_params = (
# quant_storage.itemsize if hasattr(quant_storage, "itemsize") else quant_storage.element_size()
# )
# total_numel.append(param.numel() * 2 * nb_params)
if hasattr(param, "element_size"):
num_bytes = param.element_size()
elif not hasattr(param, "quant_storage"):
num_bytes = 1
else:
num_bytes = param.quant_storage.itemsize
total_numel.append(
param.numel() * 2 * num_bytes
)
else:
total_numel.append(param.numel())
参考资料: