HuggingFace 错误 ValueError: You are trying to save a non contiguous tensor: `visual_cnn.pre.0.xx_chan

问题描述

ValueError: You are trying to save a non contiguous tensor: `visual_cnn.pre.0.xx_channel` which is not allowed. It either means you are trying to save tensors which are reference of each other in which case it's recommended to save only the full tensors, and reslice at load time, or simply call `.contiguous()` on your tensor to pack it before saving.

问题解决

自定义模型中重写以下两个方法：

class MidmPreTrainedModel(PreTrainedModel):
    # ... [other methods and properties of the class]

    def make_tensors_contiguous(self):
        for name, param in self.named_parameters():
            if not param.is_contiguous():
                param.data = param.data.contiguous()

    def save_pretrained(self, save_directory, **kwargs):
        # Make tensors contiguous
        self.make_tensors_contiguous()

        # Call the original save_pretrained method
        super().save_pretrained(save_directory, **kwargs)

参考链接：https://huggingface.co/KT-AI/midm-bitext-S-7B-inst-v1/discussions/9