open_clip.create_model_and_transforms 加载本地模型

最新推荐文章于 2024-03-27 20:19:52 发布

szZack

最新推荐文章于 2024-03-27 20:19:52 发布

阅读量2.4k

点赞数 12

分类专栏：多模态 open_clip 文章标签： open_clip 多模态

本文链接：https://blog.csdn.net/zengnlp/article/details/135644453

版权

多模态同时被 2 个专栏收录

2 篇文章 0 订阅

订阅专栏

open_clip

1 篇文章 0 订阅

订阅专栏

open_clip.create_model_and_transforms 加载本地模型

open_clip.create_model_and_transforms 加载本地模型
参考

open_clip.create_model_and_transforms 加载本地模型

代码

model, _, preprocess = open_clip.create_model_and_transforms(
        'ViT-L-14', 
        pretrained='laion/CLIP-ViT-L-14-laion2B-s32B-b82K/pytorch_model.bin'
    )

pretrained 指定路径 laion/CLIP-ViT-L-14-laion2B-s32B-b82K/pytorch_model.bin

报错

    model, _, preprocess = open_clip.create_model_and_transforms(
  File "/data/.local/lib/python3.10/site-packages/open_clip/factory.py", line 382, in create_model_and_transforms
    model = create_model(
  File "/data/.local/lib/python3.10/site-packages/open_clip/factory.py", line 288, in create_model
    load_checkpoint(model, checkpoint_path)
  File "/data/.local/lib/python3.10/site-packages/open_clip/factory.py", line 159, in load_checkpoint
    incompatible_keys = model.load_state_dict(state_dict, strict=strict)
  File "/data/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2152, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for CLIP:
        Missing key(s) in state_dict: "positional_embedding", "text_projection", "visual.class_embedding", "visual.positional_embedding", "visual.proj"

解决方法

到 https://huggingface.co/laion/CLIP-ViT-L-14-laion2B-s32B-b82K/tree/main 下载 open_clip_pytorch_model.bin
修改代码

model, _, preprocess = open_clip.create_model_and_transforms(
        'ViT-L-14', 
        pretrained='laion/CLIP-ViT-L-14-laion2B-s32B-b82K/open_clip_pytorch_model.bin'
    )

再次执行，正常运行！！