【大模型】运行Qwen1.5大模型报错AttributeError: property ‘seen_tokens‘ of ‘DynamicCache‘ object has no setter

环境

• Ubuntu22.04
• python3.11

运行模型

跑 Qwen1.5-7B-Chat-Int4 模型：

python Qwen1.5-generate.py --repo-id-or-model-path Qwen1.5-7B-Chat-Int4-ipex-llm --n-predict 128

错误信息

# python Qwen1.5-generate.py --repo-id-or-model-path Qwen1.5-7B-Chat-Int4-ipex-llm --n-predict 128
2024-04-29 12:52:41,815 - INFO - intel_extension_for_pytorch auto imported
2024-04-29 12:52:42,143 - INFO - Converting the current model to sym_int4 format......
Loading checkpoint shards:   0%|                                                                                                                              | 0/2 [00:00<?, ?it/s]/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
  return self.fget.__get__(instance, owner)()
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  8.98it/s]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead.
Traceback (most recent call last):
  File "/home/abc/llm/ipex/Qwen1.5-generate.py", line 69, in <module>
    # warmup

  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/transformers/generation/utils.py", line 1622, in generate
    result = self._sample(
             ^^^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/transformers/generation/utils.py", line 2791, in _sample
    outputs = self(
              ^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/transformers/models/qwen2/modeling_qwen2.py", line 1169, in forward
    outputs = self.model(
              ^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/ipex_llm/transformers/models/qwen2.py", line 100, in qwen2_model_forward
    return qwen2_model_forward_internal(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/ipex_llm/transformers/models/qwen2.py", line 243, in qwen2_model_forward_internal
    layer_outputs = decoder_layer(
                    ^^^^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/transformers/models/qwen2/modeling_qwen2.py", line 768, in forward
    hidden_states, self_attn_weights, present_key_value = self.self_attn(
                                                          ^^^^^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/ipex_llm/transformers/models/qwen2.py", line 298, in qwen2_attention_forward
    return forward_function(
           ^^^^^^^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/ipex_llm/transformers/models/qwen2.py", line 494, in qwen2_attention_forward_origin
    past_key_value.seen_tokens += key_states.shape[-2]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: property 'seen_tokens' of 'DynamicCache' object has no setter

重点错误

  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/ipex_llm/transformers/models/qwen2.py", line 298, in qwen2_attention_forward
    return forward_function(
           ^^^^^^^^^^^^^^^^^
  File "/home/abc/miniconda3/envs/ipex-llm/lib/python3.11/site-packages/ipex_llm/transformers/models/qwen2.py", line 494, in qwen2_attention_forward_origin
    past_key_value.seen_tokens += key_states.shape[-2]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: property 'seen_tokens' of 'DynamicCache' object has no setter

分析错误

transformers 中的 qwen2.py 的异常，猜测原因是 transformers 的版本导致的

解决方案

查看 transformers 的版本

pip list|grep transformers

输出：

transformers                4.40.0

修改 transformers 的版本

pip install transformers==4.37  -i https://mirrors.aliyun.com/pypi/simple/

估计是 4.40.0 的版本太新了，更新到 4.37 之后跑模型，就正常了。

测试

再次运行模型正常了