第二期书生浦语大模型实战营第七次课程作业

haidizym

已于 2024-04-24 12:14:46 修改

阅读量594

点赞数 6

分类专栏：大模型文章标签：笔记学习自然语言处理人工智能

于 2024-04-23 18:46:30 首次发布

本文链接：https://blog.csdn.net/haidizym/article/details/138132044

版权

大模型专栏收录该内容

15 篇文章 0 订阅

订阅专栏

【OpenCompass 大模型评测实战（一期）】：https://www.bilibili.com/video/BV1Gg4y1U7uc/
【视频地址】：https://www.bilibili.com/video/BV1Pm41127jU/
【课程文档】：https://github.com/InternLM/Tutorial/blob/camp2/opencompass/readme.md
【课程作业】：https://github.com/InternLM/Tutorial/blob/camp2/opencompass/homework.md
【opencompass】：https://github.com/open-compass/opencompass
【操作平台】：https://studio.intern-ai.org.cn/console/instance/
【自定义客观题数据集】：https://opencompass.readthedocs.io/zh-cn/latest/advanced_guides/new_dataset.html
【自定义客观题数据集】：https://opencompass.readthedocs.io/zh-cn/latest/advanced_guides/subjective_evaluation.html

基础作业

使用 OpenCompass 评测 internlm2-chat-1_8b 模型在 C-Eval 数据集上的性能
在这里插入图片描述
备注：如下两幅截图显示报错信息，安装了两个库，pip install tabulate，pip install mmengine

终于出来了–列出所有跟 InternLM 及 C-Eval 相关的配置

开始启动评测

又报错了，还是缺少库， pip install tqdm,pip install prettytable
在这里插入图片描述
截图说明最后的选择是正确的，pip install -r requirements.txt，否则这么多库啊，而且充分说明前面安装环境和资源没报错不可信，最好之前就加这句，

报错，运行出空结果，

在这里插入图片描述

备注：7b的模型可以评测，仔细研究了一下代码，发现官方教程https://github.com/InternLM/Tutorial/blob/camp2/opencompass/readme.md文档中internlm2-chat-1_8b后少了/，应该是

python run.py --datasets ceval_gen --hf-path /share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b/ --tokenizer-path /share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b/  --tokenizer-kwargs padding_side='left' truncation='left' trust_remote_code=True --model-kwargs trust_remote_code=True device_map='auto' --max-seq-len 1024 --max-out-len 16 --batch-size 2 --num-gpus 1 --debug

7b可以正常评测
在这里插入图片描述
加了/后，1.8b模型还是运行出空结果

traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
Traceback (most recent call last):
  File "/root/opencompass/opencompass/tasks/openicl_infer.py", line 156, in <module>
    inferencer.run()
  File "/root/opencompass/opencompass/tasks/openicl_infer.py", line 68, in run
    self.model = build_model_from_cfg(model_cfg)
  File "/root/opencompass/opencompass/utils/build.py", line 25, in build_model_from_cfg
    return MODELS.build(model_cfg)
  File "/root/.conda/envs/opencompass/lib/python3.10/site-packages/mmengine/registry/registry.py", line 570, in build
    return self.build_func(cfg, *args, **kwargs, registry=self)
  File "/root/.conda/envs/opencompass/lib/python3.10/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
    obj = obj_cls(**args)  # type: ignore
  File "/root/opencompass/opencompass/models/huggingface.py", line 118, in __init__
    self._load_tokenizer(path=path,
  File "/root/opencompass/opencompass/models/huggingface.py", line 134, in _load_tokenizer
    self.tokenizer = AutoTokenizer.from_pretrained(
  File "/root/.conda/envs/opencompass/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 847, in from_pretrained
    return tokenizer_class.from_pretrained(
  File "/root/.conda/envs/opencompass/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2089, in from_pretrained
    return cls._from_pretrained(
  File "/root/.conda/envs/opencompass/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2311, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/root/.cache/huggingface/modules/transformers_modules/internlm2-chat-1_8b/tokenization_internlm2_fast.py", line 131, in __init__
    super().__init__(
  File "/root/.conda/envs/opencompass/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 114, in __init__
    fast_tokenizer = convert_slow_tokenizer(slow_tokenizer)
  File "/root/.conda/envs/opencompass/lib/python3.10/site-packages/transformers/convert_slow_tokenizer.py", line 1534, in convert_slow_tokenizer
    return converter_class(transformer_tokenizer).converted()
  File "/root/.conda/envs/opencompass/lib/python3.10/site-packages/transformers/convert_slow_tokenizer.py", line 546, in __init__
    requires_backends(self, "protobuf")
  File "/root/.conda/envs/opencompass/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1438, in requires_backends
    raise ImportError("".join(failed))
ImportError: 
InternLM2Converter requires the protobuf library but it was not found in your environment. Checkout the instructions on the
installation page of its repo: https://github.com/protocolbuffers/protobuf/tree/master/python#installation and follow the ones
that match your environment. Please note that you may need to restart your runtime after installation.

ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 99049) of binary: /root/.conda/envs/opencompass/bin/python
Traceback (most recent call last):
  File "/root/.conda/envs/opencompass/bin/torchrun", line 33, in <module>
    sys.exit(load_entry_point('torch==2.0.1', 'console_scripts', 'torchrun')())
  File "/root/.conda/envs/opencompass/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/root/.conda/envs/opencompass/lib/python3.10/site-packages/torch/distributed/run.py", line 794, in main
    run(args)
  File "/root/.conda/envs/opencompass/lib/python3.10/site-packages/torch/distributed/run.py", line 785, in run
    elastic_launch(
  File "/root/.conda/envs/opencompass/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/root/.conda/envs/opencompass/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 250, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
/root/opencompass/opencompass/tasks/openicl_infer.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2024-04-23_18:23:19
  host      : intern-studio-045290
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 99049)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================
04/23 18:23:20 - OpenCompass - INFO - Partitioned into 52 tasks.
04/23 18:23:22 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-computer_network]: No predictions found.
04/23 18:23:24 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-operating_system]: No predictions found.
04/23 18:23:26 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-computer_architecture]: No predictions found.
04/23 18:23:28 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-college_programming]: No predictions found.
04/23 18:23:30 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-college_physics]: No predictions found.
04/23 18:23:32 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-college_chemistry]: No predictions found.
04/23 18:23:34 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-advanced_mathematics]: No predictions found.
04/23 18:23:36 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-probability_and_statistics]: No predictions found.
04/23 18:23:38 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-discrete_mathematics]: No predictions found.
04/23 18:23:40 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-electrical_engineer]: No predictions found.
04/23 18:23:43 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-metrology_engineer]: No predictions found.
04/23 18:23:45 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-high_school_mathematics]: No predictions found.
04/23 18:23:47 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-high_school_physics]: No predictions found.
04/23 18:23:49 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-high_school_chemistry]: No predictions found.
04/23 18:23:51 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-high_school_biology]: No predictions found.
04/23 18:23:53 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-middle_school_mathematics]: No predictions found.
04/23 18:23:55 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-middle_school_biology]: No predictions found.
04/23 18:23:57 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-middle_school_physics]: No predictions found.
04/23 18:23:59 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-middle_school_chemistry]: No predictions found.
04/23 18:24:01 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-veterinary_medicine]: No predictions found.
04/23 18:24:03 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-college_economics]: No predictions found.
04/23 18:24:06 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-business_administration]: No predictions found.
04/23 18:24:09 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-marxism]: No predictions found.
04/23 18:24:12 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-mao_zedong_thought]: No predictions found.
04/23 18:24:14 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-education_science]: No predictions found.
04/23 18:24:16 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-teacher_qualification]: No predictions found.
04/23 18:24:18 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-high_school_politics]: No predictions found.
04/23 18:24:20 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-high_school_geography]: No predictions found.
04/23 18:24:22 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-middle_school_politics]: No predictions found.
04/23 18:24:24 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-middle_school_geography]: No predictions found.
04/23 18:24:26 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-modern_chinese_history]: No predictions found.
04/23 18:24:28 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-ideological_and_moral_cultivation]: No predictions found.
04/23 18:24:30 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-logic]: No predictions found.
04/23 18:24:32 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-law]: No predictions found.
04/23 18:24:35 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-chinese_language_and_literature]: No predictions found.
04/23 18:24:37 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-art_studies]: No predictions found.
04/23 18:24:38 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-professional_tour_guide]: No predictions found.
04/23 18:24:41 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-legal_professional]: No predictions found.
04/23 18:24:43 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-high_school_chinese]: No predictions found.
04/23 18:24:45 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-high_school_history]: No predictions found.
04/23 18:24:47 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-middle_school_history]: No predictions found.
04/23 18:24:49 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-civil_servant]: No predictions found.
04/23 18:24:51 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-sports_science]: No predictions found.
04/23 18:24:53 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-plant_protection]: No predictions found.
04/23 18:24:55 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-basic_medicine]: No predictions found.
04/23 18:24:57 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-clinical_medicine]: No predictions found.
04/23 18:24:59 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-urban_and_rural_planner]: No predictions found.
04/23 18:25:01 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-accountant]: No predictions found.
04/23 18:25:03 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-fire_engineer]: No predictions found.
04/23 18:25:05 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-environmental_impact_assessment_engineer]: No predictions found.
04/23 18:25:07 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-tax_accountant]: No predictions found.
04/23 18:25:10 - OpenCompass - ERROR - /root/opencompass/opencompass/tasks/openicl_eval.py - _score - 238 - Task [opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b/ceval-physician]: No predictions found.
dataset                                         version    metric    mode    opencompass.models.huggingface.HuggingFace_Shanghai_AI_Laboratory_internlm2-chat-1_8b
----------------------------------------------  ---------  --------  ------  ---------------------------------------------------------------------------------------
ceval-computer_network                          -          -         -       -
ceval-operating_system                          -          -         -       -
ceval-computer_architecture                     -          -         -       -
ceval-college_programming                       -          -         -       -
ceval-college_physics                           -          -         -       -
ceval-college_chemistry                         -          -         -       -
ceval-advanced_mathematics                      -          -         -       -
ceval-probability_and_statistics                -          -         -       -
ceval-discrete_mathematics                      -          -         -       -
ceval-electrical_engineer                       -          -         -       -
ceval-metrology_engineer                        -          -         -       -
ceval-high_school_mathematics                   -          -         -       -
ceval-high_school_physics                       -          -         -       -
ceval-high_school_chemistry                     -          -         -       -
ceval-high_school_biology                       -          -         -       -
ceval-middle_school_mathematics                 -          -         -       -
ceval-middle_school_biology                     -          -         -       -
ceval-middle_school_physics                     -          -         -       -
ceval-middle_school_chemistry                   -          -         -       -
ceval-veterinary_medicine                       -          -         -       -
ceval-college_economics                         -          -         -       -
ceval-business_administration                   -          -         -       -
ceval-marxism                                   -          -         -       -
ceval-mao_zedong_thought                        -          -         -       -
ceval-education_science                         -          -         -       -
ceval-teacher_qualification                     -          -         -       -
ceval-high_school_politics                      -          -         -       -
ceval-high_school_geography                     -          -         -       -
ceval-middle_school_politics                    -          -         -       -
ceval-middle_school_geography                   -          -         -       -
ceval-modern_chinese_history                    -          -         -       -
ceval-ideological_and_moral_cultivation         -          -         -       -
ceval-logic                                     -          -         -       -
ceval-law                                       -          -         -       -
ceval-chinese_language_and_literature           -          -         -       -
ceval-art_studies                               -          -         -       -
ceval-professional_tour_guide                   -          -         -       -
ceval-legal_professional                        -          -         -       -
ceval-high_school_chinese                       -          -         -       -
ceval-high_school_history                       -          -         -       -
ceval-middle_school_history                     -          -         -       -
ceval-civil_servant                             -          -         -       -
ceval-sports_science                            -          -         -       -
ceval-plant_protection                          -          -         -       -
ceval-basic_medicine                            -          -         -       -
ceval-clinical_medicine                         -          -         -       -
ceval-urban_and_rural_planner                   -          -         -       -
ceval-accountant                                -          -         -       -
ceval-fire_engineer                             -          -         -       -
ceval-environmental_impact_assessment_engineer  -          -         -       -
ceval-tax_accountant                            -          -         -       -
ceval-physician                                 -          -         -       -
04/23 18:25:10 - OpenCompass - INFO - write summary to /root/opencompass/outputs/default/20240423_182100/summary/summary_20240423_182100.txt
04/23 18:25:10 - OpenCompass - INFO - write csv to /root/opencompass/outputs/default/20240423_182100/summary/summary_20240423_182100.csv

正确的答案是相似的，错误的答案各有各的错误，重新做最好
后来找到原因了，正确顺序是pip install -r requirements.txt， pip install -e .

#云盘删除opencompass文件夹，conda删除环境
bash
conda remove --name opencompass --all

cd ~
studio-conda -o internlm-base -t opencompass
source activate opencompass
git clone -b 0.2.4 https://github.com/open-compass/opencompass
cd opencompass
pip install -r requirements.txt
pip install -e .
pip install protobuf
export MKL_SERVICE_FORCE_INTEL=1
cp /share/temp/datasets/OpenCompassData-core-20231110.zip /root/opencompass/
unzip OpenCompassData-core-20231110.zip
python tools/list_configs.py internlm ceval
python run.py --datasets ceval_gen --hf-path /share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b --tokenizer-path /share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b --tokenizer-kwargs padding_side='left' truncation='left' trust_remote_code=True --model-kwargs trust_remote_code=True device_map='auto' --max-seq-len 2048 --max-out-len 16 --batch-size 4 --num-gpus 1 --debug

在这里插入图片描述

小结

踩了好几个坑，
如下是正确顺序

cd ~
studio-conda -o internlm-base -t opencompass
source activate opencompass
git clone -b 0.2.4 https://github.com/open-compass/opencompass
cd opencompass
pip install -r requirements.txt #一定要先安装所有依赖库，再安装e.
pip install -e .
pip install protobuf
export MKL_SERVICE_FORCE_INTEL=1
cp /share/temp/datasets/OpenCompassData-core-20231110.zip /root/opencompass/
unzip OpenCompassData-core-20231110.zip
python run.py --datasets ceval_gen --hf-path /share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b --tokenizer-path /share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b --tokenizer-kwargs padding_side='left' truncation='left' trust_remote_code=True --model-kwargs trust_remote_code=True device_map='auto' --max-seq-len 2048 --max-out-len 16 --batch-size 4 --num-gpus 1 --debug

进阶作业

将自定义数据集提交至OpenCompass官网
提交地址：https://hub.opencompass.org.cn/dataset-submit?lang=[object%20Object]
提交指南：https://mp.weixin.qq.com/s/_s0a9nYRye0bmqVdwXRVCg
Tips：不强制要求配置数据集对应榜单（ leaderboard.xlsx ），可仅上传 EADME_OPENCOMPASS.md 文档