[InternLM 大模型开源社区]大模型实战营第三期@202408-基础岛-第6关-OpenCompass 评测 InternLM-1.8B 实践

任务类型	任务内容	预计耗时	完成度
闯关任务	使用 OpenCompass 评测 internlm2-chat-1.8b 模型在 ceval 数据集上的性能，记录复现过程并截图	45mins	完成

• cd /root/code && vi compass_env.sh, 复制下面代码进去然后bash compass_env.sh

  conda create -n compass python=3.10 -y
  source activate compass
  [ ! -e /tmp/opencompass ] \
  && git clone https://github.com/open-compass/opencompass.git /tmp/opencompass
  cd /tmp/opencompass
  pip install -e . torch==2.0.1+cu117 torchvision==0.15.2+cu117 torchaudio==2.0.2 torchtext==0.15.2 -f https://mirror.sjtu.edu.cn/pytorch-wheels/cu117/?mirror_intel_list -f https://download.pytorch.org/whl/cu117/torch_stable.html
  
  

• unzip /share/temp/datasets/OpenCompassData-core-20231110.zip -d /tmp/opencompass

• conda activate compass && python /tmp/opencompass/tools/list_configs.py internlm ceval

  cd /tmp/opencompass \
  && python /tmp/opencompass/run.py \
      --datasets ceval_gen \
      --hf-path /root/share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b \
      --tokenizer-path /root/share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b \
      --tokenizer-kwargs padding_side='left' truncation='left' trust_remote_code=True \
      --model-kwargs device_map='auto' trust_remote_code=True \
      --max-seq-len 2048 \
      --max-out-len 16 \
      --batch-size 2 \
      --hf-num-gpus 1 \
      --debug
  

• ^ 训练开始!

• 可以从./tmp找到log