一、数据准备
解压后在opencompass文件夹下面看到数据集
可以查看到所有跟 InternLM 及 C-Eval 相关的配置
二、启动评测
找到hf_internlm2_chat_1_8b.py
from opencompass.models import HuggingFaceCausalLM
models = [
dict(
type=HuggingFaceCausalLM,
abbr='internlm2-1.8b-hf',
path="/share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b",
tokenizer_path='/share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b',
model_kwargs=dict(
trust_remote_code=True,
device_map='auto',
),
tokenizer_kwargs=dict(
padding_side='left',
truncation_side='left',
use_fast=False,
trust_remote_code=True,
),
max_out_len=100,
min_out_len=1,
max_seq_len=2048,
batch_size=8,
run_cfg=dict(num_gpus=1, num_procs=1),
)
]
配置环境变量
export MKL_THREADING_LAYER=GNU
运行程序
python run.py --datasets ceval_gen --models hf_internlm2_chat_1_8b --debug
得到如图所示的评测结果,用时约3h