第6节OpenCompass 作业

基础作业：

1.环境安装......

2.数据准备

3 执行run.py 测试internlm2-chat-7b

python run.py --datasets ceval_gen --hf-path /root/share/model_repos/internlm2-chat-7b --tokenizer-path /root/share/model_repos/internlm2-chat-7b --tokenizer-kwargs padding_side='left' truncation='left' trust_remote_code=True --model-kwargs trust_remote_code=True device_map='auto' --max-seq-len 2048 --max-out-len 16 --batch-size 4 --num-gpus 1 

4.评测结果

100%|███████████████████████████████████████████████████████████████████████████| 52/52 [04:00<00:00,  4.63s/it]
dataset                                         version    metric    mode    opencompass.models.huggingface.HuggingFace_model_repos_internlm2-chat-7b
----------------------------------------------  ---------  --------  ------  --------------------------------------------------------------------------
ceval-computer_network                          -          -         -       -
ceval-operating_system                          -          -         -       -
ceval-computer_architecture                     -          -         -       -
ceval-college_programming                       4ca32a     accuracy  gen     18.92
ceval-college_physics                           -          -         -       -
ceval-college_chemistry                         e78857     accuracy  gen     0.00
ceval-advanced_mathematics                      -          -         -       -
ceval-probability_and_statistics                -          -         -       -
ceval-discrete_mathematics                      -          -         -       -
ceval-electrical_engineer                       ae42b9     accuracy  gen     18.92
ceval-metrology_engineer                        ee34ea     accuracy  gen     50.00
ceval-high_school_mathematics                   -          -         -       -
ceval-high_school_physics                       -          -         -       -
ceval-high_school_chemistry                     -          -         -       -
ceval-high_school_biology                       -          -         -       -
ceval-middle_school_mathematics                 -          -         -       -
ceval-middle_school_biology                     -          -         -       -
ceval-middle_school_physics                     -          -         -       -
ceval-middle_school_chemistry                   -          -         -       -
ceval-veterinary_medicine                       b4e08d     accuracy  gen     39.13
ceval-college_economics                         f3f4e6     accuracy  gen     29.09
ceval-business_administration                   c1614e     accuracy  gen     30.30
ceval-marxism                                   -          -         -       -
ceval-mao_zedong_thought                        51c7a4     accuracy  gen     70.83
ceval-education_science                         591fee     accuracy  gen     62.07
ceval-teacher_qualification                     4e4ced     accuracy  gen     77.27
ceval-high_school_politics                      -          -         -       -
ceval-high_school_geography                     -          -         -       -
ceval-middle_school_politics                    -          -         -       -
ceval-middle_school_geography                   -          -         -       -
ceval-modern_chinese_history                    fc01af     accuracy  gen     65.22
ceval-ideological_and_moral_cultivation         -          -         -       -
ceval-logic                                     -          -         -       -
ceval-law                                       a110a1     accuracy  gen     37.50
ceval-chinese_language_and_literature           0f8b68     accuracy  gen     47.83
ceval-art_studies                               2a1300     accuracy  gen     66.67
ceval-professional_tour_guide                   4e673e     accuracy  gen     82.76
ceval-legal_professional                        ce8787     accuracy  gen     30.43
ceval-high_school_chinese                       -          -         -       -
ceval-high_school_history                       -          -         -       -
ceval-middle_school_history                     -          -         -       -
ceval-civil_servant                             87d061     accuracy  gen     38.30
ceval-sports_science                            -          -         -       -
ceval-plant_protection                          -          -         -       -
ceval-basic_medicine                            -          -         -       -
ceval-clinical_medicine                         -          -         -       -
ceval-urban_and_rural_planner                   95b885     accuracy  gen     58.70
ceval-accountant                                002837     accuracy  gen     34.69
ceval-fire_engineer                             bc23f5     accuracy  gen     12.90
ceval-environmental_impact_assessment_engineer  c64e2d     accuracy  gen     38.71
ceval-tax_accountant                            3a5e3c     accuracy  gen     42.86
ceval-physician                                 6e277d     accuracy  gen     51.02
01/23 11:09:52 - OpenC

进阶作业：

尝试中......

修改配置文件/root/opencompass/configs/eval_internlm_chat_turbomind.py如下字段

执行：

python run.py configs/eval_internlm_chat_turbomind.py

评测结果：