书生大模型实战营-基础关卡-6-OpenCompass 评测 InternLM-1.8B 实践

列出所有跟 InternLM 及 C-Eval 相关的配置 

测试过程挺慢的，为了防止超时，测试结果直接导出到日志

08/14 20:33:06 - OpenCompass - INFO - Loading ceval_gen: configs/datasets/ceval/ceval_gen.py
08/14 20:33:07 - OpenCompass - INFO - Loading hf_internlm2_chat_1_8b: configs/models/hf_internlm/hf_internlm2_chat_1_8b.py
08/14 20:33:07 - OpenCompass - INFO - Loading example: configs/summarizers/example.py
08/14 20:33:07 - OpenCompass - WARNING - SlurmRunner is not used, so the partition argument is ignored.
08/14 20:33:07 - OpenCompass - INFO - Partitioned into 1 tasks.
08/14 20:33:57 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-college_economics,internlm2-1.8b-hf/ceval-accountant,internlm2-1.8b-hf/ceval-tax_accountant,internlm2-1.8b-hf/ceval-physician,internlm2-1.8b-hf/ceval-civil_servant,internlm2-1.8b-hf/ceval-urban_and_rural_planner,internlm2-1.8b-hf/ceval-teacher_qualification,internlm2-1.8b-hf/ceval-college_programming,internlm2-1.8b-hf/ceval-electrical_engineer,internlm2-1.8b-hf/ceval-business_administration,internlm2-1.8b-hf/ceval-art_studies,internlm2-1.8b-hf/ceval-fire_engineer,internlm2-1.8b-hf/ceval-environmental_impact_assessment_engineer,internlm2-1.8b-hf/ceval-education_science,internlm2-1.8b-hf/ceval-professional_tour_guide,internlm2-1.8b-hf/ceval-college_chemistry,internlm2-1.8b-hf/ceval-metrology_engineer,internlm2-1.8b-hf/ceval-mao_zedong_thought,internlm2-1.8b-hf/ceval-law,internlm2-1.8b-hf/ceval-veterinary_medicine,internlm2-1.8b-hf/ceval-modern_chinese_history,internlm2-1.8b-hf/ceval-chinese_language_and_literature,internlm2-1.8b-hf/ceval-legal_professional,internlm2-1.8b-hf/ceval-logic,internlm2-1.8b-hf/ceval-middle_school_history,internlm2-1.8b-hf/ceval-plant_protection,internlm2-1.8b-hf/ceval-clinical_medicine,internlm2-1.8b-hf/ceval-computer_architecture,internlm2-1.8b-hf/ceval-middle_school_biology,internlm2-1.8b-hf/ceval-middle_school_politics,internlm2-1.8b-hf/ceval-middle_school_chemistry,internlm2-1.8b-hf/ceval-high_school_history,internlm2-1.8b-hf/ceval-computer_network,internlm2-1.8b-hf/ceval-operating_system,internlm2-1.8b-hf/ceval-college_physics,internlm2-1.8b-hf/ceval-advanced_mathematics,internlm2-1.8b-hf/ceval-high_school_physics,internlm2-1.8b-hf/ceval-high_school_chemistry,internlm2-1.8b-hf/ceval-high_school_biology,internlm2-1.8b-hf/ceval-middle_school_mathematics,internlm2-1.8b-hf/ceval-middle_school_physics,internlm2-1.8b-hf/ceval-marxism,internlm2-1.8b-hf/ceval-high_school_politics,internlm2-1.8b-hf/ceval-high_school_geography,internlm2-1.8b-hf/ceval-ideological_and_moral_cultivation,internlm2-1.8b-hf/ceval-high_school_chinese,internlm2-1.8b-hf/ceval-sports_science,internlm2-1.8b-hf/ceval-basic_medicine,internlm2-1.8b-hf/ceval-probability_and_statistics,internlm2-1.8b-hf/ceval-high_school_mathematics,internlm2-1.8b-hf/ceval-discrete_mathematics,internlm2-1.8b-hf/ceval-middle_school_geography]
08/14 20:36:09 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-college_economics]
08/14 20:42:35 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-accountant]
08/14 20:50:38 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-tax_accountant]
08/14 20:58:31 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-physician]
08/14 21:05:58 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-civil_servant]
08/14 21:14:04 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-urban_and_rural_planner]
08/14 21:21:06 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-teacher_qualification]
08/14 21:27:53 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-college_programming]
08/14 21:33:53 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-electrical_engineer]
08/14 21:39:43 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-business_administration]
08/14 21:44:49 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-art_studies]
08/14 21:49:42 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-fire_engineer]
08/14 21:54:45 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-environmental_impact_assessment_engineer]
08/14 21:59:36 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-education_science]
08/14 22:03:59 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-professional_tour_guide]
08/14 22:08:27 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-college_chemistry]
08/14 22:12:20 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-metrology_engineer]
08/14 22:16:08 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-mao_zedong_thought]
08/14 22:19:59 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-law]
08/14 22:24:00 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-veterinary_medicine]
08/14 22:27:21 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-modern_chinese_history]
08/14 22:31:05 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-chinese_language_and_literature]
08/14 22:34:30 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-legal_professional]
08/14 22:38:40 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-logic]
08/14 22:42:40 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-middle_school_history]
08/14 22:45:34 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-plant_protection]
08/14 22:48:52 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-clinical_medicine]
08/14 22:52:26 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-computer_architecture]
08/14 22:55:52 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-middle_school_biology]
08/14 22:58:57 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-middle_school_politics]
08/14 23:02:34 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-middle_school_chemistry]
08/14 23:05:45 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-high_school_history]
08/14 23:08:52 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-computer_network]
08/14 23:11:40 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-operating_system]
08/14 23:14:29 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-college_physics]
08/14 23:17:54 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-advanced_mathematics]
08/14 23:21:19 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-high_school_physics]
08/14 23:23:59 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-high_school_chemistry]
08/14 23:26:58 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-high_school_biology]
08/14 23:29:57 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-middle_school_mathematics]
08/14 23:33:13 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-middle_school_physics]
08/14 23:36:05 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-marxism]
08/14 23:38:55 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-high_school_politics]
08/14 23:42:26 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-high_school_geography]
08/14 23:45:00 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-ideological_and_moral_cultivation]
08/14 23:47:46 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-high_school_chinese]
08/14 23:51:37 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-sports_science]
08/14 23:54:01 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-basic_medicine]
08/14 23:56:46 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-probability_and_statistics]
08/15 00:00:31 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-high_school_mathematics]
08/15 00:03:21 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-discrete_mathematics]
08/15 00:05:27 - OpenCompass - INFO - Start inferencing [internlm2-1.8b-hf/ceval-middle_school_geography]
08/15 00:07:12 - OpenCompass - INFO - time elapsed: 12795.30s
08/15 00:07:15 - OpenCompass - INFO - Partitioned into 52 tasks.
08/15 00:07:18 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-computer_network]: {'accuracy': 47.368421052631575}
08/15 00:07:20 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-operating_system]: {'accuracy': 47.368421052631575}
08/15 00:07:22 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-computer_architecture]: {'accuracy': 23.809523809523807}
08/15 00:07:24 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-college_programming]: {'accuracy': 27.027027027027028}
08/15 00:07:27 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-college_physics]: {'accuracy': 42.10526315789473}
08/15 00:07:29 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-college_chemistry]: {'accuracy': 37.5}
08/15 00:07:30 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-advanced_mathematics]: {'accuracy': 26.31578947368421}
08/15 00:07:32 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-probability_and_statistics]: {'accuracy': 22.22222222222222}
08/15 00:07:35 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-discrete_mathematics]: {'accuracy': 25.0}
08/15 00:07:37 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-electrical_engineer]: {'accuracy': 27.027027027027028}
08/15 00:07:39 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-metrology_engineer]: {'accuracy': 54.166666666666664}
08/15 00:07:41 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-high_school_mathematics]: {'accuracy': 22.22222222222222}
08/15 00:07:42 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-high_school_physics]: {'accuracy': 42.10526315789473}
08/15 00:07:44 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-high_school_chemistry]: {'accuracy': 52.63157894736842}
08/15 00:07:47 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-high_school_biology]: {'accuracy': 26.31578947368421}
08/15 00:07:49 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-middle_school_mathematics]: {'accuracy': 36.84210526315789}
08/15 00:07:50 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-middle_school_biology]: {'accuracy': 80.95238095238095}
08/15 00:07:52 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-middle_school_physics]: {'accuracy': 47.368421052631575}
08/15 00:07:55 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-middle_school_chemistry]: {'accuracy': 80.0}
08/15 00:07:57 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-veterinary_medicine]: {'accuracy': 43.47826086956522}
08/15 00:07:59 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-college_economics]: {'accuracy': 32.72727272727273}
08/15 00:08:01 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-business_administration]: {'accuracy': 39.39393939393939}
08/15 00:08:03 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-marxism]: {'accuracy': 68.42105263157895}
08/15 00:08:05 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-mao_zedong_thought]: {'accuracy': 70.83333333333334}
08/15 00:08:07 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-education_science]: {'accuracy': 55.172413793103445}
08/15 00:08:09 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-teacher_qualification]: {'accuracy': 59.09090909090909}
08/15 00:08:11 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-high_school_politics]: {'accuracy': 57.89473684210527}
08/15 00:08:13 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-high_school_geography]: {'accuracy': 47.368421052631575}
08/15 00:08:15 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-middle_school_politics]: {'accuracy': 76.19047619047619}
08/15 00:08:17 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-middle_school_geography]: {'accuracy': 75.0}
08/15 00:08:19 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-modern_chinese_history]: {'accuracy': 52.17391304347826}
08/15 00:08:21 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-ideological_and_moral_cultivation]: {'accuracy': 73.68421052631578}
08/15 00:08:23 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-logic]: {'accuracy': 31.818181818181817}
08/15 00:08:26 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-law]: {'accuracy': 29.166666666666668}
08/15 00:08:28 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-chinese_language_and_literature]: {'accuracy': 47.82608695652174}
08/15 00:08:30 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-art_studies]: {'accuracy': 42.42424242424242}
08/15 00:08:32 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-professional_tour_guide]: {'accuracy': 51.724137931034484}
08/15 00:08:34 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-legal_professional]: {'accuracy': 34.78260869565217}
08/15 00:08:36 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-high_school_chinese]: {'accuracy': 36.84210526315789}
08/15 00:08:38 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-high_school_history]: {'accuracy': 65.0}
08/15 00:08:40 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-middle_school_history]: {'accuracy': 86.36363636363636}
08/15 00:08:42 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-civil_servant]: {'accuracy': 42.5531914893617}
08/15 00:08:44 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-sports_science]: {'accuracy': 52.63157894736842}
08/15 00:08:46 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-plant_protection]: {'accuracy': 40.909090909090914}
08/15 00:08:49 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-basic_medicine]: {'accuracy': 68.42105263157895}
08/15 00:08:50 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-clinical_medicine]: {'accuracy': 36.36363636363637}
08/15 00:08:52 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-urban_and_rural_planner]: {'accuracy': 52.17391304347826}
08/15 00:08:54 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-accountant]: {'accuracy': 36.734693877551024}
08/15 00:08:57 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-fire_engineer]: {'accuracy': 38.70967741935484}
08/15 00:08:59 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-environmental_impact_assessment_engineer]: {'accuracy': 51.61290322580645}
08/15 00:09:01 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-tax_accountant]: {'accuracy': 36.734693877551024}
08/15 00:09:03 - OpenCompass - INFO - Task [internlm2-1.8b-hf/ceval-physician]: {'accuracy': 42.857142857142854}
dataset                                         version    metric         mode      internlm2-1.8b-hf
----------------------------------------------  ---------  -------------  ------  -------------------
ceval-computer_network                          db9ce2     accuracy       gen                   47.37
ceval-operating_system                          1c2571     accuracy       gen                   47.37
ceval-computer_architecture                     a74dad     accuracy       gen                   23.81
ceval-college_programming                       4ca32a     accuracy       gen                   27.03
ceval-college_physics                           963fa8     accuracy       gen                   42.11
ceval-college_chemistry                         e78857     accuracy       gen                   37.5
ceval-advanced_mathematics                      ce03e2     accuracy       gen                   26.32
ceval-probability_and_statistics                65e812     accuracy       gen                   22.22
ceval-discrete_mathematics                      e894ae     accuracy       gen                   25
ceval-electrical_engineer                       ae42b9     accuracy       gen                   27.03
ceval-metrology_engineer                        ee34ea     accuracy       gen                   54.17
ceval-high_school_mathematics                   1dc5bf     accuracy       gen                   22.22
ceval-high_school_physics                       adf25f     accuracy       gen                   42.11
ceval-high_school_chemistry                     2ed27f     accuracy       gen                   52.63
ceval-high_school_biology                       8e2b9a     accuracy       gen                   26.32
ceval-middle_school_mathematics                 bee8d5     accuracy       gen                   36.84
ceval-middle_school_biology                     86817c     accuracy       gen                   80.95
ceval-middle_school_physics                     8accf6     accuracy       gen                   47.37
ceval-middle_school_chemistry                   167a15     accuracy       gen                   80
ceval-veterinary_medicine                       b4e08d     accuracy       gen                   43.48
ceval-college_economics                         f3f4e6     accuracy       gen                   32.73
ceval-business_administration                   c1614e     accuracy       gen                   39.39
ceval-marxism                                   cf874c     accuracy       gen                   68.42
ceval-mao_zedong_thought                        51c7a4     accuracy       gen                   70.83
ceval-education_science                         591fee     accuracy       gen                   55.17
ceval-teacher_qualification                     4e4ced     accuracy       gen                   59.09
ceval-high_school_politics                      5c0de2     accuracy       gen                   57.89
ceval-high_school_geography                     865461     accuracy       gen                   47.37
ceval-middle_school_politics                    5be3e7     accuracy       gen                   76.19
ceval-middle_school_geography                   8a63be     accuracy       gen                   75
ceval-modern_chinese_history                    fc01af     accuracy       gen                   52.17
ceval-ideological_and_moral_cultivation         a2aa4a     accuracy       gen                   73.68
ceval-logic                                     f5b022     accuracy       gen                   31.82
ceval-law                                       a110a1     accuracy       gen                   29.17
ceval-chinese_language_and_literature           0f8b68     accuracy       gen                   47.83
ceval-art_studies                               2a1300     accuracy       gen                   42.42
ceval-professional_tour_guide                   4e673e     accuracy       gen                   51.72
ceval-legal_professional                        ce8787     accuracy       gen                   34.78
ceval-high_school_chinese                       315705     accuracy       gen                   36.84
ceval-high_school_history                       7eb30a     accuracy       gen                   65
ceval-middle_school_history                     48ab4a     accuracy       gen                   86.36
ceval-civil_servant                             87d061     accuracy       gen                   42.55
ceval-sports_science                            70f27b     accuracy       gen                   52.63
ceval-plant_protection                          8941f9     accuracy       gen                   40.91
ceval-basic_medicine                            c409d6     accuracy       gen                   68.42
ceval-clinical_medicine                         49e82d     accuracy       gen                   36.36
ceval-urban_and_rural_planner                   95b885     accuracy       gen                   52.17
ceval-accountant                                002837     accuracy       gen                   36.73
ceval-fire_engineer                             bc23f5     accuracy       gen                   38.71
ceval-environmental_impact_assessment_engineer  c64e2d     accuracy       gen                   51.61
ceval-tax_accountant                            3a5e3c     accuracy       gen                   36.73
ceval-physician                                 6e277d     accuracy       gen                   42.86
ceval-stem                                      -          naive_average  gen                   40.59
ceval-social-science                            -          naive_average  gen                   58.21
ceval-humanities                                -          naive_average  gen                   50.16
ceval-other                                     -          naive_average  gen                   45.43
ceval-hard                                      -          naive_average  gen                   33.76
ceval                                           -          naive_average  gen                   47.03
08/15 00:09:03 - OpenCompass - INFO - write summary to /root/opencompass/outputs/default/20240814_203307/summary/summary_20240814_203307.txt
08/15 00:09:03 - OpenCompass - INFO - write csv to /root/opencompass/outputs/default/20240814_203307/summary/summary_20240814_203307.csv