【Foundation】（五）transformers之Evaluate

0、介绍

 简单易用的机器学习模型库，只需要一行代码便可加载任务任务的评估函数

1、基本使用

• 查看支持的评估函数(list_evaluation_modules)

• 加载评估函数(load)

• 查看评估函数说明(inputs_description)

• 评估指标计算(compute)

  • 全局计算(compute)

  • 迭代计算(add,add_batch)

• 计算多个评估指标(combine)

• 评估结果对比可视化(radar_plot)

import evaluate
evaluate.list_evaluation_modules(include_community=True,with_details=True)

2、加载评估函数

acc = evaluate.load('accuracy')

3、查看函数说明

print(acc.description)

print(acc.inputs_description)

print(acc)

4、评估指标计算——全局计算

acc = evaluate.load('accuracy')

results = acc.compute(references=[0,1,2,0,1,2],predictions=[0,1,1,2,1,0])
results

5、评估指标计算——迭代计算

acc = evaluate.load('accuracy')
for ref,pred in zip([0,1,0,1],(1,0,0,1)):
    acc.add(reference= ref,prediction=pred)
acc.compute()

acc = evaluate.load('accuracy')
for refs,preds in zip([[0,1],[0,1]],[[1,0],[0,1]]):
    acc.add_batch(references= refs,predictions=preds)
acc.compute()

6、多个评估指标计算

clf_metrics = evaluate.combine(['accuracy', 'f1','recall','precision'])
clf_metrics

clf_metrics.compute(predictions=[0,1,0],references=[0,1,1])

7、评估结果可视化

from evaluate.visualization import  radar_plot#目前只支持雷达图
data = [
    {'accuracy': 0.99,'f1': 0.40,'recall': 0.52,'precision': 0.85},
    {'accuracy': 0.67,'f1': 0.66,'recall': 0.55,'precision': 0.75},
    {'accuracy': 0.60,'f1': 0.20,'recall': 0.59,'precision': 0.65},
    {'accuracy': 0.50,'f1': 0.56,'recall': 0.523,'precision': 0.85},
]

model_names = ['model1', 'model2', 'model3', 'model4']

plot = radar_plot(data = data,model_names=model_names)