python计算分类模型的评价指标

目录

1、混淆矩阵

2、代码实现

2.1、OA

2.2、AA 

2.3、kappa

2.4、用户精度(User Accuracy, UA)

2.5、生产者精度(Producer Accuracy, PA)

2.6、f1_score

2.7、混淆矩阵

2.8、分类报告

---

1、混淆矩阵

以二分类问题为例，混淆矩阵表现形式如下：

• TP(True Positive，真正例)：将真实情况为正例的类型正确地预测为正例
• FN(False Negative，假反例)：将真实情况为正例的类型错误地预测为反例
• FP(False Positive，假正例)：将真实情况为反例的类型错误地预测为正例
• TN(True Negative，真反例)：将真实情况为反例的类型正确地预测为反例

2、代码实现

2.1、OA

• 总体准确度（OA）—表示总测试样本中正确分类的样本数

import numpy as np
from sklearn.metrics import accuracy_score

y_test = np.array([0,1,1,2,2,0,0,1,2,3,3,2,3,1,0,3,1,2,3,3,2,0,])
y_predict = np.array([1,1,0,2,2,2,0,2,0,3,0,2,3,1,0,2,1,2,1,3,1,3,])

# 也叫准确率
OA = accuracy_score(y_test, y_predict)
print(OA)

2.2、AA 

• 平均准确度（AA）—表示分类精度的平均值

import numpy as np
from sklearn.metrics import recall_score

y_test = np.array([0,1,1,2,2,0,0,1,2,3,3,2,3,1,0,3,1,2,3,3,2,0,])
y_predict = np.array([1,1,0,2,2,2,0,2,0,3,0,2,3,1,0,2,1,2,1,3,1,3,])

# 当average=None时，计算的是每一个类别的精度。也相当于生产者精度
# 也叫召回率
AA = recall_score(y_test, y_predict,average='macro')
print(AA)

2.3、kappa

• kappa系数（kappa）提供了关于地面真值图和分类图之间的强一致性的相互信息

import numpy as np
from sklearn.metrics import cohen_kappa_score

y_test = np.array([0,1,1,2,2,0,0,1,2,3,3,2,3,1,0,3,1,2,3,3,2,0,])
y_predict = np.array([1,1,0,2,2,2,0,2,0,3,0,2,3,1,0,2,1,2,1,3,1,3,])

kappa = cohen_kappa_score(y_test, y_predict)
print(kappa)

2.4、用户精度(User Accuracy, UA)

import numpy as np
from sklearn.metrics import precision_score

y_test = np.array([0,1,1,2,2,0,0,1,2,3,3,2,3,1,0,3,1,2,3,3,2,0,])
y_predict = np.array([1,1,0,2,2,2,0,2,0,3,0,2,3,1,0,2,1,2,1,3,1,3,])

# 当average='macro'时，计算的是平均的用户精度（UA）
# 也叫精确率
UA = precision_score(y_test, y_predict,average=None)
print(UA)

2.5、生产者精度(Producer Accuracy, PA)

import numpy as np
from sklearn.metrics import recall_score

y_test = np.array([0,1,1,2,2,0,0,1,2,3,3,2,3,1,0,3,1,2,3,3,2,0,])
y_predict = np.array([1,1,0,2,2,2,0,2,0,3,0,2,3,1,0,2,1,2,1,3,1,3,])

# 当average='macro'时，计算的是平均的生产者精度（PA）。也相当于是每个类别的精度
PA = recall_score(y_test, y_predict,average=None)
print(PA)

2.6、f1_score

import numpy as np
from sklearn.metrics import f1_score

y_test = np.array([0,1,1,2,2,0,0,1,2,3,3,2,3,1,0,3,1,2,3,3,2,0,])
y_predict = np.array([1,1,0,2,2,2,0,2,0,3,0,2,3,1,0,2,1,2,1,3,1,3,])

# 当average=None时，计算的每个类别的f1_score
f1_s = f1_score(y_test, y_predict,average="macro")
print(f1_s)

2.7、混淆矩阵

import numpy as np
from sklearn.metrics import confusion_matrix

y_test = np.array([0,1,1,2,2,0,0,1,2,3,3,2,3,1,0,3,1,2,3,3,2,0,])
y_predict = np.array([1,1,0,2,2,2,0,2,0,3,0,2,3,1,0,2,1,2,1,3,1,3,])

matrix = confusion_matrix(y_test, y_predict)
print(matrix)

2.8、分类报告

import numpy as np
from sklearn.metrics import classification_report

y_test = np.array([0,1,1,2,2,0,0,1,2,3,3,2,3,1,0,3,1,2,3,3,2,0,])
y_predict = np.array([1,1,0,2,2,2,0,2,0,3,0,2,3,1,0,2,1,2,1,3,1,3,])

report = classification_report(y_test, y_predict, digits=4)
print(report)

2.9、论文中Accuracy、Precision、Recall、F1 − score的实现

import numpy as np
from sklearn.metrics import precision_score, recall_score, f1_score, classification_report, accuracy_score,confusion_matrix

y_test = np.array([0, 1, 1, 2, 2, 0, 0, 1, 2, 3, 3, 2, 3, 1, 0, 3, 1, 2, 3, 3, 2, 0, ])
y_predict = np.array([1, 1, 0, 2, 2, 2, 0, 2, 0, 3, 0, 2, 3, 1, 0, 2, 1, 2, 1, 3, 1, 3, ])


precision = precision_score(y_test, y_predict,average='macro')
print(precision)

recall = recall_score(y_test, y_predict,average='macro')
print(recall)

f1 = f1_score(y_test, y_predict,average='macro')
print(f1)

accuracy = accuracy_score(y_test, y_predict)
print(accuracy)

print(classification_report(y_test, y_predict))

自定义根据混淆矩阵计算：

import numpy as np
from sklearn.metrics import precision_score, recall_score, f1_score, classification_report, accuracy_score,confusion_matrix

y_test = np.array([0, 1, 1, 2, 2, 0, 0, 1, 2, 3, 3, 2, 3, 1, 0, 3, 1, 2, 3, 3, 2, 0, ])
y_predict = np.array([1, 1, 0, 2, 2, 2, 0, 2, 0, 3, 0, 2, 3, 1, 0, 2, 1, 2, 1, 3, 1, 3, ])

aconfusion_matrixa_ = confusion_matrix(y_test, y_predict)

def calculate_multiclass_metrics(confusion_matrix):
    """
    根据多分类混淆矩阵计算每个类的Precision、Recall、F1得分，并返回宏观平均值和Accuracy。

    :param confusion_matrix: 多分类混淆矩阵（二维数组）
    :return: Macro Precision, Macro Recall, Macro F1得分, Accuracy
    """
    confusion_matrix = np.array(confusion_matrix)
    num_classes = confusion_matrix.shape[0]

    # 初始化指标
    precision_list = []
    recall_list = []
    f1_list = []

    # 计算每个类的指标
    for i in range(num_classes):
        TP = confusion_matrix[i, i]  # 真阳性
        FP = confusion_matrix[:, i].sum() - TP  # 假阳性
        FN = confusion_matrix[i, :].sum() - TP  # 假阴性

        # 计算Precision, Recall
        precision = TP / (TP + FP) if (TP + FP) > 0 else 0
        recall = TP / (TP + FN) if (TP + FN) > 0 else 0

        # 计算F1得分
        f1_score = (2 * precision * recall / (precision + recall)) if (precision + recall) > 0 else 0

        precision_list.append(precision)
        recall_list.append(recall)
        f1_list.append(f1_score)

    # 计算宏观平均
    macro_precision = np.mean(precision_list)
    macro_recall = np.mean(recall_list)
    macro_f1 = np.mean(f1_list)

    # 计算Accuracy
    total_correct = np.trace(confusion_matrix)  # 正确分类的样本总数
    total_samples = confusion_matrix.sum()  # 总样本数
    accuracy = total_correct / total_samples if total_samples > 0 else 0

    return macro_precision, macro_recall, macro_f1, accuracy

precision, recall, f1, acc = calculate_multiclass_metrics(aconfusion_matrixa_)

print("Precision:", precision)
print("Recall:", recall)
print("F1 Score:", f1)
print("Accuracy:", acc)