绩效指标准确性

The machine learning model is built using training data (which has input as well as output). Prediction is made on the test data (unseen data which does not have an output label) using the same model. But how do you figure out the effectiveness of the model? There must be some measures that will evaluate the performance of the model.

机器学习模型是使用训练数据(既有输入又有输出)构建的。 使用相同模型对测试数据(看不见的数据，没有输出标签)进行预测。 但是，您如何确定该模型的有效性？ 必须有一些措施可以评估模型的性能。

There are many performance metrics to evaluate the model performance, such as accuracy, precision, recall, F1-score, ROC curve, etc. each having its advantages and disadvantages. In this article, we will discuss the accuracy performance metric. It is only related to the classification task.

有许多用于评估模型性能的性能指标，例如准确性，精度，召回率，F1得分，ROC曲线等，各有其优缺点。 在本文中，我们将讨论准确性性能指标 。 它仅与分类任务有关。

One might wonder what classification is?

可能会想知道什么是分类？

When the machine learning model categorizes data into classes (output belongs to a finite set), the task is referred to as classification. Examples of the classification task are — identification of fraud transactions, detection of any disease in a patient, etc.

当机器学习模型将数据分类为类(输出属于有限集)时，该任务称为分类。 分类任务的示例是-识别欺诈交易，检测患者的任何疾病等。

Performance measure always evaluates the performance of the model on the test dataset. In this article, we will be using binary classification (where only two classes are present) since it is trivial to understand.

性能度量始终会评估模型在测试数据集上的性能。 在本文中，我们将使用二进制分类(其中只有两个类)，因为它很容易理解。

准确性 (ACCURACY)

It is the ratio of correctly classified points (prediction) to the total number of predictions. Its value ranges between 0 and 1.

它是正确分类的点(预测)与预测总数之比。 取值范围是0〜1。

EXAMPLE

例

Actual and predicted values

实际和预测值

what will be the accuracy in the above example?

上例中的准确度是多少？

Since, 2 predictions are correct out of 4, accuracy =2/4 = 0.5 or 50%. Can we conclude that if the accuracy of the model is good, the model is adequate?

由于4个预测中有2个是正确的，因此准确度= 2/4 = 0.5或50％。 我们是否可以得出结论，如果模型的准确性良好，那么模型就足够了？

cottonbro, from Pexels cottonbro ，来自Pexels

There are some loopholes in the accuracy performance metric:

准确性性能指标中存在一些漏洞：

1. Imbalanced data
   数据不平衡
2. When the model gives probability scores
   当模型给出概率分数时

1.数据不平衡 (1. Imbalanced Data)

The example discussed above is having an equal proportion of the data points of both the classes. This type of dataset, where data points are approximately equal for both the classes, is called a balanced dataset.

上面讨论的示例具有相等比例的两个类的数据点。 这种数据集在两个类中的数据点都大致相等，称为平衡数据集。

What if we have an imbalanced dataset (where the number of points of one class is much greater than the other class)? An imbalanced dataset is extremely common in real-world examples. Datasets of rare diseases have very few positive people as compared to the negative ones.

如果我们有一个不平衡的数据集(一类的点数比另一类大得多)怎么办？ 在实际示例中，不平衡的数据集极为常见。 与阴性病例相比，罕见疾病的数据集很少有阳性人群。

EXAMPLE

例

The total number of patients is 100, and 95 out of them are positive (this is an imbalanced dataset). Let’s say the machine learning model is dumb, and it gives positive output only.

患者总数为100，其中95位为阳性(这是一个不平衡的数据集)。 假设机器学习模型是愚蠢的，并且仅给出正输出。

Now, what will be the accuracy?

现在，准确度是多少？

The model will correctly classify 95 points out of 100. We will get an accuracy of 95%.

该模型将正确地对100中的95个点进行分类。我们将获得95％的准确性。

Andrea Piacquadio, from Pexels Andrea Piacquadio ，来自Pexels

DON’T GET TOO EXCITED; remember, the model is dumb.

不要太激动 ; 记住，模型是愚蠢的。

Accuracy can be misleading when dealing with imbalanced data. Therefore, it’s better not to use an accuracy performance metric with imbalanced data.

处理不平衡数据时，准确性可能会产生误导。 因此，最好不要对不平衡的数据使用准确性性能指标。

2.当模型给出概率分数时 (2. When the model gives probability scores)

Let’s say we have three models A, B, C, for binary classification, and all the three models give output as a probability score. In this example, we have used threshold as 0.5, i.e., if probability > 0.5 — prediction will be 1, else 0.

假设我们有用于二进制分类的三个模型A，B，C，所有这三个模型都将输出作为概率得分。 在此示例中，我们将阈值设为0.5，即，如果概率> 0.5，则预测为1，否则为0。

A, B, and C model with probabilistic score and the prediction based on a threshold of 0.5

具有概率得分的A，B和C模型以及基于0.5阈值的预测

Based on the threshold, all three models are giving an accuracy of 100%. But can you deduce which model is best to use based on accuracy just looking at the prediction?

根据阈值，所有三个模型的准确度均为100％。 但是，您可以只看预测就根据准确性推断出哪种模型最适合使用？

NO

没有

Therefore, accuracy is inadequate to compare models when the models give probability as the output.

因此，当模型给出概率作为输出时，准确性不足以比较模型。

结论 (CONCLUSION)

In this article, we have seen the use of accuracy performance metrics in the classification task. The accuracy metric is not appropriate when dealing with imbalanced data and when models give the probability score. These problems of accuracy can be overcome by other performance metrics such as confusion matrix, precision, recall, and F1 score, which will be discussed in the next article.

在本文中，我们已经看到了分类任务中准确性性能指标的使用。 当处理不平衡数据和模型给出概率分数时，精度度量是不合适的。 这些准确性问题可以通过其他性能指标来解决，例如混淆矩阵，精度，召回率和F1得分，这些将在下一篇文章中进行讨论。

Thanks for reading!

谢谢阅读！

翻译自: https://medium.com/@vaibhavjayaswal4180/performance-metrics-accuracy-118f728d328f