评估模型如何建立_建立和评估分类ML模型

评估模型如何建立

There are different types of problems in machine learning. Some might fall under regression (having continuous targets) while others might fall under classification (having discrete targets). Some might not have a target at all where you are just trying to learn the characteristics of data by distinguishing the data points based on their inherent features by creating clusters.

机器学习中存在不同类型的问题。 一些可能属于回归(具有连续的目标)，而另一些可能属于分类(具有离散的目标)。 有些人可能根本没有目标，而您只是试图通过创建聚类来根据数据点的固有特征来区分数据点，从而学习数据的特征。

However, this article is not about different areas of machine learning but about a very little yet important thing, which if not tended to carefully can wreak “who knows what” on your operationalized classification models and eventually, the business. Therefore, the next time when someone at work tells you that her/his model is giving ~93.23% accuracy, do not fall for it before asking the right questions.

但是，本文不是关于机器学习的不同领域，而是关于一件非常重要的事情，如果不小心的话，它可能会在您的可操作分类模型以及最终业务上造成“谁知道什么”。 因此，下一次当有人在工作时告诉您他/他的模型给出了〜93.23％的准确性时，在提出正确的问题之前，请不要掉以轻心。

So, how do we know what are the right questions?

那么，我们怎么知道正确的问题呢？

That’s a good question. Let us try to answer that by studying how to build and evaluate a classification model the right way. Everyone who has been studying machine learning is aware of all frequently used classification metrics but only a few of them know the right one to use to evaluate the performance of their classification model.

这是个好问题。 让我们尝试通过研究如何正确构建和评估分类模型来回答这一问题。 一直在研究机器学习的每个人都知道所有常用的分类指标，但是只有少数几个知道正确的方法来评估其分类模型的性能。

So, to enable you to ask the right questions, we will go through the following concepts in detail (for a classification model):

因此，为了使您能够提出正确的问题，我们将详细介绍以下概念(针对分类模型)：

1. Data Distribution (Training, Validation, Test)

   数据分配(培训，验证，测试)

2. Handling Class Imbalance.

   处理班级失衡。

3. The Right Choice of Metric For Model Evaluation

   模型评估的正确选择指标

资料分配 (Data Distribution)

While splitting data for training, validation, and test set, one should always keep in mind that all three of them must be representative of the same population. For example, in the MNIST digit classification dataset where all the digit images are grey (black and white), you train it and achieve a validation accuracy of almost 90%, but your test data has digit images of various colors (not just black and white). Now, you have a problem there. No matter what you do, there will always be a data bias. You cannot get rid of it totally but what you can do is maintain a uniformity in your validation and test data set. This is an example of a difference in the distribution of validation and test sets.

在为训练，验证和测试集拆分数据时，应始终牢记，这三个数据必须代表相同的总体。 例如，在所有数字图像均为灰色(黑白)的MNIST数字分类数据集中，您对它进行了训练并获得了将近90％的验证准确度，但是您的测试数据具有各种颜色的数字图像(不只是黑色和白色)。白色)。 现在，您在那里遇到了问题。 无论您做什么，都会有数据偏差。 您不能完全摆脱它，但是您可以做的是在验证和测试数据集中保持一致。 这是验证和测试集分布差异的一个示例。

拆分数据的正确策略 (The Right Strategy to Split Data)

Your test data set MUST always represent real-world data distribution. For example, in a binary classification problem, where you are supposed to detect positive patients for a rare disease (class 1) where 6% of the entire data set contains positive cases, then your test data should also have almost the same proportion. Make sure you follow the same distribution. This is not just the case with classification models. It holds for every type of ML modeling problem.

您的测试数据集必须始终代表真实的数据分布。 例如，在二元分类问题中，您应该检测一