软件测试 测试停止标准_停止正常测试

软件测试 测试停止标准

I see a lot of data scientists using tests such as the Shapiro-Wilk test and the Kolmogorov–Smirnov to test for normality. Stop doing this. Just stop. If you’re not yet convinced (and I don’t blame you!), let me show you why these are a waste of your time.

我看到许多数据科学家使用诸如Shapiro-Wilk检验和Kolmogorov-Smirnov检验之类的检验其正态性。 别这样 停下来。 如果您还没有说服您(并且我不怪您！)，请让我告诉您为什么这会浪费您的时间。

为什么我们关心正常性？ (Why do we care about normality?)

We should care about normality. It’s an important assumption that underpins a wide variety of statistical procedures. We should always be sure of our assumptions and make efforts to check that they are correct. However, normality tests are not the way for us to do this.

我们应该关心正常性。 这是一个重要的假设，可支持多种统计程序。 我们应该始终确保自己的假设，并努力检查它们是否正确。 但是，正常性测试不是我们执行此操作的方法。

However, in large samples (n > 30) which most of our work as data scientists is based upon the Central Limit Theorem usually applies and we need not worry about the normality of our data. But in cases where it does not apply let’s consider how we can check for normality in a range of different samples.

但是，在大样本(n> 30)中，我们作为数据科学家所做的大部分工作通常都基于中心极限定理 ，因此我们不必担心数据的正态性。 但是在不适用的情况下，让我们考虑如何检查一系列不同样本中的正态性。

小样本的正态性测试 (Normality testing in small samples)

First let us consider a small sample. Say n=10. Let’s look at the histogram for this data.

首先让我们考虑一个小样本。 假设n = 10。 让我们看一下这些数据的直方图。

Histogram of x (n=10). (Image by author)

x的直方图(n = 10)。 (作者提供的图片)

Is this normally distributed? Doesn’t really look like it — does it? Hopefully you’re with me and accept that this isn’t normally distributed. Now let’s perform the Shapiro-Wilk test on this data.

这是正态分布的吗？ 看起来不是真的吗？ 希望您与我在一起，并接受这种分布不是正态分布。 现在，我们对该数据执行Shapiro-Wilk测试。

Oh. p=0.53. No evidence to suggest that x is not normally distributed. Hmm. What do you conclude then. Well, of course, not being evidence that x is not normally distributed does not mean that x is normally distributed. What’s actually happening is that in small samples the tests are underpowered to detect deviations from normality.

哦。 p ＝ 0.53。 没有证据表明x不是正态分布。 嗯 那你得出什么结论。 好吧，当然，没有证据表明x不是正态分布并不意味着x是正态分布。 实际情况是，在小样本中，测试功能不足以检测与正常值的偏差。

Normal Q-Q Plot of x (n=10). (Image by author)

x的普通QQ图(n = 10)。 (作者提供的图片)

The best way to assess normality is through the use of a quantile-quantile plot — Q-Q plot for short. If the data is normally distributed we would expect to see a straight line. This data shows some deviation from normality, the line is not very straight. There appears to be some issues in the tail. Admittedly, without more data it is hard to say.

评估正态性的最好方法是使用分位数-分位数图 (简称QQ图)。 如果数据是正态分布的，我们期望看到一条直线。 该数据表明与正常情况有些偏差，直线不是很直。 尾部似乎有一些问题。 诚然，没有更多数据很难说。

With this data, I would have concerns about assuming normality as there appears to be some deviation in the Q-Q plot and in the histogram. But, if we had just relied on our normality test, we wouldn’t have picked this up. This is because the test is underpowered in small samples.

有了这些数据，我将担心假设正态性，因为QQ图和直方图中似乎有些偏差。 但是，如果我们只是依靠我们的正态性检验，那么我们就不会选择这种方式。 这是因为小样本中的测试功能不足。

大样本的正态性测试 (Normality testing in large samples)

Now let’s take a look at normality testing in a large sample (n=5000). Let’s take a look at a histogram.

现在，让我们看一下大样本(n = 5000)中的正态性测试。 让我们看一下直方图。

Histogram of x (n=5000). (Image by author)

x的直方图(n = 5000)。 (作者提供的图片)

I hope you’d all agree that this looks to be normally distributed. Okay, so what does the Shapiro-Wilk test say. Bazinga! p=0.001. There’s very strong evidence that x is not normally distributed. Oh dear. Well, let’s take a quick look at our Q-Q plot. Just to double check.

希望大家都同意，这种分布看起来是正态分布的。 好的，Shapiro-Wilk测试怎么说。 巴辛加！ p ＝ 0.001。 有非常有力的证据表明x 不是正态分布的。 噢亲爱的。 好吧，让我们快速看一下我们的QQ图。 只是要仔细检查。

Normal Q-Q plot for x (n=5000). (Image by author)

x的普通QQ图(n = 5000)。 (作者提供的图片)

Wow. This looks to be normally distributed. In fact, there shouldn’t be any doubt that this is normally distributed. But, the Shapiro-Wilk test says it isn’t.

哇。 这看起来是正态分布的。 其实，不应该有任何怀疑，这是正态分布的。 但是，Shapiro-Wilk测试表明并非如此。

What’s going on here? Well the Shapiro-Wilk test (and other normality tests) are designed to test for theoretical normality (i.e. the perfect bell curve). In small samples these tests are underpowered to detect quite major deviations from normality which can be easily detected through graphical methods. In larger samples these tests will detect even extremely minor deviations from theoretical normality that are not of practical concern.

这里发生了什么？ 那么Shapiro-Wilk检验(以及其他正态性检验)旨在测试理论正态性(即完美的钟形曲线)。 在小样本中，这些测试的功能不足，无法检测到很大的偏离正常值的情况，可以通过图形方法轻松地检测出这些偏离。 在较大的样本中，这些测试甚至可以检测出与理论正常性无关的极小偏差，而这在实际中并不重要。

结论 (Conclusion)

Hopefully, I have shown you that normality tests are not of practical utility for data scientists. Don’t use them. Forget about them. At best, they are useless; at worst, they are misleading. If you want to assess the normality of some data, use Q-Q plots and histograms. They’ll give you a much clearer picture about the normality of your data.

希望我已经向您展示了正态性测试对数据科学家而言并不实用。 不要使用它们。 算了吧 充其量，它们是无用的。 最糟糕的是，它们具有误导性。 如果要评估某些数据的正态性，请使用QQ图和直方图。 它们将为您提供有关数据正常性的更清晰的画面。

翻译自: https://towardsdatascience.com/stop-testing-for-normality-dba96bb73f90

软件测试 测试停止标准