Why Can We Compare Alpha and P-Value in Hypothesis Tests?

最新推荐文章于 2024-02-04 22:04:59 发布

The Well-Built City

最新推荐文章于 2024-02-04 22:04:59 发布

阅读量273

点赞数

分类专栏： Statistics Misc

本文链接：https://blog.csdn.net/Bill_Wang_01/article/details/115673440

版权

9 篇文章 0 订阅

订阅专栏

Reference: https://courses.washington.edu/p209s07/lecturenotes/Week%205_Monday%20overheads.pdf

The Motivating Question

In Hypothesis Tests, one way to know what conclusion to make (whether to reject the null hypothesis) is by comparung $\alpha$ with $p$ -value. Why can we make this comparison?

Suppose $\alpha\in[0,1]=x\%$ (usually $\alpha=5\%$ ). It simply means, assuming the null hypothesis is true (we never know whether it is true), we are allowed to reject the null hypothesis iff we observed a sample so rare that it would have occured by chance at most $x\%$ of the time.
Thus, as $\alpha$ gets larger, the minimum standard of considering a random sample as “extreme” gets looser (i.e. a sample does not have to be so rare to be considered unusual when $\alpha$ gets larger), which means it’s more unlikely to find a strong evidence against the Null Hypothesis.

Once $\alpha$ has been set, a statistic (like the difference in sample mean), which is basically a numerical summary of the samples’ data, is computed from the sample(s) we obtained.
Each statistic has an associated probability value called a $p$ -value, or the likelihood of an observed statistic occurring due to chance, given the sampling distribution of the statistic (for example, the $t$ distribution with a certain sample size $n$ ).

As we have observed, $\alpha$ determines how extreme our sample must be to reject the null hypothesis, and $p$ -value is how extreme our sample is. Moreover, the more extreme the sample needs to be or turns out to be, the smaller the two values are. Therefore, if $p$ is smaller than or equal to $\alpha$ , we know our sample is extreme enough to allow us reject $H_0$ and conclude our result (can be experiment result, research result, etc.) to be significantly different from $H_0$ .

This statsment is also equivalent to "A result that is (statsitically) significantly different from $H_0$ .
This just means a sample that can present enough (here, enough is equivalent to “statistically significant”) evidence against the null hypothesis i.e. the sample is in favor of the alternative hypothesis.
Thus, “non-significant” just means the result is in favor of the Null Hypothesis.