统计问题第76问：多重显著性检验

Question

Researchers undertook a randomised controlled trial to assess the effect of zinc supplementation on the severity and duration of diarrhoea caused by cholera in children.Children were recruited if they had watery diarrhoea and dark field microscopy examination of stool was positive for Vibrio cholerae (confirmed by stool culture). Children were randomly allocated to zinc supplementation or placebo.

The table compares the primary outcomes (duration of diarrhoea and stool output) and secondary outcomes for the zinc supplementation and placebo groups⇓. All hypothesis tests were performed using a two sided, independent samples t test, with a 0.05 (5%) critical level of significance. Zinc supplementation was found to reduce the duration and severity of diarrhoea in children with cholera.

Which of the following statements, if any, are true?

a) A type I error is when the null hypothesis is rejected in favour of the alternative hypothesis when for the total population there is no difference between treatments

b) For each hypothesis test, the maximum probability of a type I error occurring is 0.05

c) For each hypothesis test, the minimum probability of a type I error not occurring is 0.95 (95%)

d) When multiple hypothesis tests are performed, the probability of a type I error rises to more than 0.05

提示：这是一道多选题。

Answer

Answers a, b, c, and d are all true.

Hypothesis testing is used to make inferences about the population on the basis of a sample. However, it is possible that such inferences are not representative of the population. The sorts of error that can be committed when hypothesis testing—type I and II errors—have been described in a previous question. A type I error occurs if there is no difference between treatment groups in the population but there is a difference in the sample and thus the null hypothesis is rejected in favour of the alternative as a result of hypothesis testing (a is true).

Each hypothesis test was performed using thet test for independent samples.For each hypothesis test, the P value was derived by hypothetically repeating the study an infinite number of times. The P value is the proportion of these hypothetical studies that would have produced a test statistic greater or equal to the absolute value of the one calculated in the above study, assuming that the null hypothesis is true. The critical level of significance is set at 0.05 (5%). The null hypothesis would be rejected in favour of the alternative for those 5% of the infinite number of studies with the largest test statistics; hence for any hypothesis test the maximum probability of rejecting the null hypothesis is 0.05. Any hypothesis test could result in a type I error. Therefore, the maximum probability of a type I error for each hypothesis test is 0.05 (b is true). It follows that the minimum probability of a type I error not occurring for any single hypothesis test is 0.95 (95%; c is true).

The probability a type I error will occur for any of the hypothesis tests in the table is equal to one minus the probability that none of the tests will result in a type I error. The probability a single test will not result in a type I error is 0.95. The probability that none of the four hypothesis tests will result in a type I error is the product of the individual probabilities that each test will not be a type I error: (0.95×0.95×0.95×0.95)=0.95⁴=0.815. Therefore, the probability that a type I error will have occurred in the table shown is: (1-0.815)=0.185. Generally, the probability that a type I will occur is (1−0.95^x), where x is the total number of hypothesis tests. If there are two or more tests, then 0.95^x will be smaller than 0.95 and the probability of a type I error will be greater than 0.05 (d is true).

The probability a type I error will occur for a single hypothesis test is 0.05; however, when multiple hypothesis tests are performed the probability is increased. Care must be taken when research papers undertake a large number of statistical tests—ultimately some of these will result in a type I error. However, we will not know which significant findings are a type I error. Various approaches have been suggested to reduce the number of type I errors when undertaking multiple testing such as the Bonferroni method, which will be discussed in a later question.

所以答案是选择 a b c d

每天学习一点，你会更强大！‍

‍