ANOVA-学习笔记

​

5.1 One-way Analysis of Variance

5.1.1 Some notations:

$I$ samples: every treatment value corresponds to a sample $i$

${\mu }_1,{\mu }_2...{\mu }_I$: treatment means

$J_1,J_2...J_I$: sample size

$N$: Total sample,所有样本数量， $N=J_1+J_2+...+J_I$

$x_{ij}$: $j$-th observation in the i-th sample

$\overline{x_{i.}}$: Sample mean of the $i_{th}$ sample, $\overline{x_{i.}}=\frac{{\sum }_{j=1}^{Ji}{x}_{ij}}{J_i}$

${\bar{x}}_{..}$ : Sample grand mean, ${\bar{x}}_{..}=\frac{{\sum }_{i=1}^I{\sum }_{J=1}^{J_i}{x}_{ij}}{N}=\frac{{\sum }_{i=1}^I{J}_i{\bar{x}}_i}{N}$

Hypothesis is:

$H_0:{\mu }_1={\mu }_2=...={\mu }_I$ v.s. $H_1$: two or more of the ${\mu }_i$ are different.

5.1.2 Assumptions:

Treatment populations （各样本背后的总体） must

1. be normal 服从正态分布

2. have the same variance ${\sigma }^2$方差需要相等，即，方差齐性/方差同质性

Samples （随机抽样的样本）must be

3. independent from each other

Note ：

1. 在不满足正态性时可以采用非参数检验

2. 要求方差齐性的原因： ANOVA虽然叫做Analysis of Variance, 但是其目的是为了检验每个组的均值是否相同，为了具有均值上的可比性，我们需要让方差相等。

5.1.3 ANOVA Brief Introduction

方差分析的基本原理: 认为不同处理组均数间的差别主要来源于

1. 组内方差的产生只由于各个水平内部的随机变动，即不可控的随机因素造成的(random uncertainty)

2. 组间方差的产生可能由于

   1）随机误差，测量误差/个体间的差异

   2）实验条件，不同的处理造成的差异，differences between the treatment means

5.1.4 SSTr (Treatment Sum of Squares):

The spread of the sample means around the sample grand mean, it measures how different the treatment means are from each other, i.e. inter-sample variation (组间方差）

即通过衡量 对不同样本的均值 偏离样本总均值 的程度的大小，来验证不同实验组均值的差异程度.(在这里我个人的理解是，spread of the sample mean around the sample grand mean 相当于treatment means 的一个estimate，即估计值）

如果不同样本的均值非常分散(Sample means around Sample grand mean), 有很大可能是treatment means会不同，因此我们拒绝 $H_0$

SSTr 的计算公式：（前一个为定义式，后一个为方便计算式）

$$SSTr=\sum _{i=1}^I{J}_i(\overline{x_{i.}}-\overline{x_{..}}{)}^2=\sum _{i=1}^I{J}_i{\overline{x_{i.}}}^2-N{\overline{x_{..}}}^2$$

Note ：
SSTr is large, i.e. sample means spread out widely $\Rightarrow$ 我们有理由总结：treatment means differ and thus reject $H_0$
SSTr is small, i.e. sample means are all close to the sample grand mean $\Rightarrow$ 我们有理由总结：treatment means are equal.

5.1.5 SSE (Error Sum of Squares):

刚刚讲了SSTr 的计算公式和SSTr的作用，但是现在有一个问题，SSTr 多大算大，以至于让我们拒绝 $H_0$ ，认为treatment means不相等呢？

在这里我们引入SSE，让SSTr 和SSE 作比较。

SSE measures the variation in the individual sample points around their respective sample means:

$$\begin{array}{rl}SSE& =\sum _{i=1}^I\sum _{j=1}^{J_i}(x_{ij}-\overline{x_{i.}}{)}^2\\ & =\sum _{i=1}^I\sum _{j=1}^{J_i}{x}_{ij}^2-\sum _{i=1}^I{J}_i{\overline{x_{i.}}}^2\end{array}$$

Based on sample variance formula:

$$\begin{gathered} s_i^2=\frac{{\sum }_{j=1}^{J_i}(x_{ij}-\overline{x_{i.}}{)}^2}{J_i-1} \\ \Rightarrow \sum _{j=1}^{J_i}(x_{ij}-\overline{x_{i.}}{)}^2=s_i^2(J_i-1)\text{\hspace{1em}(1)} \end{gathered}$$
将（1）带入SSE的定义式，
$$SSE=\sum _{i=1}^I{s}_i^2(J_i-1)$$

5.1.6 SST (Total Sum of Squares):

observation mean value - sample grand mean value

$\begin{array}{rl}SST& =\sum _{I=1}^I\sum _{J=1}^{Ji}(x_{ij}-\overline{x..}{)}^2\\ & =\sum _{i=1}^I\sum _{J=1}^{Ji}{x}_{ij}^2-N{\overline{x_{..}}}^2\end{array}$

重要关系式：
$SST=SSTr+SSE$

5.1.7 ANOVA Table

d.f.(SSTr) = I - 1
d.f.(SSE) = N - I

$MSTr(TreatmentMeanSquare)=\frac{SSTr}{I-1}$
$MSE(ErrorMeanSquare)=\frac{SSE}{N-I}$

ANOVA Table

​方差来源	自由度	平方和	均方	F值
因素（Treatment）	I - 1	$SSTr={\sum }_{i=1}^I{J}_i(\overline{x_{i.}}-\overline{x_{..}}{)}^2$	$MSTr=\frac{SSTr}{I-1}$	$\frac{MSTr}{MSE}$
误差（Error）	N - I	$SSE={\sum }_{i=1}^I{\sum }_{j=1}^{J_i}(x_{ij}-\overline{x_{i.}}{)}^2$	$MSE=\frac{SSE}{N-I}$
总和（Total）	N - 1	$SST={\sum }_{i=1}^I{\sum }_{j=1}^{J_i}(x_{ij}-\overline{x_{..}}{)}^2$

核心要点：
1. 假设检验：

$\{\begin{array}{c}{H}_0:{\mu }_0={\mu }_1=...={\mu }_I\\ H_1:\text{Two or more means are not equal}\end{array}$

2. 假设检验的test statistics：$F=\frac{MSTr}{MSE}$

3. 假设检验的Criteria:
-If $H_0$ is right, F is near 1
-If $H_0$ is false, F ＞ 1 (right tailed)

4. 对应的原理：

$\{\begin{array}{c}E(SSTr)=(I-1){\sigma }^2\mathrm{if}{H}_0\mathrm{is}\mathrm{right}\\ E(SSTr)>(I-1){\sigma }^2\mathrm{if}{H}_0\mathrm{is}\mathrm{wrong}\end{array}$

但是不管假设是否正确，$E(SSE)=(N-I){\sigma }^2$

$⟹$$\{\begin{array}{c}E(MSTr)=\frac{(I-1){\sigma }^2}{I-1}={\sigma }^2\mathrm{if}{H}_0\mathrm{is}\mathrm{right}\\ E(MSTr)>{\sigma }^2\mathrm{if}{H}_0\mathrm{is}\mathrm{wrong}\end{array}$

但是不管假设是否正确，$E(MSE)=E(\frac{SSE}{N-I})=\frac{(N-I){\sigma }^2}{N-I}={\sigma }^2$

$⟹$ $\{\begin{array}{c}若H_0正确，F值接近1\\ 若H_0错误，F值>1\end{array}$

5. F test
假设经过计算得到F-value为$f$，significance level = $\alpha$
$F_{\alpha }(I-1,N-I)<f$,then we say we reject $H_0$ under $\alpha$ significance level.Therefore there is indication from the data that there is significant difference among the treatment means.

6. F test 对应的p-value 转换
p(F= f-value) <0.05, there is moderate evidence against $H_0$ in favor of $H_1$.
There is significant difference among the treatment means.

5.2 An Alternate Parameterization

现在介绍另外一种比较treatment means的方法。

拆分

• Assume $X_{ij}\sim N({\mu }_i,{\sigma }^2)$
• Error term ${\epsilon }_{ij}\sim N(0,{\sigma }^2)$
• For each $X_{ij}$, 将其拆分成两个部分
  $X_{ij}={\mu }_i+{\epsilon }_{ij}$

定义新的变量

• Define population grand mean
  $\mu =\frac{1}{I}{\sum }_{i=1}^I{\mu }_i$

• Define i-th treatment effect $${\alpha }_i={\mu }_i-\mu \text{\hspace{1em}(2)}$$
  we can easily get ${\sum }_{i=1}^I{\alpha }_i=0$
  From(2), we know ${\mu }_i={\alpha }_i+\mu$

• Thus, we can change the equation of $X_{ij}$ into $$X_{ij}={\mu }_i+{\epsilon }_{ij}=\mu +{\alpha }_i+{\epsilon }_{ij}$$where ${\sum }_{i=1}^I{\alpha }_i=0$

• For the one-way ANOVA,
  $$\begin{gathered} H_0:{\mu }_1={\mu }_2=...={\mu }_I\iff \\ {H}_0:{\alpha }_1={\alpha }_2=...={\alpha }_I=0 \end{gathered}$$

为什么都等于0？
$\because {\mu }_i={\alpha }_i+\mu$
$\mu$ is fixed,
${\mu }_1=...={\mu }_I$,
$\therefore {\alpha }_1=...={\alpha }_I$
$又\because {\sum }_{i=1}^I{\alpha }_i=0$
$\therefore {\alpha }_1={\alpha }_2=...={\alpha }_I=0$

5.3 Comparison with Random Effects Model

固定效应模型和随机效应模型的对比

Item	Differences	$H_0$
Fixed Effect Model	Treatments are chosen deliberately by the experimenters. Interest is on specific treatment means	${\mu }_1=...={\mu }_I$
Random Effect Model	Treatments are chosen at random from a population of possible treatments. No particular interest.	For every treatment in the population, treatment means are equal.

和固定效应模型一样，随机效应模型也假设 population of treatment means is normal.

5.4 Unbalanced Designed?

Balanced: $J_1=J_2=...=J_I$
Balanced 实际上更好，因为不同水平下的样本相同更有助于保证方差相等。
在平衡设计实验中，不平等方差的影响通常不会很大，it’s more robust
if it’s unbalanced, use

1. nonparametric test
2. Welch Test