【学习笔记】数理统计习题十二

Q1: Consider the multiple linear regression model
$$\mathbf{Y}=\mathbf{X}\mathbf{\beta }+\mathbf{ϵ},$$
where $\mathbf{Y}=(y_1,\dots ,y_n{)}^{\top }$, $\mathbf{\beta }=({\beta }_0,\dots ,{\beta }_{p-1}{)}^{\top }$, $\mathbf{X}$ is the $n\times p$ design matrix, and $\mathbf{ϵ}=({ϵ}_1,\dots ,{ϵ}_n{)}^{\top }$. Assume that $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}(X)=p<n$, $E[\mathbf{ϵ}]=0$, and $\mathrm{V}\mathrm{a}\mathrm{r}[\mathbf{ϵ}]={\sigma }^2{I}_n$ with $\sigma >0$.

(a). Show that the covariance matrix of the least squares estimates is diagonal if and only if the columns of $\mathbf{X}$, ${\mathbf{X}}_1,\dots ,{\mathbf{X}}_p$, are orthogonal, that is ${\mathbf{X}}_i^{\top }{\mathbf{X}}_j=0$ for $i\ne j$.

(b). Let ${\hat{y}}_i$ and ${\widehat{ϵ}}_i$ be the fitted values and the residuals, respectively. Show that $n{\sigma }^2={\sum }_{i=1}^n\mathrm{V}\mathrm{a}\mathrm{r}[{\hat{y}}_i]+{\sum }_{i=1}^n\mathrm{V}\mathrm{a}\mathrm{r}[{\widehat{ϵ}}_i]$.

(c). Suppose further that $\mathbf{ϵ}\sim N(0,{\sigma }^2{I}_n)$, and you use F test to handle the hypothesis
$$H_0:{\beta }_1={\beta }_2=\cdots ={\beta }_{p-1}=0vs.H_1:\sum _{i=1}^{p-1}{\beta }_i^2\ne 0.$$If the coefficient of determination $R^2=0.58$, $p=5$ and $n=15$, is the null rejected at the significance level $\alpha =0.05$?
($F_{0.95}(4,10)=3.48,F_{0.95}(5,10)=3.33,t_{0.95}(10)=1.81$)

解：
  (a) 可知最小二乘估计为$\hat{\beta }=(X^{T}X{)}^{-1}{X}^{T}Y$，有$Var[\hat{\beta }]={\sigma }^2(X^{T}X{)}^{-1}$，如果$Var[\hat{\beta }]$是对角的，于是$X^{T}X$也是对角的，这表明矩阵$X\text{X}X$的列$X_1,X_2,\cdots ,X_n$是正交的，反之亦然。
  (b) 残差为$$\widehat{ϵ}=Y-X\hat{\beta }=Y-X(X^{T}X{)}^{-1}{X}^{T}Y=(I_n-P)Y$$其中$P=X(X^{T}X$