【推导结果】如何得到 回归均方误差 & 估计系数的标准误

对线性回归模型系数标准差标准误的理解

1.生成数据

y	x	e
3.6	1	0.63
3.4	2	-1.38
7.6	3	1.01
7.4	4	-1.01
11.6	5	1.38
11.4	6	-0.63

2.回归

$$y={\beta }_0+{\beta }_{1}x+ϵ$$

$$y_i={\beta }_0+{\beta }_1{x}_i+e_i$$

reg y x

      Source |       SS           df       MS      Number of obs   =         6
-------------+----------------------------------   F(1, 4)         =     34.60
       Model |   57.422285         1   57.422285   Prob > F        =    0.0042
    Residual |  6.63771505         4  1.65942876   R-squared       =    0.8964
-------------+----------------------------------   Adj R-squared   =    0.8705
       Total |  64.0600001         5      12.812   Root MSE        =    1.2882

------------------------------------------------------------------------------
           y | Coefficient  Std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
           x |   1.811429   .3079359     5.88   0.004     .9564615    2.666396
       _cons |       1.16   1.199238     0.97   0.388    -2.169618    4.489618
------------------------------------------------------------------------------

3.计算回归的标准误差

（1）SSE\SSR\SST

$SSE$: Sum of Squares Error,
$$SSE=\sum _{i=1}^n(\widehat{y_i}-y_i{)}^2=\sum _{i=1}^n(e_i-\bar{e}{)}^2$$
在本示例中，$SSE=(3.6-2.97{)}^2+(3.4-4.78{)}^2+(7.6-6.95{)}^2+(7.4-8.41{)}^2+(11.6-10.22{)}^2+(11.4-12.03{)}^2=6.637713$

$SSR$: Sum of Squares of the Regression
$$SSR=\sum _{i=1}^n(\widehat{y_i}-\bar{y}{)}^2$$
$SST$: Total Sum of Squares
$$SST=\sum _{i=1}^n(y_i-\bar{y}{)}^2$$

（2）MSE

回归的标准误差为：
$$s^2=MSE=\frac{SSE}{n-K}=\frac{{\sum }_{i=1}^n(e_i-\bar{e}{)}^2}{n-K}$$

$$s=\sqrt{MSE}$$

$$s^2=\frac{6.637713}{6-2}=1.6594282;s=1.288188$$

（3）SE

$$S_{\hat{\beta }}=\sqrt{\frac{s^2}{{\sum }_{i=1}^n(x_i-\bar{x})}}$$

$$S_{\hat{\beta }}=\sqrt{\frac{\frac{1}{n-2}{\sum }_{i=1}^n\widehat{e^2}}{{\sum }_{i=1}^n(x_i-\bar{x})}}$$

$$S_{\hat{\beta }}=\sqrt{\frac{\frac{1}{4}\times 6.637713}{(1-3.5{)}^2+(2-3.5{)}^2+(3-3.5{)}^2+(4-3.5{)}^2+(5-3.5{)}^2+(6-3.5{)}^2}}$$

SE为何会很大？

• 样本少，分母可能大
• 极端值多
• X没有足够的variation

---

Appendix

1. simulation code

clear 
set obs 6
gen y = 3.6 in 1 
replace y = 3.4 in 2 
replace y = 7.6 in 3
replace y = 7.4 in 4
replace y = 11.6 in 5
replace y = 11.4 in 6
gen x = _n

reg y x
predict xb

gen e = y - xb
format %9.2f xb 
format %9.2f e 
egen addtext_mean = rowmean(y xb)
forv i = 1/6{
	su add in `i',d
	global y`i' = r(mean)
	su e in `i',d
	global e`i' = r(mean)
}

tw (scatter y x, mlab(y) mlabp(1)) /// 
   (lfit y x) /// 
   (scatter xb x, mlab(xb) mlabp(1)) /// 
   (rspike y xb x) ,legend(off) /// 
   text($y1 0.9 "0.63",size(vsmall) color(red)) /// 
   text($y2 1.9 "-1.38",size(vsmall) color(red)) /// 
   text($y3 2.9 "1.01",size(vsmall) color(red)) /// 
   text($y4 3.9 "-1.01",size(vsmall) color(red)) /// 
   text($y5 4.9 "1.38",size(vsmall) color(red)) /// 
   text($y6 5.9 "-0.63",size(vsmall) color(red)) 

2.序列相关 同方差 or 异方差

对于①参数线性②不存在“严格多重共线性”③随机抽样④严格外生性⑤“球形扰动项”(条件同方差+不存在自相关)五个假定均能够满足时

OLS估计量为BLUE，最优无偏线性估计量

此时，x的协方差矩阵为：
$$Var(\widehat{{\beta }_1}|x)=Var({\beta }_1+\frac{\sum (x_i-\bar{x})e_i}{\sum (x_i-\bar{x})}|x)$$

$$Var(\widehat{{\beta }_1}|x)=\frac{Var(\sum (x_i-\bar{x})e_i|x)}{[\sum (x_i-\bar{x}{)}^2{]}^2}$$

• 倘若序列无关，那么和的方差即等价于方差的和，假设$Var(e_i|x)={\sigma }^2$

KaTeX parse error: Unknown column alignment: * at position 71: … \begin{array}{*̲*lr**} …

• 序列相关：

$$\widehat{{\sigma }^2}=\frac{\sum e_i^2}{n-k-1}$$

3.calculate SE in matlab

sqrt(inv(X'*X)*1.6594282)