困惑我好久的Multivariate Linear Regression 求导问题

Suppose we have (${\vec{x}}^{(i)},y^{(i)}$) with sample size $N$, where ${\vec{x}}^{(i)}\in {\mathbb{R}}^D$.
$$\hat{y}=\sum _{j=1}^D{\beta }_j{x}_j$$
$$\mathcal{L}(a,b)=\frac{1}{2}(a-b{)}^2$$
$$\begin{array}{rl}\epsilon ({\beta }_0,{\beta }_1,...,{\beta }_D)& =\frac{1}{N}\sum _{i=1}^N\mathcal{L}({\hat{y}}^{(i)},y^{(i)})\\ & =\frac{1}{2N}\sum _{i=1}^N({\hat{y}}^{(i)},y^{(i)}{)}^2\\ & =\frac{1}{2N}\sum _{i=1}^N(\sum _{j=1}^D{\beta }_j{x}_j^{(i)}-y^{(i)}{)}^2\end{array}$$

Take Derivative with respect to $w_j$:
$$\begin{array}{rl}\frac{\partial \epsilon }{\partial {\beta }_j}& =\frac{1}{N}\sum _{i=1}^N{x}_j^{(i)}({\hat{y}}^{(i)}-y^{(i)})\\ & =\frac{1}{N}\sum _{i=1}^N{x}_j^{(i)}(\sum _{j^{\prime }=1}^D{\beta }_{j^{\prime }}{x}_{j^{\prime }}^{(i)}-y^{(i)})(这部分注意：你就是这里不明白)\\ & =\frac{1}{N}\sum _{j^{\prime }=1}^D(\sum _{i=1}^N{x}_j^{(i)}{x}_{j^{\prime }}^{(i)}){\beta }_{j^{\prime }}-\frac{1}{N}\sum _{i=1}^N{x}_j^{(i)}{y}^{(i)}\end{array}$$
Let $A_{j{j}^{\prime }}=\frac{1}{N}{\sum }_{i=1}^N{x}_j^{(i)}{x}_{j^{\prime }}^{(i)}\in {\mathbb{R}}^D$ and $c_j=\frac{1}{N}{\sum }_{i=1}^N{x}_j^{(i)}{y}^{(i)}\in {\mathbb{R}}^D$. Then:
$$\begin{array}{rl}\frac{\partial \epsilon }{\partial {\beta }_j}& =\frac{1}{N}\sum _{j^{\prime }=1}^D(\sum _{i=1}^N{x}_j^{(i)}{x}_{j^{\prime }}^{(i)}){\beta }_{j^{\prime }}-\frac{1}{N}\sum _{i=1}^N{x}_j^{(i)}{y}^{(i)}\\ & =\frac{1}{N}\sum _{j^{\prime }=1}^D{A}_{j{j}^{\prime }}{\beta }_{j^{\prime }}-c_j\stackrel{set}{=}0\end{array}$$
Let $X\in {\mathbb{R}}^{N\times D}$ , $A=\frac{1}{N}{X}^{T}X$ and $c=\frac{1}{N}{X}^{T}y$
$$X=\left[\begin{array}{c}{x}^{(1)T}\\ x^{(2{)}^T}\\ .\\ .\\ x^{(n{)}^T}\end{array}\right]\text{\hspace{1em}(3)}$$
$$\begin{array}{rl}\frac{\partial \epsilon }{\partial {\beta }_j}& =\frac{1}{N}\sum _{j^{\prime }=1}^D(\sum _{i=1}^N{x}_j^{(i)}{x}_{j^{\prime }}^{(i)}){\beta }_{j^{\prime }}-\frac{1}{N}\sum _{i=1}^N{x}_j^{(i)}{y}^{(i)}\\ & =\frac{1}{N}\sum _{j^{\prime }=1}^D{A}_{j{j}^{\prime }}{\beta }_{j^{\prime }}-c_j\\ & =A\beta -c\stackrel{set}{=}0\end{array}$$
$$\hat{\beta }=A^{-1}c=(X^{T}X{)}^{-1}{X}^{T}t$$
终于解决了！

一种更简单的方法是直接在risk做变换：
$$\begin{array}{rl}\epsilon ({\beta }_0,{\beta }_1,...,{\beta }_D)& =\frac{1}{2N}\sum _{i=1}^N(\sum _{j=1}^D{\beta }_j{x}_j^{(i)}-y^{(i)}{)}^2\\ & =\frac{1}{2N}[X\beta -y{]}^T[X\beta -y]\end{array}$$
Finally, the MLE estimate is $\hat{\beta }=(X^{T}X{)}^{-1}{X}^{T}y$

This is only a estimate from one single training data, but we really want to get the true error or prediction error, which can be defined as:
$$\begin{array}{rl}{\epsilon }_{true}({\beta }_0,{\beta }_1,...,{\beta }_D)& =\frac{1}{2}E(\sum _{j=1}^D{\beta }_j{\mathbf{x}}_j-\mathbf{y}{)}^2\\ & =\frac{1}{2}{\int }_{\mathbf{x}}(\sum _{j=1}^D{\beta }_j{\mathbf{x}}_j-\mathbf{y}{)}^{2}p(\mathbf{x})d\mathbf{x}\end{array}$$
If want to read more about bias-variance in linear regression model, read the following:
https://courses.cs.washington.edu/courses/cse546/12wi/slides/cse546wi12LinearRegression.pdf