机器学习--- 均方误差损失(Mean Squared Error, MSE) --＞[附代码]

一、简介

• 欧氏距离损失经常用在线性回归问题（求解的是连续问题）。
• 回归问题解决的是对具体数值的预测，比如房价预测、销量预测等等。
• 解决回归问题的神经网络一般只有一个输出节点，这个节点的输出值就是预测值。

二、数学推导

1. 假设训练数据 $X$, 训练数据的 $label为Y$。
2. 预测函数为：$f(x_i)=\widehat{y_i}=w{x}_i+b$
3. 损失函数：
   $$\begin{array}{rl}Los{s}_{MSE}(y,\hat{y})& =\frac{1}{m}\underset{(w,b)}{argmin}\sum _{i=1}^m(f(x_i)-y_i{)}^2\\ & =\frac{1}{m}\underset{(w,b)}{argmin}\sum _{i=1}^m(y_i-w{x}_i-b{)}^2\end{array}$$

2.1 导数计算

4. 计算导数 $\frac{\partial loss}{\partial w},\frac{\partial loss}{\partial b}$
   
   $$\begin{array}{r}\frac{\partial loss}{\partial w}=2\sum _{i=1}^m{x}_i[f(x_i)-y_i]\end{array}$$
   $$\begin{array}{r}\frac{\partial loss}{\partial b}=2\sum _{i=1}^m[f(x_i)-y_i]\end{array}$$

5. 求解新的 $m和b$
   
   $$\{\begin{array}{l}b=\frac{1}{m}{\sum }_{i=1}^m[y_i-w{x}_i]\\ \\ w=\frac{{\sum }_{i=1}^m{y}_i(x_i-\bar{x})}{{\sum }_{i=1}^m{x}_i^2-\frac{1}{m}({\sum }_{i=1}^m{x}_i{)}^2}\end{array}$$

---

首先求解 $b$：

$$\begin{array}{rl}\frac{\partial loss}{\partial b}& =2\sum _{i=1}^m[f(x_i)-y_i]\\ & =2\sum _{i=1}^m[w{x}_i+b-y_i]\\ & =2(mb-\sum _{i=1}^m[y_i-w{x}_i])\end{array}$$

令 $\frac{\partial loss}{\partial b}=0$
$$\begin{array}{rl}上式& =0\\ mb& =\sum _{i=1}^m[y_i-w{x}_i]\\ b& =\frac{1}{m}\sum _{i=1}^m[y_i-w{x}_i]\end{array}$$

---

然后求解 $w$：

$$\begin{array}{rl}\frac{\partial loss}{\partial w}& =2\sum _{i=1}^m{x}_i[f(x_i)-y_i]\\ & =2(w\sum _{i=1}^m{x}_i^2-\sum _{i=1}^m(y_i-b)x_i)\end{array}$$

令 $\frac{\partial loss}{\partial w}=0$
$$\begin{array}{rl}上式& =0\\ w\sum _{i=1}^m{x}_i^2& =\sum _{i=1}^m(y_i-b)x_i\\ 将b& =\frac{1}{m}\sum _{i=1}^m[y_i-w{x}_i]代入上式\\ w\sum _{i=1}^m{x}_i^2& =\sum _{i=1}^m{y}_i{x}_i-\sum _{i=1}^m{x}_i\frac{1}{m}\sum _{i=1}^m[y_i-w{x}_i]\\ w\sum _{i=1}^m{x}_i^2& =\sum _{i=1}^m{y}_i{x}_i-\frac{1}{m}\sum _{i=1}^m{x}_i\sum _{i=1}^m{y}_i+\frac{w}{m}(\sum _{i=1}^m{x}_i{)}^2\\ w(\sum _{i=1}^m{x}_i^2-\frac{1}{m}(\sum _{i=1}^m{x}_i{)}^2)& =\sum _{i=1}^m{y}_i{x}_i-\frac{1}{m}\sum _{i=1}^m{x}_i\sum _{i=1}^m{y}_i\\ w(\sum _{i=1}^m{x}_i^2-\frac{1}{m}(\sum _{i=1}^m{x}_i{)}^2)& =\sum _{i=1}^m{y}_i{x}_i-\bar{x}\sum _{i=1}^m{y}_i\\ w& =\frac{{\sum }_{i=1}^m{y}_i(x_i-\bar{x})}{{\sum }_{i=1}^m{x}_i^2-\frac{1}{m}({\sum }_{i=1}^m{x}_i{)}^2}\end{array}$$

三、代码实现

3.1 python 代码实现

import numpy as np

# 模型公式
y_hat = np.dot(X, w) + b

# 损失函数
loss = np.sum((y_hat - y) ** 2) / num_train

# 参数的偏导
dw = np.dot(X.T, (y_hat - y)) / num_train
db = np.sum((y_hat - y)) / num_train

3.2 torch 代码实现

import torch

loss_fn = torch.nn.MSELoss(reduce=False, size_average=False,reduction='mean')
loss = loss_fn(input.float(), target.float())

'''
reduction-三个值，
	none: 不使用约简；
	mean:返回loss和的平均值；
	sum:返回loss的和。
默认：mean。
'''

参考资料