深度学习Deep learning小白入门笔记——20230707

最新推荐文章于 2024-08-03 10:51:11 发布

孤光一点萤❂

最新推荐文章于 2024-08-03 10:51:11 发布

阅读量69

点赞数

分类专栏： DeepLearning 文章标签：深度学习笔记人工智能

本文链接：https://blog.csdn.net/weixin_45698813/article/details/131605116

版权

8 篇文章 0 订阅

订阅专栏

Deep Learning 2023/07/07

$L (b, w) = y = b + w x$

$L (b, w)$

$\frac{1}{N}\sum_{n}e_n$

e为误差值

$\mid y-\hat{y} \,\,\mid$

$(y-\hat{y})^2$

如果真实值和预测值都是概率分布，则会使用交叉熵作为损失函数用以度量两者之间的差异

$w^*,b^*=arg\min_{w,b}L$

随机选取初始点

计算L损失函数在初始点对w的导

如果计算在这一点的斜率小于零(负数)则增大w

如果计算在这一点的斜率大于零(正数)则减小w

$\eta\frac{\partial L}{\partial w}\mid_{w = w^0}$

η: learning rate

$\Rightarrow y=b+\sum_{i} c_i sigmoid(b_i+w_ix)$

$y=b+\sum_j w_jx_j\Rightarrow y=b+\sum_i c_i sigmoid(b_i+\sum_jw_{ij}x_j)$

Backpropagation: an efficient way to compute ∂L/∂w in neural network.
- Gradient Descent
- Chain Rule
  
  Case 1 :
  $g(x)\ \ \ \ \ \ \ z=h(y) \\ \Delta x \rightarrow \Delta y \rightarrow \Delta z \ \ \ \ \ \ \frac{dz}{dx} = \frac{dz}{dy}\frac{dy}{dx}$
  Case 2 :
  $\ \ \ \ \ y = h(s) \ \ \ \ \ z = k(x,y) \\ \frac{dz}{ds} = \frac{\partial z}{\partial x}\frac{dx}{ds}+\frac{\partial z}{\partial y}\frac{dy}{ds}$
- $L(\theta)=\sum_{n=1}^NC^n(\theta) \rightarrow \frac{\partial L(\theta)}{\partial w}=\sum_{n=1}^N\frac{\partial C^n(\theta)}{\partial w}$

1958: Perceptron (linear model 感知机，一种人工神经网络)
1969: Perceptron has limitation
1980s: Multi-layer perceptron
- Do not have significant difference from DNN today.
1986: Backpropagation
- Usually more than 3 hidden layers is not helpful.
1989: 1 hidden layer is “good enough”, why deep?
2006: RBM initialization(breakthrough)
2009: GPU
2011: Start to be popular in speech recognition
2012: win ILSVRC image competition