![](https://img-blog.csdnimg.cn/20201014180756927.png?x-oss-process=image/resize,m_fixed,h_64,w_64)
李宏毅2020机器学习
Zaгathustra
这个作者很懒,什么都没留下…
展开
-
李宏毅 deep learning theory P4 2-1 when gradient is zero
前情回顾虽然神经网络不是convex的,但是有没有这样一种可能,定义出的loss function的local minima都是接近的原创 2021-05-12 15:58:11 · 118 阅读 · 0 评论 -
李宏毅 Deep learning theory P3 1-3 is deep better than shallow?
原创 2021-05-12 14:28:00 · 125 阅读 · 0 评论 -
李宏毅 Deep learning theory P2 1-2 Potential of deep
最佳状况,有些pattern是不可能出现的deep的产生更多线段的感觉是在折纸只增加深度,宽度不变,线段的个数是指数上升的只增加宽度,不增加深度,线段的个数是线性变化的增加每层噪声来观察结果,靠前层数的权值影响较大,类似于折纸,前面的几层是在折纸,后面的几层是在剪纸using deep structure to fit functions刚才的理论给出的是upper bound,所以shallow可能并没有在最...原创 2021-05-12 11:25:12 · 132 阅读 · 0 评论 -
李宏毅deep learning theory P1 can shallow network fit any function
shallow的network已经可以fit any function,deep的有什么优势deep的network和shallow训练出来的有什么不同1. can shallow network fit any function?用神经网络去拟合一个L-lipschitz的函数,使得最大的差值error小于epsilon,需要分L/epsilon个segment,需要L/epsilon个神经元拟合因此对于任何L-lipschitz的函数都可以用一层hid...原创 2021-05-11 11:06:24 · 152 阅读 · 0 评论 -
李宏毅2020机器学习深度学习 43-44 network compression: network pruning
network compression意义:一些设备,比如可穿戴设备资源(内存和算力)有限,因此需要compress这些network to fit these devicesQ:why larger network is easier to optimize?A plausible explanation is that a large network contains many small networks. Every group of initial parameters原创 2021-01-31 16:16:27 · 234 阅读 · 1 评论 -
李宏毅2020机器学习深度学习 14 Tips for Training DNN
It's not the problem of underfitting, it is just not welltrained.Why bigger network doesn't have the same good performance asthesmaller one? Maybe it has the ability but it is not well trained.Dropout : when there are good results on training data...原创 2021-01-30 23:18:43 · 251 阅读 · 0 评论 -
李宏毅2020机器学习深度学习 13 backpropogation
There is no difference between gradient descent and backpropogation. Backpropogation makes gradient descent more effective.Chain rule原创 2021-01-29 18:28:05 · 109 阅读 · 0 评论 -
李宏毅2020机器学习深度学习 12 brief introduction of deep learning
deep learning attracts lots of attentionI believe you have seen lots of exciting results before. There are more and more deep learning applications, for example, you can see the growing deep learning trends at Google.The history of deep learning: Ups a原创 2021-01-28 23:17:53 · 248 阅读 · 0 评论