【LLM入门】Building makemore Part 5: Building a WaveNet

from视频： https://www.youtube.com/watch?v=t3YJ5hKiMQ0

Chapters:
intro
00:00:00 intro
00:01:40 starter code walkthrough
00:06:56 let’s fix the learning rate plot【均值平滑】
00:09:16 pytorchifying our code: layers, containers, torch.nn, fun bugs
implementing wavenet
00:17:11 overview: WaveNet【分组融合】
00:19:33 dataset bump the context size to 8
00:19:55 re-running baseline code on block_size 8
00:21:36 implementing WaveNet
00:37:41 training the WaveNet: first pass
00:38:50 fixing batchnorm1d bug【dim=(0,1)】
00:45:21 re-training WaveNet with bug fix
00:46:07 scaling up our WaveNet
conclusions
00:46:58 experimental harness
00:47:44 WaveNet but with “dilated causal convolutions”【引出 卷积】
00:51:34 torch.nn
00:52:28 the development process of building deep neural nets
00:54:17 going forward
00:55:26 improve on my loss! how far can we improve a WaveNet on this data?

00:06:56 let’s fix the learning rate plot

00:17:11 overview: WaveNet

00:21:36 implementing WaveNet

00:38:50 fixing batchnorm1d bug

C在最后一个维度！但是在pytorch中，C是第1维度！

00:47:44 WaveNet but with “dilated causal convolutions”