fastai入门教程和基本概念

课程页
fast.ai releases new deep learning course, four libraries, and 600-page book
https://www.fast.ai/2020/08/21/fastai2-launch/

fastai applications - quick start
https://docs.fast.ai/quick_start.html

教程页
https://docs.fast.ai/tutorial.vision

程序页
https://colab.research.google.com/github/fastai/fastai/blob/master/nbs/23_tutorial.vision.ipynb
可点击 “复制到云盘硬盘” 复制到自己的谷歌硬盘

The 1cycle policy learn.lr_find() learn.recorder.plot() learn.fit(2, 5e-3) learn.fit_one_cycle(2, 5e-2)
https://fastai1.fast.ai/callbacks.one_cycle.html
How Do You Find A Good Learning Rate
https://sgugger.github.io/how-do-you-find-a-good-learning-rate.html

Understanding Fastai’s fit_one_cycle method
https://iconof.com/1cycle-learning-rate-policy/

CLRs are not computationally expensive and eliminate the need to find the best learning rate value—the optimal learning rate will fall somewhere between the minimum and maximum bounds.
“The essence of this learning rate policy comes from the observation that increasing the learning rate might have a short term negative effect and yet achieve a longer term beneficial effect.” Smith
Cyclical Learning Rates are effective because they can successfully negotiate saddle points, which typically have small gradients (flat surfaces) and can slow down training when learning rate is small. The best way to overcome such obstacles is to speed up and to move fast until a curved surface is found. The increasing learning rate of CLRs does just that, efficiently.
Concretely, in super-convergence, learning rate starts at a low value, increases to a very large value and then decreases to a value much lower than its initial one.

Picking Out Components of a Path
https://realpython.com/python-pathlib/

import pathlib
pathlib.Path.cwd()
PosixPath(’/home/gahjelle/realpython/’)

path
PosixPath(’/home/gahjelle/realpython/test.md’)

path.name
‘test.md’

path.stem
‘test’

path.suffix
‘.md’

path.parent
PosixPath(’/home/gahjelle/realpython’)

path.parent.parent
PosixPath(’/home/gahjelle’)

path.anchor
‘/’

作者：有糖吃可好
链接：https://www.zhihu.com/question/310138804/answer/581039350
来源：知乎
著作权归作者所有。商业转载请联系作者获得授权，非商业转载请注明出处。

unfreeze 在fastai课程中使用的是预训练模型，模型卷积层的权重已经提前在ImageNet 上训练好了，在使用的时候一般只需要在预训练模型最后一层卷积层后添加自定义的全连接层即可。卷积层默认是freeze的，即在训练阶段进行反向传播时不会更新卷积层的权重，只会更新全连接层的权重。在训练几个epoch之后，全连接层的权重已经训练的差不多了，但accuracy还没有达到你的要求，这时你可以调用unfreeze然后再进行训练，这样在进行反向传播时便会更新卷积层的权重(一般不会对卷积层权重进行较大的更新，只会进行一点点的微调，越靠前的卷积层调整的幅度越小，所以有了differential learning rate 这一想法)precompute 当precompute=True时，会提前计算出每一个训练样本（不包括增强样本）在预训练模型最后一层卷积层的activation， 并将其缓存下来，之后在训练阶段进行前向传播的时候，直接将precompute 的activation 作为后面全连接层(FC Layer)的输入，这样便省去前面卷积层进行前向传播的计算量，减少训练所需时间（这种优势在epoch比较大的时候能够显著0提高训练速度）。当precompute=False时，则不会提前计算训练样本的activation，每一个epoch都需要重新将训练样本+增强样本(前提是进行了增强操作)进行卷积层的前向传播，然后进行反向传播更新对应的权重。
总结一下：当precompute=True时：

1. 增强样本不会参与训练；
2. 不管你的模型是unfreeze还是freeze都不会更新卷积层的权重，因为activation已经提前计算好了，如果你想要更新卷积层权重，必须把precompute设置为False。 当 precompute=False时 ： 如果有增强样本，增强样本这时便会参与模型训练； 如果模型为unfreeze状态，在进行反向传播的时候便会更新卷积层的权重。

Below are the steps we followed to build a State of the Art CLASSIFIER:-

ENABLE DATA AUGMENTATION AND SET PRECOMPUTE = TRUE.
USE lr_find() TO FIND THE HIGHEST LEARNING RATE WHERE LOSS IS STILL CLEARLY IMPROVING
TRAIN LAST LAYER FROM PRECOMPUTED ACTIVATIONS FOR COUPLE OF EPOCHS.
TRAIN LAST LAYER WITH DATA AUGMENTATION (i.e PRECOMPUTE=FALSE) FOR 2–3 EPOCH WITH CYCLE_LEN=1.
UNFREEZE ALL THE LAYERS.
SET EARLIER LAYERS TO LOWER LEARNING RATE THAN NEXT HIGHER LAYERS AND TRAIN IT.
USE lr_find() AGAIN.
TRAIN FULL NETWORK WITH cycle_mult=2 UNTILL OVERFITTING.

下面是我们通常采取的8个步骤:

1.启用数据增强，并设置预计算（precompute）=True
2.使用lrfind()找到最高的学习速率，其中损失仍在明显改善。
3.为1-2个epoch的预计算激活训练最后一层
4.用数据增强的方法(即预计算=False)进行最后一层的训练，使用cyclelen=1的2-3个周期
5.解冻所有的层
6.将较早的层设置为比下一个更高层的3x-10x降低的学习速率
7.再次使用lr_find()
8.用cycle_mult=2训练整个网络，直到过度拟合

img_f = fnames[0]
img = open_image(img_f) ＃原形式
img.show(figsize=(5,5))

get_y_fn = lambda x: path_lbl/f’{x.stem}_P{x.suffix}’
通常很明显。这里，我猜是文件名 + _P，我创建了一个简单的函数，取文件名，然后加上 _P ，放到另一个目录里 (path_lbl) ，然后尝试打开它，成功了。
f = lambda a,b,c:a+b+c 中的关键字lambda表示匿名函数，
#冒号:之前的a,b,c表示它们是这个函数的参数。
#匿名函数不需要return来返回值，表达式本身结果就是返回值。

##图像显示方法

mask = open_mask(get_y_fn(img_f))
mask.show(figsize=(5,5), alpha=1)

cam_fn = fnames[5]
cam_img = PILImage.create(cam_fn)
cam_img.resize((224,96))
cam_img.show
test_eq(cam_img.size, (128,96))
tmask = Transform(PILMask.create)
mask_fn = path/‘labels’/f’{cam_fn.stem}P{cam_fn.suffix}’
mask = tmask(mask_fn)
test_eq(type(mask), PILMask)
test_eq(mask.size, (128,96))
**,axs = plt.subplots(1,3, figsize=(16,15))**
cam_img.show(ctx=axs[0], title=‘image’)
mask.show(alpha=1, ctx=axs[1], vmin=1, vmax=30, title=‘mask’)
cam_img.show(ctx=axs[2], title=‘superimposed’)
mask.show(ctx=axs[2], vmin=1, vmax=30);

dls.show_batch(max_n=4)
learn.show_results(max_n=6, figsize=(15,17))

两种数据加载方式

dls = SegmentationDataLoaders.from_label_func(
path, bs=8, fnames = fnames, label_func = label_func, codes = codes
)

camvid = DataBlock(blocks=(ImageBlock, MaskBlock(codes)),
get_items = get_image_files,
get_y = label_func,
splitter=RandomSplitter(),
batch_tfms=aug_transforms(size=(120,160)))
dls = camvid.dataloaders(path/“images”, path=path, bs=8)

learn = unet_learner(dls, resnet34)
learn.fine_tune(3)

learn.fine_tune()
It’s a new method in fastai v2. So in fastai v1 you will still have to use these steps separately.
fit_one_cycle
unfreeze
fit_one_cycle