model capacity
depth越大 更能抽象出更多特征,模型表达能力越强
case 1 estimated < ground-truth under-fitting
case 2 estimated > ground-truth over-fitting
over fitting 现象更多
如何检测和减少overfitting
TrainSet Val Set TestSet
valset 挑选模型参数
划分数据集 train 参数 True or False
torch.utils.data.random_split(train_db,[50000,10000]) 将train_db 划分一部分出来作为val_db
K-fold cross validation 60k数据划分为K份,每次取一份来做validation,其他数据做train
减轻overfitting
1\more data
2\constraint model complexity
shallow model
regulaization
3\Dropout
4\ data argumentation
5\early stopping
Regularization 损失函数增加一项 L1正则化或者L2正则化 使用L1范数 L2范数
也叫做weight decay
optim.SGD(net.parameters(), lr = learning_rate, weight_decay=0.01) # 正则化参数0.01
L1正则化
regularization_loss = 0
for par