转:Link
L2 regularization:
optimizer = torch.optim.Adam(model.parameters(), lr=1e-4, weight_decay=1e-5)
lambd = torch.tensor(1.)
l2_reg = torch.tensor(0.)
for param in model.parameters():
l2_reg += torch.norm(param)
loss += lambd * l2_reg
L1 regularization:inclulde weight only
L1_reg = torch.tensor(0., requires_grad=True)
for name, param in model.named_parameters():
if 'weight' in name:
L1_reg = L1_reg + torch.norm(param, 1)
total_loss = total_loss + 10e-4 * L1_reg