目录
LAMB优化器
Layer-wise Adaptive Moments optimizer for Batching training (LAMB)
LAMB 优化器无需用户调整超参数,用户只需输入学习率即可。
Implementation of https://arxiv.org/abs/1904.00962 for large batch, large learning rate training.
pytorch-lamb/lamb.py at master · cybertronai/pytorch-lamb · GitHub
调用示例:
optimizer = Lamb(model.parameters(), lr=args.lr, w