0. 简介
pytorch lightning通过提供LightningModule和LightningDataModule,使得在用pytorch编写网络模型时,加载数据、分割数据集、训练、验证、测试、计算指标的代码全部都能很好的组织起来,显得主程序调用时,代码简洁可读性大幅度提升。
1. pytorch lightning的安装
pip install pytorch-lightning
conda install pytorch-lightning -c conda-forge
2. 定义一个网络模型模型:LightningModule
通过继承LightningModule,并实现几个关键的函数,使得模型在训练、验证和测试过程中能进行模块化调用,具体细节完全被自定义的函数封装,整体十分简洁。定义一个LightningModule的基类,可以实现的函数如下:
from pytorch_lightning import LightningModule
class MyModel(LightningModule):
"""
The only required methods in the LightningModule are:
init
training_step
configure_optimizers
"""
def __init__(self, *args, **kwargs): pass
def forward(self, *args, **kwargs): pass
def training_step(self, batch, batch_idx, optimizer_idx, hiddens): pass
def training_step_end(self, *args, **kwargs): pass # 接受train_step的返回值
def training_epoch_end(self, outputs): pass # 接受train_step一整个epoch的返回值的列表
def validation_step(self, batch, batch_idx, dataloader_idx): pass # model.eval() and torch.no_grad() are called automatically
def validation_step_end(self, *args, **kwargs): pass # 接受validation_step的返回值
def validation_epoch_end(self, outputs)
def test_step(self, batch, batch_idx, dataloader_idx): pass # model.eval() and torch.no_grad() are called automatically
def test_step_end(self, *args, **kwargs): pass # 接收test_step的返回值
def test_epoch_end(self, outputs): pass
def configure_optimizers(self, ): pass
def any_extra_hook(...): pass # 指代任意其他的可重载函数
其中,必须实现的函数只有__init__() 、training_step()、configure_optimizers()。
3. 定义一个数据模型:LightningDataModule
通过定义LightningDataModule的子类,数据集分割、加载的代码将整合在一起,可以实现的方法有:
class MyDataModule(LightningDataModule):
def __init__(self):
super().__init__()
def prepare_data(self):
# download, split, etc...
# only called on 1 GPU/TPU in distributed
def setup(self,stage:str): # stage: "fit", "test", 【暂时不知道验证步骤叫什么名字,可以自己打印一下】
# make assignments here (val/train/test split)
# called on every process in DDP
def train_dataloader(self):
train_split = Dataset(...)
return DataLoader(train_split)
def val_dataloader(self):
val_split = Dataset(...)
return DataLoader(val_split)
def test_dataloader(self):
test_split = Dataset(...)
return DataLoader(test_split)
4. 使用pytorch lightning的API开始训练
def main():
model = MyModule()
data_module = MyDataModule()
trainer = pytorch_lightning.Trainer(...) # some arugments, 根据需要传入你的参数
trainer.fit(module, datamodule=data_module)
trainer.test(module, datamodule=data_module, verbose=True)
if __name__ == "__main__":
main()
具体实现都通过类封装之后,主函数就显得简洁多了。