accelerate 笔记：多个gpu training，一个gpu infer

首先，test_loader 不参与accelerate的prepare

if accelerator.is_main_process:
    # Set up test dataset without any accelerator call

其次，只在main_process的地方进行infer

for _, (image, target) in enumerate(train_loader):
    accelerator.wait_for_everyone()
    model.train()
    ...
    # model eval
    model.eval()
    if accelerator.is_main_process:
        # Single GPU evaluation. Don't use accelerate calls