Pytorch distributed RuntimeError: Address already in use

Pytorch distributed RuntimeError: Address already in use

如果是使用pytorch distributed 单机多卡训练方式，出现该错误，非常好解决。

Traceback (most recent call last):
  File "main1.py", line 279, in <module>
    train(args, io,root)
  File "main1.py", line 53, in train
    torch.distributed.init_process_group('nccl', init_method='env://')
  File "/home/labpos/anaconda3/envs/ldr/lib/python3.6/site-packages/torch/distributed/distributed_c10d.py", line 400, in init_process_group
    store, rank, world_size = next(rendezvous(url))
  File "/home/labpos/anaconda3/envs/ldr/lib/python3.6/site-packages/torch/distributed/rendezvous.py", line 143, in _env_rendezvous_handler
    store = TCPStore(master_addr, master_port, world_size, start_daemon)
RuntimeError: Address already in use

在启动分布式训练时，加上端口号（任意）即可解决：

CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2  --master_port 29501  main.py