PaddleDetection多卡训练配置(Linux环境)

PaddleDetection多卡训练配置(Linux环境)

1.查看当前设备显卡数量

nvidia-smi

2.设置可用显卡

假设从上一步查询到本机有4张显卡

# 设置4张卡可用
export CUDA_VISIBLE_DEVICES=0,1,2,3

3.执行多卡训练

选择设备0, 1, 2, 3

python -m paddle.distributed.launch --selected_gpus 0,1,2,3 tools/train_multi_machine.py -c configs/yolov3_mobilenet_v1_roadsign.yml

4.reader文件上batch_size与显卡的关系

使用TrainReader配置举例

TrainReader:
  inputs_def:
    fields: ['image', 'im_info', 'im_id', 'gt_bbox', 'gt_class', 'is_crowd', 'gt_mask']
  dataset:
    !COCODataSet
    image_dir: train2017
    anno_path: annotations/instances_train2017.json
    dataset_dir: dataset/coco
  sample_transforms:
  - !DecodeImage
    to_rgb: true
  - !RandomFlipImage
    prob: 0.5
    is_mask_flip: true
  - !NormalizeImage
    is_channel_first: false
    is_scale: true
    mean: [0.485,0.456,0.406]
    std: [0.229, 0.224,0.225]
  - !ResizeImage
    target_size: 800
    max_size: 1333
    interp: 1
    use_cv2: true
  - !Permute
    to_bgr: false
    channel_first: true
  batch_transforms:
  - !PadBatch
    pad_to_stride: 32
    use_padded_im_info: false
  # batch_size设置的是每张显卡的batch_size是多少
  # 假设4张卡，batch_size为1，训练等同于单卡batch_size为4训练。
  batch_size: 1
  shuffle: true
  worker_num: 2
  drop_last: false
  use_process: false