【关于torch中的多线程num_works设置相关问题】

【关于torch中的多线程num_works设置相关问题】

1、首先要搞明白的一点是，多线程是指CPU工作的线程数，现在的电脑CPU 一般都会有10个线程以上，如下图所示：

2、查看电脑的总线程

ctrl+shift+esc任务管理器，cpu界面->右键->将图形更改->逻辑处理器，有多少方块就有多少线程。

这里显示有20线程

3、接下来是如何在训练中找出最佳num_works数值

参考链接: Pytorch之DataLoader的num_works参数设置
这里是以Mnist数据集为例，BATCH_SIZE = 100，循环不同的num_works，计算一轮训练的耗时。（其他数据更换train_set、train_loader 的参数即可）

import time
import torch.utils.data as DataLoader
import torchvision
import torchvision.transforms as transforms
 
if __name__ == '__main__':
    BATCH_SIZE = 100
    transform = transforms.Compose([transforms.ToTensor(),
                                    transforms.Normalize((0.5,), (0.5,))])
    train_set = torchvision.datasets.MNIST('data', download=False, train=True, transform=transform)
    
    # data loaders
    train_loader = DataLoader.DataLoader(train_set, batch_size=BATCH_SIZE, shuffle=True)
    
    for num_workers in range(10):   # 查看总线程数，ctrl+shift+esc任务管理器，cpu界面->右键->将图形更改->逻辑处理器，有多少方块就有多少线程。
        train_loader = DataLoader.DataLoader(train_set, batch_size=BATCH_SIZE, shuffle=True, num_workers=num_workers)
        # training ...
        start = time.time()
        for epoch in range(1):
            for step, (batch_x, batch_y) in enumerate(train_loader):
                pass
        end = time.time()
        print('num_workers is {} and it took {} seconds'.format(num_workers, end - start))

输出结果如下：

num_workers is 0 and it took 3.3641223907470703 seconds
num_workers is 1 and it took 4.425052881240845 seconds
num_workers is 2 and it took 2.7799184322357178 seconds
num_workers is 3 and it took 2.262011766433716 seconds
num_workers is 4 and it took 2.056213855743408 seconds
num_workers is 5 and it took 1.9935059547424316 seconds
num_workers is 6 and it took 2.13820219039917 seconds
num_workers is 7 and it took 2.7871010303497314 seconds
num_workers is 8 and it took 3.03206205368042 seconds
num_workers is 9 and it took 3.1266887187957764 seconds

可以看出，BATCH_SIZE = 100，最佳的num_works = 5时间最短。