Torch中多GPU运行代码学习

Main.lua
	-- Copyright (c) 2014, Facebook, Inc.
	-- All rights reserved.
	--
	-- This source code is licensed under the BSD-style license found in the
	-- LICENSE file in the root directory of this source tree. An additional grant
	-- of patent rights can be found in the PATENTS file in the same directory.
	--
	require 'torch'
	require 'cutorch'
	require 'paths'
	require 'xlua'
	require 'optim'
	require 'nn'
	torch.setdefaulttensortype('torch.FloatTensor')
	local opts= paths.dofile('opts.lua')
	opt = opts.parse(arg)
	nClasses = opt.nClasses
	paths.dofile('util.lua')
	paths.dofile('model.lua')
	opt.imageSize = model.imageSizeor opt.imageSize
	opt.imageCrop = model.imageCropor opt.imageCrop
	print(opt)
	cutorch.setDevice(opt.GPU)-- by default, use GPU 1
	torch.manualSeed(opt.manualSeed)
	print('Saving everything to: ' .. opt.save)
	os.execute('mkdir -p ' .. opt.save)
	paths.dofile('data.lua')
	paths.dofile('train.lua')
	paths.dofile('test.lua')
	epoch = opt.epochNumber
	for i=1,opt.nEpochsdo
	train()
	test()
	epoch = epoch+ 1
	end

 

The training scripts come with several options which can be listed by running the script with the flag --help

th main.lua --help

To run the training, simply run main.lua By default, the script runs 1-GPU AlexNet with the CuDNN backend and 2 data-loader threads.

th main.lua -data [imagenet-folder with train and val folders]

For 2-GPU model parallel AlexNet + CuDNN, you can run it this way:

th main.lua -data [imagenet-folder with train and val folders] -nGPU 2 -backend cudnn -netType alexnet

Similarly, you can switch the backends to 'cunn' to use a different set of CUDA kernels.

You can also alternatively train OverFeat using this following command:

th main.lua -data [imagenet-folder with train and val folders] -netType overfeat

# multi-GPU overfeat (let's say 2-GPU)

th main.lua -data [imagenet-folder with train and val folders] -netType overfeat -nGPU 2