Windows libtorch C++部署GPU版

libtorch C++ for cuda in Windows

---

环境准备

• python(模型转换)
• VS 2017
• CUDA/CUDNN

安装cuda

• 查看电脑cuda版本

• 下载对应版本CUDA Toolkit

  • 最新版本

    

  • 历史版本

• 添加环境变量

• 检查

nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Thu_Jun_11_22:26:48_Pacific_Daylight_Time_2020
Cuda compilation tools, release 11.0, V11.0.194
Build cuda_11.0_bu.relgpu_drvr445TC445_37.28540450_0

安装cudnn

• 下载地址
• 解压文件夹，将解压后的文件夹下的文件夹bin、include、lib拷贝到cuda安装目录下，与之相对应的文件夹

安装libtorch

• 下载地址
• cuda版本是11.0，低版本cuda没有办法安装高版本的libtorch，因此安装cuda10.2对应的安装包

VS编译

• Python模型转换

def save(net, input, save_path):
    net.eval()
    traced_script_module = torch.jit.trace(net, input)
    traced_script_module.save(save_path)

• 环境配置

  • C/C++——常规——附加包含目录

    

  • 链接器——常规——附加库目录

  

  • 链接器——输入——附加依赖项

• 编译

int main(int argc, char *argv[]) {

	bool running_gpu_mode = true;
	torch::DeviceType device_type;
	bool flag = torch::cuda::is_available();
	if (torch::cuda::is_available() && running_gpu_mode) {
		device_type = torch::kCUDA;
	}
	else {
		device_type = torch::kCPU;
	}
	torch::Device device(device_type);

	std::string model_path = "E:/Programms/Win PyTorch C++/model/netCRNN_gpu.pt";
	torch::jit::script::Module module = torch::jit::load(model_path);
	if (torch::cuda::is_available() && running_gpu_mode) {
		module.to(at::kCUDA);
	}

	cv::Mat img, img_trans;
	std::string img_path = "E:/Programms/Win PyTorch C++/data/1.jpg";
	img = cv::imread(img_path, -1);
	cv::cvtColor(img, img_trans, cv::COLOR_BGR2GRAY);
	cv::resize(img_trans, img_trans, cv::Size(140, 32));

	// Change the Image into Tensor for prediction
	torch::Tensor tensor_image = torch::from_blob(img_trans.data, { 32, 140, 1 }, torch::kByte);
	tensor_image = tensor_image.permute({ 2,0,1 });
	tensor_image = tensor_image.toType(torch::kFloat);
	// distribution between 0 and 1
	tensor_image = tensor_image.div(255);
	// normalize, value between -1 and 1
	tensor_image = tensor_image.sub(0.5);
	tensor_image = tensor_image.div(0.5);
	tensor_image = tensor_image.unsqueeze(0);
	//std::cout<<tensor_image<<std::endl;
	if (running_gpu_mode && torch::cuda::is_available()) {
		torch::Device device(device_type);
		tensor_image = tensor_image.to(device);
	}

	std::vector<torch::jit::IValue> inputs;
	inputs.push_back(tensor_image);
	at::Tensor result = module.forward({ tensor_image }).toTensor();
	auto res = result.max(2);
	auto conf = std::get<0>(res).squeeze(1);
	auto pred = std::get<1>(res).squeeze(1);
	int size = 0;
	int plateNum[10];
	float plateConf[10];
	for (int i = 0; i < 32; i++) {
		if (pred[i].item<int>() != 0 && !(i > 0 && (pred[i - 1].item<int>() == pred[i].item<int>()))) {

			plateNum[size] = pred[i].item<int>() - 1;
			plateConf[size] = conf[i].item<float>();
			std::cout << alphabet[pred[i].item<int>() - 1];
			size++;
		}
	}
	std::cout << std::endl;

	system("pause");

	return 0;
}

QA

• Torch::cuda::is_available()返回为False

链接器中 -> 命令行 -> 其他选项

/INCLUDE:?warp_size@cuda@at@@YAHXZ

• 0x00007FFEEE704ED9 处(位于 TorchGPU.exe 中)有未经处理的异常: Microsoft C++ 异常: c10::Error，位于内存位置 0x0000005E710FE0D0 处。

这个问题原因我看网上有很多种，我的是因为模型路径是相对路径，同样的Code，CPU版本可以，GPU版会报错，改成绝对路径就可以了。

参考链接

[1] Torch::cuda::is_available return false in libtorch 1.5

[2] C++部署Pytorch（Libtorch）出现问题、错误汇总