CUDA Environment Variables: CUDA_VISIBLE_DEVICES

1. To ensure that a GPU version TensorFlow process only runs on CPU.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os

os.environ["CUDA_VISIBLE_DEVICES"] = "-1"

import tensorflow as tf
from tensorflow.python.client import device_lib

print(device_lib.list_local_devices())

strong@foreverstrong:~/git_workspace/MonoGRNet$ python test.py 
2019-06-20 13:57:53.576570: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2019-06-20 13:57:53.579780: E tensorflow/stream_executor/cuda/cuda_driver.cc:406] failed call to cuInit: CUDA_ERROR_NO_DEVICE
2019-06-20 13:57:53.579819: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:158] retrieving CUDA diagnostic information for host: foreverstrong
2019-06-20 13:57:53.579826: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:165] hostname: foreverstrong
2019-06-20 13:57:53.579847: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:189] libcuda reported version is: 384.130.0
2019-06-20 13:57:53.579873: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:369] driver version file contents: """NVRM version: NVIDIA UNIX x86_64 Kernel Module  384.130  Wed Mar 21 03:37:26 PDT 2018
GCC version:  gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9) 
"""
2019-06-20 13:57:53.579898: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:193] kernel reported version is: 384.130.0
2019-06-20 13:57:53.579904: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:300] kernel version seems to match DSO: 384.130.0
[name: "/device:CPU:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 5213009322066253468
]
strong@foreverstrong:~/git_workspace/MonoGRNet$

2. CUDA C++ Programming Guide - CUDA Environment Variables

https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html

Variable:
CUDA_VISIBLE_DEVICES

Values:
A comma-separated sequence of GPU identifiers
逗号分隔的 GPU 标识符序列

Description:
GPU identifiers are given as integer indices or as UUID strings. GPU UUID strings should follow the same format as given by nvidia-smi, such as GPU-8932f937-d72c-4106-c12f-20bd9faed9f6. However, for convenience, abbreviated forms are allowed; simply specify enough digits from the beginning of the GPU UUID to uniquely identify that GPU in the target system. For example, CUDA_VISIBLE_DEVICES=GPU-8932f937 may be a valid way to refer to the above GPU UUID, assuming no other GPU in the system shares this prefix.
GPU 标识符以整数索引或 UUID 字符串的形式给出。GPU UUID 字符串应遵循与 nvidia-smi 相同的格式，例如 GPU-8932f937-d72c-4106-c12f-20bd9faed9f6。为方便起见，允许使用缩写形式。只需从 GPU UUID 的开头指定足够的数字，即可唯一标识目标系统中的 GPU。例如，假设系统中没有其他 GPU 共享此前缀，则 CUDA_VISIBLE_DEVICES = GPU-8932f937 可能是引用上述 GPU UUID 的有效方式。

Only the devices whose index is present in the sequence are visible to CUDA applications and they are enumerated in the order of the sequence. If one of the indices is invalid, only the devices whose index precedes the invalid index are visible to CUDA applications. For example, setting CUDA_VISIBLE_DEVICES to 2,1 causes device 0 to be invisible and device 2 to be enumerated before device 1. Setting CUDA_VISIBLE_DEVICES to 0,2,-1,1 causes devices 0 and 2 to be visible and device 1 to be invisible.
只有序列中存在索引的设备对 CUDA 应用程序可见，并且它们按序列的顺序枚举。如果其中一个索引无效，则只有索引位于无效索引之前的设备才对 CUDA 应用程序可见。例如，将 CUDA_VISIBLE_DEVICES 设置为 2,1 会导致设备 0 不可见，设备 2 将在设备 1 之前枚举。将 CUDA_VISIBLE_DEVICES 设置为 0,2,-1,1 会导致设备0 和 2 可见，设备 1 不可见。

CUDA_VISIBLE_DEVICES=1
只有编号为 1 的 GPU 对程序是可见的，在代码中 gpu[0] 指的就是这块 GPU。
CUDA_VISIBLE_DEVICES=0,2,3
只有编号为 0,2,3 的 GPU 对程序是可见的，在代码中 gpu[0] 指的是第 0 块，gpu[1] 指的是第 2 块，gpu[2] 指的是第 3 块。
CUDA_VISIBLE_DEVICES=2,0,3
只有编号为 0,2,3 的 GPU 对程序是可见的，在代码中 gpu[0] 指的是第 2 块，gpu[1] 指的是第 0 块，gpu[2] 指的是第 3 块。

Python 中使用示例，注意赋值为字符串，不是整型。

import os
os.environ["CUDA_VISIBLE_DEVICES"] = "0,1,2,3"
import tensorflow as tf

References

[1] Yongqiang Cheng, https://yongqiang.blog.csdn.net/