全局管理单例类Caffe

Caffe单例子

Caffe采用单例模式(singleton)为每个线程保存相关的变量和句柄.
在不同线程下的Caffe对象可以通过Get()函数进行访问.

static boost::thread_specific_ptr<Caffe> thread_instance_;
Caffe& Caffe::Get() {
  if (!thread_instance_.get()) {
    thread_instance_.reset(new Caffe());
  }
  return *(thread_instance_.get());
}

依照编译模式的不同可以将程序归纳为两种运行模式:
1. CPU_ONLY 模式
2. CPU + GPU 模式

第一种模式仅仅使用CPU作计算,因此不涉及cuda,第二种模式涉及到GPU编程.因此两种模式下类的定义和实现会有所不同.

数据成员

Caffe类有4或6个数据成员:

class Caffe {
public:
  enum Brew { CPU, GPU };
protected:
#ifndef CPU_ONLY
  cublasHandle_t cublas_handle_;
  curandGenerator_t curand_generator_;
#endif
  shared_ptr<RNG> random_generator_;
  Brew mode_;
  int solver_count_;
  bool root_solver_;
};

在GPU+CPU模式下定义了有两个额外的成员:
* cublasHandle_t : 指向cuBLAS库环境的指针.cuBLAS库相关信息需要通过cublasCreate()来创建,cublasDestroy()来销毁.
* curandGenerator_t : 用于生成相应的随机数指针.

其他共有的数据成员:
* Brew mode_:程序运行的模式,GPU或者CPU.
* shared_ptr<RNG> random_generator_ : 用于产生随机数
* int solver_count_ : 并行训练器的数目
* bool root_solver_ : 主训练器

Getters 和 Setters

Caffe有如下几个访问和设置数据成员的函数.

inline static RNG& rng_stream() {
  if (!Get().random_generator_) {
    Get().random_generator_.reset(new RNG());
  }
  return *(Get().random_generator_);
}
#ifndef CPU_ONLY
  inline static cublasHandle_t cublas_handle() { return Get().cublas_handle_; }
  inline static curandGenerator_t curand_generator() {
    return Get().curand_generator_;
  }
#endif
inline static Brew mode() { return Get().mode_; }
inline static void set_mode(Brew mode) { Get().mode_ = mode; }
static void set_random_seed(const unsigned int seed);
static void SetDevice(const int device_id);
static void DeviceQuery();
static bool CheckDevice(const int device_id);
static int FindDevice(const int start_id = 0);
inline static int solver_count() { return Get().solver_count_; }
inline static void set_solver_count(int val) { Get().solver_count_ = val; }
inline static bool root_solver() { return Get().root_solver_; }
inline static void set_root_solver(bool val) { Get().root_solver_ = val; }

同样,这些函数的实现也依赖于程序的编译模式.

随机数生成

随机数是Caffe对象要管理的主要内容之一.Caffe类提供了一个产生随机数的统一接口,而这个借口实现依赖于多种随机数的生成方式.

通常,随机数的产生通过一组协作的类来实现: 随机数引擎(engines) 和 随机数分布(distribution).通常随机数引擎产生的随机数不能直接使用,需要通过分布类型的协作才能产生期望的随机值.随机数引擎(engines)类的实例化常常会用到随机数种子(seed),即 random_engines e(seed);

随机数种子产生函数 Caffe::cluster_seedgen

seed是产生随机数重要的参数之一, 函数cluster_seedgen利用Linux下的熵池文件(urandom)或者时间戳(熵池文件不可用时)来产生随机数种子.

int64_t cluster_seedgen(void) {
  int64_t s, seed, pid;
  FILE* f = fopen("/dev/urandom", "rb");
  if (f && fread(&seed, 1, sizeof(seed), f) == sizeof(seed)) {
    fclose(f);
    return seed;
  }
  LOG(INFO) << "System entropy source not available, "
              "using fallback algorithm to generate seed instead.";
  if (f)
    fclose(f);
  pid = getpid();
  s = time(NULL);
  seed = std::abs(((s * 181) * ((pid - 83) * 359)) % 104729);
  return seed;
}

随机数引擎:RNG嵌套类

在Caffe类中定义了RNG类用做于随机数生成器, RNG的初始化依赖随机数种子seed.
类RNG的定义如下:

class RNG {
public:
  RNG();
  explicit RNG(unsigned int seed);
  explicit RNG(const RNG&);
  RNG& operator=(const RNG&);
  void* generator();
private:
  class Generator;
  shared_ptr<Generator> generator_;
};

Class Generator是RNG的嵌套类. 它是一个随机数引擎.定义为:

typedef boost::mt19937 rng_t;
class Caffe::RNG::Generator {
public:
  Generator() : rng_(new caffe::rng_t(cluster_seedgen())) {}
  explicit Generator(unsigned int seed) : rng_(new caffe::rng_t(seed)) {}
  caffe::rng_t* rng() { return rng_.get(); }
private:
  shared_ptr<caffe::rng_t> rng_;
};

Generator的构造函数通过cluster_seedgen()产生随机种子(random seeding)或给定seed产生一个boost::mt19937对象.
RNG的实现由是否定义CPU_ONLY而存在差别.

1. 在CPU_ONLY模式下的实现:

Caffe::RNG::RNG() : generator_(new Generator()) { }
Caffe::RNG::RNG(unsigned int seed) : generator_(new Generator(seed)) { }
void* Caffe::RNG::generator() {
  return static_cast<void*>(generator_->rng());
}
Caffe::RNG& Caffe::RNG::operator=(const RNG& other) {
  generator_ = other.generator_;
  return *this;
}

2.　在 CPU + GPU 模式下的实现:

Caffe::RNG::RNG() : generator_(new Generator()) { }
Caffe::RNG::RNG(unsigned int seed) : generator_(new Generator(seed)) { }
Caffe::RNG& Caffe::RNG::operator=(const RNG& other) {
  generator_.reset(other.generator_.get());
  return *this;
}
void* Caffe::RNG::generator() {
  return static_cast<void*>(generator_->rng());
}

两者的不同之处仅在于赋值操作的实现:
- generator_ = other.generator_;
- generator_.reset(other.generator_.get());

Caffe::Caffe的构造函数和析构函数

不同模式下,显然会有不同的构造函数和西沟函数:

• CPU_ONLY 模式下

构造函数不涉及cuda实现的细节,随机数的实现采用boost rng.

Caffe::Caffe()
    : random_generator_(), mode_(Caffe::CPU),
      solver_count_(1), root_solver_(true) { }
Caffe::~Caffe() { }

• GPU + CPU 模式下

构造函数需要实现和curand,cublas handler相关初始化细节.随机数的产生依靠curand实现.

Caffe::Caffe()
    : cublas_handle_(NULL), curand_generator_(NULL), random_generator_(),
    mode_(Caffe::CPU), solver_count_(1), root_solver_(true) {
  if (cublasCreate(&cublas_handle_) != CUBLAS_STATUS_SUCCESS) {
    LOG(ERROR) << "Cannot create Cublas handle. Cublas won't be available.";
  }
  if (curandCreateGenerator(&curand_generator_, CURAND_RNG_PSEUDO_DEFAULT)
      != CURAND_STATUS_SUCCESS ||
      curandSetPseudoRandomGeneratorSeed(curand_generator_, cluster_seedgen())
      != CURAND_STATUS_SUCCESS) {
    LOG(ERROR) << "Cannot create Curand generator. Curand won't be available.";
  }
}
Caffe::~Caffe() {
  if (cublas_handle_) CUBLAS_CHECK(cublasDestroy(cublas_handle_));
  if (curand_generator_) {
    CURAND_CHECK(curandDestroyGenerator(curand_generator_));
  }
}

Aside curandGenerator_t:用于生成相应的随机数指针,通过调用
curandCreateGenerator(curandGenerator_t* generator, curandRngType_t rng_type)
来指定随机数生成算法.
通过函数
curandSetPseudoRandomGeneratorSeed(curandGenerator_t generator, unsigned long long seed)
来设定随机数的初始值(种子).
然后通过
curandGenerateUniform(curandGenerator_t generator, float* outputPtr, size_t num)
产生特定个数的随机数.
最后调用curandDestroyGenerotor释放随机数指针变量.

Aside cuBLAS context: cuBLAS库环境需要调用cublasCreate()函数来对句柄(handle)进行初始化.并且这个句柄需要传递给随后调用的cuBLAS库函数.当一个程序不再使用cuBLAS库环境,必须通过调用cublasDestory()函数来对这个context占用的资源进行释放.即在使用cuBLAS之前需要先申请相应的库资源,在使用完毕之后,释放该资源.这样做的好处在于,它能允许用户在使用多线程和多GPU的情况下,精确地控制每个库环境.例如cudaSetDevice()函数可以将不同的主机线程关联到不同的GPU设备上.在这种情况下,为每个线程可以为其关联的设备建立不同的cuBLAS库环境.同时,每个库环境有其特定的handle,所以通过不同的句柄,就可以将计算任务分发给不同的GPU设备.如果需要在同一个线程中,需要调用不同的GPU设备进行运算,那么可以首先需要通过cudaSetDevice()来指定GPU,然后通过cublasCreate()建立一个cuBLAS库环境.

Aside cublasCreate(cublasHandle_t *handle)
该函数用于初始化一个cuBLAS库环境.这个库环境绑定在当前的GPU设备上.如果需要使用多GPU设备,那么可以为每个设备建立一个cuBLAS库环境(多GPU).另外也可以为同一个设备建立多个不同配置的库环境(例如多线程,单GPU).

Aside cublasDestroy(cublasHandle_t handle)
该函数与cublasCreate()成对出现,用于释放建立的cuBLAS库环境.当不再使用该cuBLAS库环境时,调用cublasDestroy来释放资源.同时cublasDestroy会顺带调用cublasDeviceSynchronize来对设备进行同步,所以建议一个库环境应该尽可能长地使用,从而让cublasCreate/cublasDestroy调用的次数尽可能小.

访问数据成员

Caffe类管理一个线程下的相关信息.这些信息的访问通过简单的成员函数实现,即前面提到的Getters和Setters函数.

设置随机数种子

static void set_random_seed(const unsigned int seed);

• CPU_ONLY 模式

void Caffe::set_random_seed(const unsigned int seed) {
  // RNG seed
  Get().random_generator_.reset(new RNG(seed));
}

• GPU + CPU 模式

void Caffe::set_random_seed(const unsigned int seed) {
  // Curand seed
  static bool g_curand_availability_logged = false;
  if (Get().curand_generator_) {
    CURAND_CHECK(curandSetPseudoRandomGeneratorSeed(curand_generator(),
        seed));
    CURAND_CHECK(curandSetGeneratorOffset(curand_generator(), 0));
  } else {
    if (!g_curand_availability_logged) {
        LOG(ERROR) <<
            "Curand not available. Skipping setting the curand seed.";
        g_curand_availability_logged = true;
    }
  }
  // RNG seed
  Get().random_generator_.reset(new RNG(seed));
}

访问GPU设备

显然,在CPU_ONLY模式下,这些GPU设备将不存在,因此也无法访问.

• CPU_ONLY 模式

void Caffe::SetDevice(const int device_id) {
  NO_GPU;
}
void Caffe::DeviceQuery() {
  NO_GPU;
}
bool Caffe::CheckDevice(const int device_id) {
  NO_GPU;
  return false;
}
int Caffe::FindDevice(const int start_id) {
  NO_GPU;
  return -1;
}

• GPU + CPU 模式

常规模式下,GPU设备的访问可以通过cuda自带的库函数.

void Caffe::SetDevice(const int device_id) {
  int current_device;
  CUDA_CHECK(cudaGetDevice(&current_device));
  if (current_device == device_id) {
    return;
  }
  // The call to cudaSetDevice must come before any calls to Get, which
  // may perform initialization using the GPU.
  CUDA_CHECK(cudaSetDevice(device_id));
  if (Get().cublas_handle_) {
    CUBLAS_CHECK(cublasDestroy(Get().cublas_handle_));
  }
  if (Get().curand_generator_) {
    CURAND_CHECK(curandDestroyGenerator(Get().curand_generator_));
  }
  CUBLAS_CHECK(cublasCreate(&Get().cublas_handle_));
  CURAND_CHECK(curandCreateGenerator(&Get().curand_generator_,
      CURAND_RNG_PSEUDO_DEFAULT));
  CURAND_CHECK(curandSetPseudoRandomGeneratorSeed(Get().curand_generator_,
      cluster_seedgen()));
}
void Caffe::DeviceQuery() {
  cudaDeviceProp prop;
  int device;
  if (cudaSuccess != cudaGetDevice(&device)) {
    printf("No cuda device present.\n");
    return;
  }
  CUDA_CHECK(cudaGetDeviceProperties(&prop, device));
  LOG(INFO) << "Device id:                     " << device;
  LOG(INFO) << "Major revision number:         " << prop.major;
  LOG(INFO) << "Minor revision number:         " << prop.minor;
  LOG(INFO) << "Name:                          " << prop.name;
  LOG(INFO) << "Total global memory:           " << prop.totalGlobalMem;
  LOG(INFO) << "Total shared memory per block: " << prop.sharedMemPerBlock;
  LOG(INFO) << "Total registers per block:     " << prop.regsPerBlock;
  LOG(INFO) << "Warp size:                     " << prop.warpSize;
  LOG(INFO) << "Maximum memory pitch:          " << prop.memPitch;
  LOG(INFO) << "Maximum threads per block:     " << prop.maxThreadsPerBlock;
  LOG(INFO) << "Maximum dimension of block:    "
      << prop.maxThreadsDim[0] << ", " << prop.maxThreadsDim[1] << ", "
      << prop.maxThreadsDim[2];
  LOG(INFO) << "Maximum dimension of grid:     "
      << prop.maxGridSize[0] << ", " << prop.maxGridSize[1] << ", "
      << prop.maxGridSize[2];
  LOG(INFO) << "Clock rate:                    " << prop.clockRate;
  LOG(INFO) << "Total constant memory:         " << prop.totalConstMem;
  LOG(INFO) << "Texture alignment:             " << prop.textureAlignment;
  LOG(INFO) << "Concurrent copy and execution: "
      << (prop.deviceOverlap ? "Yes" : "No");
  LOG(INFO) << "Number of multiprocessors:     " << prop.multiProcessorCount;
  LOG(INFO) << "Kernel execution timeout:      "
      << (prop.kernelExecTimeoutEnabled ? "Yes" : "No");
  return;
}
bool Caffe::CheckDevice(const int device_id) {
  bool r = ((cudaSuccess == cudaSetDevice(device_id)) &&
            (cudaSuccess == cudaFree(0)));
  cudaGetLastError();
  return r;
}
int Caffe::FindDevice(const int start_id) {
  int count = 0;
  CUDA_CHECK(cudaGetDeviceCount(&count));
  for (int i = start_id; i < count; i++) {
    if (CheckDevice(i)) return i;
  }
  return -1;
}

cublasGetErrorString 和 curandGetErrorString

Caffe::cublasGetErrorString

const char* cublasGetErrorString(cublasStatus_t error) {
  switch (error) {
  case CUBLAS_STATUS_SUCCESS:
    return "CUBLAS_STATUS_SUCCESS";
  case CUBLAS_STATUS_NOT_INITIALIZED:
    return "CUBLAS_STATUS_NOT_INITIALIZED";
  case CUBLAS_STATUS_ALLOC_FAILED:
    return "CUBLAS_STATUS_ALLOC_FAILED";
  case CUBLAS_STATUS_INVALID_VALUE:
    return "CUBLAS_STATUS_INVALID_VALUE";
  case CUBLAS_STATUS_ARCH_MISMATCH:
    return "CUBLAS_STATUS_ARCH_MISMATCH";
  case CUBLAS_STATUS_MAPPING_ERROR:
    return "CUBLAS_STATUS_MAPPING_ERROR";
  case CUBLAS_STATUS_EXECUTION_FAILED:
    return "CUBLAS_STATUS_EXECUTION_FAILED";
  case CUBLAS_STATUS_INTERNAL_ERROR:
    return "CUBLAS_STATUS_INTERNAL_ERROR";
#if CUDA_VERSION >= 6000
  case CUBLAS_STATUS_NOT_SUPPORTED:
    return "CUBLAS_STATUS_NOT_SUPPORTED";
#endif
#if CUDA_VERSION >= 6050
  case CUBLAS_STATUS_LICENSE_ERROR:
    return "CUBLAS_STATUS_LICENSE_ERROR";
#endif
  }
  return "Unknown cublas status";
}

Caffe::curandGetErrorString

const char* curandGetErrorString(curandStatus_t error) {
  switch (error) {
  case CURAND_STATUS_SUCCESS:
    return "CURAND_STATUS_SUCCESS";
  case CURAND_STATUS_VERSION_MISMATCH:
    return "CURAND_STATUS_VERSION_MISMATCH";
  case CURAND_STATUS_NOT_INITIALIZED:
    return "CURAND_STATUS_NOT_INITIALIZED";
  case CURAND_STATUS_ALLOCATION_FAILED:
    return "CURAND_STATUS_ALLOCATION_FAILED";
  case CURAND_STATUS_TYPE_ERROR:
    return "CURAND_STATUS_TYPE_ERROR";
  case CURAND_STATUS_OUT_OF_RANGE:
    return "CURAND_STATUS_OUT_OF_RANGE";
  case CURAND_STATUS_LENGTH_NOT_MULTIPLE:
    return "CURAND_STATUS_LENGTH_NOT_MULTIPLE";
  case CURAND_STATUS_DOUBLE_PRECISION_REQUIRED:
    return "CURAND_STATUS_DOUBLE_PRECISION_REQUIRED";
  case CURAND_STATUS_LAUNCH_FAILURE:
    return "CURAND_STATUS_LAUNCH_FAILURE";
  case CURAND_STATUS_PREEXISTING_FAILURE:
    return "CURAND_STATUS_PREEXISTING_FAILURE";
  case CURAND_STATUS_INITIALIZATION_FAILED:
    return "CURAND_STATUS_INITIALIZATION_FAILED";
  case CURAND_STATUS_ARCH_MISMATCH:
    return "CURAND_STATUS_ARCH_MISMATCH";
  case CURAND_STATUS_INTERNAL_ERROR:
    return "CURAND_STATUS_INTERNAL_ERROR";
  }
  return "Unknown curand status";
}