对 SuperNeurons 的复现记录

论文

[1801.04380] SuperNeurons: Dynamic GPU Memory Management for Training Deep Neural Networks

源码

GitHub - linnanwang/superneurons-release: this is the release repository of superneurons

环境

• Ubuntu 22.04
• GeForce RTX 2080 Ti
• CUDA v10.2
• cuDNN v7.6.5（不能用8.0以上版本）

编译

1、安装 libjpeg：Linux中安装libjpeg库(两种应用)

2、安装 gflags

git clone https://github.com/gflags/gflags.git
cd gflags/
mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DBUILD_SHARED_LIBS=ON -DBUILD_STATIC_LIBS=ON -DGFLAGS_NAMESPACE=google ..
make
sudo make install

3、安装 glog v0.3.5（不能用太高版本。std::exchange是在 C++14 中引入的，如果使用C++11，那么它就不是标准提供的。最开始以为是 C++ 版本的问题，后来想了下，是安装的 glog 版本太高了，源码里的 glog 支持的是 C++11，所以应该降低 glog 版本。）

# 下载：https://github.com/google/glog/releases/tag/v0.3.5
tar -xzf glog-0.3.5.tar.gz
cd glog-0.3.5/
mkdir build && cd build
../configure
make
sudo make install

4、下载源代码

git clone https://github.com/linnanwang/superneurons-release.git

5、修改 superneurons-release/config.linux

cd superneurons-release/
vi config.linux

修改后的内容如下（根据自己安装的位置）：

CUDA_ROOT_DIR=/usr/local/cuda
CUDNN_ROOT_DIR=/usr/local/cuda

GLOG_ROOT=/usr/local

6、编译安装

mkdir build && cd build
cmake ..
make -j8

测试

下载数据集 CIFAR-10，解压

tar -xzf cifar-10-binary.tar.gz

数据集路径：/home/username/dataset/cifar-10-batches-bin

创建文件夹superneurons-release/dataset/cifar，存放处理过的数据集。

cd build/tools/
./convert_cifar10 /home/username/dataset/cifar-10-batches-bin superneurons-release/dataset/cifar 1 1
./compute_image_mean superneurons-release/dataset/cifar/cifar10_train_image_0.bin superneurons-release/dataset/cifar/cifar_train.mean

修改文件superneurons-release/testing/cifar10.cpp中数据集的路径

//	train_mean_file = (char *) "/data/lwang53/dataset/cifar/cifar_train.mean";
//	train_image_bin = (char *) "/data/lwang53/dataset/cifar/cifar10_train_image_0.bin";
//	train_label_bin = (char *) "/data/lwang53/dataset/cifar/cifar10_train_label_0.bin";
//	test_image_bin  = (char *) "/data/lwang53/dataset/cifar/cifar10_test_image_0.bin";
//	test_label_bin  = (char *) "/data/lwang53/dataset/cifar/cifar10_test_label_0.bin";
    
train_mean_file = (char *) "../../dataset/cifar/cifar_train.mean";
train_image_bin = (char *) "../../dataset/cifar/cifar10_train_image_0.bin";
train_label_bin = (char *) "../../dataset/cifar/cifar10_train_label_0.bin";
test_image_bin  = (char *) "../../dataset/cifar/cifar10_test_image_0.bin";
test_label_bin  = (char *) "../../dataset/cifar/cifar10_test_label_0.bin";

在superneurons-release/build目录下

make
cd testing/
./cifar10

参考

• 关于./configure、cmake、make、make install 命令的介绍
• glog+gflag的安装
• Ubuntu20.04安装gflags
• undefined reference to ‘FlagRegisterer::FlagRegisterer’ · Issue #203

备忘

superneurons-release/scripts/config.linux暂时没有修改

CUDA_ROOT_DIR=/usr/local/cuda
CUDNN_ROOT_DIR=/usr/local/cuda

LIBJPEG_INCLUDE_DIR = /usr/local/include
LIBJPEG_LIB_FILE = /usr/local/lib/libjpeg.so

GLOG_ROOT=/usr/local

MNIST