全案超详细LightBGM-GPU 安装

笔者今天进行lightBGM的使用, 全网检索, 没有一个好用的, 不多团队的力量就是强大, 经过几个消失的探索, 终于整理出了一份GPU版的安装教程, 另付测试代码

lightgbm GPU源码编译

可以官网查看

git clone --recursive https://github.com/microsoft/LightGBM ; cd LightGBM
mkdir build ; cd build
# if you have installed NVIDIA CUDA to a customized location, you should specify paths to OpenCL headers and library like the following:
cmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda/include/ ..
# 官网用的下面的命令，咱们在此就弃用下面的改为上面指定路径去执行
# cmake -DUSE_GPU=1 ..
make -j4

这需要注意, Submodule path 'compute': checked out '36c89134d4013b2e5e45bc55656a18bd6141995a', 不用处理, 继续进行

依赖安装

sudo apt-get install --no-install-recommends git cmake build-essential libboost-dev libboost-system-dev libboost-filesystem-dev 
pip install setuptools wheel numpy scipy scikit-learn -U

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-0GjoPUih-1592057337002)(assets/image-20200613213643510.png)]

出现如下错误, 对错误包进行修改版本

pip install setuptools wheel numpy scipy scikit-learn -U --user

pip uninstall graphviz
pip install graphviz==0.8.2 --user

pip uninstall protobuf
pip install protobuf==3.7.0 --user

pip uninstall typing
pip install typing==3.6.6 --user

最后安装前端

sudo pip install lightgbm --install-option=--gpu --install-option="--opencl-include-dir=/usr/local/cuda/include/" --install-option="--opencl-library=/usr/local/cuda/lib64/libOpenCL.so"

链接OpenCL和python包

# 终端手动添加路径文件
mkdir -p /etc/OpenCL/vendors && echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd

# 这里可能会出现使用管理员权限, 可通过密码进入root
sudo passwd

测试安装代码一可以在这里找到

下面给出完整的安装流程
https://github.com/DragonYong/Jun–Dragon/blob/master/lightBGM_test.ipynb

mist@C-000058-GPU:~$ git clone --recursive https://github.com/microsoft/LightGBM ; cd LightGBM

Cloning into 'LightGBM'...

remote: Enumerating objects: 8, done.

remote: Counting objects: 100% (8/8), done.

remote: Compressing objects: 100% (7/7), done.

remote: Total 18109 (delta 1), reused 4 (delta 1), pack-reused 18101

Receiving objects: 100% (18109/18109), 12.15 MiB | 2.95 MiB/s, done.

Resolving deltas: 100% (13219/13219), done.

Submodule 'include/boost/compute' (https://github.com/boostorg/compute) registered for path 'compute'

Cloning into '/home/mist/LightGBM/compute'...

remote: Enumerating objects: 21728, done.     

remote: Total 21728 (delta 0), reused 0 (delta 0), pack-reused 21728     

Receiving objects: 100% (21728/21728), 8.51 MiB | 556.00 KiB/s, done.

Resolving deltas: 100% (17565/17565), done.

Submodule path 'compute': checked out '36c89134d4013b2e5e45bc55656a18bd6141995a'

mist@C-000058-GPU:~/LightGBM$ mkdir build ; cd build

mist@C-000058-GPU:~/LightGBM/build$ cmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda/include/ ..

-- The C compiler identification is GNU 7.4.0

-- The CXX compiler identification is GNU 7.4.0

-- Check for working C compiler: /usr/bin/cc

-- Check for working C compiler: /usr/bin/cc -- works

-- Detecting C compiler ABI info

-- Detecting C compiler ABI info - done

-- Detecting C compile features

-- Detecting C compile features - done

-- Check for working CXX compiler: /usr/bin/c++

-- Check for working CXX compiler: /usr/bin/c++ -- works

-- Detecting CXX compiler ABI info

-- Detecting CXX compiler ABI info - done

-- Detecting CXX compile features

-- Detecting CXX compile features - done

-- Found OpenMP_C: -fopenmp (found version "4.5") 

-- Found OpenMP_CXX: -fopenmp (found version "4.5") 

-- Found OpenMP: TRUE (found version "4.5")  

-- Looking for CL_VERSION_2_2

-- Looking for CL_VERSION_2_2 - not found

-- Looking for CL_VERSION_2_1

-- Looking for CL_VERSION_2_1 - not found

-- Looking for CL_VERSION_2_0

-- Looking for CL_VERSION_2_0 - not found

-- Looking for CL_VERSION_1_2

-- Looking for CL_VERSION_1_2 - found

-- Found OpenCL: /usr/local/cuda/lib64/libOpenCL.so (found version "1.2") 

-- OpenCL include directory: /usr/local/cuda/include

-- Found Boost: /usr/local/include (found suitable version "1.69.0", minimum required is "1.56.0") found components: filesystem system 

-- Performing Test MM_PREFETCH

-- Performing Test MM_PREFETCH - Success

-- Using _mm_prefetch

-- Performing Test MM_MALLOC

-- Performing Test MM_MALLOC - Success

-- Using _mm_malloc

-- Configuring done

-- Generating done

-- Build files have been written to: /home/mist/LightGBM/build

mist@C-000058-GPU:~/LightGBM/build$ make -j4

Scanning dependencies of target lightgbm

Scanning dependencies of target _lightgbm

[ 3%] Building CXX object CMakeFiles/lightgbm.dir/src/main.cpp.o

[ 3%] Building CXX object CMakeFiles/lightgbm.dir/src/application/application.cpp.o

[ 4%] Building CXX object CMakeFiles/_lightgbm.dir/src/application/application.cpp.o

[ 6%] Building CXX object CMakeFiles/lightgbm.dir/src/boosting/boosting.cpp.o

[ 8%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/boosting.cpp.o

[ 9%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/gbdt.cpp.o

[ 11%] Building CXX object CMakeFiles/lightgbm.dir/src/boosting/gbdt.cpp.o

[ 12%] Building CXX object CMakeFiles/lightgbm.dir/src/boosting/gbdt_model_text.cpp.o

[ 14%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/gbdt_model_text.cpp.o

[ 16%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/gbdt_prediction.cpp.o

[ 17%] Building CXX object CMakeFiles/lightgbm.dir/src/boosting/gbdt_prediction.cpp.o

[ 19%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/prediction_early_stop.cpp.o

[ 20%] Building CXX object CMakeFiles/lightgbm.dir/src/boosting/prediction_early_stop.cpp.o

[ 22%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/bin.cpp.o

[ 24%] Building CXX object CMakeFiles/lightgbm.dir/src/io/bin.cpp.o

[ 25%] Building CXX object CMakeFiles/lightgbm.dir/src/io/config.cpp.o

[ 27%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/config.cpp.o

[ 29%] Building CXX object CMakeFiles/lightgbm.dir/src/io/config_auto.cpp.o

[ 30%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/config_auto.cpp.o

[ 32%] Building CXX object CMakeFiles/lightgbm.dir/src/io/dataset.cpp.o

[ 33%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/dataset.cpp.o

[ 35%] Building CXX object CMakeFiles/lightgbm.dir/src/io/dataset_loader.cpp.o

[ 37%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/dataset_loader.cpp.o

[ 38%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/file_io.cpp.o

[ 40%] Building CXX object CMakeFiles/lightgbm.dir/src/io/file_io.cpp.o

[ 41%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/json11.cpp.o

[ 43%] Building CXX object CMakeFiles/lightgbm.dir/src/io/json11.cpp.o

[ 45%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/metadata.cpp.o

[ 46%] Building CXX object CMakeFiles/lightgbm.dir/src/io/metadata.cpp.o

[ 48%] Building CXX object CMakeFiles/lightgbm.dir/src/io/parser.cpp.o

[ 50%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/parser.cpp.o

[ 51%] Building CXX object CMakeFiles/lightgbm.dir/src/io/tree.cpp.o

[ 53%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/tree.cpp.o

[ 54%] Building CXX object CMakeFiles/lightgbm.dir/src/metric/dcg_calculator.cpp.o

[ 56%] Building CXX object CMakeFiles/_lightgbm.dir/src/metric/dcg_calculator.cpp.o

[ 58%] Building CXX object CMakeFiles/lightgbm.dir/src/metric/metric.cpp.o

[ 59%] Building CXX object CMakeFiles/_lightgbm.dir/src/metric/metric.cpp.o

[ 61%] Building CXX object CMakeFiles/lightgbm.dir/src/network/linker_topo.cpp.o

[ 62%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/linker_topo.cpp.o

[ 64%] Building CXX object CMakeFiles/lightgbm.dir/src/network/linkers_mpi.cpp.o

[ 66%] Building CXX object CMakeFiles/lightgbm.dir/src/network/linkers_socket.cpp.o

[ 67%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/linkers_mpi.cpp.o

[ 69%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/linkers_socket.cpp.o

[ 70%] Building CXX object CMakeFiles/lightgbm.dir/src/network/network.cpp.o

[ 72%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/network.cpp.o

[ 74%] Building CXX object CMakeFiles/lightgbm.dir/src/objective/objective_function.cpp.o

[ 75%] Building CXX object CMakeFiles/_lightgbm.dir/src/objective/objective_function.cpp.o

[ 77%] Building CXX object CMakeFiles/lightgbm.dir/src/treelearner/data_parallel_tree_learner.cpp.o

[ 79%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/data_parallel_tree_learner.cpp.o

[ 80%] Building CXX object CMakeFiles/lightgbm.dir/src/treelearner/feature_parallel_tree_learner.cpp.o

[ 82%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/feature_parallel_tree_learner.cpp.o

[ 83%] Building CXX object CMakeFiles/lightgbm.dir/src/treelearner/gpu_tree_learner.cpp.o

[ 85%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/gpu_tree_learner.cpp.o

[ 87%] Building CXX object CMakeFiles/lightgbm.dir/src/treelearner/serial_tree_learner.cpp.o

[ 88%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/serial_tree_learner.cpp.o

[ 90%] Building CXX object CMakeFiles/lightgbm.dir/src/treelearner/tree_learner.cpp.o

[ 91%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/tree_learner.cpp.o

[ 93%] Building CXX object CMakeFiles/lightgbm.dir/src/treelearner/voting_parallel_tree_learner.cpp.o

[ 95%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/voting_parallel_tree_learner.cpp.o

[ 96%] Building CXX object CMakeFiles/_lightgbm.dir/src/c_api.cpp.o

[ 98%] Linking CXX executable ../lightgbm

[100%] Linking CXX shared library ../lib_lightgbm.so

[100%] Built target lightgbm

[100%] Built target _lightgbm

mist@C-000058-GPU:~/LightGBM/build$ sudo apt-get install --no-install-recommends git cmake build-essential libboost-dev libboost-system-dev libboost-filesystem-dev 

Reading package lists... Done

Building dependency tree    

Reading state information... Done

build-essential is already the newest version (12.4ubuntu1).

git is already the newest version (1:2.17.1-1ubuntu0.5).

The following additional packages will be installed:

 cmake-data libarchive13 libboost-filesystem1.65-dev

 libboost-filesystem1.65.1 libboost-system1.65-dev libboost-system1.65.1

 libboost1.65-dev libjsoncpp1 liblzo2-2 librhash0 libuv1

Suggested packages:

 cmake-doc ninja-build lrzip libboost-doc libboost1.65-doc

 libboost-atomic1.65-dev libboost-chrono1.65-dev libboost-container1.65-dev

 libboost-context1.65-dev libboost-coroutine1.65-dev

 libboost-date-time1.65-dev libboost-exception1.65-dev libboost-fiber1.65-dev

 libboost-graph1.65-dev libboost-graph-parallel1.65-dev

 libboost-iostreams1.65-dev libboost-locale1.65-dev libboost-log1.65-dev

 libboost-math1.65-dev libboost-mpi1.65-dev libboost-mpi-python1.65-dev

 libboost-numpy1.65-dev libboost-program-options1.65-dev

 libboost-python1.65-dev libboost-random1.65-dev libboost-regex1.65-dev

 libboost-serialization1.65-dev libboost-signals1.65-dev

 libboost-stacktrace1.65-dev libboost-test1.65-dev libboost-thread1.65-dev

 libboost-timer1.65-dev libboost-type-erasure1.65-dev libboost-wave1.65-dev

 libboost1.65-tools-dev libmpfrc++-dev libntl-dev

The following NEW packages will be installed:

 cmake cmake-data libarchive13 libboost-dev libboost-filesystem-dev

 libboost-filesystem1.65-dev libboost-filesystem1.65.1 libboost-system-dev

 libboost-system1.65-dev libboost-system1.65.1 libboost1.65-dev libjsoncpp1

 liblzo2-2 librhash0 libuv1

0 upgraded, 15 newly installed, 0 to remove and 61 not upgraded.

Need to get 12.4 MB of archives.

After this operation, 145 MB of additional disk space will be used.

Do you want to continue? [Y/n] y

Get:1 https://mirrors.tuna.tsinghua.edu.cn/ubuntu bionic/main amd64 liblzo2-2 amd64 2.08-1.2 [48.7 kB]

Get:2 https://mirrors.tuna.tsinghua.edu.cn/ubuntu bionic-updates/main amd64 cmake-data all 3.10.2-1ubuntu2.18.04.1 [1332 kB]

Get:3 https://mirrors.tuna.tsinghua.edu.cn/ubuntu bionic-updates/main amd64 libarchive13 amd64 3.2.2-3.1ubuntu0.6 [288 kB]

Get:4 https://mirrors.tuna.tsinghua.edu.cn/ubuntu bionic/main amd64 libjsoncpp1 amd64 1.7.4-3 [73.6 kB]

Get:5 https://mirrors.tuna.tsinghua.edu.cn/ubuntu bionic/main amd64 librhash0 amd64 1.3.6-2 [78.1 kB]

Get:6 https://mirrors.tuna.tsinghua.edu.cn/ubuntu bionic/main amd64 libuv1 amd64 1.18.0-3 [64.4 kB]

Get:7 https://mirrors.tuna.tsinghua.edu.cn/ubuntu bionic-updates/main amd64 cmake amd64 3.10.2-1ubuntu2.18.04.1 [3152 kB]

Get:8 https://mirrors.tuna.tsinghua.edu.cn/ubuntu bionic/main amd64 libboost1.65-dev amd64 1.65.1+dfsg-0ubuntu5 [7218 kB]

Get:9 https://mirrors.tuna.tsinghua.edu.cn/ubuntu bionic/main amd64 libboost-dev amd64 1.65.1.0ubuntu1 [3128 B]

Get:10 https://mirrors.tuna.tsinghua.edu.cn/ubuntu bionic/main amd64 libboost-system1.65.1 amd64 1.65.1+dfsg-0ubuntu5 [10.5 kB]

Get:11 https://mirrors.tuna.tsinghua.edu.cn/ubuntu bionic/main amd64 libboost-filesystem1.65.1 amd64 1.65.1+dfsg-0ubuntu5 [40.3 kB]

Get:12 https://mirrors.tuna.tsinghua.edu.cn/ubuntu bionic/main amd64 libboost-system1.65-dev amd64 1.65.1+dfsg-0ubuntu5 [11.8 kB]

Get:13 https://mirrors.tuna.tsinghua.edu.cn/ubuntu bionic/main amd64 libboost-filesystem1.65-dev amd64 1.65.1+dfsg-0ubuntu5 [52.1 kB]

Get:14 https://mirrors.tuna.tsinghua.edu.cn/ubuntu bionic/main amd64 libboost-filesystem-dev amd64 1.65.1.0ubuntu1 [2960 B]

Get:15 https://mirrors.tuna.tsinghua.edu.cn/ubuntu bionic/main amd64 libboost-system-dev amd64 1.65.1.0ubuntu1 [3060 B]

Fetched 12.4 MB in 1s (11.4 MB/s)        

debconf: unable to initialize frontend: Dialog

debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 15.)

debconf: falling back to frontend: Readline

Selecting previously unselected package liblzo2-2:amd64.

(Reading database ... 26005 files and directories currently installed.)

Preparing to unpack .../00-liblzo2-2_2.08-1.2_amd64.deb ...

Unpacking liblzo2-2:amd64 (2.08-1.2) ...

Selecting previously unselected package cmake-data.

Preparing to unpack .../01-cmake-data_3.10.2-1ubuntu2.18.04.1_all.deb ...

Unpacking cmake-data (3.10.2-1ubuntu2.18.04.1) ...

Selecting previously unselected package libarchive13:amd64.

Preparing to unpack .../02-libarchive13_3.2.2-3.1ubuntu0.6_amd64.deb ...

Unpacking libarchive13:amd64 (3.2.2-3.1ubuntu0.6) ...

Selecting previously unselected package libjsoncpp1:amd64.

Preparing to unpack .../03-libjsoncpp1_1.7.4-3_amd64.deb ...

Unpacking libjsoncpp1:amd64 (1.7.4-3) ...

Selecting previously unselected package librhash0:amd64.

Preparing to unpack .../04-librhash0_1.3.6-2_amd64.deb ...

Unpacking librhash0:amd64 (1.3.6-2) ...

Selecting previously unselected package libuv1:amd64.

Preparing to unpack .../05-libuv1_1.18.0-3_amd64.deb ...

Unpacking libuv1:amd64 (1.18.0-3) ...

Selecting previously unselected package cmake.

Preparing to unpack .../06-cmake_3.10.2-1ubuntu2.18.04.1_amd64.deb ...

Unpacking cmake (3.10.2-1ubuntu2.18.04.1) ...

Selecting previously unselected package libboost1.65-dev:amd64.

Preparing to unpack .../07-libboost1.65-dev_1.65.1+dfsg-0ubuntu5_amd64.deb ...

Unpacking libboost1.65-dev:amd64 (1.65.1+dfsg-0ubuntu5) ...

Selecting previously unselected package libboost-dev:amd64.

Preparing to unpack .../08-libboost-dev_1.65.1.0ubuntu1_amd64.deb ...

Unpacking libboost-dev:amd64 (1.65.1.0ubuntu1) ...

Selecting previously unselected package libboost-system1.65.1:amd64.

Preparing to unpack .../09-libboost-system1.65.1_1.65.1+dfsg-0ubuntu5_amd64.deb ...

Unpacking libboost-system1.65.1:amd64 (1.65.1+dfsg-0ubuntu5) ...

Selecting previously unselected package libboost-filesystem1.65.1:amd64.

Preparing to unpack .../10-libboost-filesystem1.65.1_1.65.1+dfsg-0ubuntu5_amd64.deb ...

Unpacking libboost-filesystem1.65.1:amd64 (1.65.1+dfsg-0ubuntu5) ...

Selecting previously unselected package libboost-system1.65-dev:amd64.

Preparing to unpack .../11-libboost-system1.65-dev_1.65.1+dfsg-0ubuntu5_amd64.deb ...

Unpacking libboost-system1.65-dev:amd64 (1.65.1+dfsg-0ubuntu5) ...

Selecting previously unselected package libboost-filesystem1.65-dev:amd64.

Preparing to unpack .../12-libboost-filesystem1.65-dev_1.65.1+dfsg-0ubuntu5_amd64.deb ...

Unpacking libboost-filesystem1.65-dev:amd64 (1.65.1+dfsg-0ubuntu5) ...

Selecting previously unselected package libboost-filesystem-dev:amd64.

Preparing to unpack .../13-libboost-filesystem-dev_1.65.1.0ubuntu1_amd64.deb ...

Unpacking libboost-filesystem-dev:amd64 (1.65.1.0ubuntu1) ...

Selecting previously unselected package libboost-system-dev:amd64.

Preparing to unpack .../14-libboost-system-dev_1.65.1.0ubuntu1_amd64.deb ...

Unpacking libboost-system-dev:amd64 (1.65.1.0ubuntu1) ...

Setting up libuv1:amd64 (1.18.0-3) ...

Setting up cmake-data (3.10.2-1ubuntu2.18.04.1) ...

Setting up librhash0:amd64 (1.3.6-2) ...

Setting up libboost-system1.65.1:amd64 (1.65.1+dfsg-0ubuntu5) ...

Processing triggers for libc-bin (2.27-3ubuntu1) ...

Setting up libboost1.65-dev:amd64 (1.65.1+dfsg-0ubuntu5) ...

Setting up liblzo2-2:amd64 (2.08-1.2) ...

Setting up libjsoncpp1:amd64 (1.7.4-3) ...

Setting up libarchive13:amd64 (3.2.2-3.1ubuntu0.6) ...

Setting up libboost-filesystem1.65.1:amd64 (1.65.1+dfsg-0ubuntu5) ...

Setting up libboost-system1.65-dev:amd64 (1.65.1+dfsg-0ubuntu5) ...

Setting up libboost-dev:amd64 (1.65.1.0ubuntu1) ...

Setting up cmake (3.10.2-1ubuntu2.18.04.1) ...

Setting up libboost-system-dev:amd64 (1.65.1.0ubuntu1) ...

Setting up libboost-filesystem1.65-dev:amd64 (1.65.1+dfsg-0ubuntu5) ...

Setting up libboost-filesystem-dev:amd64 (1.65.1.0ubuntu1) ...

Processing triggers for libc-bin (2.27-3ubuntu1) ...

mist@C-000058-GPU:~/LightGBM/build$ pip install setuptools wheel numpy scipy scikit-learn -U

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple

Collecting setuptools

 Downloading https://pypi.tuna.tsinghua.edu.cn/packages/95/95/f657b6e17f00c3f35b5f68b10e46c3a43af353d8856bd57bfcfb1dbb3e92/setuptools-47.1.1-py3-none-any.whl (583kB)

   |████████████████████████████████| 583kB 7.4MB/s 

Collecting wheel

 Downloading https://pypi.tuna.tsinghua.edu.cn/packages/8c/23/848298cccf8e40f5bbb59009b32848a4c38f4e7f3364297ab3c3e2e2cd14/wheel-0.34.2-py2.py3-none-any.whl

Collecting numpy

 Downloading https://pypi.tuna.tsinghua.edu.cn/packages/b3/a9/b1bc4c935ed063766bce7d3e8c7b20bd52e515ff1c732b02caacf7918e5a/numpy-1.18.5-cp36-cp36m-manylinux1_x86_64.whl (20.1MB)

   |████████████████████████████████| 20.1MB 57.8MB/s 

Collecting scipy

 Downloading https://pypi.tuna.tsinghua.edu.cn/packages/dc/29/162476fd44203116e7980cfbd9352eef9db37c49445d1fec35509022f6aa/scipy-1.4.1-cp36-cp36m-manylinux1_x86_64.whl (26.1MB)

   |████████████████████████████████| 26.1MB 410kB/s 

Collecting scikit-learn

 Downloading https://pypi.tuna.tsinghua.edu.cn/packages/d9/3a/eb8d7bbe28f4787d140bb9df685b7d5bf6115c0e2a969def4027144e98b6/scikit_learn-0.23.1-cp36-cp36m-manylinux1_x86_64.whl (6.8MB)

   |████████████████████████████████| 6.9MB 53.7MB/s 

Requirement already satisfied, skipping upgrade: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn) (0.14.0)

Collecting threadpoolctl>=2.0.0

 Downloading https://pypi.tuna.tsinghua.edu.cn/packages/f7/12/ec3f2e203afa394a149911729357aa48affc59c20e2c1c8297a60f33f133/threadpoolctl-2.1.0-py3-none-any.whl

ERROR: mxnet-cu100 1.5.1.post0 has requirement graphviz<0.9.0,>=0.8.1, but you'll have graphviz 0.13 which is incompatible.

ERROR: chainer 6.4.0 has requirement protobuf<3.8.0rc1,>=3.0.0, but you'll have protobuf 3.10.0 which is incompatible.

ERROR: chainer 6.4.0 has requirement typing<=3.6.6, but you'll have typing 3.7.4.1 which is incompatible.

Installing collected packages: setuptools, wheel, numpy, scipy, threadpoolctl, scikit-learn

 Found existing installation: setuptools 41.4.0

  Uninstalling setuptools-41.4.0:

ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/bin/easy_install'

Consider using the `--user` option or check the permissions.



WARNING: You are using pip version 19.3.1; however, version 20.1.1 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

mist@C-000058-GPU:~/LightGBM/build$ pip install setuptools wheel numpy scipy scikit-learn -U --user

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple

Collecting setuptools

 Using cached https://pypi.tuna.tsinghua.edu.cn/packages/95/95/f657b6e17f00c3f35b5f68b10e46c3a43af353d8856bd57bfcfb1dbb3e92/setuptools-47.1.1-py3-none-any.whl

Collecting wheel

 Using cached https://pypi.tuna.tsinghua.edu.cn/packages/8c/23/848298cccf8e40f5bbb59009b32848a4c38f4e7f3364297ab3c3e2e2cd14/wheel-0.34.2-py2.py3-none-any.whl

Collecting numpy

 Using cached https://pypi.tuna.tsinghua.edu.cn/packages/b3/a9/b1bc4c935ed063766bce7d3e8c7b20bd52e515ff1c732b02caacf7918e5a/numpy-1.18.5-cp36-cp36m-manylinux1_x86_64.whl

Collecting scipy

 Using cached https://pypi.tuna.tsinghua.edu.cn/packages/dc/29/162476fd44203116e7980cfbd9352eef9db37c49445d1fec35509022f6aa/scipy-1.4.1-cp36-cp36m-manylinux1_x86_64.whl

Collecting scikit-learn

 Using cached https://pypi.tuna.tsinghua.edu.cn/packages/d9/3a/eb8d7bbe28f4787d140bb9df685b7d5bf6115c0e2a969def4027144e98b6/scikit_learn-0.23.1-cp36-cp36m-manylinux1_x86_64.whl

Collecting threadpoolctl>=2.0.0

 Using cached https://pypi.tuna.tsinghua.edu.cn/packages/f7/12/ec3f2e203afa394a149911729357aa48affc59c20e2c1c8297a60f33f133/threadpoolctl-2.1.0-py3-none-any.whl

Requirement already satisfied, skipping upgrade: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn) (0.14.0)

ERROR: mxnet-cu100 1.5.1.post0 has requirement graphviz<0.9.0,>=0.8.1, but you'll have graphviz 0.13 which is incompatible.

ERROR: chainer 6.4.0 has requirement protobuf<3.8.0rc1,>=3.0.0, but you'll have protobuf 3.10.0 which is incompatible.

ERROR: chainer 6.4.0 has requirement typing<=3.6.6, but you'll have typing 3.7.4.1 which is incompatible.

Installing collected packages: setuptools, wheel, numpy, scipy, threadpoolctl, scikit-learn

 WARNING: The scripts easy_install and easy_install-3.6 are installed in '/home/mist/.local/bin' which is not on PATH.

 Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

 WARNING: The script wheel is installed in '/home/mist/.local/bin' which is not on PATH.

 Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

 WARNING: The scripts f2py, f2py3 and f2py3.6 are installed in '/home/mist/.local/bin' which is not on PATH.

 Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

Successfully installed numpy-1.18.5 scikit-learn-0.23.1 scipy-1.4.1 setuptools-47.1.1 threadpoolctl-2.1.0 wheel-0.34.2

WARNING: You are using pip version 19.3.1; however, version 20.1.1 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

mist@C-000058-GPU:~/LightGBM/build$ pip uninstall graphviz

Uninstalling graphviz-0.13:

 Would remove:

  /usr/local/lib/python3.6/dist-packages/graphviz-0.13.dist-info/*

  /usr/local/lib/python3.6/dist-packages/graphviz/*

Proceed (y/n)? y

ERROR: Exception:

Traceback (most recent call last):

 File "/usr/lib/python3.6/shutil.py", line 550, in move

  os.rename(src, real_dst)

PermissionError: [Errno 13] Permission denied: '/usr/local/lib/python3.6/dist-packages/graphviz-0.13.dist-info/' -> '/tmp/pip-uninstall-4d_2j1tb'



During handling of the above exception, another exception occurred:



Traceback (most recent call last):

 File "/usr/local/lib/python3.6/dist-packages/pip/_internal/cli/base_command.py", line 153, in _main

  status = self.run(options, args)

 File "/usr/local/lib/python3.6/dist-packages/pip/_internal/commands/uninstall.py", line 79, in run

  auto_confirm=options.yes, verbose=self.verbosity > 0,

 File "/usr/local/lib/python3.6/dist-packages/pip/_internal/req/req_install.py", line 755, in uninstall

  uninstalled_pathset.remove(auto_confirm, verbose)

 File "/usr/local/lib/python3.6/dist-packages/pip/_internal/req/req_uninstall.py", line 394, in remove

  moved.stash(path)

 File "/usr/local/lib/python3.6/dist-packages/pip/_internal/req/req_uninstall.py", line 283, in stash

  renames(path, new_path)

 File "/usr/local/lib/python3.6/dist-packages/pip/_internal/utils/misc.py", line 338, in renames

  shutil.move(old, new)

 File "/usr/lib/python3.6/shutil.py", line 562, in move

  rmtree(src)

 File "/usr/lib/python3.6/shutil.py", line 486, in rmtree

  _rmtree_safe_fd(fd, path, onerror)

 File "/usr/lib/python3.6/shutil.py", line 444, in _rmtree_safe_fd

  onerror(os.unlink, fullname, sys.exc_info())

 File "/usr/lib/python3.6/shutil.py", line 442, in _rmtree_safe_fd

  os.unlink(name, dir_fd=topfd)

PermissionError: [Errno 13] Permission denied: 'METADATA'

mist@C-000058-GPU:~/LightGBM/build$ pip uninstall graphviz --user



Usage:  

 pip uninstall [options] <package> ...

 pip uninstall [options] -r <requirements file> ...



no such option: --user

mist@C-000058-GPU:~/LightGBM/build$ pip uninstall graphviz

Uninstalling graphviz-0.13:

 Would remove:

  /usr/local/lib/python3.6/dist-packages/graphviz-0.13.dist-info/*

  /usr/local/lib/python3.6/dist-packages/graphviz/*

Proceed (y/n)? y

ERROR: Exception:

Traceback (most recent call last):

 File "/usr/lib/python3.6/shutil.py", line 550, in move

  os.rename(src, real_dst)

PermissionError: [Errno 13] Permission denied: '/usr/local/lib/python3.6/dist-packages/graphviz-0.13.dist-info/' -> '/tmp/pip-uninstall-ykrfmcr4'



During handling of the above exception, another exception occurred:



Traceback (most recent call last):

 File "/usr/local/lib/python3.6/dist-packages/pip/_internal/cli/base_command.py", line 153, in _main

  status = self.run(options, args)

 File "/usr/local/lib/python3.6/dist-packages/pip/_internal/commands/uninstall.py", line 79, in run

  auto_confirm=options.yes, verbose=self.verbosity > 0,

 File "/usr/local/lib/python3.6/dist-packages/pip/_internal/req/req_install.py", line 755, in uninstall

  uninstalled_pathset.remove(auto_confirm, verbose)

 File "/usr/local/lib/python3.6/dist-packages/pip/_internal/req/req_uninstall.py", line 394, in remove

  moved.stash(path)

 File "/usr/local/lib/python3.6/dist-packages/pip/_internal/req/req_uninstall.py", line 283, in stash

  renames(path, new_path)

 File "/usr/local/lib/python3.6/dist-packages/pip/_internal/utils/misc.py", line 338, in renames

  shutil.move(old, new)

 File "/usr/lib/python3.6/shutil.py", line 562, in move

  rmtree(src)

 File "/usr/lib/python3.6/shutil.py", line 486, in rmtree

  _rmtree_safe_fd(fd, path, onerror)

 File "/usr/lib/python3.6/shutil.py", line 444, in _rmtree_safe_fd

  onerror(os.unlink, fullname, sys.exc_info())

 File "/usr/lib/python3.6/shutil.py", line 442, in _rmtree_safe_fd

  os.unlink(name, dir_fd=topfd)

PermissionError: [Errno 13] Permission denied: 'METADATA'

mist@C-000058-GPU:~/LightGBM/build$ sudo pip uninstall graphviz

WARNING: The directory '/home/mist/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

Uninstalling graphviz-0.13:

 Would remove:

  /usr/local/lib/python3.6/dist-packages/graphviz-0.13.dist-info/*

  /usr/local/lib/python3.6/dist-packages/graphviz/*

Proceed (y/n)? y

 Successfully uninstalled graphviz-0.13

mist@C-000058-GPU:~/LightGBM/build$ pip install graphviz==0.8.2 --user

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple

Collecting graphviz==0.8.2

 Downloading https://pypi.tuna.tsinghua.edu.cn/packages/05/e4/8fcc76823534d47f079c0ff1b3d8b57784e8fba63ceb1ded32c9f4dd993c/graphviz-0.8.2-py2.py3-none-any.whl

Installing collected packages: graphviz

Successfully installed graphviz-0.8.2

WARNING: You are using pip version 19.3.1; however, version 20.1.1 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

mist@C-000058-GPU:~/LightGBM/build$ sudo pip uninstall protobuf

WARNING: The directory '/home/mist/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

Uninstalling protobuf-3.10.0:

 Would remove:

  /usr/local/lib/python3.6/dist-packages/google/protobuf/*

  /usr/local/lib/python3.6/dist-packages/protobuf-3.10.0-py3.6-nspkg.pth

  /usr/local/lib/python3.6/dist-packages/protobuf-3.10.0.dist-info/*

Proceed (y/n)? y

 Successfully uninstalled protobuf-3.10.0

mist@C-000058-GPU:~/LightGBM/build$ pip install protobuf==3.7.0 --user

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple

Collecting protobuf==3.7.0

 Downloading https://pypi.tuna.tsinghua.edu.cn/packages/c5/60/ca38e967360212ddbb004141a70f5f6d47296e1fba37964d8ac6cb631921/protobuf-3.7.0-cp36-cp36m-manylinux1_x86_64.whl (1.2MB)

   |████████████████████████████████| 1.2MB 3.8MB/s 

Requirement already satisfied: setuptools in /home/mist/.local/lib/python3.6/site-packages (from protobuf==3.7.0) (47.1.1)

Requirement already satisfied: six>=1.9 in /usr/local/lib/python3.6/dist-packages (from protobuf==3.7.0) (1.12.0)

ERROR: chainer 6.4.0 has requirement typing<=3.6.6, but you'll have typing 3.7.4.1 which is incompatible.

Installing collected packages: protobuf

Successfully installed protobuf-3.7.0

WARNING: You are using pip version 19.3.1; however, version 20.1.1 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

mist@C-000058-GPU:~/LightGBM/build$ sudo pip uninstall typing

WARNING: The directory '/home/mist/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

Uninstalling typing-3.7.4.1:

 Would remove:

  /usr/local/lib/python3.6/dist-packages/typing-3.7.4.1.dist-info/*

  /usr/local/lib/python3.6/dist-packages/typing.py

Proceed (y/n)? y

 Successfully uninstalled typing-3.7.4.1

mist@C-000058-GPU:~/LightGBM/build$ pip install typing==3.6.6 --user

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple

Collecting typing==3.6.6

 Downloading https://pypi.tuna.tsinghua.edu.cn/packages/4a/bd/eee1157fc2d8514970b345d69cb9975dcd1e42cd7e61146ed841f6e68309/typing-3.6.6-py3-none-any.whl

Installing collected packages: typing

Successfully installed typing-3.6.6

WARNING: You are using pip version 19.3.1; however, version 20.1.1 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

mist@C-000058-GPU:~/LightGBM/build$ sudo pip install lightgbm --install-option=--gpu --install-option="--opencl-include-dir=/usr/local/cuda/include/" --install-option="--opencl-library=/usr/local/cuda/lib64/libOpenCL.so"

/usr/local/lib/python3.6/dist-packages/pip/_internal/commands/install.py:283: UserWarning: Disabling all use of wheels due to the use of --build-options / --global-options / --install-options.

 cmdoptions.check_install_build_global(options)

WARNING: The directory '/home/mist/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

WARNING: The directory '/home/mist/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple

Collecting lightgbm

 Downloading https://pypi.tuna.tsinghua.edu.cn/packages/d2/90/4a75304dae29ae4b2497ed01b12e7c0a8f12f981c086f086b5416d28bf07/lightgbm-2.3.1.tar.gz (679kB)

   |████████████████████████████████| 686kB 1.3MB/s 

Requirement already satisfied: numpy in /home/mist/.local/lib/python3.6/site-packages (from lightgbm) (1.18.5)

Requirement already satisfied: scipy in /home/mist/.local/lib/python3.6/site-packages (from lightgbm) (1.4.1)

Requirement already satisfied: scikit-learn in /home/mist/.local/lib/python3.6/site-packages (from lightgbm) (0.23.1)

Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn->lightgbm) (0.14.0)

Requirement already satisfied: threadpoolctl>=2.0.0 in /home/mist/.local/lib/python3.6/site-packages (from scikit-learn->lightgbm) (2.1.0)

Skipping wheel build for lightgbm, due to binaries being disabled for it.

Installing collected packages: lightgbm

  Running setup.py install for lightgbm ... done

Successfully installed lightgbm-2.3.1

WARNING: You are using pip version 19.3.1; however, version 20.1.1 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

mist@C-000058-GPU:~/LightGBM/build$ 

mist@C-000058-GPU:~/LightGBM/build$ # 终端手动添加路径文件

mist@C-000058-GPU:~/LightGBM/build$ mkdir -p /etc/OpenCL/vendors && echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd

mkdir: cannot create directory ‘/etc/OpenCL’: Permission denied

mist@C-000058-GPU:~/LightGBM/build$ 

mist@C-000058-GPU:~/LightGBM/build$ su 

Password: 

su: Authentication failure

mist@C-000058-GPU:~/LightGBM/build$ su

Password: 

su: Authentication failure

mist@C-000058-GPU:~/LightGBM/build$ sudo passwd

Enter new UNIX password: 

Retype new UNIX password: 

passwd: password updated successfully

mist@C-000058-GPU:~/LightGBM/build$ su

Password: 

(base) root@C-000058-GPU:/home/mist/LightGBM/build# # 终端手动添加路径文件

(base) root@C-000058-GPU:/home/mist/LightGBM/build# mkdir -p /etc/OpenCL/vendors && echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd

(base) root@C-000058-GPU:/home/mist/LightGBM/build# 

(base) root@C-000058-GPU:/home/mist/LightGBM/build# Connection to 47.103.52.232 closed by remote host.

Connection to 47.103.52.232 closed.

让我们一起分享，共同成长，分享使我们在编程路上并不孤独。快来扫描二维码，与博主一起快乐学习吧！