Caffe2 - Detectron 代码环境构建尝试

1.首先下载caffe2，pytorch已经包含了caffe2，不过pytorch1.4是最后一板支持py2的。
caffe2官方installation教程
conda install pytorch-nightly -c pytorch
不太行
尝试源码安装

sudo apt-get update
sudo apt-get install -y --no-install-recommends \
      build-essential \
      cmake \
      git \
      libgoogle-glog-dev \
      libgtest-dev \
      libiomp-dev \
      libleveldb-dev \
      liblmdb-dev \
      libopencv-dev \
      libopenmpi-dev \
      libsnappy-dev \
      libprotobuf-dev \
      openmpi-bin \
      openmpi-doc \
      protobuf-compiler \
      python-dev \
      python-pip                          (系统默认python已经执行)
pip install --user \
      future \
      numpy \
      protobuf \
      typing \
      hypothesis  (系统默认python已经执行)
sudo apt-get install -y --no-install-recommends \
      libgflags-dev \
      cmake (系统默认python已经执行)

libgflags2根据系统选择
对于 Ubuntu 14.04
sudo apt-get install -y --no-install-recommends libgflags2
对于 Ubuntu 16.04
sudo apt-get install -y --no-install-recommends libgflags-dev(系统默认python已经执行)
然后
git clone https://github.com/pytorch/pytorch.git && cd pytorch(系统默认python已经执行)
git submodule update --init --recursive(系统默认python已经执行)
sudo python setup.py install(系统默认python已经执行)
如果出现如下报错:

bootstrap.cc:7:18: fatal error: nccl.h: 没有那个文件或目录
compilation terminated.
transport/net_ib.cc:7:18: fatal error: nccl.h: 没有那个文件或目录
compilation terminated.
In file included from misc/nvmlwrap.cc:7:0:
include/nvmlwrap.h:10:18: fatal error: nccl.h: 没有那个文件或目录
compilation terminated.
Makefile:110: recipe for target '/home/asber/Documents/pytorch/build/nccl/obj/misc/nvmlwrap.o' failed
make[1]: *** [/home/asber/Documents/pytorch/build/nccl/obj/misc/nvmlwrap.o] Error 1
make[1]: *** 正在等待未完成的任务....
Makefile:110: recipe for target '/home/asber/Documents/pytorch/build/nccl/obj/bootstrap.o' failed
make[1]: *** [/home/asber/Documents/pytorch/build/nccl/obj/bootstrap.o] Error 1
Makefile:110: recipe for target '/home/asber/Documents/pytorch/build/nccl/obj/transport/net_ib.o' failed
make[1]: *** [/home/asber/Documents/pytorch/build/nccl/obj/transport/net_ib.o] Error 1
---
Makefile:25: recipe for target 'src.build' failed
make: *** [src.build] Error 2
[29/3537] Building CXX object third_party/...dir/__/src/google/protobuf/stubs/time.cc.o
ninja: build stopped: subcommand failed.

解决方案：更新本机的nccl库
https://cloud.tencent.com/developer/article/1384540
https://www.cnblogs.com/pprp/p/9540644.html

按照此处:https://docs.nvidia.com/deeplearning/sdk/nccl-install-guide/index.html
sudo dpkg -i nccl-repo-.deb之后需要
sudo apt-key add /var/nccl-repo-2.5.6-ga-cuda9.0/7fa2af80.pub
根据自己下载的页面提示
sudo apt install libnccl2=2.5.6-1+cuda9.0 libnccl-dev=2.5.6-1+cuda9.0
安装OK

cd pytorch
mkdir build
cd build
cmake …
############################################################################################
(若是报‘no module yaml’ 错误，需要安装sudo pip install pyyaml ,然后重新 cmake …)
############################################################################################
sudo make -j8 install
(个人认为这个不用 因为python setup.py install的时候已经build过了)

编译完成，进行测试，出现success,即为成功。
cd ~ && python -c ‘from caffe2.python import core’ 2>/dev/null && echo “Success” || echo “Failure”
####检测caffe2是否安装成功，出现大于0，就是成功安装
python -c ‘from caffe2.python import workspace; print(workspace.NumCudaDevices())’

测试报错:

from caffe2.python import core
from past.builtins import basestring
ImportError: No module named past.builtins

解决方案pip2 install future
pip2 install future的时候报错

File “/usr/local/bin/pip2”, line 5, in
from pip._internal.cli.main import main
ImportError: No module named main

解决方案:https://www.cnblogs.com/chax/p/9409353.html

asber@asber-X550VX:~/Documents$ cd ~ && python -c ‘from caffe2.python import core’ 2>/dev/null && echo “Success” || echo “Failure”
Success

2.python依赖
pip install numpy pyyaml matplotlib opencv-python>=3.0 setuptools Cython mock(系统默认python已经执行)

3.COCO API 安装：

# COCOAPI=/home/asber/Documents/PaddleDetection-release-0.1/cocoapi
git clone https://github.com/cocodataset/cocoapi.git $COCOAPI(系统默认python已经执行)
cd $COCOAPI/PythonAPI(系统默认python已经执行)
# Install into global site-packages
make install(系统默认python已经执行)
# Alternatively, if you do not have permissions or prefer
# not to install the COCO API into global site-packages
python2 setup.py install --user(系统默认python已经执行)

修改环境变量
(不一定)
sudo gedit ~/.bashrc
COCOAPI=/path/to/install/cocoapi(系统默认python已经执行)
source ~/.bashrc

4.Detectron 安装

Clone Detectron reposity：
# DETECTRON=/path/to/clone/detectron 在Documents下
git clone https://github.com/facebookresearch/detectron $DETECTRON  (系统默认python已经执行)

()不一定)
pip install -r  requirements.txt(系统默认python已经执行)

设置 Python 模块：
cd $DETECTRON && make
output:
>copying build/lib.linux-x86_64-2.7/detectron/utils/cython_bbox.so -> detectron/utils
>copying build/lib.linux-x86_64-2.7/detectron/utils/cython_nms.so -> detectron/utils
>Creating /home/asber/.local/lib/python2.7/site-packages/Detectron.egg-link (link to .)
>Adding Detectron 0.0.0 to easy-install.pth file
>Installed /home/asber/Documents/detectron
>Processing dependencies for Detectron==0.0.0
>Finished processing dependencies for Detectron==0.0.0

cd detectron/detectron/tests
测试 Detectron 安装，如 SpatialNarrowAsOp test：
python2 test_spatial_narrow_as_op.py

python输出

[E init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
Found Detectron ops lib: /usr/local/lib/python2.7/dist-packages/torch/lib/libcaffe2_detectron_ops_gpu.so
...
----------------------------------------------------------------------
Ran 3 tests in 7.273s

OK

---

##################根据自己路径修改对应的地方，并执行指令 ##################
python tools/infer_simple.py
–cfg configs/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml
–output-dir /tmp/detectron-visualizations
–image-ext jpg
–wts https://dl.fbaipublicfiles.com/detectron/35861858/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml.02_32_51.SgT4y1cO/output/train/coco_2014_train:coco_2014_valminusminival/generalized_rcnn/model_final.pkl
demo

RuntimeError: [enforce fail at context_gpu.cu:496] error == cudaSuccess. 2 vs 0. Error at: /home/asber/Documents/pytorch/caffe2/core/context_gpu.cu:496: out of memory
Error from operator: 
input: "gpu_0/res3_3_branch2c_bn" input: "gpu_0/res3_2_branch2c_bn" output: "gpu_0/res3_3_sum" name: "" type: "Sum" device_option { device_type: 1 device_id: 0 }frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, void const*) + 0x67 (0x7f26e468a947 in /usr/local/lib/python2.7/dist-packages/caffe2/python/../../torch/lib/libc10.so)

6、Detectron简单使用

参考

https://github.com/facebookresearch/Detectron/blob/master/GETTING_STARTED.md

训练相关的我们放到之后去说，这里先说下利用已经训练好的模型和算法框架来进行目标的检测。

模型进行推理测试.
https://blog.csdn.net/zziahgf/article/details/79141879
https://www.cnblogs.com/zhencv/p/8384419.html

MASKRCNN
https://blog.csdn.net/Xiongchao99/article/details/79167353#0-tsina-1-94209-397232819ff9a47a7b7e80a40613cfe1
https://www.cnblogs.com/zhencv/p/8384419.html

参考文章
ubuntu16.04 caffe2 maskrcnn
https://www.cnblogs.com/acti/p/9931912.html
https://www.cnblogs.com/zealousness/p/8757353.html
https://blog.csdn.net/Xiongchao99/article/details/79167353#0-tsina-1-94209-397232819ff9a47a7b7e80a40613cfe1
https://blog.csdn.net/hyk900817/article/details/80253841
https://blog.csdn.net/m0_37644085/article/details/84962718
https://zhuanlan.zhihu.com/p/58100164
https://blog.csdn.net/lfs666666/article/details/88129352
https://www.cnblogs.com/zhencv/p/8384419.html
https://blog.csdn.net/qq_29462849/article/details/85841388