ubuntu 14.04+ GTX 1070+cuda 8.0 + cudnn 5.1+opencv3.0的安装参见我前两篇博客,详细介绍了!
上一篇写了如何接着配置caffe框架. 本文重点为caffe-ssd的安装与配置细节(注意此时电脑的配置为ubuntu 14.04+ GTX 1070+cuda 8.0 + cudnn 5.1+opencv3.0+caffe)
一、安装ssd(大部分依赖库已经在配置caffe的时候完成)
在/home下新建文件夹caffe-ssd(区别于caffe)
cd caffe-ssd
git clone https://github.com/weiliu89/caffe.git
cd caffe
git checkout ssd
一、配置ssd
cd /home/**(您服务器的名字)/caffe
cp Makefile.config.example Makefile.config
打开Makefile.config,修改之处为
use_cudnn:=1取消注释
opencv_version:=3取消注释
with_python_layer:=1取消注释
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
改为
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
修改Makefile文件
LIBRARIES += glog gflags protobuf boost_system boost_filesystem boost_regex m hdf5_hl hdf5
改为
LIBRARIES += glog gflags protobuf boost_system boost_filesystem boost_regex m hdf5_hl hdf5 opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs
然后编译
用下面的方法开始不成功就放弃了
cd /home//caffe
mkdir build
cd build
cmake ..
make all -j16
make install
make runtest
make pycaffe
其中在cmake ..的时候发现opencv还是找的opencv2.4.8版本,在caffe-ssd/caffe里面的CMakeLists.txt添加
set(OpenCV_DIR /home/shan/opencv-3.0.0/build)
这样就能找到3.0版本的了,改完之后接着用下面的方法:
根据readme.md里的指令编译
make -j8
make py
make test -j8
make runtest -j8
在Make -j8 过程会出现error:
1、json_parser_read.hpp:257:264: error: ‘type name’ declared as function returning an array escape
解决:sudo gedit/usr/include/boost/property_tree/detail/json_parser_read.hpp将257行的escape代码段注释掉即可
2、cannot find -lopencv_videoio
cannot find -lopenblas
像这种问题是找不到libopencv_videoio.so、libopenblas.so等文件
解决方法是:首先在计算机中查找看是否有libopencv_videoio等文件
若没有,就sudo apt-get install libopencv_videoio-dev
若有,就把文件拷贝到/usr/lib下 ,如果不行,就拷贝到/usr/local/lib
对于我的问题libopencv_videoio.so能找到,进行拷贝
sudo cp /home/shan/opencv-3.0.0/build/lib/libopencv_videoio.so /usr/lib
sudo cp /home/shan/opencv-3.0.0/build/lib/libopencv_videoio.so.3.0 /usr/lib
sudo cp /home/shan/opencv-3.0.0/build/lib/libopencv_videoio.so.3.0.0 /usr/lib
sudo cp /home/shan/opencv-3.0.0/build/lib/libopencv_videoio_pch_dephelp.a /usr/lib
而libopenblas找不到,
sudo apt-get install libopenblas-dev
再make -j8没问题
make py
make test -j8
make runtest -j8
出现问题: error while loading shared libraries: libcudart.so.8.0: cannot open shared object file: No such file or directory
解决办法:首先确认/etc/profile中的路径包含了cuda8.0的安装路径及相应的库文件
sudo gedit /etc/profile
export PATH=$PATH:/usr/local/cuda-8.0/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-8.0/lib64
export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/cuda-8.0/lib64
export PYTHONPATH=~/caffe-ssd/caffe/python:$PYTHONPATH
export PKG_CONFIG_PATH=/usr/local/cuda-8.0/pkgconfig:/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH
$source /etc/profile
使配置文件生效,再次执行。
若仍提示相同的错误,则执行以下命令,将相应的库文件复制到/usr/lib
sudo cp /usr/local/cuda-8.0/lib64/libcudart.so.8.0 /usr/local/lib/libcudart.so.8.0 && sudo ldconfig
sudo cp /usr/local/cuda-8.0/lib64/libcublas.so.8.0 /usr/local/lib/libcublas.so.8.0 && sudo ldconfig
sudo cp /usr/local/cuda-8.0/lib64/libcurand.so.8.0 /usr/local/lib/libcurand.so.8.0 && sudo ldconfig
还是不行!!!
sudo cp /usr/local/cuda-8.0/lib64/libcudart.so.8.0 /usr/lib/libcudart.so.8.0 && sudo ldconfig
成功!!!
到了第二天不知道动了什么,。,不好使了,,删除caffe-ssd包,重新下载编译,编译不过去了,,又用这个法编译过去了,,真是醉了
cd /home//caffe
mkdir build
cd build
cmake ..
make all -j16
make install
make runtest
make pycaffe
接下来,下载数据集和训练
### Preparation
1. Download [fully convolutional reduced (atrous) VGGNet](https://gist.github.com/weiliu89/2ed6e13bfd5b57cf81d6). By default, we assume the model is stored in `$CAFFE_ROOT/models/VGGNet/`
2. Download VOC2007 and VOC2012 dataset. By default, we assume the data is stored in `$HOME/data/`
```Shell
# Download the data.
cd $HOME/data
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtest_06-Nov-2007.tar
# Extract the data.
tar -xvf VOCtrainval_11-May-2012.tar
tar -xvf VOCtrainval_06-Nov-2007.tar
tar -xvf VOCtest_06-Nov-2007.tar
数据集的获取:通过解压数据集到/data下最靠谱,如果通过拷贝以前保存的数据集到/data下,在/VOC2012/JPEGImage下2011_001232.jpg是拷贝不过来的
3.生成LMDB文件
cd $CAFFE_ROOT
# Create the trainval.txt, test.txt, and test_name_size.txt in data/VOC0712/
./data/VOC0712/create_list.sh
# You can modify the parameters in create_data.sh if needed.
# It will create lmdb files for trainval and test with encoded original image:
# - $HOME/data/VOCdevkit/VOC0712/lmdb/VOC0712_trainval_lmdb
# - $HOME/data/VOCdevkit/VOC0712/lmdb/VOC0712_test_lmdb
# and make soft links at examples/VOC0712/
./data/VOC0712/create_data.sh
```
### 训练
1. Train your model and evaluate the model on the fly.
```Shell
# It will create model definition files and save snapshot models in:
# - $CAFFE_ROOT/models/VGGNet/VOC0712/SSD_300x300/
# and job file, log file, and the python script in:
# - $CAFFE_ROOT/jobs/VGGNet/VOC0712/SSD_300x300/
# and save temporary evaluation results in:
# - $HOME/data/VOCdevkit/results/VOC2007/SSD_300x300/
# It should reach 72.* mAP at 60k iterations.
python examples/ssd/ssd_pascal.py 训练模型
```
打开caffe/examples/ssd/ssd_pascal.py这个文件,找到gpus=’0,1,2,3’这一行,如果您的服务器有一块显卡,则将123删去,如果有两个显卡,则删去23,以此类推。如果您服务器没有gpu支持,则注销以下几行,程序会以cpu形式训练。(这个是解决问题cudasuccess(10vs0)的方法)
#Ifnum_gpus >0:
# batch_size_per_device =int(math.ceil(float(batch_size) / num_gpus))
#iter_size =int(math.ceil(float(accum_batch_size) / (batch_size_per_device * num_gpus)))
# solver_mode =P.Solver.GPU# device_id =int(gpulist[0])
保存后终端运行:
cd /home/**(您服务器的名字)/caffe
python examples/ssd/ssd_pascal.py
如果出现问题cudasuccess(2vs0)则说明您的显卡计算量有限,再次打开caffe/examples/ssd/ssd_pascal.py这个文件,找到batch_size =32这一行,修改数字32,可以修改为16,或者8,甚至为4(相信大家这个修改可以理解,我就不作说明了),保存后再次终端运行python examples/ssd/ssd_pascal.py
If you don't have time to train your model, you can download a pre-trained model at [here](http://www.cs.unc.edu/~wliu/projects/SSD/models_VGGNet_VOC0712_SSD_300x300.tar.gz).
2. Evaluate the most recent snapshot.
# If you would like to test a model you trained, you can do:
python examples/ssd/score_ssd_pascal.py演示detection的训练结果,数值在0.718左右
```
3. Test your model using a webcam. Note: press <kbd>esc</kbd> to stop.
# If you would like to attach a webcam to a model you trained, you can do:
python examples/ssd/ssd_pascal_webcam.py演示网络摄像头识别效果
```
4. Check out `examples/ssd_detect.ipynb` or `examples/ssd/ssd_detect.cpp` on how to detect objects using a SSD model.
### Models
1. Models trained on VOC0712: [SSD300](http://www.cs.unc.edu/~wliu/projects/SSD/models_VGGNet_VOC0712_SSD_300x300.tar.gz), [SSD500](http://www.cs.unc.edu/~wliu/projects/SSD/models_VGGNet_VOC0712_SSD_500x500.tar.gz)
2. Models trained on MSCOCO trainval35k: [SSD300](http://www.cs.unc.edu/~wliu/projects/SSD/models_VGGNet_coco_SSD_300x300.tar.gz), [SSD500](http://www.cs.unc.edu/~wliu/projects/SSD/models_VGGNet_coco_SSD_500x500.tar.gz)
3. Models trained on ILSVRC2015 trainval1: [SSD300](http://www.cs.unc.edu/~wliu/projects/SSD/models_VGGNet_ilsvrc15_SSD_300x300.tar.gz), [SSD500](http://www.cs.unc.edu/~wliu/projects/SSD/models_VGGNet_ilsvrc15_SSD_500x500.tar.gz) (46.4 mAP on val2)
########################################################################
在python examples/ssd/ssd_pascal.py时有一些小问题产生,按照网上的都解决,,但具体的忘了,好像
sudo python examples/ssd/ssd_pascal.py 这样就可以训练数据了
运行python examples/ssd/score_ssd_pascal.py 和 python examples/ssd/ssd_pascal_webcam.py出现错误:
层与层之间的维度信息不匹配,这时候训练迭代次数将近40000次,然后继续运行
Python example/ssd/ssd_pascal.py等到迭代到了68000多次,再次运行,成功!!!
至此,所有配置工作圆满完成!!!