系统：Ubuntu 14
Kintinuous + Elastic Fusion代码来源：https://github.com/mp3guy
准备工作
安装git
- sudo apt-get install git
- git config --global user.name 你的git username
- git config --global user.email 你的邮箱
直接在ubuntu软件中心下载安装，但是配置Elastic Fusion需要升级到3.2，否则自带的是2.8
参考：http://askubuntu.com/questions/610291/how-to-install-cmake-3-2-on-ubuntu-14-04
- sudo apt-get install build-essential
- wget http://www.cmake.org/files/v3.2/cmake-3.2.2.tar.gz
- tar xf cmake-3.2.2.tar.gz
- cd cmake-3.2.2
- ./configure
- make
- sudo make install
如果提示找不到bin/cmake，那么sudo apt-get upgrade就可以了
配置Kinect运行环境（一个参考，实际上我并没有用Kinect跑，最后用的是Xtion）
参考：http://www.cc.gatech.edu/grads/a/ahuaman3/docs/tutorials/software/build/perception.html
OpenNI2安装：
- sudo apt-get install g++
- sudo apt-get install python
- sudo apt-get install libusb-1.0-0-dev
- sudo apt-get install libudev-dev
- sudo apt-get install openjdk-6-jdk
- sudo apt-get install freeglut3-dev
- sudo apt-get install graphviz
- sudo apt-get install doxygen
- git clone https://github.com/occipital/OpenNI2
- cd OpenNI2
- make
在其中加入：
- UBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02b0", MODE:="0666", OWNER:="root", GROUP:="video"
- SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02ad", MODE:="0666", OWNER:="root", GROUP:="video"
- SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02ae", MODE:="0666", OWNER:="root", GROUP:="video"
- SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02c2", MODE:="0666", OWNER:="root", GROUP:="video"
- SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02be", MODE:="0666", OWNER:="root", GROUP:="video"
- SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02bf", MODE:="0666", OWNER:="root", GROUP:="video"
- git clone https://github.com/OpenKinect/libfreenect.git
- cd libfreenect
- mkdir build; cd build
- cmake .. -DBUILD_OPENNI2_DRIVER=ON
- make
- cp -L lib/OpenNI2-FreenectDriver/libFreenectDriver.so ${OPENNI2_DIR}/Bin/x64-Release/OpenNI2/Drivers #将驱动拷贝到OpenNI2的解压文件夹下，OPENNI2_DIR：OpenNI2的解压文件夹
插上kinect后，使用lsusb命令查看确保有以下三个：
Xbot camera
Xbot motor
Xbot audio
安装成功检查：
- cd ${OPENNI2_DIR}/Bin/x64-Release/
- ./NiViewer
- git clone https://github.com/mp3guy/Kintinuous
1.新系统：
找到build.sh，在属性中设置为可执行
./build.sh
等待配置成功
2.非新系统：
安装CUDA：
下载deb文件 https://developer.nvidia.com/cuda-downloads查看是否成功运行cuda- sudo dpkg -i cuda-repo-ubuntu1404_7.5-18_amd64.deb
- sudo apt-get update
- sudo apt-get install cuda
- sudo reboot
- cd /usr/local/cuda/samples
- sudo make all -j4
- cd /usr/local/cuda/samples/bin/x86_64/linux/release
- sudo ./deviceQuery
CUDA Device Query (Runtime API) version (CUDART static linking)
Detected 1 CUDA Capable device(s)我在这里遇到了一定问题，找了很久，发现是Secure Boot没关掉，关掉之后就好了。解决的流程如下：
运行deviceQuery：no capable device
尝试解决：
1.怀疑是内核问题
add in: GRUB_CMDLINE_LINUX_DEFAULT with: pci=nocrs pci=realloc
依然：no capable device
2.装最新的nvidia 367
新问题：CUDA driver version is insufficient for CUDA runtime version
3.查看当前的kernel版本
kernel:3.19.0-65-generic
4.查看一些参数
sudo dpkg --list | grep nvidia
lspci | grep -i nvidia 设备里的确有nvidia显卡
lsmod 发现video一览只有i915_bpo，说明nvidia独显并没有被启用
sudo modprobe nvidia-367 手动启用nvidia显卡，提示ERROR: could not insert 'nvidia_367': Required key not available
终于找到问题：不是显卡驱动没装好，而是无法启动显卡（被BIOS禁了？）
5.装回nvidia 352
sudo apt-get remove --purge nvidia-*
(***提示cmake-qt-gui依赖cmake 2.8出错，卸载cmake-qt-gui。卸载后cmake命令没出错，其他情况会不会出错有待观察)
sudo add-apt-repository ppa:xorg-edgers/ppa -y
sudo apt-get update
sudo apt-get install nvidia-352
6.查看安装的版本
sudo dpkg --list | grep nvidia
ii nvidia-352 352.93-0ubuntu1 amd64 NVIDIA binary driver - version 352.93
ii nvidia-modprobe 352.93-0ubuntu1 amd64 Load the NVIDIA kernel driver and create device files
ii nvidia-opencl-icd-352 352.93-0ubuntu1 amd64 NVIDIA OpenCL ICD
ii nvidia-prime 0.6.2 amd64 Tools to enable NVIDIA's Prime
ii nvidia-settings 367.35-0ubuntu0~gpu14.04.1 amd64 Tool for configuring the NVIDIA graphics driver
安装正确，运行devieQuery依然提示no device found，回到原点
7.现在的问题：
sudo modprobe nvidia-352 手动启用nvidia显卡，提示ERROR: could not insert 'nvidia_352': Required key not available
网上给出的解决方法：turn off the secure boot feature of the motherboard
8.尝试解决：开机F12，secure boot勾选为disable，legacy勾选
sudo modprobe nvidia-352 成功！
nvidia-settings 显示独显成功！
deviceQuery 检测到显卡！应该可以了！
9.新问题：kinect无法连接了。。。（之前是可以的）
lsusb 能看到XBOX的三个设备
从头重新配置，发现有几个依赖包没有安装（可能之前哪一步把他卸载了）
sudo apt-get install openjdk-6-jdk
sudo apt-get install graphviz
sudo apt-get install doxygen
之后NiViewer可以正常运行了
安装依赖项：
- sudo apt-get install -y cmake-qt-gui git build-essential libusb-1.0-0-dev libudev-dev openjdk-7-jdk freeglut3-dev python-vtk libvtk-java libglew-dev cuda-7-5 libsuitesparse-dev
安装pcl：
- sudo add-apt-repository ppa:v-launchpad-jochen-sprickerhof-de/pcl
- sudo apt-get update
- sudo apt-get install libpcl-all
安装opengl：
- sudo apt-get install build-essential
- sudo apt-get install libgl1-mesa-dev
- sudo apt-get install libgl1-mesa-dev
- sudo apt-get install libglu1-mesa-dev
- sudo apt-get install freeglut3-dev
安装opencv：
（不可以直接在ubuntu的apt-get里面获取opencv，因为ubuntu提供的opencv没有nonfree模块）
下载http://opencv.org/downloads.html
- sudo apt-get install build-essential libgtk2.0-dev libjpeg-dev libtiff4-dev libjasper-dev libopenexr-dev cmake python-dev python-numpy python-tk libtbb-dev libeigen2-dev yasm libfaac-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev libx264-dev libqt4-dev libqt4-opengl-dev sphinx-common texlive-latex-extra libv4l-dev libdc1394-22-dev libavcodec-dev libavformat-dev libswscale-dev
- unzip opencv-2.4.9.zip
- cd opencv-2.4.9
- mkdir build
- cd build
- cmake -D BUILD_NEW_PYTHON_SUPPORT=OFF -D WITH_OPENCL=OFF -D WITH_OPENMP=ON -D INSTALL_C_EXAMPLES=OFF -D BUILD_DOCS=OFF -D BUILD_EXAMPLES=OFF -D WITH_QT=OFF -D WITH_OPENGL=OFF -D WITH_VTK=OFF -D BUILD_PERF_TESTS=OFF -D BUILD_TESTS=OFF -D WITH_CUDA=OFF -D BUILD_opencv_gpu=OFF ..
- make
- sudo make install
安装Dlib：
- sudo apt-get install libboost-dev
- git clone https://github.com/dorian3d/DLib
- cd DLib/build
- cmake ..
- make
- sudo make install
安装DBoW2：
- git clone https://github.com/dorian3d/DBoW2#dbow2
- cd DBoW2/build
- cmake ..
- make
- sudo make install
安装DLoopDetector：
- git clone https://github.com/dorian3d/DLoopDetector
- cd DLoopDetector/build
- cmake ..
- make
- sudo make install
安装iSAM：
- svn co https://svn.csail.mit.edu/isam
- cd isam
- make
安装Pangolin：
- sudo apt-get install libglew-dev
- sudo apt-get install libboost-dev libboost-thread-dev libboost-filesystem-dev
- sudo apt-get install libpython2.7-dev
- sudo apt-get install ffmpeg libavcodec-dev libavutil-dev libavformat-dev libswscale-dev
- sudo apt-get install libdc1394-22-dev libraw1394-dev
- sudo apt-get install libjpeg-dev libpng12-dev libtiff5-dev libopenexr-dev
- git clone https://github.com/stevenlovegrove/Pangolin
- cd Pangolin/build
- cmake -DCPP11_NO_BOOST=1 ..
- make -j
运行Kintinuous：
作者的一些说明：
The GUI (Kintinuous) can take a bunch of parameters when launching it from the command line. They are as follows:
-c : Loads a camera calibration file specified by a depth_intrinsics matrix in OpenCV format.
-l : Processes the specified .klg log file.
-v : Loads DBoW vocabulary file.
-p : Loads ground truth poses to use instead of estimated pose.
-gpu : Sets which GPU should be used by CUDA.
-n : Number of frames to process.
-t : Voxel threshold for volume shifting (default 14).
-cw : Removes voxels below this threshold when extracting slices (default 8).
-lt : Disallow loop closures within this time period of the last (default 30s).
-s : Size of the fusion volume (default 6m).
-dg : Rate of pose sampling for deformation (default 0.8m).
-il : Inlier threshold for RANSAC (default 0.35).
-it : Residual threshold for pose graph optimisation (default 10).
-sm : Static mode (disable volume shifting).
-f : Flip RGB/BGR.
-od : Perform online deformation (required for loop closure).
-m : Enable mesh generation.
-no : Disable overlap of extracted slices.
-nos : Remove overlap when saving map.
-r : Use RGB tracking only.
-ri : Use combined ICP+RGB tracking.
-d : Enable dynamic cube positioning.
-dc : Disable color weighting by angle.
-fl : Subsample pose graph for faster loop closure.
-fod : Enable fast odometry.
以上所有配置以后就可以正常运行kintinuous了，运行的命令可以看上面的作者说明，比如：
- ./Kintinuous
默认从华硕Xtion中读取图像，如果没有Xtion则会报错找不到设备
没有话说Xtion的可以试试作者给的几个log，在原网址上有下载。
使用log的运行方法如下：
- ./Kintinuous -s 7 -v ../vocab.yml.gz -l loop.klg -ri -fl -od
# author has provided a sample dataset loop.klg which you can run easily with Kintinuous
配置ElasticFusion
准备工作和相关的依赖项包括开源包的配置方法和KIntinous一样，只需要额外安装开源包openni2
安装OpenNI2:- git clone https://github.com/occipital/OpenNI2.git
- cd OpenNI2
- make -j8
- cd ..
生成 ./ElasticFusion
注意源码里有三个项目，Core、GPUTest和GUI
如果要使用作者的程序，则需要把三个项目安装顺序依次配置成功
- cd ../Core
- mkdir build
- cd build
- cmake -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-7.5 ../src
- make -j8
- cd ../../GPUTest
- mkdir build
- cd build
- cmake -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-7.5 ../src
- make -j8
- cd ../../GUI
- mkdir build
- cd build
- cmake -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-7.5 ../src
- make -j8
- cd ../../myfusion
- mkdir build
- cd build
- cmake -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-7.5 ../src
- make -j8
直接调用以下命令，默认从华硕Xtion中获取图像并开始三位重建
- ./ElasticFusion
也可以使用参数控制，一些Elastci Fusion的运行参数如下：
- -cal : Loads a camera calibration file specified as fx fy cx cy. 载入相机参数（默认参数为华硕Xtion的参数）
- -l : Processes the specified .klg log file. 不使用摄像头读取，使用log文件模拟相机的图像采集过程
- -p : Loads ground truth poses to use instead of estimated pose. 读取ground truth的位姿来作为三位重建的依据之一（默认会自动估计当前的位置，如果传入ground truth的位姿则直接使用传入的位姿）
- -c : Surfel confidence threshold (default 10). Surfel的信任阈值
- -d : Cutoff distance for depth processing (default 3m). 多少距离以上的深度数据视为无用数据（单位mm，默认为3000，即3m）
- -i : Relative ICP/RGB tracking weight (default 10). ICP/RGB的tracking阈值
- -ie : Local loop closure residual threshold (default 5e-05).
- -ic : Local loop closure inlier threshold (default 35000).
- -cv : Local loop closure covariance threshold (default 1e-05).
- -pt : Global loop closure photometric threshold (default 115).
- -ft : Fern encoding threshold (default 0.3095). 关键帧编码阈值
- -t : Time window length (default 200).
- -s : Frames to skip at start of log. 如果使用log，跳过初始帧的数目
- -e : Cut off frame of log.
- -f : Flip RGB/BGR.
- -icl : Enable this if using the ICL-NUIM dataset (flips normals to account for negative focal length on that data).
- -o : Open loop mode.
- -rl : Enable relocalisation.
- -fs : Frame skip if processing a log to sim
- ulate real-time.
- -q : Quit when finished a log.
- -fo : Fast odometry (single level pyramid).
- -nso : Disables SO(3) pre-alignment in tracking.
- -r : Rewind and loop log forever.
- -ftf : Do frame-to-frame RGB tracking.
- -sc : Showcase mode (minimal GUI).
使用Elastic Fusion的CoreAPI
作者还提供了使用他的Core作为基础库的方法。
首先编译Core项目
- cd ../Core
- mkdir build
- cd build
- cmake -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-7.5 ../src
- make -j8
然后在CMakelist.txt中加入
- find_package(efusion REQUIRED)
- include_directories(${EFUSION_INCLUDE_DIR})
- target_link_libraries(MyProject ${EFUSION_LIBRARY})
在头文件中加入
- #include <ElasticFusion.h>
初始化
- Resolution::getInstance(640, 480); #图像大小
- Intrinsics::getInstance(528, 528, 320, 240); #相机参数
- pangolin::Params windowParams; #GUI窗口
- windowParams.Set("SAMPLE_BUFFERS", 0);
- windowParams.Set("SAMPLES", 0);
- pangolin::CreateWindowAndBind("Main", width, height, windowParams);
- glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
- glPixelStorei(GL_PACK_ALIGNMENT, 1);
使用封装好的ElasticFusion类，可以精确定义每一个参数
默认的构造函数
- ElasticFusion eFusion;
精确定义每一个参数后调用构建函数
- float confidence = 10.0f; //fusion的confidence阈值
- float depth = 3.0f; //去掉depth大于某个阈值的帧
- float icp = 10.0f; //icp的阈值
- float icpErrThresh = 5e-05; //icp错误阈值
- float covThresh = 1e-05;
- float photoThresh = 115;
- float fernThresh = 0.3095f; //新fern的阈值
- int timeDelta = 200;
- int icpCountThresh = 35000;
- bool openLoop = 0; //open loop模式：不开启
- bool iclnuim = 0; //使用icl dataset:不使用
- bool reloc = 0; //重定位模式：不开启
- bool fastOdom = 0; //Fast odometry (single level pyramid) mode :不开启
- bool so3 = 1; //SO(3) pre-alignment in tracking：开启
- bool frameToFrameRGB = 0; //只做rgb图像的tracking：不开启
- int timestamp = 0;
- std::string savefilename = "test";
- ElasticFusion eFusion(openLoop ? std::numeric_limits<int>::max() / 2 : timeDelta,icpCountThresh, icpErrThresh, covThresh, !openLoop, iclnuim, reloc,photoThresh, confidence, depth, icp, fastOdom, fernThresh, so3,frameToFrameRGB, savefilename);
处理输入帧：
- eFusion.processFrame(rgb, depth, timestamp, currentPose, weightMultiplier);
可选参数1:currentPose——先验的current pose,比如传入了标定好的ground truth三维坐标数据,则可以用上一时刻和这一时刻的三位坐标来估计相机姿态
- Eigen::Matrix4f * currentPose;//当前的位姿
- currentPose = new Eigen::Matrix4f;
- currentPose->setIdentity();
- *currentPose = groundTruthOdometry->getTransformation(timestamp); }
- eFusion.processFrame(rgb, depth, timestamp, currentPose, weightMultiplier);
- delete currentPose;
可选参数2:weightMultiplier (full frame fusion weight)——默认为1.f 可以自己调整,比如针对前面跳过的帧数对参数进行调整
- float weightMultiplier = framesToSkip + 1;
- eFusion.processFrame(rgb, depth, timestamp, currentPose, weightMultiplier);
一个简单的使用CoreAPI的程序：
- #include "myfusion_base.h"
- int main() {
- //图像的尺寸以及相机参数
- int width = 640;
- int height = 480;
- int camera_fx = 528;
- int camera_fy = 528;
- int camera_cx = 320;
- int camera_cy = 240;
- cout << "Initializing ..." << endl;
- Resolution::getInstance(width, height);
- Intrinsics::getInstance(camera_fx, camera_fy, camera_cx, camera_cy);
- cout << "Initializing done." << endl;
- cout << "Setting parameters..." << endl;
- float confidence = 10.0f; //fusion的confidence阈值
- float depth = 3.0f; //去掉depth大于某个阈值的帧
- float icp = 10.0f; //icp的阈值
- float icpErrThresh = 5e-05; //icp错误阈值
- float covThresh = 1e-05;
- float photoThresh = 115;
- float fernThresh = 0.3095f; //新fern的阈值
- int timeDelta = 200;
- int icpCountThresh = 35000;
- //int start = 1;
- //int end = std::numeric_limits<unsigned short>::max(); //Funny bound, since we predict times in this format really!
- bool openLoop = 0; //open loop模式：不开启
- bool iclnuim = 0; //使用icl dataset:不使用
- bool reloc = 0; //重定位模式：先做重建，不开启重定位
- bool fastOdom = 0; //Fast odometry (single level pyramid) mode :不开启
- bool so3 = 1; //SO(3) pre-alignment in tracking：开启
- bool frameToFrameRGB = 0; //只做rgb图像的tracking：不开启
- int timestamp = 0;
- std::string savefilename = "test";
- cout << "Setting parameters done." << endl;
- cout << "Building eFusion..." << endl;
- pangolin::Params windowParams;
- windowParams.Set("SAMPLE_BUFFERS", 0);
- windowParams.Set("SAMPLES", 0);
- pangolin::CreateWindowAndBind("Main", width, height, windowParams);
- glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
- glPixelStorei(GL_PACK_ALIGNMENT, 1);
- ElasticFusion eFusion(
- openLoop ? std::numeric_limits<int>::max() / 2 : timeDelta,
- icpCountThresh, icpErrThresh, covThresh, !openLoop, iclnuim, reloc,
- photoThresh, confidence, depth, icp, fastOdom, fernThresh, so3,
- frameToFrameRGB, savefilename);
- cout << "Building eFusion done" << endl;
- //待处理文件的位置和下标
- std::string filedir = "../pic/";
- int file_start = 1;
- int file_end = 782;
- vector<Eigen::Matrix4f> pose;
- //三位重建
- for (int file_ptr = file_start; file_ptr <= file_end; file_ptr = file_ptr +1) {
- cout << "Processing frame : "<<file_ptr << " ..."<<endl;
- FRAME f = readFrame(file_ptr, filedir);
- unsigned char * rgb = f.rgb;
- unsigned short * dep = f.dep;
- eFusion.processFrame(rgb, dep, timestamp);
- Eigen::Matrix4f currPose = eFusion.getCurrPose();
- pose.push_back(currPose);
- cout << "current pose is : " << endl<<currPose << endl;
- }
- cout << "Processing frames done." << endl<<endl;
- cout<<"Saving Elastic-Fusion model..."<<endl;
- cout<<"Saving Elastic-Fusion model done"<<endl<<endl;
- /*
- //查看处理的结果
- int time = eFusion.getTick();//查看此时eFusion的系统时间
- cout<<"time : "<<time<<endl;
- Ferns keyfern = eFusion.getFerns();//关键帧dataset
- int ld_num = eFusion.getDeforms();//局部deformations的数量
- Deformation ld = eFusion.getLocalDeformation();//局部deformation图
- int gd_num = eFusion.getFernDeforms();//全局deformations的数量
- GlobalModel gm = eFusion.getGlobalModel();//全局deformation model:
- int CloudPoint_num = eFusion.globalModel.lastCount(); //点云的点数量
- Eigen::Vector4f * mapData = eFusion.globalModel.downloadMap(); //点云图
- for (unsigned int i = 0; i < CloudPoint_num; i++) {
- Eigen::Vector4f pos = mapData[(i * 3) + 0];
- if (pos[3] > eFusion.confidenceThreshold) {
- //这是个有效的顶点,validCount++;
- }
- }//调用点云中的每一个点
- */
- cout << "saving cloud points..." << endl;
- eFusion.savePly(); //保存当前的点云图至ply
- cout << "cloud point has saved to " << savefilename << ".ply" << endl;
- return 0;
- }
- FRAME readFrame(int index, std::string filedir) {
- FRAME f;
- // 文件目录
- string rgbDir = filedir + "rgb/";
- string depthDir = filedir + "depth/";
- // 文件后缀
- string rgbExt = ".png";
- string depthExt = ".png";
- stringstream ss;
- // 读rgb图
- ss << rgbDir << index << rgbExt;
- string filename;
- ss >> filename;
- f.rgb_mat = cv::imread(filename);
- f.rgb = f.rgb_mat.data;
- // 读depth图
- ss.clear();
- filename.clear();
- ss << depthDir << index << depthExt;
- ss >> filename;
- f.dep_mat = cv::imread(filename, -1);
- f.dep = (unsigned short*) f.dep_mat.data;
- f.frameID = index;
- return f;
- }
扫一扫
9175
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
