使用BundleFusion重建個人數據集【全網最詳細】

寫在前面

推薦使用的項目代碼：

GitHub - FangGet/BundleFusion_Ubuntu_Pangolin: a porting for BundleFusion working on ubuntu, with Pangolin as visualizer

本人電腦配置：

本人電腦配置是Ubuntu 20.04+cuda 11.7+ 2080TI

使用iphone 進行場景的拍攝，獲取RGBD數據

 配置github環境

安裝docker環境

首先我認爲用電腦配置環境，我失敗了，所以我使用github推薦的docker進行配置，首先安裝docker和nvidia-docker

sudo apt-get install docker
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)    && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -    && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
sudo apt-get install -y nvidia-docker2
sudo systemctl restart docker

docker pull

sudo nvidia-docker pull nvidia/cudagl:11.4.0-devel-ubuntu20.04

docker build

sudo nvidia-docker build -t bundlefusion-cu11.4-cudagl:latest .

但是這個部分我遇到了很多的問題，比如說dockerfile中就是unable to fetch

因此我總是在改dockerfile，修改的如下，主要還是install 一些包什麼的

# FROM nvidia/cuda:11.6.0-devel-ubuntu20.04
FROM nvidia/cudagl:11.4.0-devel-ubuntu20.04
#https://hub.docker.com/r/nvidia/cudagl/

RUN apt update &&  \
    DEBIAN_FRONTEND="noninteractive" apt install -y --no-install-recommends  \
    wget unzip git make cmake gcc clang gdb libeigen3-dev libncurses5-dev libncursesw5-dev libfreeimage-dev \
    # libs for FFMPEG functionality in OpenCV
    libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libgtk-3-dev pkg-config && \
    rm -rf /var/lib/apt/lists/*

# Download and install OpenCV
ENV OPENCV_VERSION=4.5.4

# RUN wget -O opencv.zip https://github.com/opencv/opencv/archive/refs/tags/${OPENCV_VERSION}.zip && \
#     unzip opencv.zip && \
RUN git clone -v --progress https://github.com/opencv/opencv.git && \
    cd opencv && \
    git checkout tags/${OPENCV_VERSION} -b v${OPENCV_VERSION} && \
    mkdir -p build &&  \
    cd build && \
    #cmake -D WITH_CUDA=ON \
    cmake \
    -D BUILD_EXAMPLES=OFF -D BUILD_opencv_apps=OFF -D BUILD_DOCS=OFF -D BUILD_PERF_TESTS=OFF -D BUILD_TESTS=OFF .. && \
    # -D BUILD_EXAMPLES=OFF -D BUILD_opencv_apps=OFF -D BUILD_DOCS=OFF -D BUILD_PERF_TESTS=OFF -D BUILD_TESTS=OFF ../opencv-${OPENCV_VERSION} && \
    make -j 48 && \
    make install && \
    cd .. && \
    # rm opencv.zip && \
    # rm -rf opencv-${OPENCV_VERSION} && \
    rm -rf build

# Download and install pangolin
ENV PANGOLIN_VERSION=0.6
RUN wget http://archive.ubuntu.com/ubuntu/pool/universe/g/gcc-8/libgfortran-8-dev_8.4.0-3ubuntu2_amd64.deb
RUN apt-get update && apt-get install -y --no-install-recommends \
    gcc-8-base \
    libgcc-8-dev \
    libgfortran5 \
 && dpkg -i libgfortran-8-dev_8.4.0-3ubuntu2_amd64.deb
RUN apt-get update && apt-get install -y --no-install-recommends gcc-8-base libgcc-8-dev libgfortran5
RUN apt-get install -y libboost1.71-dev libboost-date-time1.71.0 libboost-serialization1.71-dev
RUN wget http://archive.ubuntu.com/ubuntu/pool/main/b/boost1.71/libboost-date-time1.71-dev_1.71.0-6ubuntu6_amd64.deb
RUN dpkg -i libboost-date-time1.71-dev_1.71.0-6ubuntu6_amd64.deb
RUN apt-get install -f
RUN apt-get install -f && apt update && apt install -y  libgl1-mesa-dev libglew-dev libboost-all-dev
RUN apt update --fix-missing && apt install -y  libgl1-mesa-dev libglew-dev libboost-all-dev
RUN git clone -v --progress https://github.com/stevenlovegrove/Pangolin.git && \
    cd Pangolin && \
    git checkout tags/v${PANGOLIN_VERSION} -b v${PANGOLIN_VERSION} && \
    mkdir -p build &&  \
    cd build && \
    cmake .. && \
    make -j48 && \
    make install && \
    cd .. && \
    rm -rf build

#pangolon fix #replace "GL/glew.h" with "/usr/include/GL/glew.h"
COPY pangolin_fix.h /usr/local/include/pangolin/gl/glplatform.h

WORKDIR /app


ENV NVIDIA_DRIVER_CAPABILITIES graphics,utility,compute


COPY CMakeLists.txt zParametersBundlingDefault.txt zParametersDefault.txt ./
# ADD cmake cmake
ADD include include
ADD src src
ADD example example


RUN mkdir build && \
    cd build &&  \
    cmake -DVISUALIZATION=ON .. &&  \
    make -j48

# RUN apt update && apt install -y mesa-utils
# ENTRYPOINT ["./build/bundle_fusion_example", "./zParametersDefault.txt", "./zParametersBundlingDefault.txt"] 

就此就已經安裝好環境了

跑通

建議使用github中推薦的方式，先下載office2數據集，再進行試跑

記得修改run_docker.sh中的docker名字

export DATASETS=path/to/dataset
./run_docker.sh

 跑通的結果如下啦：

修改個人數據集

我是使用IPHONE進行拍攝的RGBD數據

主要是修改color和深度圖轉爲名字爲這樣的格式，pose位姿就是單位矩陣就好了：frame-000000.color.jpg,frame-000000.depth.png,frame-000000.pose.txt,記得還要加info.txt

info.txt修改如下

m_versionNumber = 4
m_sensorName = StructureSensor
m_colorWidth = 1440
m_colorHeight = 1920
m_depthWidth = 1440
m_depthHeight = 1920
m_depthShift = 1000
m_calibrationColorIntrinsic = 1364 0 719 0 0 1364 966 0 0 0 1 0 0 0 0 1 
m_calibrationColorExtrinsic = 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 
m_calibrationDepthIntrinsic = 1364 0 719 0 0 1364 966 0 0 0 1 0 0 0 0 1  
m_calibrationDepthExtrinsic = 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 
m_frames.size = 878

至於修改圖片的名字的代碼如下：請修改好base_path

import os
import shutil

# 设置原始文件夹和目标文件夹
base_path = 'xxxxx'
color_folder = os.path.join(base_path, 'color')
depth_folder = os.path.join(base_path, 'depth')
data_folder = os.path.join(base_path, 'data')

# 创建data文件夹，如果不存在的话
if not os.path.exists(data_folder):
    os.makedirs(data_folder)

# 单位矩阵内容
identity_matrix = '1 0 0 0\n0 1 0 0\n0 0 1 0\n0 0 0 1'

# 重命名color和depth文件，并创建pose文件
for folder_name in ['color', 'depth']:
    folder_path = os.path.join(base_path, folder_name)
    for filename in sorted(os.listdir(folder_path)):
        if filename.endswith('.png'):
            # 提取文件编号
            frame_number = int(filename.split('.')[0])-1
            # 设置新的文件名格式
            new_filename = f'frame-{frame_number:06d}.{folder_name[:]}.png'
            new_filename_jpg = new_filename.replace('.png', '.jpg') if 'color' in folder_name else new_filename
            new_filepath = os.path.join(data_folder, new_filename_jpg)

            # 复制文件到data文件夹并重命名
            shutil.copyfile(os.path.join(folder_path, filename), new_filepath)

            # 如果是彩色图像，还需要将其转换为JPG
            if 'color' in folder_name:
                os.rename(new_filepath, new_filepath.replace('.png', '.jpg'))

            # 创建pose文件
            pose_filename = f'frame-{frame_number:06d}.pose.txt'
            pose_filepath = os.path.join(data_folder, pose_filename)
            with open(pose_filepath, 'w') as f:
                f.write(identity_matrix)

print("Dataset conversion is complete.")

然後是修改zParametersDefault.txt和zParametersBundlingDefault中的圖片大小参数、相機內參，還有PrimeSenseSensor.cpp文件，然后将“RGBDSensor::init(xx,xx,xx,xx,xx)”更改

具體請看：BundleFusion_Ubuntu_Pangolin usage_fangget/bundlefusion_ubuntu_pangolin-CSDN博客

 修改完再進行docker build一下，再跑run docker 的sh，就好啦

對於數據的後處理

後期再寫吧，希望有人催更hhhh