MIAS-LCEC Toolbox Installation Instruction

MIAS-LCEC Installation Instruction

Project Website：MIAS Group

Article：
Online, Target-Free LiDAR-Camera Extrinsic Calibration via Cross-Modal Mask Matching

Toolbox Code：ZWhuang666/MIAS-LCEC (github.com)

Toolbox Demo：bilibili video

Dataset Download：[Google Drive ] [BaiDuYun]

Introduction

This project consists of two different codes package, C++and Python, and requires the installation of dependencies for both C++ and Python.

Suggestion Operation System

• Ubuntu 22.04
• Ros2 humble

C++ Dependencies

• ROS2

• PCL

• OpenCV

• Iridescence

• GLFW (zlib/libpng license)

• gl3w (Public domain)

• Dear ImGui (MIT license)

• ImGuizmo (MIT license)

• implot (MIT license)

• Eigen (MPL2 license)

• portable-file-dialogs (WTFPL license)

Python Dependencies

• Pytorch
• MobileSAM
• timm

Install C++ dependencies

# Install dependencies for Iridescence
sudo apt-get install -y libglm-dev libglfw3-dev libpng-dev libjpeg-dev libeigen3-dev libboost-filesystem-dev libboost-program-options-dev

# Install Iridescence for visualization
##Option1, copy the iridescence attached with this software (under the path "~/MIAS-LCEC/bin/iridescence") to your computer.
mkdir iridescence/build && cd iridescence/build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
sudo make install
##option2, download iridescence.this method might have some problem if iridescence updated.
git clone https://github.com/koide3/iridescence --recursive
mkdir iridescence/build && cd iridescence/build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
sudo make install

# in some case the zvision can't find iridescence package, please add the /usr/local/lib into the ld.so.conf using below command
sudo vim /etc/ld.so.conf

# make the config into effective
sudo /sbin/ldconfig -v

Install Python Environments

A new environment must be created and all the dependencies shall be installed under this environment, this document use pytorch_env as the environment name , users can change this environment name

# Install miniconda3
# download Miniconda3-py39_4.12.0-Linux-x86_64.sh from website
bash Miniconda3-py39_4.12.0-Linux-x86_64.sh

# create and activate pytorch environment
# make sure the python version consistent with the python version of ROS2
conda create -n pytorch_env python=3.10
conda activate pytorch_env

# install pytorch and cuda in the pytorch environment
# make sure the cuda version keep consistent in below two commands
# make sure the Nvida driver is installed correctly
nvidia-smi
conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=11.8 -c pytorch -c nvidia

# install pytorch for CPU if Nvidia display card is not available
## Option 1
conda install torch
## Option 2
pip install torch
## Option 3
pip install torch-2.1.2+cpu-cp310-cp310-linux_x86_64.whl

# install MobileSam
# option1:download MobileSam and run below command
cd ~/MIAS-LCEC/bin/MobileSAM
pip install -e.
# option2
pip install git+https://github.com/ChaoningZhang/MobileSAM.git

# install timm/pyautogui/pygame
pip install timm

Download and configure MIAS-LCEC

Download MIAS-LCEC from GitHub and configure the environment path of the program.

# clone git source
cd ~
git clone https://github.com/ZWhuang666/MIAS-LCEC.git
cd ~/MIAS-LCEC/

# activate your conda environment and make the configurtaion
conda activate pytorch_env
python config.py

If correctly operated, the file “~/MIAS-LCEC/bin/python/config.json” will be like:

{
    "path": {
        "ros": "/bin/python",
        "conda": "your path to conda/envs/pytorch_env/lib/python3.10/site-packages",
        "sam": "~/MIAS-LCEC/bin/MobileSAM",
    },
    "Debug": {}
}

Please make sure to do conda activate [your conda path] before tapping python config.py. If the program failed to run python code, please check your conda environment path using:

conda env list

and write the correct path to “~/MIAS-LCEC/bin/python/config.json” manually.