基于激光雷达点云的3d对象检测实现，共2部分，共2部分

最新推荐文章于 2024-05-30 19:17:05 发布

weixin_26706653

最新推荐文章于 2024-05-30 19:17:05 发布

阅读量397

点赞数

文章标签： python java

原文链接：https://towardsdatascience.com/lidar-point-cloud-based-3d-object-detection-implementation-with-colab-part-2-of-2-f3ad55c3f38c

版权

VoxelNet a point cloud based 3D object detection algorithm is implemented using google colab.

VoxelNet基于点云的3D对象检测算法是使用Google colab实现的。

In my previous article, I have explained crucial concepts required to implement the VoxelNet an end-to-end learning model for the 3d object detection you can find here

在上一篇文章中，我已经解释了实现VoxelNet 3D对象检测的端到端学习模型所需的关键概念。

In continuation with the previous article, we will work out to implement the VoxelNet algorithm for the 3d object detection using KITTI point cloud data

在前一篇文章的基础上，我们将努力实现使用KITTI点云数据实现VoxelNet算法用于3d对象检测的方法。

Step 1: Processing the KITTI dataset to train the model[Detailed steps]

步骤1：处理KITTI数据集以训练模型[详细步骤]

The following steps are crucial to prepare the data to train the model. KITTI dataset needs to be downloaded, cropped, processed, and save in the drive.

以下步骤对于准备训练模型的数据至关重要。 KITTI数据集需要下载，裁剪，处理并保存在驱动器中。

登录google colab ，打开笔记本，目前不需要GPU。

Note: in case if GPU is selected, note that each user is provided with ~30GB of virtual space and the entire dataset has a size of ~40+ GB and then to unzip dataset memory will run out. In the next steps we will download the KITTI datset, process, crop and create .zip and move to the drive to use the data for future

注意：如果选择了GPU，请注意，每个用户都拥有大约30GB的虚拟空间，整个数据集的大小约为40+ GB，然后解压缩数据集的内存将用光。在接下来的步骤中，我们将下载KITTI数据集，进行处理，裁剪并创建.zip，然后移至驱动器以将数据用于将来

Required dataset
所需数据集

Velodyne point clouds (29 GB): input data to VoxelNet
Velodyne点云(29 GB)：向VoxelNet输入数据

2. Training labels of the object data set (5 MB): input label to VoxelNet

2.对象数据集的训练标签(5 MB)：输入标签到VoxelNet

3. Camera calibration matrices of the object data set (16 MB): for visualization of predictions

3.对象数据集的相机校准矩阵(16 MB)：用于可视化预测

4. Left color images of the object data set (12 GB): for visualization of predictions

4.对象数据集的左侧彩色图像(12 GB)：用于可视化预测

#clone the voxelnet git repo
!git clone https://github.com/gkadusumilli/Voxelnet.git
#change the cwd
%cd /content/drive/My Drive/Voxelnet/crop_data

To download the datasets (.zip) directly to the colab virtual machine [~ 15–20 mins]
要将数据集(.zip)直接下载到colab虚拟机中[〜15–20分钟]

#Data label file
!wget (enter the link)#Calib file
!wget (enter the link )#Velodyne file
!wget (enter the link)# Image file
!wget (enter the link)

Unzip the datasets(.zip) to the folder (~20- 30 mins)
将数据集(.zip)解压缩到文件夹(约20至30分钟)

#Unzip the velodyne training folder
!unzip /content/Voxelnet/crop_data/data_object_velodyne.zip 'training/*' -d /content/Voxelnet/crop_data#Unzip the image training folder
!unzip /content/Voxelnet/crop_data/data_object_image_2.zip 'training/*' -d /content/Voxelnet/crop_data#unzip the object label
!unzip /content/Voxelnet/crop_data/data_object_label_2.zip#unzip the data object calib
!unzip /content/Voxelnet/crop_data/data_object_calib.zip 'training/*' -d /content/Voxelnet/crop_data

cropping point cloud data for training and validation. Point clouds outside the image coordinates are removed. cropped point cloud will overwrite the existing raw point cloud [40–45 mins]
裁剪点云数据以进行培训和验证。图像坐标外部的点云将被删除。裁剪的点云将覆盖现有的原始点云[40-45分钟]

#to run the 'crop.py' lower version of scipy is needed
!pip install scipy==1.1.0#run crop.py
!python crop.py

Create the validation data to evaluate the model[~10–15 mins]
创建验证数据以评估模型[〜10-15分钟]

#create a folder 'validation' and copy the content in training #folder!mkdir /content/Voxelnet/crop_data/validation%cp -av /content/Voxelnet/crop_data/training /content/Voxelnet/crop_data/validation/

we will split the training as per the protocols here [2–3 mins]
我们将按照此处的协议划分培训[2-3分钟]

Final step…

最后一步…

create .zip the folder of the processed, cropped data [30–40mins]
创建.zip处理的裁剪数据的文件夹[30–40分钟]

!zip -r /content/VoxelNet/data/data_lidar.zip /content/VoxelNet/crop_data

Move the .zip folder to the drive [~5 mins]
将.zip文件夹移至驱动器[〜5分钟]

#rename the folder as you need in the drive, I've stored in dtive with the folder named 'AI'!mv "/content/VoxelNet/data/data_lidar.zip" "/content/gdrive/My Drive/AI"

Here is the link to the colab jupyter notebook with all the above-mentioned steps.

这是所有上述步骤的colab jupyter笔记本的链接。

步骤2：训练模型 (Step 2: Training the model)

log in to google colab , create a new notebook
登录到谷歌合作实验室，创建一个新的笔记本
To access the GPU: Click on runtime > Change runtime type > GPU
要访问GPU：单击运行时>更改运行时类型> GPU

Note: The colab GPU runtime is approximate ~12 hours post that, the runtime will be disconnected, the data stored will be lost. Since in our case each epoch would take ~2 hours and need to train for more than 20 epochs to observe the preliminary results. So will use google drive as a path to store the code, checkpoints, prediction results, and so on

注意：colab GPU运行时大约在12小时后，运行时将被断开，存储的数据将丢失。由于在我们的案例中，每个时期都需要2个小时左右的时间，因此需要训练20个以上的时期才能观察到初步结果。因此，将使用Google驱动器作为存储代码，检查点，预测结果等的路径

Change the current working directory(CWD) to drive
将当前工作目录(CWD)更改为驱动器

#voxelnet is the folder name, you can rename as you need
%cd /content/drive/My Drive/Voxelnet

VoxelNet implementation needs several dependencies. So we will clone the entire repository.
VoxelNet的实现需要几个依赖项。因此，我们将克隆整个存储库。

!git clone https://github.com/gkadusumilli/Voxelnet.git

Building the files
建立档案

%cd /content/drive/My Drive/Voxelnet!python setup.py build_ext --inplace

Unzip the processed dataset in the crop_data folder [35–40 mins]
将处理后的数据集解压缩到crop_data文件夹中[35–40分钟]

%cd /content/drive/My Drive/Voxelnet/crop_data#Locate the zip folder in the drive and unzip using !unzip command
!unzip "/content/drive/My Drive/AI/data_lidar.zip"

Training the model
训练模型

%cd /content/drive/My Drive/Voxelnet/

Important argument parse

重要参数解析

parser = argparse.ArgumentParser()
  parser.add_argument("--strategy", default="all", help="Distributed or centralized training (options : all for all available gpus or string of gpus numbers separated by commas like '0,1')", type=str)


  parser.add_argument("--batch_size", default=2, help="Total batch_size", type=int)
  parser.add_argument("--n_epochs", default=160, help="Total Number of epochs to train the model", type=int)


  parser.add_argument("--learning_rate", default=1e-3, help="Learning rate", type=float)
  parser.add_argument("--small_addon_for_BCE", default=1e-6, help="Small addon to add to the binary asymetric cross entropy for the loss", type=float)
  parser.add_argument("--max_gradient_norm", default=5.0, help="Maximum gradient norm to clip into", type=float)
  parser.add_argument("--alpha_bce", default=1.5, help="Alpha BCE", type=float)
  parser.add_argument("--beta_bce", default=1.0, help="Beta BCE", type=float)
  parser.add_argument("--huber_delta", default=3.0, help="Huber loss epsilon", type=float)


  parser.add_argument("--dump_vis", default="no", help="Boolean to save the viz (images, heatmaps, birdviews) of the dump test (yes or no)", type=str2bool)
  parser.add_argument("--data_root_dir", default="", help="Data root directory", type=str)
  parser.add_argument("--model_dir", default="", help="Directory to save the models, the viz and the logs", type=str)
  parser.add_argument("--model_name", default="", help="Model Name", type=str)
  parser.add_argument("--dump_test_interval", default=-1, help="Launch a dump test every n epochs", type=int)
  parser.add_argument("--summary_interval",default=-1, help="Save the training summary every n steps", type=int)
  parser.add_argument("--summary_val_interval", default=-1, help="Run an evaluation of the model and save the summary  every n steps and", type=int)
  parser.add_argument("--summary_flush_interval", default=-1, help="Flush the summaries every n steps", type=int)
  parser.add_argument("--ckpt_max_keep", default=11, help="Max checkpoints to keep", type=int)

!python train.py \--strategy="all" \--n_epochs=16 \--batch_size=2 \--learning_rate=0.001 \--small_addon_for_BCE=1e-6 \--max_gradient_norm=5 \--alpha_bce=1.5 \--beta_bce=1 \--huber_delta=3 \#if dump_vis == yes, boolean to save visualization results
--dump_vis="no" \--data_root_dir="/content/drive/My Drive/Voxelnet/crop_data" \--model_dir="model" \--model_name="model6" \--dump_test_interval=3 \--summary_interval=2 \--summary_val_interval=40 \--summary_flush_interval=20 \--ckpt_max_keep=10 \

使用张量板可视化日志目录 (Visualizing the log dir with the tensorboard)

%load_ext tensorboard#summary_logdir is the logdir name
%tensorboard --logdir summary_logdir

The below snap is the logdir results @ epoch6

下面的快照是@ diroch6的logdir结果

评估模型 (Evaluating the model)

!python predict.py \
--strategy="all" \
--batch_size=2 \
--dump_vis="yes" \
--data_root_dir="../DATA_DIR/T_DATA/" \
--dataset_to_test="validation" \
--model_dir="model" \
--model_name="model6" \
--ckpt_name="" \

The code to download, crop, process the KITTI dataset can be found here
下载，裁剪，处理KITTI数据集的代码可在此处找到
The code to implement from step 2 can be found here
可在此处找到第2步要实施的代码

Here are some of the results obtained after training the model for 30 epochs.

这是训练模型30个纪元后获得的一些结果。