【论文笔记】Fully automated region of interest segmentation pipeline for UAV based RGB images

Fully automated region of interest segmentation pipeline for UAV based RGB images

keyword：无人机 目标检测 ROI提取 YOLOV4

1.摘要

Unmanned Aerial Vehicles (UAVs) have exhibited its potential for efficient and noninvasive crop data acquisition in high throughput crop phenotyping. In general, for analysis of phenotypic traits, there is a need for extracting the region of interest (RoI) from
images captured by UAVs. It involves the generation of orthomosaic, which is a complicated and time-intensive process. In this study, a fully automated AI-based pipeline has
been proposed for the RoI segmentation from raw RGB images acquired via UAV. The
proposed pipeline achieves a near real-time processing speed compared to the other
baseline methods. The key feature of the pipeline is the introduction of Sub-Paths, in which
the original UAV flight path is divided into several small paths which facilitates parallel
processing. The image quality of the extracted RoI has been examined using blind/referenceless image spatial quality evaluator (BRISQUE) and natural image quality evaluator
(NIQE). The performance of the proposed pipeline is exemplified with the Leaf Area Index
(LAI) estimation on five datasets containing three different crop types and growth stages.
Regression analysis has also been performed on the estimated LAI values. Average R2,
RMSE, and correlation scores of the estimates are observed to be 0.68, 0.033, and 0.83,
respectively
该方法的关键特点是引入了子路径，其中将原无人机的飞行路径划分为几条小路径，便于并行处理。利用盲/无参考图像空间质量评价器(brisque)和自然图像质量评价器(NIQE)对提取的RoI的图像质量进行了检测。对包含三种不同作物类型和生长阶段的5个数据集进行了叶面积指数(Lai)估计，验证了该流程的性能。回归分析也得到了证实。

2.Introduction

A fully automated, fast end-to-end AI-based RoI segmentation pipeline from UAV-based RGB images is developed in this article. The concept of UAV Sub-Paths is leveraged where in the original flight path is disintegrated into several small paths, and all of them can be processed parallelly,thereby increasing efficiency substantially. Processing time would be proportional to the number of Sub-Paths obtained.
After Sub-Path identification, a density-based approach is proposed to identify the optimal Sub-Paths, which further lead to a decrease in the runtime of the pipeline. The image quality of the RoIs obtained from both the orthomosaic and the pipeline is analysed with no-reference image quality metrics. The proposed pipeline is tested with the Leaf Area Index (LAI) estimation for five different datasets consisting of three types of crops, namely Sorghum, Pigeon Pea, and Pearl Millet, to ensure varied vegetation. Regression analysis is performed on the experimental results to demonstrate the effectiveness of the pipeline.

3.Proposed pipeline

数据采集、子路径检测、基于密度的子路径选择、子图检测、重复子图去除和RoI分割。

3.1. Field study and data acquisition

在10米高度以5km/h的飞行速度拍摄的，从萌发到收获时间，每周定期，重叠度80%。

3.2. UAV Sub-Path detection

本研究提出了一种基于人工智能的ROI全自动分割方法，该方法不需要生成正射影像，可以直接在无人机原始影像上进行ROI分割。与其他分割方法相比，本研究提出的方法能够对无人机影像进行实时处理，简化了数据处理流程，提高了无人机影像的ROI分割效率。

本研究提出的ROI分割方法的最大特点是将无人机航线分为若干个“子航线”，并对每个子航线同时处理。本研究提出的方法包含以下几个步骤，分别为子航线探测、基于点云的子航线选取、子田块探测、删除重复子田块和ROI分割。其中， YOLOv4目标检测方法被用于ROI自动分割。ROI分割之后，利用无参考影像空间质量评估和自然影像质量评估方法分别对分割后的影像质量进行了评估，评估结果显示，使用本文方法分割的ROI影像质量和基于正射影像分割的ROI影像质量一致。因此，本研究提出的方法可用于基于无人机可见光影像的ROI自动分割。

4.result and val

此外，本研究还利用ROI分割后的结果估测高粱、豌豆和狼尾草等不同作物在不同生育时期的叶面积指数，并通过与地面实际测量值对比，评价叶面积指数的估测精度。拟合分析结果显示，估测叶面积指数的平均R2、RMSE和相关系数分别为0.68，0.033和0.83。综上所述，本研究提出的ROI分割方法可以替代传统的基于正射影像的ROI分割方法，提高ROI分割效率。