SLAM硬件加速之FPGA-base ORB(1)——Image Pyramid

残血到处浪

已于 2023-03-31 13:10:08 修改

阅读量740

点赞数

文章标签：计算机视觉 vr fpga开发

于 2023-03-22 10:04:52 首次发布

本文链接：https://blog.csdn.net/weixin_40817208/article/details/129694647

版权

该系列主要是总结一下自己看到的几篇关于FPGA加速论文。

背景

我们知道在VSLAM里视觉前端需要的计算资源开始占比是很大的，比如在ORB-SLAM中，ORB算子是一个关键的计算瓶颈，这部分的计算量可能占整个SLAM框架的50%甚至更多，特别在需要多摄像头(4摄)，高分辨率的应用场景中。因此出现了挺多针对ORB的硬件加速的方案，本系列主要针对的是基于FPGA的加速策略。

ORB pipeline

ORB可以拆解成多个小模块的组合：

Image Pyramid

FAST Detector

Feature Point Sorting

Non-Maximum

Suppression Rotated BRIEF

Matching

后面会针对这些模块来拆解不同模块的各种方案。

Image Pyramid

图像金字塔是为了尺度不变性。

Match counts with increasing pyramid heights for three different pairs of images (three levels per octave

Match counts with increasing pyramid levels per octave for three different pairs of images (9 levels total)

图片出自论文[2]

可以看到不同的图片金字塔层数和缩放比例会带来不一样的匹配效果，在计算量和精度直接进行tradeoff，可以看出金字塔层数在8，缩放比例在1.2～1.25都是不错的选择，在ORB-SLAM3的配置选择为scaleFactor: 1.2,nLevels: 8

关于这部分的优化并没有太多论文提及，下面介绍在论文[10]中提到的方法：

该方法采用的缩放比例是1.25，这样上一层5个像素映射下一层的4个像素，除了第一个像素直接映射，其他像素可以通过移位加来实现。

SLAM硬件加速之FPGA-base ORB(2)——FAST Detector

Reference

[1]FPGA-based ORB Feature Extraction for Real-Time Visual SLAM. 2017

[2]FPGA Acceleration of Multilevel ORB Feature Extraction for Computer Vision. 2017

[3]A New FPGA Architecture of FAST and BRIEF Algorithm for On-Board Corner Detection and Matching. 2018

[4]High-throughput and area-optimized architecture for rbrief feature extraction. 2018

[5]eSLAM: An Energy-Efficient Accelerator for Real-Time ORB-SLAM on FPGA Platform. 2019

[6]An Efficient Real-Time FPGA-Based ORB Feature Extraction for an UHD Video Stream for Embedded Visual SLAM. 2022

[7]HcveAcc: A High-Performance and Energy-Efficient Accelerator for Tracking Task in VSLAM System. 2020 [8]ac2SLAM: FPGA Accelerated High-Accuracy SLAM with Heapsort and Parallel Keypoint Extractor. 2023

[9]Area-Time Efficient Streaming Architecture for FAST and BRIEF Detector. 2018

[10]A Low Latency Feature Extraction Accelerator with Reduced Internal Memory. 2017

[11]A 42fps Full-HD ORB Feature Extraction Accelerator with Reduced Memory Overhead.2017

[12]A Flexible and Efficient Real-Time ORB-Based Full-HD Image Feature Extraction Accelerator. 2020

[13]ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras. 2016