3d指向检测 ros_3d_pointing_detection

 

 Introduction

The workflow of this project is:

1. Detect 2D human joints from color image by Openpose.
2. Compute 3D human joints from 2D joints and depth image.
3. Detect target objects by YOLO.
4. If the right arm is fully stretched, then the person is doing a "pointing" action. (The arm is stretched if the angle between the upper arm and forearm is smaller than the threshold.)
5. The pointing direction(ray) is defined as the vector from shoulder to wrist.
6. The 3D pixel that is in front of the wrist and is very close to the pointing ray is where the person is pointing to.
7. Backproject this 3D pixel to 2D image. If this 2D pixel is in one of the objects' bounding boxes, then we know the person is pointing to that object! Done!

https://github.com/felixchenfy/ros_3d_pointing_detection

依赖项 openpose 3d版

https://github.com/felixchenfy/ros_openpose_rgbd