文章目录
Paper information
GanHand: Predicting Human Grasp Affordances in Multi-Object Scenes
Paper
Code
Video
Introduction
Target
Given a single RGB image of a scene with an arbitrary number of objects, we aim to predict human grasp affordances,
- i.e. predict multiple plausible solutions of how a human would grasp each one of the observed objects
- This implies producing valid hand configurations showing several contact points with the target object but no intersection with other elements of the scene.
Contribution
- estimates the 3D shape/pose of the objects
- predicts the best grasp type according to a taxonomy with 33 classes
- refines the hand configuration given by the grasping class, through an optimization of the 51 parameters of the MANO model
- YCB affordance dataset
Previous work
1 33-class grasp taxonomy
2 differentiable hand model
Affordance: opportunities of interaction in the scene
Method
Notation | Meaning | Notes |
---|---|---|
I I I | image | |
M M M | model | |
C C C | grasp type | a coarse hand representation, within the 33-grasp taxonomy |
P P P | hand pose | |
V V V | hand shape | set of vertices of hand mesh |
H H H | ={P, V} | 51-DoF parameters of MANO model |
H C H_C HC | representative hand configuration of class C C C | |
T o b j e c t T_{object} Tobject |