Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping...

Research Topic

A robotic pick-and-place system that is capable of grasping and recognizing both known and novel objects in cluttered environments.
这里的novel object是指不需要task-specific training data for novel objects

Contribution

文章的贡献就是 提出了两个framework

• object-agnostic grasping framework: visual observations $\to$ actions
  得到dense pixel-wise probability maps of the affordances for four different grasping primitive actions.
  (affordance 和 probability of picking success差不多意思)
• cross domain image classification framework
  match observed images to product images，起到object recognition的作用

Related Work

有两大类的方法：Recognition followed by Model-based Grasping

1. classical solutions for robotic picking——standard two-step approach:

   1. recognition and pose estimation
   2. model-based grasp planning

   这些方法都需要3D models of the objects during test time, and/or training data with the physical objects themselves.
   因此都很难扩展到novel objects.

   这篇文章提出的grasping framework其实没有object identity这步，因此自然也就object-agnostic了。

2. Recognition in parallel with Object-Agnostic Grasping
   这类方法就是exploit local features without object identity了
   they still need some form of in-place object recognition in order to associate grasp proposals with object identities.

3. Active Perception
   exploiting control strategies for acquiring data to improve perception.
   这篇文章也是受这种思想启发，follow a $grasp-first-then-recognize$ work flow.

System Overview

overall approach

1. uses FCNs to infer the pixel-wise affordances of four different grasping primitive action
2. 选择affordance最高的primitive action, pick up the object, isolated it from the clutter, hold it up in front of the camera
3. recognizes its category, and place it in the appropriate bin

system setup

• 6 DOF ABB IRB1600id robot arm
• 自己设计的末端装置，既能抓也能吸
• RealSense SR300 RGB-D cameras

Multi-Affordance Grasping