论文笔记:7DOF

4.1 The set-up of simulation environment

In order to meet the demand for a large dataset, we proposed a new method to generate images and ground truth labels from CAD models by simulation. Then we use the method to ShapeNetSem, a subset of ShapeNet , which include more than 12k objects of 82 categories. Fortunately, shapeNetSem provides the density and static friction coefficient of most objects and we can utilize these information to generate robust grasp candidates. And we get a new dataset with more than 400k images of 12k objects and **** grasp positions, which have already marked whether it is a successful grasping position. The main process we generate data is illustrated on Figure. We used Blender to generate RGBD images with Cycles Render and chosed pyBullet library for physics simulation.

4.2 Data creation and annotations

A.Environment setting

To reduce the impact of background, we create the world with a brightness and a plane without texture . Then we set a Lamp at a fixed location. Later we import an object from CAD models. Due to the objects in ShapeNetSem have a large range of scales and some posture of the objects is unstable in practice. So we rescale the model and the longest side of the objects’ axis alignedbounding box is no longer than 150mm and the shortest side is no larger than 60mm. We drop the object from a point above the plane and get the stable position and orientation. Both the pyBullet Simulation environment and blender keep the same configuration.
For each objects, wet set three different distances between the camera and the object and with each distance we create 12 views around the objects and 1 view directly above the object.

B. Image rendering

We use Blender with Cycles engine to rendered RGB and depth images. We can get the depth  buffer in Blender. Cause the depth buffer of Cycles camera model is the distance between a given point and the camera instead of the camera plane. We change the depth buffer in Blender to the actual depth. Since we can transform Camera coordinate system to pixel coordinate system  with the camera intrinsic matrix, K by:
  $$\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=[K]\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\end{array}\right]$$,
  where (u,v) is the point in pixel coordinate and \((X_c,Y_c,Z_c)\)is the point in camera Coordinate. And  with the  pixel coordinate system and intrisic matirx K, we calculate the Camera coordinate system:
  $$\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\end{array}\right]=[K{]}^{-1}\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]$$
  .The conversion from depth buffer \(z^n\) to real depth \(z^e\) is given by \(f(z^e) = f^e(zn)=\frac{z^n×Z_c}{X_c2+Y_c^2+Z_c2}\).
For each view, a mask image separating the object and the background is provided .