Self-Calibration for D400 Series

 

Self-Calibration for D400 Series

Intel® RealSense™ Self-Calibration for D400 Series Depth Cameras

Background

• Prerequisites
  • Firmware version 5.12.02.100
  • LibRS version 2.33
• New calibration features with firmware
  1. Self-calibration
     1. Restore depth performance (relative error)
  2. Tare
     1. Restore depth accuracy (absolute error)
  3. Health-Check

     1. Optional after calibration; gives a direct measure of calibration state and validates that a device is calibrated well

• Benefits
  • On-chip, zero load on host CPU
  • Works on any OS
  • Very fast (<2 sec)
  • Low power
  • Can work target-less (but works best with specific types of target)
  • Requires no motion or repositioning during calibration
  • USB2 & and USB3
  • Secure IP
• Corrects either intrinsic or extrinsic distortions, but not both at the same time
  • Can select which to run
  • Intrinsic – default and recommended
  • Both will correct the central 20% FOV, but differ in how they correct edges of FOV
• Option to reset to original factory calibration

Setup

Recommended Ideal Conditions

• Target
  • Ideally use a target, exact target doesn't matter as long as it's random textured pattern (see Appendix A of white paper for example)
  • Target does NOT have to be perpendicular to camera for calibration
• Distance
  • Place camera far enough away that it is beyond minimum working distance (0.3 m for D415)
  • Must be close enough that 35% of target texture is visible in image
• Turn off IR (recommended)

Beyond Ideal Conditions

• Works well under a wide variety of conditions
• Constraint: needs to be good texture in the scene
• As long as a healthy camera has a good depth map with high fill ratio, then scene is well suited for calibration
• Conditions under which "good" depth would be expected in the central 20% ROI should provide good self-calibration results
• Does not need to be flat
• Can be static or moving
• Examples of scenes that Intel has confirmed work well (Fig. 6 of white paper)
  • Ideal scene: well textured flat target with projector ON
  • White flat wall with projector ON
  • Textured carpet
  • Cluttered desktop
  • Flower on a table top
  • Rocks on a patio, outside in bright sunlight
  • Face indoor with projector turned ON
  • Face indoors without projector
  • Face indoors without projector

Notes on using a white wall

• Not recommended to use a white wall for the D415 (regular laser pattern of projector causes problems because it needs to be textured scene)
• For D415, use textured surface, projector turned OFF, and standard speed
• Special calibration mode for using a white wall with no texture or laser pattern projector turned on 
  • Speed settings - "white wall"
  • Use "High Accuracy depth settings"

Self-Calibration

• Restores depth performance
  • focuses on ability of cameras to see objects and report back their position
  • improves precision, or relative error
• Health-Check can be done once self-cal is done
  • No target required
  • Indicates extent of how good the calibration is through a number
  • Tells whether or not calibration should be re-done
  • Absolute value of <0.25 is acceptable
  • Accurate and can be relied on without requiring a flat wall characterization in Depth Quality Tool

Self-Calibration Using Command-Line 

// Commands to run self-calibration // 1. Get pointer to current calibration table, before running on-chip calibration CalTableBefore = rs2_get_calibration_table(); // 2. Runs the on-chip self-calibration routine that returns pointer to new calibration table. CalTableAfter = rs2_run_on_chip_calibration(); // 3. Toggle between calibration tables to assess which is better. (OPTIONAL) rs2_set_calibration_table(CalTable); // Optional to validate with Depth Quality Tool before permanently burning to FW in Step 4 // 4. Burns the calibration to FW persistently. rs2_write_calibration

Self-Calibration Using RealSense Viewer

• Functions can be run on RealSense Viewer and Depth Quality Tool rather than command window
• RGB imager can be left on or off during cal
• Steps
  1. Set up camera as previously described
  2. Speed and mode (extrinsic/intrinsic) can be selected
  3. Select on-chip calibration
     1. viewer automatically sets to 256x144 resolution (returns to normal resolution when calibration is done)
  4. Provide Health-Check results
     1. User can toggle between original and new calibration before deciding whether to apply or dismiss new calibration
  5. Select "Apply New" to burn to flash
     1. "Dismiss" cancels to keep original calibration
• Health-Check number is returned as normalized "Calibration Error" 

Tare-Calibration 

• Restores depth accuracy, or absolute error
  • corrects absolute distance measurement (slope and offset of depth measurement vs distance)
  • Compare to tare with a scale: remove bias and reset to a known value
• Recommended to run self-cal before running Tare
• DOES require a flat surface of a known distance
• User must input a ground truth measurement 
• Tips on making a good ground truth measurement (can be a challenge)
  • Measured perpendicular distance from left camera lens, 2mm recessed from the front of the D415 windows
  • Make sure target is completely flat and perpendicular to the camera
  • If using a laser range finder, ensure it has proper resolution
• Change presets to "High Accuracy Depth" mode (recommended)
• Similar process as running self-calibration

Tare-Calibration Using Command Line

// Commands to run tare-calibration // 1. Call tare function - argument is ground truth depth rs2_run_tare_calibration <ground truth depth> // 2. Burns the calibration to FW persistently. rs2_write_calibration

Tare-Calibration Using RealSense Viewer

• Steps
  1. Set up as previously described (note: must use a flat surface at a known distance)
  2. Select Tare Calibration
  3. Select intrinsic/extrinsic mode
  4. Enter ground truth distance
  5. All other settings can be left at default
  6. Once completed, user can choose between new and original calibrations