各相机模型(针孔+鱼眼)综述

Overview

欢迎访问 持续更新：https://cgabc.xyz/posts/c60702bf/

Lens Projections

• About the various projections of the photographic objective lenses
• Models for the various classical lens projections
• 鱼眼相机成像、校准和拼接(笔记)
• Computer Generated Angular Fisheye Projections

• Perspective and fisheye imaging process

Optics: Terminology

• Dioptric: All elements are refractive (lenses)
• Catoptric: All elements are reflective (mirrors)
• Catadioptric: Elements are refractive and reflective (mirrors + lenses)

Papers

• Straight Lines Have to be Straight: Automatic Calibration and Removal of Distortion from Scenes of Structured Environments
• A Generic Camera Model and Calibration Method for Conventional, Wide-Angle, and Fish-Eye Lenses
• Single View Point Omnidirectional Camera Calibration from Planar Grids

Related Code

• Supported models in Kalibr

• Distortion Models in ROS (distortion_models.h)

  • plumb_bob: a 5-parameter polynomial approximation of radial and tangential distortion
  • rational_polynomial: an 8-parameter rational polynomial distortion model
  • equidistant (lunar)

• hengli/camodocal

  • Pinhole camera model
  • Unified projection model
  • Equidistant fish-eye model

• uzh-rpg/rpg_vikit/vikit_common: support pinhole, atan and omni camera

• ethz-asl/aslam_cv2: support pinhole and unified peojection and radtan, fisheye and equidistant distortion

• cggos/okvis_cg: support pinhole peojection and radtan and equidistant distortion

• ptam_cg/src/ATANCamera.cc: support ATAN camera model

Pinhole camera

• pinhole model (rectilinear projection model) + radial-tangential distortion

The Pinhole camera model is the most common camera model for consumer cameras. In this model, the image is mapped onto a plane through perspective projection. The projection is defined by the camera intrinsic parameters such as focal length, principal point, aspect ratio, and skew.

Fisheye camera

OpenCV fisheye camera model

• pinhole model (rectilinear projection model) + fisheye distortion

The Fisheye camera model is a camera model utilized for wide field of view cameras. This camera model is neccessary because the pinhole perspective camera model is not capable of modeling image projections as the field of view approaches 180 degrees.

Given a point $X=[x_c{y}_c{z}_c]$ from the camera $z_c=1$ plane in camera coordinates, the pinhole projection is:

$$\{\begin{array}{l}r=\sqrt{x_c^2+y_c^2}\\ \theta =atan2(r,|z_c|)=atan2(r,1)=atan(r)\end{array}$$

in another way

$$f=r^{\prime }\cdot tan(\theta )\text{where}{r}^{\prime }=\sqrt{u^2+v^2}$$

fisheye distortion:

$${\theta }_d=\theta (1+k1\cdot {\theta }^2+k2\cdot {\theta }^4+k3\cdot {\theta }^6+k4\cdot {\theta }^8)$$

The distorted point coordinates are

$$\{\begin{array}{l}{x}_d=\frac{{\theta }_d\cdot x_c}{r}\\ y_d=\frac{{\theta }_d\cdot y_c}{r}\end{array}$$

convert into pixel coordinates, the final pixel coordinates vector

$$\{\begin{array}{l}u=f_x(x_d+\alpha y_d)+c_x\\ v=f_y\cdot y_d+c_y\end{array}$$

write in matrix form

$$\begin{array}{r}\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}{f}_x& \alpha & c_x\\ 0& f_y& c_y\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{x}_d\\ y_d\\ 1\end{array}\right]\end{array}$$

• Fisheye camera model (OpenCV)
• FisheyeCameraModel (theia-sfm)

ATAN model

• pinhole model (rectilinear projection model) + FOV distortion
• ptam_cg/src/ATANCamera.cc

This is an alternative representation for camera models with large radial distortion (such as fisheye cameras) where the distance between an image point and principal point is roughly proportional to the angle between the 3D point and the optical axis. This camera model is first proposed in Straight Lines Have to be Straight: Automatic Calibration and Removal of Distortion from Scenes of Structured Environments.

Given a point $X=[x_c{y}_c{z}_c]$ from the camera $z_c=1$ plane in camera coordinates, the pinhole projection is:

$$r=\sqrt{\frac{x_c^2+y_c^2}{z_c^2}}=\sqrt{x_c^2+y_c^2}$$

FOV distortion:

$$r_d=\frac{1}{\omega }arctan(2\cdot r\cdot tan(\frac{\omega }{2}))$$

where $\omega$ is the FOV distortion coefficient

The distorted point coordinates are

$$\{\begin{array}{l}{x}_d=\frac{r_d}{r}\cdot x_c\\ y_d=\frac{r_d}{r}\cdot y_c\end{array}$$

convert into pixel coordinates, the final pixel coordinates vector

$$\begin{array}{r}\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}{f}_x& 0& c_x\\ 0& f_y& c_y\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{x}_d\\ y_d\\ 1\end{array}\right]\end{array}$$

Equidistant fish-eye model

Omnidirectional Camera