OpenCV学习之stereoRectify()函数及initUndistortRectifyMap()函数参数说明

OpenCV学习之stereoRectify()函数及initUndistortRectifyMap()函数

内容：关于OpenCV中提供的stereoRectify（）函数参数说明。

一、stereoRectify()函数

CV_EXPORTS_W void stereoRectify(InputArray cameraMatrix1, 
InputArray distCoeffs1,
InputArray cameraMatrix2, 
InputArray distCoeffs2,
Size imageSize, InputArray R, InputArray T,
OutputArray R1, OutputArray R2,
OutputArray P1, OutputArray P2,
OutputArray Q, int flags = CALIB_ZERO_DISPARITY,
double alpha = -1, Size newImageSize =Size(),
CV_OUT Rect* validPixROI1 = 0, CV_OUT Rect* validPixROI2 = 0 );

/** @brief Computes rectification transforms for each head of a calibrated stereo camera.
为每一个校正过的双目相机计算修正变换。
@param cameraMatrix1 First camera matrix.
@param cameraMatrix1 第一个相机内参矩阵
@param distCoeffs1 First camera distortion parameters.
@param distCoeffs1 第一个相机畸变参数
@param cameraMatrix2 Second camera matrix.
@param cameraMatrix1 第二个相机内参矩阵
@param distCoeffs2 Second camera distortion parameters.
@param distCoeffs1 第二个相机畸变参数
@param imageSize Size of the image used for stereo calibration.
@param imageSize 用于相机标定的图像尺寸
@param R Rotation matrix from the coordinate system of the first camera to the second camera,
@param R 从第一个相机到第二个相机坐标系系统的旋转矩阵
see @ref stereoCalibrate.
@param T Translation vector from the coordinate system of the first camera to the second camera,
@param T 从第一个相机到第二个相机的坐标系的平移矩阵
see @ref stereoCalibrate.
@param R1 Output 3x3 rectification transform (rotation matrix) for the first camera. This matrix brings points given in the unrectified first camera’s coordinate system to points in the rectified first camera’s coordinate system. In more technical terms, it performs a change of basis from the unrectified first camera’s coordinate system to the rectified first camera’s coordinate system.
@param R1 为第一个相机输出3x3的修正矩阵（旋转矩阵）。该矩阵将未经过校正的第一个摄像机坐标系中给出的点引入到经过校正的第一个摄像机坐标系中的点。用更专业的术语来说，它执行了一个基础的改变，从未矫正的第一个摄像机的坐标系到矫正的第一个摄像机的坐标系。
@param R2 Output 3x3 rectification transform (rotation matrix) for the second camera. This matrix brings points given in the unrectified second camera’s coordinate system to points in the rectified second camera’s coordinate system. In more technical terms, it performs a change of basis from the unrectified second camera’s coordinate system to the rectified second camera’s coordinate system.
@param R2 为第二个相机输出3x3的校正变换(旋转矩阵)。该矩阵将未校正的第二摄像机坐标系中给定的点引入已校正的第二摄像机坐标系中的点。用更专业的术语来说，它执行了一个基础的改变，从未矫正的第二个摄像机的坐标系到矫正的第二个摄像机的坐标系。
@param P1 Output 3x4 projection matrix in the new (rectified) coordinate systems for the first camera, i.e. it projects points given in the rectified first camera coordinate system into therectified first camera’s image.
@param P1 为第一个摄像机输出新的(经校正的)坐标系统中的3x4投影矩阵，即它将经校正的第一个摄像机坐标系统中给定的点投影到被校正的第一个摄像机的图像中。
@param P2 Output 3x4 projection matrix in the new (rectified) coordinate systems for the second camera, i.e. it projects points given in the rectified first camera coordinate system into the rectified second camera’s image.
@param P2 为第二个摄像机输出新的(经校正的)坐标系统中的3x4投影矩阵，即将经校正的第一个摄像机坐标系统中给定的点投影到经校正的第二个摄像机的图像中。
@param Q Output 4x4 disparity-to-depth mapping matrix (see @ref reprojectImageTo3D).
@param Q 输出4x4的视差-深度映射矩阵(参见@ref reprojectImageTo3D)。
@param flags Operation flags that may be zero or CALIB_ZERO_DISPARITY . If the flag is set, the function makes the principal points of each camera have the same pixel coordinates in the rectified views. And if the flag is not set, the function may still shift the images in the horizontal or vertical direction (depending on the orientation of epipolar lines) to maximize the useful image area.
@param flags 操作标志，可以是零或CALIB_ZERO_DISPARITY。如果设置了标志，该函数将使每个摄像机的主点在校正视图中具有相同的像素坐标。如果没有设置标志，函数仍然可以在水平或垂直方向(取决于极线的方向)移动图像，以最大化有用的图像区域。
@param alpha Free scaling parameter. If it is -1 or absent, the function performs the default scaling. Otherwise, the parameter should be between 0 and 1. alpha=0 means that the rectified images are zoomed and shifted so that only valid pixels are visible (no black areas after rectification). alpha=1 means that the rectified image is decimated and shifted so that all the pixels from the original images from the cameras are retained in the rectified images (no source image pixels are lost). Any intermediate value yields an intermediate result between those two extreme cases.
@param alpha 自由的尺度参数。如果是-1或缺席，函数将执行默认的缩放。否则，参数应该在0和1之间。alpha=0表示校正后的图像被缩放和移动，只有有效像素可见(校正后没有黑色区域)。alpha=1表示对校正后的图像进行抽取和移位，使所有来自相机的原始图像的像素都保留在校正后的图像中(无源图像像素丢失)。任何中间值都是这两个极端情况的中间结果。
@param newImageSize New image resolution after rectification. The same size should be passed to initUndistortRectifyMap (see the stereo_calib.cpp sample in OpenCV samples directory). When (0,0) is passed (default), it is set to the original imageSize . Setting it to a larger value can help you preserve details in the original image, especially when there is a big radial distortion.
@param newImageSize 校正后图像分辨率。同样的大小也应该传递给initUndistortRectifyMap (参见OpenCV样本目录中的stereo_calib.cpp样本)。当传递(0,0)时(默认)，将其设置为原始imageSize。将其设置为较大的值可以帮助您保存原始图像的细节，特别是当有一个大的径向失真。
@param validPixROI1 Optional output rectangles inside the rectified images where all the pixels are valid. If alpha=0 , the ROIs cover the whole images. Otherwise, they are likely to be smaller (see the picture below).
@param validPixROI1 矫正图像内的可选的输出矩形，其中所有像素是有效的。如果alpha=0, ROI覆盖整个图像。否则，它们可能会更小(见下图)。
@param validPixROI2 Optional output rectangles inside the rectified images where all the pixels are valid. If alpha=0 , the ROIs cover the whole images. Otherwise, they are likely to be smaller(see the picture below).
@param validPixROI2 矫正图像内的可选的输出矩形，其中所有像素是有效的。如果alpha=0, ROI覆盖整个图像。否则，它们可能会更小(见下图)。
The function computes the rotation matrices for each camera that (virtually) make both camera image planes the same plane. Consequently, this makes all the epipolar lines parallel and thus simplifies the dense stereo correspondence problem. The function takes the matrices computed by stereoCalibrate as input. As output, it provides two rotation matrices and also two projection matrices in the new coordinates. The function distinguishes the following two cases:
该函数为每个摄像机计算旋转矩阵，使两个摄像机的图像平面(实际上)成为同一平面。因此，这使得所有的极线平行，从而简化了稠密的立体对应问题。该函数以stereoCalibrate计算的矩阵作为输入。作为输出，它提供了新坐标中的两个旋转矩阵和两个投影矩阵。该函数区分了以下两种情况:
Horizontal stereo: the first and the second camera views are shifted relative to each other mainly along the x-axis (with possible small vertical shift). In the rectified images, the corresponding epipolar lines in the left and right cameras are horizontal and have the same y-coordinate. P1 and P2 look like:
**水平双目:第一和第二摄像机视图主要沿x轴相对移动(可能有较小的垂直移动)。在校正后的图像中，左右相机对应的极线是水平的，且y坐标相同。P1和P2看起来像:
\f[\texttt{P1} = \begin{bmatrix}
f & 0 & cx_1 & 0 \
0 & f & cy & 0 \
0 & 0 & 1 & 0
\end{bmatrix}\f]
\f[\texttt{P2} = \begin{bmatrix}
f & 0 & cx_2 & T_xf \
0 & f & cy & 0 \
0 & 0 & 1 & 0
\end{bmatrix} ,\f]
where \fKaTeX parse error: Undefined control sequence: \f at position 4: T_x\̲f̲ is a horizontal shift between the cameras and \fKaTeX parse error: Undefined control sequence: \f at position 10: cx_1=cx_2\̲f̲ if
CALIB_ZERO_DISPARITY is set.
Vertical stereo: the first and the second camera views are shifted relative to each other mainly in the vertical direction (and probably a bit in the horizontal direction too). The epipolar lines in the rectified images are vertical and have the same x-coordinate. P1 and P2 look like:
垂直立体视觉:第一和第二摄像机的视图主要在垂直方向上相对移动(可能在水平方向上也有一点)。经校正的图像中的极线是垂直的，并且有相同的x坐标。P1和P2看起来像:
\f[\texttt{P1} = \begin{bmatrix}
f & 0 & cx & 0 \
0 & f & cy_1 & 0 \
0 & 0 & 1 & 0
\end{bmatrix}\f]
\f[\texttt{P2} = \begin{bmatrix}
f & 0 & cx & 0 \
0 & f & cy_2 & T_yf \
0 & 0 & 1 & 0
\end{bmatrix},\f]
where \fKaTeX parse error: Undefined control sequence: \f at position 4: T_y\̲f̲ is a vertical shift between the cameras and \fKaTeX parse error: Undefined control sequence: \f at position 10: cy_1=cy_2\̲f̲ if
CALIB_ZERO_DISPARITY is set.
As you can see, the first three columns of P1 and P2 will effectively be the new “rectified” camera matrices. The matrices, together with R1 and R2 , can then be passed to initUndistortRectifyMap to initialize the rectification map for each camera.
如你所见，P1和P2的前三列将有效地成为新的“校正”相机矩阵。矩阵，连同R1和R2，然后可以被传递到initUndistortRectifyMap，以初始化每个摄像机的双目校正映射。
See below the screenshot from the stereo_calib.cpp sample. Some red horizontal lines pass through the corresponding image regions. This means that the images are well rectified, which is what most stereo correspondence algorithms rely on. The green rectangles are roi1 and roi2 . You see that their interiors are all valid pixels.
下面是来自stereo_calib.cpp示例的屏幕截图。一些红色的水平线穿过相应的图像区域。这意味着图像得到了很好的校正，这是大多数立体对应算法所依赖的。绿色的矩形是roi1和roi2。你可以看到它们的内部都是有效的像素。

二、initUndistortRectifyMap()函数

**

CV_EXPORTS_W void initUndistortRectifyMap(
InputArray cameraMatrix, 
InputArray distCoeffs,
InputArray R, InputArray newCameraMatrix,
Size size, int m1type, OutputArray map1, OutputArray map2 );

**

/** @brief Computes the undistortion and rectification transformation map.
/** @brief 计算未失真和校正变换映射。
The function computes the joint undistortion and rectification transformation and represents the result in the form of maps for remap. The undistorted image looks like original, as if it is captured with a camera using the camera matrix=newCameraMatrix and zero distortion. In case of a monocular camera, newCameraMatrix is usually equal to cameraMatrix, or it can be computed by #getOptimalNewCameraMatrix for a better control over scaling. In case of a stereo camera, newCameraMatrix is normally set to P1 or P2 computed by #stereoRectify .
该函数计算联合的未失真和校正变换，并将结果以映射的形式表示出来。未失真的图像看起来像原始的，就好像它是用相机捕捉的，该相机矩阵为newCameraMatrix并且零失真。对于单目相机，newCameraMatrix通常等于cameraMatrix，或者它可以通过#getOptimalNewCameraMatrix计算，以更好地控制缩放。对于立体相机，newCameraMatrix通常设置为P1或P2由#stereoRectify()计算。
Also, this new camera is oriented differently in the coordinate space, according to R. That, for example, helps to align two heads of a stereo camera so that the epipolar lines on both images become horizontal and have the same y- coordinate (in case of a horizontally aligned stereo camera).
此外，根据R，新相机在坐标空间的方向是不同的，例如，帮助对齐立体相机的两个头部，使两幅图像上的极线变成水平的，有相同的y坐标(在水平对齐立体相机的情况下)。
The function actually builds the maps for the inverse mapping algorithm that is used by remap. That is, for each pixel \fKaTeX parse error: Undefined control sequence: \f at position 7: (u, v)\̲f̲ in the destination (corrected and rectified) image, the function computes the corresponding coordinates in the source image (that is, in the original image from camera). The following process is applied:
该函数实际上为remap使用的逆映射算法构建映射。也就是说，对于目标(修正和矫正后)图像中的每个像素 (u, v)，该函数计算源图像(即来自摄像机的原始图像)中的对应坐标。采用以下流程:
\f[
\begin{array}{l}
x \leftarrow (u - {c’}_x)/{f’}_x \
y \leftarrow (v - {c’}_y)/{f’}_y \
{[X,Y,W]} ^T \leftarrow R^{-1}*[x , y , 1]^T \
x’ \leftarrow X/W \
y’ \leftarrow Y/W \
r^2 \leftarrow x’^2 + y’^2 \
x’’ \leftarrow x’ \frac{1 + k_1 r^2 + k_2 r^4 + k_3 r^6}{1 + k_4 r^2 + k_5 r^4 + k_6 r^6}

• 2p_1 x’ y’ + p_2(r^2 + 2 x’^2) + s_1 r^2 + s_2 r^4\
  y’’ \leftarrow y’ \frac{1 + k_1 r^2 + k_2 r^4 + k_3 r^6}{1 + k_4 r^2 + k_5 r^4 + k_6 r^6}
• p_1 (r^2 + 2 y’^2) + 2 p_2 x’ y’ + s_3 r^2 + s_4 r^4 \
  s\vecthree{x’’’}{y’’’}{1} =
  \vecthreethree{R_{33}(\tau_x, \tau_y)}{0}{-R_{13}((\tau_x, \tau_y)}
  {0}{R_{33}(\tau_x, \tau_y)}{-R_{23}(\tau_x, \tau_y)}
  {0}{0}{1} R(\tau_x, \tau_y) \vecthree{x’’}{y’’}{1}\
  map_x(u,v) \leftarrow x’’’ f_x + c_x \
  map_y(u,v) \leftarrow y’’’ f_y + c_y
  \end{array}
  \f]
  where \fKaTeX parse error: Undefined control sequence: \f at position 85: …_x, \tau_y]]]])\̲f̲
  are the distortion coefficients.

In case of a stereo camera, this function is called twice: once for each camera head, after stereoRectify, which in its turn is called after #stereoCalibrate. But if the stereo camera was not calibrated, it is still possible to compute the rectification transformations directly from the fundamental matrix using #stereoRectifyUncalibrated. For each camera, the function computes homography H as the rectification transformation in a pixel domain, not a rotation matrix R in 3D space. R can be computed from H as \f[\texttt{R} = \texttt{cameraMatrix} ^{-1} \cdot \texttt{H} \cdot \texttt{cameraMatrix}\f]
where cameraMatrix can be chosen arbitrarily.
对于立体相机，这个函数被调用两次:对每个相机头部调用一次，在stereoRectify后，在#stereoCalibrate后调用。但是，如果立体摄像机没有校准，仍然可以使用#stereoRectifyUncalibrated.从基本矩阵直接计算校正变换。对于每个摄像机，函数计算单应性H作为像素域中的校正变换，而不是三维空间中的旋转矩阵R。R可以从H计算为\f[\texttt{R} = \texttt{cameraMatrix} ^{-1} \cdot \texttt{H} \cdot \texttt{cameraMatrix
其中cameraMatrix可以自由选择。
@param cameraMatrix Input camera matrix
@param cameraMatrix 输入相机矩阵
\fKaTeX parse error: Undefined control sequence: \vecthreethree at position 3: A=\̲v̲e̲c̲t̲h̲r̲e̲e̲t̲h̲r̲e̲e̲{f_x}{0}{c_x}{0… .
@param distCoeffs Input vector of distortion coefficients
@param distCoeffs 输入畸变参数矩阵
\fKaTeX parse error: Undefined control sequence: \f at position 85: …_x, \tau_y]]]])\̲f̲
of 4, 5, 8, 12 or 14 elements. If the vector is NULL/empty, the zero distortion coefficients are assumed. 如果矢量为NULL/空，则假定零失真系数。
@param R Optional rectification transformation in the object space (3x3 matrix). R1 or R2 ,computed by #stereoRectify can be passed here. If the matrix is empty, the identity transformation is assumed. In cvInitUndistortMap R assumed to be an identity matrix.
@param R 对象空间(3x3矩阵)的可选校正变换。R1或R2，通过#stereoRectify计算，这里可以跳过。如果该矩阵为空，则假定采用恒等变换。在cvInitUndistortMap中，R假设为单位矩阵。
@param newCameraMatrix New camera matrix
@param newCameraMatrix 新相机矩阵
\fKaTeX parse error: Undefined control sequence: \vecthreethree at position 4: A'=\̲v̲e̲c̲t̲h̲r̲e̲e̲t̲h̲r̲e̲e̲{f_x'}{0}{c_x'}….
@param size Undistorted image size.未失真图像尺寸
@param m1type Type of the first output map that can be CV_32FC1, CV_32FC2 or CV_16SC2, see #convertMaps
@param m1type 第一个输出映射类型可以是CV_32FC1、CV_32FC2或CV_16SC2的请参阅#convertMaps
@param map1 The first output map.
@param map1 第一个输出映射
@param map2 The second output map.
@param map2 第二个输出映射
*/