First calibration example - Corner extraction, calibration, additional tools(一)

This section takes you through a complete calibration example based on a total of 20 (and 25) images of a planar checkerboard.
This example lets you learn how to use all the features of the toolbox: loading calibration images, extracting image corners, running the main calibration engine, displaying the results, controlling accuracies, adding and suppressing images, undistorting images, exporting calibration data to different formats... This example is highly recommended for someone who is just starting using the toolbox.

这部分带你熟悉一个基于20张（和25张）二维棋盘的标定的完整的例子。这个例子帮助你了解怎样利用工具箱的特性：加载标定图像，运行主标定引擎，显示结果，控制精度，添加和减少图片，无畸变图片，输出标定不同格式的数据...这个例子特别适合刚开始用这个工具箱的人。

Download the calibration images all at once calib_example.zip (4461Kb zipped) or one by one, and store the 20 images into a seperate folder named calib_example.

• From within matlab, go to the example folder calib_example containing the images.
  
  
• Reading the images:
  
  Click on the Image names button in the Camera calibration tool window. Enter the basename of the calibration images (Image) and the image format (tif).
  All the images (the 20 of them) are then loaded in memory (through the command Read images that is automatically executed) in the variables I_1,I_2 ,...,I_20. The number of images is stored in the variablen_ima (=20 here).
  The matlab window should look like this:
  

  在Matlab工作环境下，将当前工作路径切换到calib_example中。启动工具箱后点击Image names按钮，提示加载图片的信息。

  

  The complete set of images is also shown in thumbnail format (this images can always be regenerated by runningmosaic):

  这些图片同样以很小的大小显示出来：

  

• Extract the grid corners:
  
  Click on the Extract grid corners button in the Camera calibration tool window.
  在标定工具箱上点击Extract grid cornder按钮。
  
  
  Press "enter" (with an empty argument) to select all the images (otherwise, you would enter a list of image indices like[2 5 8 10 12] to extract corners of a subset of images). Then, select the default window size of the corner finder:wintx=winty=5 by pressing "enter" with empty arguments to the wintx and winty question. This leads to a effective window of size 11x11 pixels.
  先选择图像（默认为全选），再选择默认的corner finder窗口大小（就是十字光标），wintx=winy=5,比较合适。光标的大小恰好是11X11个像素。
  
  
  The corner extraction engine includes an automatic mechanism for counting the number of squares in the grid. This tool is specially convenient when working with a large number of images since the user does not have to manually enter the number of squares in both x and y directions of the pattern. On some very rare occasions however, this code may not predict the right number of squares. This would typically happen when calibrating lenses with extreme distortions. At this point in the corner extraction procedure, the program gives the option to the user to disable the automatic square counting code. In that special mode, the user would be prompted for the square count for every image. In this present example, it is perfectly appropriate to keep working in the default mode (i.e. with automatic square counting activated), and therefore, simply press "enter" with an empty argument. (NOTE: it is generally recommended to first use the corner extraction code in this default mode, and then, if need be, re-process the few images with "problems")
  角落提取引擎含有自动计算方块个数的系统。这个引擎特别适合多图像的情况。避免了在X方向和Y方向手动输入方块个数的麻烦。
  
  问你是否使用自动方块检测机制吗？推荐第一次时使用。

• The first calibration image is then shown on Figure 2:

  

  Click on the four extreme corners on the rectangular checkerboard pattern. The clicking locations are shown on the four following figures (WARNING: try to click accurately on the four corners, at most 5 pixels away from the corners. Otherwise some of the corners might be missed by the detector).
  点击四个最边上的角点。最多偏离5个像素，否则可能会丢失掉部分角点。

  Ordering rule for clicking: The first clicked point is selected to be associated to the origin point of the reference frame attached to the grid. The other three points of the rectangular grid can be clicked in any order. This first-click rule is especially important if you need to calibrate externally multiple cameras (i.e. compute the relative positions of several cameras in space). When dealing with multiple cameras, the same grid pattern reference frame needs to be consistently selected for the different camera images (i.e. grid points need to correspond across the different camera views). For example, it is a requirement to run the stereo calibration toolboxstereo_gui.m (tryhelp stereo_gui and visit the fifth calibration example page for more information).
  点击角点的顺序：第一个角点十分重要，其它三个可以任意顺序。

  

  The boundary of the calibration grid is then shown on Figure 2:
  显示出标定边界：

   

   

  Enter the sizes dX and dY in X and Y of each square in the grid (in this case,dX=dY=30mm=default values):
  
  
  输入每个方块的dX和dY大小，默认是30mm。

  Note that you could have just pressed "enter" with an empty argument to select the default values. The program automatically counts the number of squares in both dimensions, and shows the predicted grid corners in absence of distortion:
  注意：你可以直接按“回车键”来选择默认值。程序会自动计算方块的数量（X方向和Y方向），并且显示出来。

  

  

  If the predicted corners are close to the real image corners, then the following step may be skipped (if there is not much image distortion). This is the case in that present image: the predicted corners are close enough to the real image corners. Therefore, it is not necessary to "help" the software to detect the image corners by entering a guess for radial distortion coefficient. Press "enter", and the corners are automatically extracted using those positions as initial guess.
  如果预测的角点接近真实图片上的角点，那么接下来的步骤可以省略（如果没有太大的图像变形的话）。现在预测的跟实际很接近，不需要通过键入径向畸变系数（radial distortion coefficient）来“帮助”软件。按下“回车键”，角点自动被提取出来。

  

  The image corners are then automatically extracted, and displayed on figure 3 (the blue squares around the corner points show the limits of the corner finder window):
  角点被自动提取出来，并显示在figure 3上（围着角点的蓝色方块显示了角点的边界）。

  

  The corners are extracted to an accuracy of about 0.1 pixel.
  Follow the same procedure for the 2^nd, 3^rd, ... , 14^th images. For example, here are the detected corners of image 2, 3, 4, 5, 6 and 7:
  角点精度在0.1个像素。

  对第2，3直到第14副图像，做同样的步骤。

  

  Observe the square dimensions dX, dY are always kept to their original values (30mm).
  Sometimes, the predicted corners are not quite close enough to the real image corners to allow for an effective corner extraction. In that case, it is necessary to refine the predicted corners by entering a guess for lens distortion coefficient. This situation occurs at image 15. On that image, the predicted corners are:

  有时候，预测的角点跟实际角点不是很接近。这样的话，要通过径向畸变系数重新预测角点。这种情况发生在第15副图像：

  

  Observe that some of the predicted corners within the grid are far enough from the real grid corners to result into wrong extractions. The cause: image distortion. In order to help the system make a better guess of the corner locations, the user is free to manually input a guess for the first order lens distortion coefficient kc (to be precise, it is the first entry of the full distortion coefficient vectorkc described at thispage). In order to input a guess for the lens distortion coefficient, enter a non-empty string to the questionNeed of an initial guess for distortion? (for example1). Enter then a distortion coefficient ofkc=-0.3 (in practice, this number is typically between -1 and 1).
  观察到一些预测角点跟实际偏差很大，提取角点出错。原因是：图像失真。为了帮助系统预测正确，用户必须手动输入校正系数kc。为了输入一个校正系数的估计，输入一个非空字符串到问题：Need of an initial guess for distortion? 然后，输入校正系数kc=-0.3（系数在-1到1之间）。

  

  According to this distortion, the new predicted corner locations are:

  根据校正，新的预测角点是：

  

  If the new predicted corners are close enough to the real image corners (this is the case here), input any non-empty string (such as1) to the questionSatisfied with distortion?. The subpixel corner locations are then computed using the new predicted locations (with image distortion) as initial guesses:
  如果新的预测接近实际角点位置（这个就是），输入任意非空字符串（如1）到问题：Satisfied with distortion? 这样一来，启用新预测角点。

  

  If we had not been satisfied, we would have entered an empty-string to the questionSatisfied with distortion? (by directly pressing "enter"), and then tried a new distortion coefficientkc. You may repeat this process as many times as you want until satisfied with the prediction (side note: the values of distortion used at that stage are only used to help corner extraction and will not affect at all the next main calibration step. In other words, these values are neither used as final distortion coefficients, nor used as initial guesses for the true distortion coefficients estimated through the calibration optimization stage).

  如果没有满意，输入一个空字符串到问题：Satisfied with distortion?（直接按“回车键”），再输入一个新的校正系数kc。重复这个过程直到满意为止（边注：这个矫正值只是用来帮助角点提取，不会影响到接下来的校正步骤。换句话说，这些值不会用作最终校正系数，也不会用于校正优化步骤）。

  The final detected corners are shown on Figure 3:

  最后一副预测角点图像：

  

  Repeat the same procedure on the remaining 5 images (16 to 20). On these images however, do not use the predicted distortion option, even if the extracted corners are not quite right. In the next steps, we will correct them (in this example, we could have not used this option for image 15, but that was quite useful for illustration).
  在剩下的5副图像（16到20）重复上述步骤。在这5副图像上，不要使用预测变形选项，尽管提取的角点不是十分准确。在接下来的步骤中，我们将修正他们（第15副图像也可以不用预测变形选项，但是为了功能演示）。

  
  After corner extraction, the matlab data file calib_data.mat is automatically generated. This file contains all the information gathered throughout the corner extraction stage (image coordinates, corresponding 3D grid coordinates, grid sizes, ...). This file is only created in case of emergency when for example matlab is abruptly terminated before saving. Loading this file would prevent you from having to click again on the images.
  完成角点提取后，matlab数据文件calib_data.mat自动生成。该文件包含角点提取步骤获得的全部信息（图像坐标，对应的3D网格坐标，网格大小，...）。该文件是为了防止紧急情况发生，如matlab在保存前突然退出。加载该文件可以避免重新点击所有图像。

  
  During your own calibrations, when there is a large amount of distortion in the image, the program may not be able to automatically count the number of squares in the grid. In that case, the number of squares in both X and Y directions have to be entered manually. This should not occur in this present example.
  在你自己校正过程中，当图像失真较为严重时，程序可能不能自动计算出方块的个数。这种情况下，X方向和Y方向上的方块个数必须手动给出。本例子不会出现这种情况。

  
  Another problem may arise when performing your own calibrations. If the lens distortions are really too severe (for fisheye lenses for example), the simple guiding tool based on a single distortion coefficientkc may not be sufficient to provide good enough initial guesses for the corner locations. For those few difficult cases, a script program is included in the toolbox that allows for a completely manual corner extraction (i.e. one click per corner). The script file is calledmanual_corner_extraction.m (in memory efficient mode, you should usemanual_corner_extraction_no_read.m instead) and should be executed AFTER the traditional corner extaction code (the script relies on data that were computed by the traditional corner extraction code -square count, grid size, order of points, ...- even if the corners themselves were wrongly detected). Obviously, this method for corner extraction could be extremely time consuming when applied on a lot of images. It therefore recommended to use it as a last resort when everything else has failed. Most users should never have to worry about this, and it will not happen in this present calibration example.
  另一个问题可能在校正过程中出现。如果透镜畸变（lens distortion）十分严重（比如鱼眼镜头），基于矫正系数kc的简单指导工具可能不够提供足够好的角点预测。对于这些很少见的困难的情况，工具箱提供一个脚本程序来允许完全手动角点提取（如：一次点击提取一个角点）。这个脚本程序叫做manual_corner_extraction.m（在省内存环境下，你应该用manual_corner_extraction_no_read.m），应该在普通角点提取代码之后运行（因为这个脚本依赖于普通角点提取计算出的方块数，网格大小，角点顺序...尽管角点预测是错误的）。显然，当有很多图像时，这种方法提取角点是很费时间的。因此推荐当所有其它工具都不能用时才使用。大多数用户不必担心，本例子也不会出现这种情况。

  
  
• Main Calibration step:
  
  After corner extraction, click on the button Calibration of the Camera calibration tool to run the main camera calibration procedure.

  完成角点提取后，点击摄像头校正工具箱上的Calibration按钮来运行主要校正步骤。
  Calibration is done in two steps: first initialization, and then nonlinear optimization.

  校正分两步：第一，初始化。第二，非线性最优化。
  The initialization step computes a closed-form solution for the calibration parameters based not including any lens distortion (program name:init_calib_param.m).
  The non-linear optimization step minimizes the total reprojection error (in the least squares sense) over all the calibration parameters (9 DOF for intrinsic: focal, principal point, distortion coefficients, and 6*20 DOF extrinsic => 129 parameters). For a complete description of the calibration parameters, click on that link. The optimization is done by iterative gradient descent with an explicit (closed-form) computation of the Jacobian matrix (program name:go_calib_optim.m).

  初始化步骤计算校正参数的封闭解，不包括任何镜头失真（程序名称：init_calib_param.m）。非线性最优化步骤使所有参数的二次投影误差最小（9个内部自由度（DOF）：焦点，投影中心，失真系数，和6X20个外部自由度，一共129个参数）。关于参数的详细描述，点击link链接。优化步骤是通过梯度向下遍历，导数矩阵，即雅各比矩阵（Jacobian matrix）来实现的（程序名称：go_calib_optim.m）。
  

  The Calibration parameters are stored in a number of variables. For a complete description of them, visit thispage. Notice that the skew coefficientalpha_c and the 6th order radial distortion coefficient (the last entry ofkc) have not been estimated (this is the default mode). Therefore, the angle between the x and y pixel axes is 90 degrees. In most practical situations, this is a very good assumption. However, later on, a way of introducing the skew coefficientalpha_c in the optimization will be presented.

  校正参数存储在一系列的变量中。详细信息请访问这个网页。注意斜交系数alpha_c和第6个径向失真系数没有被估计。因此，x方向和y方向的角度是90度。在大多数实际情况中，这是个很好的假设。但是，稍后，一种引入斜交系数alpha_c的最优化将介绍。

  Observe that only 11 gradient descent iterations are required in order to reach the minimum. This means only 11 evaluations of the reprojection function + Jacobian computation and inversion. The reason for that fast convergence is the quality of the initial guess for the parameters computed by the initialization procedure.
  For now, ignore the recommendation of the system to reduce the distortion model. The reprojection error is still too large to make a judgement on the complexity of the model. This is mainly because some of the grid corners were not very precisely extracted for a number of images.
  Click on Reproject on images in the Camera calibration tool to show the reprojections of the grids onto the original images. These projections are computed based on the current intrinsic and extrinsic parameters. Input an empty string (just press "enter") to the question Number(s) of image(s) to show ([] = all images) to indicate that you want to show all the images:
  为了达到最小化，观察仅仅11个梯度下降遍历。这意味着仅仅评估11个二次投影方程和导数矩阵及其转制。收敛的快慢取决于初始化步骤中初始参数估计的好坏。现在，忽略系统减少畸变模型的方法。二次投影误差依然太大。这主要是因为一些网格角点没有精确的提取出来。点击Reproject on images按钮来显示原始图像的二次投影。这些投影是基于当前的内部参数和外部参数。输入一个空字符串（按“回车键”）到问题：Number(s) of image(s) to show ([] = all images) 来表示你想显示所有图片。

  
   The following figures shows the first four images with the detected corners (red crosses) and the reprojected grid corners (circles).
  接下显示4张图片，上面有检测到的角点（红色十字）和二次投影的角点（圆圈）。

  

  

  The reprojection error is also shown in the form of color-coded crosses:

  二次投影误差同样显示如下：
  

  In order to exit the error analysis tool, right-click on anywhere on the figure (you will understand later the use of this option).

  如果想退出误差分析工具，图像上任意位置右键。
  Click on Show Extrinsic in the Camera calibration tool. The extrinsic parameters (relative positions of the grids with respect to the camera) are then shown in a form of a 3D plot:
  点击工具箱上的Show Extrinsic。外部参数（网格对于摄像机的相对坐标，即摄像机坐标系）将显示出来：

  

  On this figure, the frame (O_c,X_c,Y_c,Z_c) is the camera reference frame. The red pyramid corresponds to the effective field of view of the camera defined by the image plane. To switch from a "camera-centered" view to a "world-centered" view, just click on the Switch to world-centered view button located at the bottom-left corner of the figure.
  在这幅图上，坐标系是摄像机坐标系。红色的金字塔对应着摄像头的视角。如果想将摄像机坐标系转换成世界坐标系，点击Switch to world-centered view

  

  On this new figure, every camera position and orientation is represented by a green pyramid. Another click on theSwitch to camera-centered view button turns the figure back to the "camera-centered" plot.
  这张图中，每张图片的位置和方向用一个绿色的小金字塔表示。

  Looking back at the error plot, notice that the reprojection error is very large across a large number of figures. The reason for that is that we have not done a very careful job at extracting the corners on some highly distorted images (a better job could have been done by using the predicted distortion option). Nevertheless, we can correct for that now by recomputing the image corners on all images automatically. Here is the way it is going to be done: press on theRecomp. corners button in the main Camera calibration tool and select once again a corner finder window size ofwintx = winty = 5 (the default values):
  回头看一下误差图表，发现有一些二次误差很大。原因是我们没有在一些失真较大的图像上仔细提取角点（一个较好的工作是用预测失真选项）。无论如何，我们可以自动重新计算所有图像角点。方法是：点击Recomp.corners按钮，选择新的角点检测大小，wintx=winty=5

  

  To the question Number(s) of image(s) to process ([] = all images) press "enter" with an empty argument to recompute the corners on all the images. Enter then the mode of extraction: the automatic mode (auto) uses the re-projected grid as initial guess locations for the corner, the manual mode lets the user extract the corners manually (the traditional corner extraction method). In the present case, the reprojected grid points are very close to the actual image corners. Therefore, we select the automatic mode: press "enter" with an empty string. The corners on all images are then recomputed. your matlab window should look like:
  

  Run then another calibration optimization by clicking on Calibration:

  运行另外一个最优化：

  

  Observe that only six iterations were necessary for convergence, and no initialization step was performed (the optimization started from the previous calibration result). The two values0.12668 and0.12604 are the standard deviation of the reprojection error (in pixel) in both x and y directions respectively. Observe that the uncertainties on the calibration parameters are also estimated. The numerical values are approximately three times the standard deviations.
  After optimization, click on Save to save the calibration results (intrinsic and extrinsic) in the matlab fileCalib_Results.mat
  观察到只有6个迭代会收敛，而且没有初始化过程（最优化步骤使用前面的校准结果）。0.12668和0.12604分别是X方向和Y方向上的标准差。观察到矫正系数的不确定性同样被估计。数值结果大约是三倍的标准差。最优化完成后，点击Save按钮来保存校正的结果，结果保存在Calib_results.mat。

  

  For a complete description of the calibration parameters, click on that link.
  Once again, click on Reproject on images to reproject the grids onto the original calibration images. The four first images look like:
  再一次，点击Reproject on images按钮来二次投影网格到原始的校正图像。前四副图像如下：

  

  Click on Analyse error to view the new reprojection error (observe that the error is much smaller than before):

  点击Analyse error来查看新的二次投影误差（这次误差比前次少很多）：

  

  After right-clicking on the error figure (to exit the error-analysis tool), click onShow Extrinsic to show the new 3D positions of the grids with respect to the camera:

  右键点击误差窗口（退出误差分析工具），点击Show Extinsic来显示新的3D网格位置：

  

  
  A simple click on the Switch to world-centered view button changes the figure to this:
  点击Switch to world-centered view按钮转换成世界坐标系：

  

  The tool Analyse error allows you to inspect which points correspond to large errors. Click onAnalyse error and click on the figure region that is shown here (upper-right figure corner):
  误差分析窗口允许你查看哪个点的误差最大。点击Analyse error来显示：

  

  After clicking, the following information appears in the main Matlab window:
  

  This means that the corresponding point is on image 18, at the grid coordinate (0,0) in the calibration grid (at the origin of the pattern). The following image shows a close up of that point on the calibration image (before, exit the error inspection tool by clicking on the right mouse button anywhere within the figure):
  这说明是第18副图像对应的点，网格坐标是（0，0）。

  

  The error inspection tool is very useful in cases where the corners have been badly extracted on one or several images. In such a case, the user can recompute the corners of the specific images using a different window size (larger or smaller).
  误差查看工具是非常有用的，可以查看哪个角点被错误的提取。这样以来，用户可以采用一个不同的窗口大小（更大，或者更小）来重新计算特定图像的角点。
  For example, let us recompute the image corners using a window size (wintx=winty=9) for all 20 images except for images 20 (usewintx=winty=5), images 5, 7, 8, 19 (usewintx=winty=7), and images 18 (usewintx=winty=8). The extraction of the corners should be performed with three calls ofRecomp. corners. At the first call ofRecomp. corners, selectwintx=winty=9, choose to process images 1, 2, 3, 4, 6, 9, 10, 11, 12, 13, 14, 15, 16 and 17, and select the automatic mode (the reprojections are already very close to the actual image corners):
  

  At the second call of Recomp. corners, select wintx=winty=8, choose to process image 18 and select once again the automatic mode:
  

  At the third call of Recomp. corners, select wintx=winty=7, choose to process images 5, 7, 8 and 19 and select once again the automatic mode:
  

  Re-calibrate by clicking on Calibration:

  

  Observe that the reprojection error (0.11689,0.11500) is slightly smaller than the previous one. In addition, observe that the uncertainties on the calibration parameters are also smaller. Inspect the error by clicking onAnalyse error:
  可以看到二次投影误差是（0.11689，0.11500），要比前一个要小。另外，关于校正参数的不确定性也减小了。点击Analyse error来查看误差：

  

  Let us look at the previous point of interest on image 18, at the grid coordinate (0,0) in the calibration grid. For that, click onReproject on images and select to show image 18 only (of course, before that, you must exit the error inspection tool by right-cliking within the window):
  让我们来看一下第18副图片感兴趣的点，在网格坐标（0，0）处。点击Reproject on images按钮，选择只显示第18副图片（当然在这之前要右键关闭误差查看工具）。

  

  A close view at the point of interest (on image 18) shows a smaller reprojection error:

  显示误差减少：

  

  Click once again on Save to save the calibration results (intrinsic and extrinsic) in the matlab fileCalib_Results.mat
  保存校正结果在Calib_Results.mat。

  

  Observe that the previous calibration result file was copied under Calib_Results_old0.mat (just in case you want to use it later on).