韩国国家视觉实验室

제목 : Single-camera catadioptric stereo system

A simple and successful design for a panoramic stereo system that uses a single camera and multiple mirrors is presented. A wide-baseline stereo enhances the 3D reconstruction accuracy even with a single camera. The feasibility of the system has been demonstrated with experiments in an indoor environment.

To obtain a wide FOV, conic mirrors are useful. In particular, hyperboloidal mirrors, which have two foci, satisfy the SVP constraint [1]. The entire light ray directed to the focus inside the mirror (the effective viewpoint) reflects and converges at the other focus (the effective pinhole). Then, we can convert the projected image to a panoramic image seen from the effective viewpoint by establishing the relationship between the world point and its projection onto the image plane. In the coaxial panoramic stereo, once the correspondence between the panoramic images has been established, depth computation in panorama is straightforward with the disparity of the image points [2]. For the design of the panoramic stereo system, it is desirable to keep the SVP constraint, widen the baseline, use a single camera, and reduce the system size. The proposed system achieves these design factors by the SVP catadioptric image formation scheme and the light ray folding scheme.

To demonstrate the feasibility of the system, an experimental mirror setup was built as shown in following Figure. The length of the baseline was set to 120mm, which is adequate for indoor usage. The diameter of the mirror was set to 80mm. The vertical FOV was set to more than   for both upward and downward views. A 1/1.8 inch CCD camera with 1200 horizontal scan lines and a lens with 12mm focal length was used to capture the image. Following figure shows a catadioptric stereo image of our laboratory captured by the given experimental setup. All the corresponding points between each view are radially collinear. This fact eases the search process for the stereo matching. To convert the input image to stereo panoramic views, we computed the correspondence between the light ray directions from the virtual viewpoint and its projection in the acquired image with equations (1-3). Fig. 4(b, c) shows a rectified panoramic stereo pair. Stereo matching was conducted with simple window-based correlation search. Following figure shows the disparity map, in which nearby objects such as tables and a robot appear brighter than distant objects such as bookshelves and racks. Following figures are results of 3D reconstruction with the panoramic images and disparity map. We can observe the 3D sensation with the change of the viewing direction [ Demo ] Following figure is the vertical projection of the 3D reconstruction with a height of 10cm near the level of tabletop. We can observe the linear edges of nearby tables and objects on them. The disparity map and depth map show that the reconstruction accuracy is quite good. It reports that the maximum depth error is under 3% of the depth itself, except the ambiguous regions (specular region, textureless region, and a region with repetitive texture). The result demonstrates that the proposed compact, wide-baseline panoramic stereo system with a single camera can provide sufficient accuracy for various machine vision applications in which the panoramic range measurements are important. It can be widely used for robotic applications (environment recognition, path planning, and obstacle avoidance), virtual reality, surveillance systems, and military applications.

Gijeong Jang, Sungho Kim, Inso Kweon, "Single camera catadioptric stereo system", The 6th Workshop on Omnidirectional Vision, Camera Networks and Non-classical cameras (OMNIVIS2005)

[1] BAKER, S., and NAYAR, S.: 'A theory of single-viewpoint catadioptric image formation', Int. J. Computer Vision, 1999, 35, (2), pp. 175-196 [2] GLUCKMAN, J., NAYAR, S. K., and THORESZ, K. J.: 'Real-time omnidirectional and panoramic stereo', Proc. Image Understanding Workshop, November 1998, Monterey, pp. 299-303