关闭

Panorama Stitching

1078人阅读 评论(0) 收藏 举报
分类:

Panorama Stitching

[Data Acquisition] [Feature Matching] [Image Matching and Validation] [Blending]

[Results] [Downloads] [References]


Data Acquisition

We use Nikon D70 to capture the images. Nikon D70 is a high-end professional camera which all intrinsic parameters are controllable. Tripod is used in the capture. Also Nikon provides a remote control program Nikon Capture 4.0 [Nikon]. The original captured image is at least 1504x1000, which is too big for our application, and thus the images are scaled down using Batch Thumbs in our experiments.

Feature Extraction

Our implementation is based on [Lowe04]. There are some differences between the algorithm details mentioned in class and in the lecture slides. They are described in the following.

Feature Detection

Gaussian Map and Difference of Gaussian

In the SIFT algorithm, the Gaussian images will have the factor k difference in neighbor layer. The value of k that mentioned in class is

,

but the value of k used in the reference code is

.

So the actual procedure of generation of the DoGs is following,

The k value of generation of Gaussian images has a significant impact on the value of features. Use the k-value of the reference matlab code can almost find 4 times features.

Local Adjustment

The method of local adjustment mentioned in[Lowe04] uses 3D vector (x, y , sigma) to adjust the feature position, however we found using 2D vector (x, y) has better performance.(See Figure.). On the other hand, the contrast threshold that suggested by [Lowe04] is considered too high. This issue is also mentioned in the reference code [SIFTTutorial] and [AutoPano].

Feature Description

We think the details of describing features are not clear in [Lowe04] . The following are the details that we think must be clarified.

After finding the main orientation of a feature, we must compute the histogram of the gradients in the window as local description. Actually, in our implementation and reference code, we rotate our window and resample the gradient, and then compute the angle and the magnitude. Note that resample pre-computed dx and dy instead of the angle and the magnitude would result better performance.

The local description is 4x4 sample array, and the each sample is composed of a 8 dimension vector which is the histogram of the gradient of the pixels in the 4x4 pixel small window. In order to have robust histograms, each pixel in the big 16x16 window is multiplied by a bilinear weight and added to the histograms of nearest 4 samples from 4x4 sample array. See Fig2.

Feature Matching

K-D Tree

We implement our own k-d tree according to the description in [Beis97] and [Brown03] with some modifications. In our implementation, the node-split criterion is max-spread instead of max-variance in the paper. This modification can greatly speed up the process while the performance is almost the same. The original work doesn't mention many details, and here we build one tree for each image independently.

In the current setting, each feature has at most one match in each image, and we determine its validation by examining the distance ratio between the two best matches. There are at most 4 features in 4 different images matched for each feature.

Below are some matching result where the image is rotated and scaled. We connect all matched features between the two images. We can see most connection lines pass through a single point.

Image Matching and Validation

We implement a reduced version of David's work in [Brown03]. We limit the variables in the image matching while the validation of the matches is preserved, and thus the input images can be out of order and images which do not belong to the panorama are rejected.

RANSAC

After the feature matching pairs are determined, we apply RANSAC [Fischler81] to estimate the homography. We find if we estimate the translational motion, we can enlarge the threshold and also speed up the process. The full homography is then estimated by the inliners indicated by RANSAC.

Match Validation

In [Brown03], a probabilistic based measure is applied to determine if the image belongs to the panorama. However, we find that SIFT is a very robust feature descriptor and fake feature matches can be removed easily, as described in [Lowe04]. Furthermore, after RANSAC, if the image matching does not exist, the number of inliners is always fewer than 10 in our experiments. Therefore, we determine the validation of the image by a fixed threshold. If the number of inliners is less than the threshold, we rule out this image.

Blending

We use the two-band blending method that is mentioned in [Lowe04] and [Burt83] to blend the image. According to [Lowe04], the image is separated into the high-band and low-band image. The high-band image is blended by choosing the information that has the maximum weight. Low-band is blended by weighted sum. Because the transform parameters are not integers, we must resample image in order to blend the single warped image into the panorama. This way increases the difficulty of coding, so we just truncate the float-point number into integer. Thanks to multi-band technique, the results are good.

We shift every column of the panorama to compensate the drift effect when blending [lec05_stitching]. But this method produces aliasing effect, so we interpolate the pixel value vertically. A result without drift compensation is shown below. Also we find that this drift error is inevitable in the panorama mosaic because the coordinate is distorted by cylinder transformation. For example, two features lie on one a horizontal line become lying on a curve after transformation.

Results

We apply our implementation to both public test images and images we captured. Some data are snapshotted in World of Warcraft. We notice only by using our own data, there would be defects at the top or bottom of the panorama, and we think they are contributed by many sources.

First we didn't apply the radial distortion compensation, so the pictures are distorted around the corners. Second, we can not make the pitch angle zero in the capture, so the captured images do not lie on the cylinder perfectly. We believe the second reason make major part to the distortion. In fact this problem exists in most data, and many implementations avoid this problem by reducing the vertical field of view.

From our files

From test files

 

Downloads

Captured images

Click images to download the zipped package. (Including the config file for our program)

Note that we don't provide the original images of wow1 due to the possible copyright issue.

[Gym] [Lib] [101] [Wow01]

Source codes and execute files

To learn how to use, please check readme.txt

Stitching. (.NET project) [here]

References

    1.  Nikon official website. http://www.nikonusa.com
    2.  David G. Lowe, Distinctive image features from scale-invariant keypoints, in International Journal of Computer Vision, 60, 2 (2004), pp. 91-110.
    3.  Jeffrey S. Beis and David G. Lowe, Shape indexing using approximate nearest-neighbour search in high-dimensional spaces, in Conference on Computer Vision and Pattern Recognition, Puerto Rico, June, 1997, pp. 1000-1006.
    4. Martin A. Fischler and Robert C. Bolles, Random sample consensus: A paradigm for
      model fitting with applications to image analysis and automated cartography
      , in Communications
      of the ACM
      , 24, 6 (1981), 381-395.
    5.  M. Brown and D. G. Lowe. Recognising panoramas, In Proceedings of the 9th International Conference on Computer Vision (ICCV2003), Nice, France, October 2003, pp. 1218-1225.
    6.  P. J. Burt and Edward H. Adelson, A multiresolution spline with application to image mosaics, in ACM Transactions on Graphics, no. 4, vol. 2, 1983, pp. 217-236.
    7. Matlab SIFT tutorial ftp://ftp.cs.utoronto.ca/pub/jepson/teaching/vision/2503/SIFTtutorial.zip
    8. AutoPano, http://autopano.kolor.com/
    9. IVR file format. http://www2h.biglobe.ne.jp/~hnakamur/technolab/livepic/livepic_spec/spec.htm

     


© 2006 Chia-Kai Liang and Chihyuan Chung, NTUEE

0
0
查看评论

OpenCV学习笔记(五十一)——imge stitching图像拼接stitching

stitching是OpenCV2.4.0一个新模块,功能是实现图像拼接,所有的相关函数都被封装在Stitcher类当中。这个类当中我们可能用到的成员函数有createDefault、estimateTransform、composePanorama、stitch。其内部实现的过程是非常繁琐的,需要...
  • yang_xian521
  • yang_xian521
  • 2012-05-22 17:26
  • 64177

CS131-PA1 全景拼接Panorama Stitching

全景拼接(Panoramic stitching)是机器视觉应用比较早也比较成功的方向,目前有大量的应用,比如google的街道全景图,手机的全景图像拼接,各种拼接软件(Photosynth and AutoStitch)等等. 本文方法是通过计算不同图像sift特征,并计算出图像之间的仿射关系,...
  • rain2211
  • rain2211
  • 2016-12-31 13:59
  • 870

全景影像拼接程序 影像拼接 图像拼接vc++程序

  • 2011-05-06 13:20
  • 1.54MB
  • 下载

【计算机视觉】stitching_detail算法介绍

已经不负责图像拼接相关工作,有技术问题请自己解决,谢谢。 一、stitching_detail程序运行流程       1.命令行调用程序,输入源图像以及程序的参数       2.特征点检测,判断是使用surf还是orb...
  • LG1259156776
  • LG1259156776
  • 2017-11-06 16:41
  • 242

imge stitching图像拼接stitching

直接代码: #include #include #include #include // 新版本写在下面文件中: #include //#include "opencv2/features2d/features2d.hpp" #include #include us...
  • u014365862
  • u014365862
  • 2016-12-02 11:41
  • 619

全景图 HelloPanorama 的使用

HelloPanorama是由国外一名老外写的,所以很久没有更新,最近开始做全景视图,开始使用他所提供的库,将遇到的问题说明一下 首先下载HelloPanorama项目,在gitHub可以搜索到, https://github.com/heroims/HelloPanoramaGL 下载完成后,发现...
  • qq_37240033
  • qq_37240033
  • 2017-04-11 09:02
  • 556

观察者与陀螺仪传感器的应用(基于PanoramaImageView)

原作者的PanoramaImageView项目github地址:https://github.com/gjiazhe/PanoramaImageView实现原理:首先将自定义PanoramaImageView的ScaleType手动置为CENTER_CROP:将图片等比例缩放,让图像的短边与Imag...
  • enjoy_azad
  • enjoy_azad
  • 2017-01-02 18:31
  • 603

OpenCV2学习笔记(十六):Stitching图像拼接

图像拼接stitching是OpenCV2.4.0出现的一个新模块,所有的相关函数都被封装在Stitcher类当中。这里演示一个Stitch类的实例。关于Stitcher类的详细介绍,可以参考: http://docs.opencv.org/2.4.2/modules/stitching/doc/h...
  • liyuefeilong
  • liyuefeilong
  • 2015-03-26 11:24
  • 7837

stitching via

转载】stitching via的作用   在PADS里面有Via的属性为stitching,stitiching 的意思为缝合的意思,所以也叫做缝合孔。 指的是那些没有走线的过孔。如果我们选择一个网络,添加过孔的话,那孔的属性默认为stitching。 ...
  • gtkknd
  • gtkknd
  • 2014-12-27 17:58
  • 3497

stitching_detail算法介绍(转载)

一、stitching_detail程序运行流程       1.命令行调用程序,输入源图像以及程序的参数       2.特征点检测,判断是使用surf还是orb,默认是surf。       3.对图...
  • mengfanteng
  • mengfanteng
  • 2015-11-19 14:56
  • 895
    个人资料
    • 访问:1944933次
    • 积分:22288
    • 等级:
    • 排名:第389名
    • 原创:102篇
    • 转载:1385篇
    • 译文:6篇
    • 评论:261条
    联系方式
    个人邮箱: xuxiduo@zju.edu.cn
    QQ群:
    1)OpenCV俱乐部
        186168905

    2) 视频/音频/图像/算法/ML
        群1:148111910

        群2:157103105

    备注:加群需要回答问题,避免广告党。
    如果你是博客看到后加的,请注明“博客”并回答问题,只注明”博客“不回答问题的恕不加入。答案为和群相关的任何技术名词,不能出现1)和2)中的任何字眼
    博客专栏
    文章分类
    最新评论