PanoramicImage Mosaic

PanoramicImage Mosaic

ｆｒｏｍ：　http://pages.cs.wisc.edu/~csverma/CS766_09/ImageMosaic/imagemosaic.html

Chaman Singh Verma  and Mon-Ju

---

---

Contents:

1. Introduction
2. Theory and Implementation Details
   1. Cylindrical Image Warping
   2. SIFT Feature Detections
   3. Ransac Translation.
      
3. Post Processing:
   1. Image Blending
      1. Feathering Algorithm.
      2. Image Pyramid Algorithm.
   2. Drift Correction.
4. Performance and Results
5. Software usage.
   
6. References.

Introduction

Generating full view panoramic imagesis important for both commercialand artistic value. Since the inception of photography many specificdevices have been invented to create panoramic images but with theavailability of inexpensive digital camera, the desire to create fullpanoramic images is overwhelming and importance of automatic imagestitching is quite high.

In this project, we create panoramic images using Cylindrical Warping.Cylindrical warping is easiest to implement, but it has stringentrequirement that all the images must be taken with level camera or withthe known tilted angle. With this method, full homography calculationsare not needed only the translation along the angular direction isrequired to create the panoramic images.

Forward Warping:

From image coordinates (x,y), the projected coordinates on the cylinder(x', y') are given by :

Inverse Warping:

inverse mapping from cylindricalcoordinates (x',y') to the image (x,y) is :

In forward warping the source imageismapped onto cylindrical surface, but it can create holes in thedestinationimage ( because some pixels may never get mapped there). therefore weuse inverse mapping where each pixel in the destination image is mappedto the source image. Since either of the mapping is unlikely to beexact on thepixel values, bilinear interpolation is used to calculate the colors atthe destination pixels.

Radial Distortion:

Because of the thick lens that areoften used in the camera, it isnecessary to correct the radial distortions in the image.  One ofthe simplified distortion model that is commonly used is

                                                          Xd = Xu( 1 + k1*r^2 + k2*r^4)
                                                          Yd = Yu( 1 + k1*r^2 + k2*r^4 )

Where (Xd,Yd) are the distorted image position and (Xu,Yu) are theundistorted correct position. The values (k1,k2) that depends on thecamera can be calibrated using some standard techniques.

Both inverse mapping and radial distortion correction requiresinterpolation to calculate the color values at the destination pixel.oInterpolation is not only expensive, they also smooth the features,therefore, this step must be minimized to get high quality finalimages. In this application there are two places where interpolatedvalues are required (1) Cylindrical to image in inverse warping (2)From undistorted image values to distorted images values. We cancombine these two steps and calculate cylindrical image values directlyfrom the distorted images and therefore, avoid the intermediateinterpolation values.

SIFT Feature detection

We directly use SIFT algorithm to produce features in every image. EachSIFT descriptor is 128 char long. These features are matched withneighboring image to estimate the translation. Since there could besmall number of outliers that has potential to misalign the finalimages, we use RANSAC algorithm to eliminate these outliers from thefinal estimation.

Ransac Translation

Ransac algorithm is general purposealgorithm that can be used tocalculate full homography in the presense of outliers. The use ofcylindrical warping has the advantage that only the translation motionhas to calculated on the warped image. Also for the translationestimation, only one feature is sufficient. Ransac estimation countsthe inliners based on some tolerance value ( d < eps) which dependson the noise present in the images. Since our image capturing was highquality, only two pixel tolerance was sufficient to get goodestimation. ( We found that that in our images only 5-10% outliers).

Image Blending:

When different images are stitchedtogether, for various reasons(changed lighting conditions, vignette effects) the adjacent pixelintensities differ enough to produce artifacts as shown in thefollowing pictures. To remove these artifacts, we experimented with twoalgorithms (1) Feathering (2) Image pyramids.

                               
                  

Feathering or center weighting  image blending:

In this simplest approach, the thepixel values in the blended regionsare weighted average from the two overlapping images. Sometimes thissimple approach doesn't work( for example in the presence of exposuredifferences ). But in our case, all the images were taken at the sametime and using high quality tripods, therefore, this simple algorithmproduces excellant results.
   

PB(i,j) = (1-w)*PA(i,j) + w*PB(i,j)

 

Pyramid Blening:

Laplacian pyramid is an algorithm usingGaussian to blend the image while keeping the significant feature inthe mean time. It downsizes the image into different levels (sizes)with Gaussian. Later it expands the Gaussian in to the lower leverand subtracts from the image in that lever to acquire the Laplacianimage.

After generatingLaplacian pyramids forthe overlap images A and B, we combine the two images in differentLaplacian levels by combining partial images from each of them.

Afterward, we expand the LS from the toplevel () to the next level (N-1) and add it to the original Laplacianimage in the corresponding layer () to generate the latest Laplacianimage in the corresponding layer (). We repeat this step until reachingground level () and the final result will be the blending image

Apple (Image A)	Orange (Image B)
FeatheringBlend	LaplacePyramid Blend

Drift Correction:

Very often the first image and the last image in the view don't alignproperly. This misalignment can be adjusted by shear warping and otherbundle adjustment algorithm. In our case, we didn't have to do anyadjustment as the misaligment was only 1-2 pixels.

Results:

We used Canon SX100 and Kedan tripods provided by Dr. Li Zhang. Somespecific information of the camera are as follows:

resolution    :  480x640
focal length :  678.05421
K1               :  -0.22982
K2               :   0.22952

Here is the panoramic image generated from our implemention: panorama1 and panorama2

Software and Usage Guide:

The entire source code is written in C++. The only dependencies to usethis software are "SIFT" and ImageiMagick++ which must be downloadedseparately.   Monju has developed Pyramid Blending code inMatlab


Click here to download the entire source code.

Usage:

Step 1:  Create one folder, set DataSet.
Step 2:  Inside the Dataset folder create some new directories;
            DataSet/RawImages :   All the raw images goes in thisdirectories
            DataSet/WarpImages:   All cylindrical warped images arestored in this directory.
            DataSet/PNGImages :   All cylindrical images are converted ingray scale for feature detection.
            DataSet/Keys            :  All features keys from the Sift programs are stored in thisdirectory.
      
Once all the directories are created execute the command
                                      executable <root directory name>

It will generate a panormic image "panimage.jpg" which must be croppedusing ImageMagick.

Reference software

• SIFT: ScaleInvariant Feature Transform
• AutoStitch:Automatic image stitching

References

1. R. Szeliski and H.-Y. Shum. Creating full view panoramic image mosaics and texture-mapped models,SIGGRAPH 1997, pp251-258.
2. M. Brown, D. G. Lowe, Recognising Panoramas, ICCV 2003.

Group Contribution

We worked as a group but distributed the work as follows:

Chaman Singh :  Develop the entire C++ code including warping,ransac, feathering (in C++) , results, performance analysis and
                           document writing.
MonJu             :   Image taking, Pyramid blending code in matlab, featheringcode in matlab.