Matting

 

Matting [Introduction] [Bayesian & Robust Matting] [Flash Matting] [System Diagram] [Download] [Reference] Introduction This is our results for VFX 2008's final project. We implement 3 matting algorithms, compare the resulting alphamaps, and show some of our observations in this report. In fact, we've also implemented Poisson matting, but the result was not promising. So we decide not to include it in our reports. Bayesian & Robust Matting Both Bayesian and Robust Matting utilize the foreground and background colors to predict the current unknown region's alpha. Two main differences are: the method of pixel sampling (see the upper figures) the way of computing alpha from foreground and background.(see the lower figures) Although Robust Matting has the term "robust" in its name, its results are not so robust in our implementation. As we can see the figure below, Robust Matting can be easily influenced by some ourliers because of its pair-wise point candidate selection. On the contrary, although GMM (used in Bayesian matting) may not describe the distribution of the foreground/background points well, it is more tolerable to the outliers. Therefore, we've found that the trimap influence the result more in Robust matting than in Bayesian matting. See the figure of comparison between Bayesian and Robust Matting. Another major difference between Bayesian matting and Robust matting is that the former is a sampling approach (solve the alphamap point by point) and the latter is a combination of sampling approach and propagation approach (solve the alphamap globally). Robust matting utilizes random walk to make its alphamap more globally reasonable. However, we find the performance would not be be enhanced much, because the effect is somehow like a balance between the initial guess and the smoothed initial guess. We can observe this phenomenon by adjusting the ratio between the data cost and the smooth cost as in the following figure. Therefore, do not expect the random walk step would greatly refine the result. The initial guess is much more important. Flash Matting Why do we need Flash Matting? If the background color is too similar to the foreground color, the result would be poor no matter we use Bayesian matting or Robust matting. In flash matting, it introduces a way to provide more information other than pixel color by utilizing a flash light. Flash Matting uses an additional flashed image. The algorithm is strongly based on the assumption that the foreground objects become brighter after the flash whereas the background objects remain the same. It modifies Bayesian Matting to jointly consider the additional information in optimization to yield a better result. However, the result is not good when the assumption does not hold. We can see the figure below. Some part of hair is close to black color, and the brighter amount in this part is not as much as in other regions. Some other solutions should be provided to make up for this shortage. Besides, in the case of horse with colorful backlight, the brightened part is background, so the initial setting of background and foreground is reverse. Some bad blockwise estimation occur in both Bayesian and Flash Matting. It is becuase both of them characterize background and foreground color distributions as GMM. If the number of Gaussians is less than the number of actual color clusters, some bad blockwise estimation occurs. System Diagram Since we implement three matting algorithms in the same program, we have to differentiate these three methods before adopting any of them to compute alphamap. We describe our system diagram as the figure below. Besides of given trimap, we allow user to tune the trimap by a simple gui interface and utilize different threshold in auto-generating trimap by Flash Matting. Download Source Code and Executive Files To learn how to use, please check readme.txt in zipped file. Matting (VS 2005 project) [ download] Executive (matting.exe)  [ download] Presentation Slides   Slide [ download] Reference Yung-Yu Chuang, Brian Curless, David Salesin, and Richard Szeliski, "A Bayesian Approach to Digital Matting." CVPR, pp. 264-271, IEEE Computer Society, 2001. J. Wang and M. F. Cohen, "Optimized Color Sampling for Robust Matting." IEEE Computer Vision and Pattern Recognition or CVPR, pp. 1-8, 2007. Jian Sun, Yin Li, Sing Bing Kang, and Heung-Yeung Shum, "Flash matting." ACM Transactions on Graphics, 25(3), pp. 772-778, July 2006.

 

http://mpac.ee.ntu.edu.tw/~sutony/vfx_matting/index.html