纹理合成

Texture Synthesis

By  ERIC  |  Published:  OCTOBER 23, 2014

What in this post is actually part of my computational photography homework, because I’m recently preparing for interviews, so for reviewing it, I re-implemented this method.

WHAT IS IT

Texture synthesis is another very interesting application of image processing. What it does is, given a texture sample, generate new texture that is similar with the given sample, similar here means for human observer, the new generated texture appears to be the same kind of texture.

For example, if we have the following image.

what we want is to fill the black region using the same texture with middle part. A trivial solution is just copy the whole image (or part of it), and put it into the unfilled region, and the output image is something like this.

Actually we feel not that happy with this result, the blockiness is clearly perceivable, and looks like some tiles in the bathroom. We want the texture expands automatically.

This image looks better, although it is also kind of weird, but we can hardly tell that this image is made by tiling some tiles, it is more likely that to be a whole image. This is exactly what texture synthesis does.

ALGORITHM

01	function FindMatches(Template,SampleImage)

02	ValidMask = 1s where Template is filled, 0s otherwise

03	GaussMask = Gaussian2D(WindowSize,Sigma)

04	TotWeight = sum i,j GaussiMask(i,j)*ValidMask(i,j)

05	for i,j do

06	for ii,jj do

07	dist = (Template(ii,jj)-SampleImage(i-ii,j-jj))^2

08	SSD(i,j) = SSD(i,j) + dist*ValidMask(ii,jj)*GaussMask(ii,jj)

09

end

10	SSD(i,j) = SSD(i,j) / TotWeight

11

end

12	PixelList = all pixels (i,j)where SSD(i,j)<=min(SSD)*(1+ErrThreshold)

13	return PixelList

14

end

01	function GrowImage(SampleImage,Image,WindowSize)

02	while Image not filled do

03	progress = 0

04	PixelList = GetUnfilledNeighbors(Image)

05	foreach Pixel in PixelList do

06	Template = GetNeighborhoodWindow(Pixel)

07	BestMatches = FindMatches(Template, SampleImage)

08	BestMatch = RandomPick(BestMatches)

09	if (BestMatch.error < MaxErrThreshold) then

10	Pixel.value = BestMatch.value

11	progress = 1

12

end

13

end

14	if progress == 0

15	then MaxErrThreshold = MaxErrThreshold * 1.1

16

end

17	return Image

18

end

Something need to pay attention:

1. Window size is manually selected, if the texture repeats, then we can use the unit size of the texture.
2. Function GetUnfilledNeighbors() returns a list of all unfilled pixels that have filled pixels as their neighbors. The list is randomly permuted and then sorted by decreasing number of filled neighbor pixels.
3. When calculating neighbors, we use the 8-direction neighbors.
4. If the current pixel is at the edge of image (one or some of its neighbors doesn’t exist), then just flag the non-exist neighbors, and calculate regularly.
5. GetNeigborhoodWindow() returns a window of size WindowSize around a given pixel (returns flagged windows when at edges).
6. RandomPick() picks an element randomly from the list.
7. Gaussian2D() generates a 2D Gaussian in a window of given a size centered in the center and with a given standard deviation (in pixels).
8. The recommended constant values are: ErrThreshold = 0.1, MaxErrThreshold = 0.3, Sigma = WindowSize/6 (3sigma for each side).

 SOURCE CODE

 https://github.com/xingdi-eric-yuan/texture-synthesis

RESULTS AND  MORE

Some results (including using former code)

1. radishes (100 * 100) VS expanded radishesss (111 * 111)

   

2. black hole filling

      

3. barbed wire fence (75 * 75) VS expanded baaaaarbed wire fence (225 * 225)

   

For testing this algorithm, you can use the Brodatz Textures, which includes 112 gray scale texture images.

Because the above github version code is today’s new implemented code, I didn’t include something like hole or foreground object removal or multi-channel image processing.

For implementing foreground object removal, just alter the map generating part, no matter hole removal or image expansion, all we need to do is set the part of we want to be zero in map.

For implementing 3-channel image texture synthesis, just split the image into 3 different channels, and do what we do with single channel image 3 times, and then merge the 3 channels result as the final result image.

Lastly, there are some people use image pyramid to do texture synthesis, although I never tried it, but I believe it is a good idea to synthesis the information of images on each layer of whatever pyramid, maybe I’ll try this the other day 

This entry was posted in Algorithm, OpenCV, Vision  and tagged Computer Vision, texture synthesis. Bookmark the  permalink.  Post a comment or leave a trackback:  Trackback URL.