互相关python程序,使用python和互相关进行图像配准

I got two images showing exaktly the same content: 2D-gaussian-shaped spots. I call these two 16-bit png-files "left.png" and "right.png". But as they are obtained thru an slightly different optical setup, the corresponding spots (physically the same) appear at slightly different positions. Meaning the right is slightly stretched, distorted, or so, in a non-linear way. Therefore I would like to get the transformation from left to right.

So for every pixel on the left side with its x- and y-coordinate I want a function giving me the components of the displacement-vector that points to the corresponding pixel on the right side.

In a former approach I tried to get the positions of the corresponding spots to obtain the relative distances deltaX and deltaY. These distances then I fitted to the taylor-expansion up to second order of T(x,y) giving me the x- and y-component of the displacement vector for every pixel (x,y) on the left, pointing to corresponding pixel (x',y') on the right.

To get a more general result I would like to use normalized cross-correlation. For this I multiply every pixelvalue from left with a corresponding pixelvalue from right and sum over these products. The transformation I am looking for should connect the pixels that will maximize the sum. So when the sum is maximzied, I know that I multiplied the corresponding pixels.

I really tried a lot with this, but didn't manage. My question is if somebody of you has an idea or has ever done something similar.

import numpy as np

import Image

left = np.array(Image.open('left.png'))

right = np.array(Image.open('right.png'))

# for normalization (http://en.wikipedia.org/wiki/Cross-correlation#Normalized_cross-correlation)

left = (left - left.mean()) / left.std()

right = (right - right.mean()) / right.std()

Please let me know if I can make this question more clear. I still have to check out how to post questions using latex.

Thank you very much for input.

I'm afraid, in most cases 16-bit images appear just black (at least on systems I use) :( but of course there is data in there.

UPDATE 1

I try to clearify my question. I am looking for a vector-field with displacement-vectors that point from every pixel in left.png to the corresponding pixel in right.png. My problem is, that I am not sure about the constraints I have.

$ \vec{r} + \vec{d}(\vec{r}) = \vec{r}\prime $

where vector r (components x and y) points to a pixel in left.png and vector r-prime (components x-prime and y-prime) points to the corresponding pixel in right.png. for every r there is a displacement-vector.

What I did earlier was, that I found manually components of vector-field d and fitted them to a polynom second degree:

$ \left(\begin{array}{c}x \\ y\end{array}\right) + \left(\begin{array}{c}\Delta x(x,y) \\ \Delta y(x,y)\end{array}\right)=\left(\begin{array}{c}x\prime \\ y\prime \end{array}\right) $

So I fitted:

$ \Delta x(x,y) = K_0 + K_1\cdot x + K_2 \cdot y + K_3 \cdot x^2 + K_4 \cdot xy + K_5 \cdot y^2 $

and

$ \Delta y(x,y) = K_6 + K_7\cdot x + K_8 \cdot y + K_9 \cdot x^2 + K_{10} \cdot xy + K_{11} \cdot y^2 $

Does this make sense to you? Is it possible to get all the delta-x(x,y) and delta-y(x,y) with cross-correlation? The cross-correlation should be maximized if the corresponding pixels are linked together thru the displacement-vectors, right?

UPDATE 2

So the algorithm I was thinking of is as follows:

Deform right.png

Get the value of cross-correlation

Deform right.png further

Get the value of cross-correlation and compare to value before

If it's greater, good deformation, if not, redo deformation and do something else

After maximzied the cross-correlation value, know what deformation there is :)

About deformation: could one do first a shift along x- and y-direction to maximize cross-correlation, then in a second step stretch or compress x- and y-dependant and in a third step deform quadratic x- and y-dependent and repeat this procedure iterativ?? I really have a problem to do this with integer-coordinates. Do you think I would have to interpolate the picture to obtain a continuous distribution?? I have to think about this again :( Thanks to everybody for taking part :)

解决方案

OpenCV (and with it the python Opencv binding) has a StarDetector class which implements this algorithm.

As an alternative you might have a look at the OpenCV SIFT class, which stands for Scale Invariant Feature Transform.

Update

Regarding your comment, I understand that the "right" transformation will maximize the cross-correlation between the images, but I don't understand how you choose the set of transformations over which to maximize. Maybe if you know the coordinates of three matching points (either by some heuristics or by choosing them by hand), and if you expect affinity, you could use something like cv2.getAffineTransform to have a good initial transformation for your maximization process. From there you could use small additional transformations to have a set over which to maximize. But this approach seems to me like re-inventing something which SIFT could take care of.

To actually transform your test image you can use cv2.warpAffine, which also can take care of border values (e.g. pad with 0). To calculate the cross-correlation you could use scipy.signal.correlate2d.

Update

Your latest update did indeed clarify some points for me. But I think that a vector field of displacements is not the most natural thing to look for, and this is also where the misunderstanding came from. I was thinking more along the lines of a global transformation T, which applied to any point (x,y) of the left image gives (x',y')=T(x,y) on the right side, but T has the same analytical form for every pixel. For example, this could be a combination of a displacement, rotation, scaling, maybe some perspective transformation. I cannot say whether it is realistic or not to hope to find such a transformation, this depends on your setup, but if the scene is physically the same on both sides I would say it is reasonable to expect some affine transformation. This is why I suggested cv2.getAffineTransform. It is of course trivial to calculate your displacement Vector field from such a T, as this is just T(x,y)-(x,y).

The big advantage would be that you have only very few degrees of freedom for your transformation, instead of, I would argue, 2N degrees of freedom in the displacement vector field, where N is the number of bright spots.

If it is indeed an affine transformation, I would suggest some algorithm like this:

identify three bright and well isolated spots on the left

for each of these three spots, define a bounding box so that you can hope to identify the corresponding spot within it in the right image

find the coordinates of the corresponding spots, e.g. with some correlation method as implemented in cv2.matchTemplate or by also just finding the brightest spot within the bounding box.

once you have three matching pairs of coordinates, calculate the affine transformation which transforms one set into the other with cv2.getAffineTransform.

apply this affine transformation to the left image, as a check if you found the right one you could calculate if the overall normalized cross-correlation is above some threshold or drops significantly if you displace one image with respect to the other.

if you wish and still need it, calculate the displacement vector field trivially from your transformation T.

Update

It seems cv2.getAffineTransform expects an awkward input data type 'float32'. Let's assume the source coordinates are (sxi,syi) and destination (dxi,dyi) with i=0,1,2, then what you need is

src = np.array( ((sx0,sy0),(sx1,sy1),(sx2,sy2)), dtype='float32' )

dst = np.array( ((dx0,dy0),(dx1,dy1),(dx2,dy2)), dtype='float32' )

result = cv2.getAffineTransform(src,dst)