图像算法---Image Deformation Using Moving Least Squares

原文地址：http://vision.gel.ulaval.ca/~jflalonde/cours/4105/h14/tps/results/project/jingweicao/index.html

Final Project: Image Deformation Using Moving Least Squares

Jingwei Cao

Project Description

In this project, I implemented the paper "Image deformation using moving least square" which is proposed by Scott Schaefer. This method consists of three type of deformation:affine deformation,similarity deformation and rigid deformation. And it also use both controlling point sets and controlling line segment sets to specify the morph.

Part1: Moving Least Squares Deformation

The MLS (Moving Least Squares) Deformation is used to find the best transformation function  f that maps  p  to  q .  p  is a set of controlled handles and  q  is the deformed positions of the controlled handles. The function  f(v)  needs to satisfy three conditions:(1) Interpolation: The handles  p  should map directly to  q  under deformation (i.e. f(pi)=qi )(2) Smoothness: f should produce smooth deformations (3)Identity: If the deformed handles  q  are the same as the  p , then  f  should be the identity function.(i.e.,  qi=pi ⇒f(v)=v )

1 Affine Deformation

Affine transform:  lv(x)=xM+T
Translation can be removed:  T=q∗−p∗M
p∗=ΣiwipiΣiwi   q∗=ΣiwiqiΣiwi
We can easily get:  Iv(x)=(x−p∗)M+q∗
The new cost function:
Σiwi|p^iM−q^i|2  where  p^i=pi−p∗   q^i=qi−q∗
M  could be different class of transformations
Solution: M=(Σip^Tiwip^i)−1Σjwjp^Tjq^j
The deformation function:
fa(v)=ΣjAjq^j+q∗  
Aj=(v−p∗)(Σip^Tiwip^i)−1p^Ti
Aj can be precomputed
Contains non-uniform scaling and shear

2 Similarity Deformation

A special subset of affine transformations
Translation,Rotation,Constraints: Uniform-Scaling
Requirements:  MTM=λ2I
Define  M=(M1,M2) , where  M2=M⊥1
Cost function (Least squares problem) still quadratic in  M
Σiwi|(p^i−p^⊥i)M−q^Ti|2 , where  (x,y)⊥=(−y,x)
Solution for matrix  M
M=1μsΣiwi(p^i−p^⊥i)(q^Ti,−qi^⊥T) , where  μs=Σiwip^ip^Ti
Solution for deformation function
fs(v)=Σiq^i(1μsAi)+q∗ ,where  Ai=wi(p^i−p^⊥i)(v−p∗−(v−p∗)⊥)T

3 Rigid Deformation

Solution for Matrix  M
M=Σi(p^i−p^⊥i)(q^Ti,−qi^⊥T)(Σiwiq^ip^Ti)2+(Σiwiq^ip^⊥Ti)2√
Solution for deformation function
fr(v)=|v−p∗|f→r(v)|f→r(v)|+q∗ , where  f→r(v)=Σiq^iAi  and  Ai  is the same as in similarity deformations.

4 Experimental Results

I do the similar experiment but with the opposite direction. The puppet leans towards right.

The author's results

My own results

Original Image	Affine Deformation	Similarity Deformation	Rigid Deformation

Some interesting result

Monalisa	Deformed Monalisa

Girrafe	Deformed Girrafe

Generally speaking, the MLS image deformation using controlled points produces the result as I have expected. The rigid deformation can produce the most realistic result compared with other methods in my implementation but with the lowest speed.

Part2: Deformation with Line Segments

Handles are control curves instead of control points
Cost function
Σi∫10wi(t)|pi(t)M+T−qi(t)|2  and  wi(t)=|p′i(t)||pi(t)−v|2α
T still can be removed
T=q∗−p∗M
T=q∗−p∗M
Σi∫10wi(t)|p^i(t)M−q^i(t)|2
where  p^i(t)=pi(t)−p∗  and  q^i(t)=qi(t)−q∗

1 Affine Lines

Represent line segments  p^i(t) , q^i(t)  as matrix products
p^i(t)=(1−t,t)(a^ib^i)  and  q^i(t)=(1−t,t)(c^id^i)
Cost function
Σi∫10|(1−t,t)((a^ib^i)M−(c^id^i))|2
Minimizer
M=(Σi(a^ib^i)TWi(a^ib^i))−1Σj(a^ib^i)TWj(c^id^i)
Wi=(δ00iδ01iδ01iδ11i)  where  δ00i=∫10wi(t)(1−t)2dt
δ01i=∫10wi(t)(1−t)tdt  and  δ11i=∫10wi(t)t2dt
Deformation
fa(v)=ΣjAj(c^id^i)+q∗
Aj=(v−p∗)(Σi(a^ib^i)TWi(a^ib^i))−1(a^jb^j)TWj

2 Similarity Lines

Cost function
Σi∫10|(1−t001−tt00t)⎛⎝⎜⎜⎜⎜⎜a^i−a^⊥ib^i−b^⊥i⎞⎠⎟⎟⎟⎟⎟M−(c^Tid^Ti)|2
Minimizer
M=1μsΣj⎛⎝⎜⎜⎜⎜⎜a^i−a^⊥ib^i−b^⊥i⎞⎠⎟⎟⎟⎟⎟TWj(c^Tjd^Tjc^⊥Tjd^⊥Tj)
where  μs=Σia^ia^Tiδ00i+2a^ib^Tiδ01i+b^ib^Tiδ11i
Wj=⎛⎝⎜⎜⎜⎜⎜δ00j0δ01j00δ00j0δ01jδ01j0δ11j00δ01j0δ11j⎞⎠⎟⎟⎟⎟⎟
Deformation
fs(v)=Σj(c^j,d^j)(1μsAj)+q∗
μs=Σia^ia^Tiδ00i+2a^ib^Tiδ01i+b^ib^Tiδ11i
Aj=Wj⎛⎝⎜⎜⎜⎜⎜a^i−a^⊥ib^i−b^⊥i⎞⎠⎟⎟⎟⎟⎟(v−p∗−(v−p∗)⊥)T

3 Rigid Lines

Derive from similarity lines
μr=|Σj(a^Tj−a^⊥Tjb^Tj−b^⊥Tj)Wj(c^Tjd^Tj)|
Deformation
fr(v)=|v−p∗|f→r(v)|f→r(v)|+q∗
f→r(v)=Σj(c^j−d^j)Aj
Aj=Wj⎛⎝⎜⎜⎜⎜⎜a^i−a^⊥ib^i−b^⊥i⎞⎠⎟⎟⎟⎟⎟(v−p∗−(v−p∗)⊥)T

4 Experimental Results

Original Image	Affine Line
Similarity Image	Rigid Line

Unfortunately, my implementation of line segments deformation produces some unharmonious morph on the image with some bizarre artifacts at the bottom of the image region. In the rigid line deformation, the profile of the pisa tower bends too much. I think the reason is the definition of the line segments. This is really a tricky part which will affect the results.

附上一个DEMO连接：源码连接https://github.com/cxcxcxcx/imgwarp-opencv

网页版DEMO连接http://chenxing.name/fun/imgwarp-js/