原文地址:http://vision.gel.ulaval.ca/~jflalonde/cours/4105/h14/tps/results/project/jingweicao/index.html
Final Project: Image Deformation Using Moving Least Squares
Jingwei Cao
Project Description
In this project, I implemented the paper "Image deformation using moving least square" which is proposed by Scott Schaefer. This method consists of three type of deformation:affine deformation,similarity deformation and rigid deformation. And it also use both controlling point sets and controlling line segment sets to specify the morph.
Part1: Moving Least Squares Deformation
The MLS (Moving Least Squares) Deformation is used to find the best transformation function f that maps p to q . p is a set of controlled handles and q is the deformed positions of the controlled handles. The function f(v) needs to satisfy three conditions:(1) Interpolation: The handles p should map directly to q under deformation (i.e. f(pi)=qi )(2) Smoothness: f should produce smooth deformations (3)Identity: If the deformed handles q are the same as the p , then f should be the identity function.(i.e., qi=pi ⇒f(v)=v )
1 Affine Deformation
Affine transform:
lv(x)=xM+T
Translation can be removed:
T=q∗−p∗M
p∗=ΣiwipiΣiwi
q∗=ΣiwiqiΣiwi
We can easily get:
Iv(x)=(x−p∗)M+q∗
The new cost function:
Σiwi|p^iM−q^i|2
where
p^i=pi−p∗
q^i=qi−q∗
M
could be different class of transformations
Solution:
M=(Σip^Tiwip^i)−1Σjwjp^Tjq^j
The deformation function:
fa(v)=ΣjAjq^j+q∗
Aj=(v−p∗)(Σip^Tiwip^i)−1p^Ti
Aj
can be precomputed
Contains non-uniform scaling and shear
2 Similarity Deformation
A special subset of affine transformations
Translation,Rotation,Constraints: Uniform-Scaling
Requirements:
MTM=λ2I
Define
M=(M1,M2)
, where
M2=M⊥1
Cost function (Least squares problem) still quadratic in
M
Σiwi|(p^i−p^⊥i)M−q^Ti|2
, where
(x,y)⊥=(−y,x)
Solution for matrix
M
M=1μsΣiwi(p^i−p^⊥i)(q^Ti,−qi^⊥T)
, where
μs=Σiwip^ip^Ti
Solution for deformation function
fs(v)=Σiq^i(1μsAi)+q∗
,where
Ai=wi(p^i−p^⊥i)(v−p∗−(v−p∗)⊥)T
3 Rigid Deformation
Solution for Matrix
M
M=Σi(p^i−p^⊥i)(q^Ti,−qi^⊥T)(Σiwiq^ip^Ti)2+(Σiwiq^ip^⊥Ti)2√
Solution for deformation function
fr(v)=|v−p∗|f→r(v)|f→r(v)|+q∗
, where
f→r(v)=Σiq^iAi
and
Ai
is the same as in similarity deformations.
4 Experimental Results
I do the similar experiment but with the opposite direction. The puppet leans towards right.
The author's results
My own results
Original Image | Affine Deformation | Similarity Deformation | Rigid Deformation |
Some interesting result
Monalisa | Deformed Monalisa |
Girrafe | Deformed Girrafe |
Generally speaking, the MLS image deformation using controlled points produces the result as I have expected. The rigid deformation can produce the most realistic result compared with other methods in my implementation but with the lowest speed.
Part2: Deformation with Line Segments
Handles are control curves instead of control points
Cost function
Σi∫10wi(t)|pi(t)M+T−qi(t)|2
and
wi(t)=|p′i(t)||pi(t)−v|2α
T still can be removed
T=q∗−p∗M
T=q∗−p∗M
Σi∫10wi(t)|p^i(t)M−q^i(t)|2
where
p^i(t)=pi(t)−p∗
and
q^i(t)=qi(t)−q∗
1 Affine Lines
Represent line segments
p^i(t)
,
q^i(t)
as matrix products
p^i(t)=(1−t,t)(a^ib^i)
and
q^i(t)=(1−t,t)(c^id^i)
Cost function
Σi∫10|(1−t,t)((a^ib^i)M−(c^id^i))|2
Minimizer
M=(Σi(a^ib^i)TWi(a^ib^i))−1Σj(a^ib^i)TWj(c^id^i)
Wi=(δ00iδ01iδ01iδ11i)
where
δ00i=∫10wi(t)(1−t)2dt
δ01i=∫10wi(t)(1−t)tdt
and
δ11i=∫10wi(t)t2dt
Deformation
fa(v)=ΣjAj(c^id^i)+q∗
Aj=(v−p∗)(Σi(a^ib^i)TWi(a^ib^i))−1(a^jb^j)TWj
2 Similarity Lines
Cost function
Σi∫10|(1−t001−tt00t)⎛⎝⎜⎜⎜⎜⎜a^i−a^⊥ib^i−b^⊥i⎞⎠⎟⎟⎟⎟⎟M−(c^Tid^Ti)|2
Minimizer
M=1μsΣj⎛⎝⎜⎜⎜⎜⎜a^i−a^⊥ib^i−b^⊥i⎞⎠⎟⎟⎟⎟⎟TWj(c^Tjd^Tjc^⊥Tjd^⊥Tj)
where
μs=Σia^ia^Tiδ00i+2a^ib^Tiδ01i+b^ib^Tiδ11i
Wj=⎛⎝⎜⎜⎜⎜⎜δ00j0δ01j00δ00j0δ01jδ01j0δ11j00δ01j0δ11j⎞⎠⎟⎟⎟⎟⎟
Deformation
fs(v)=Σj(c^j,d^j)(1μsAj)+q∗
μs=Σia^ia^Tiδ00i+2a^ib^Tiδ01i+b^ib^Tiδ11i
Aj=Wj⎛⎝⎜⎜⎜⎜⎜a^i−a^⊥ib^i−b^⊥i⎞⎠⎟⎟⎟⎟⎟(v−p∗−(v−p∗)⊥)T
3 Rigid Lines
Derive from similarity lines
μr=|Σj(a^Tj−a^⊥Tjb^Tj−b^⊥Tj)Wj(c^Tjd^Tj)|
Deformation
fr(v)=|v−p∗|f→r(v)|f→r(v)|+q∗
f→r(v)=Σj(c^j−d^j)Aj
Aj=Wj⎛⎝⎜⎜⎜⎜⎜a^i−a^⊥ib^i−b^⊥i⎞⎠⎟⎟⎟⎟⎟(v−p∗−(v−p∗)⊥)T
4 Experimental Results
Original Image | Affine Line |
Similarity Image | Rigid Line |
Unfortunately, my implementation of line segments deformation produces some unharmonious morph on the image with some bizarre artifacts at the bottom of the image region. In the rigid line deformation, the profile of the pisa tower bends too much. I think the reason is the definition of the line segments. This is really a tricky part which will affect the results.