Digital Image Processing Lecture 7: Geometric Transformation March 16, 2005 Prof. Charlene Tsai
Jan 20, 2016
Digital Image Processing Lecture 7: Geometric Transformation
March 16, 2005
Digital Image Processing Lecture 7: Geometric Transformation
March 16, 2005
Prof. Charlene TsaiProf. Charlene Tsai
Digital Image Processing Lecture 7 2
ReviewReview
A geometric transform of an image consists of two basic steps Step1: determining the pixel co-ordinate transformation Step2: determining the brightness of the points in the
digital grid.
In the previous lecture, we discussed brightness interpolation and some variations of affine transformation.
A geometric transform of an image consists of two basic steps Step1: determining the pixel co-ordinate transformation Step2: determining the brightness of the points in the
digital grid.
In the previous lecture, we discussed brightness interpolation and some variations of affine transformation.
(x,y)T(x,y)
Digital Image Processing Lecture 7 3
Affine Transformation (con’d)Affine Transformation (con’d)
An affine transformation is equivalent to the composed effects of translation, rotation and scaling.
The general affine transformation is commonly expressed as below:
An affine transformation is equivalent to the composed effects of translation, rotation and scaling.
The general affine transformation is commonly expressed as below:
0th order coefficients 1st order coefficients
1
1
'
'
y
xAB
y
x
Digital Image Processing Lecture 7 4
General FormulationGeneral Formulation
A geometric transform is a vector function T that maps the pixel (x,y) to a new position (x’,y’):
Tx(x,y) and Ty(x,y) are usually polynomial equations.
This transform is linear with respect to the coefficients ark and brk.
A geometric transform is a vector function T that maps the pixel (x,y) to a new position (x’,y’):
Tx(x,y) and Ty(x,y) are usually polynomial equations.
This transform is linear with respect to the coefficients ark and brk.
m
r
rm
k
krrky
m
r
rm
k
krrkx yxbyxTyxayxT
0 00 0
, ,
yxTyyxTx yx , , ''
Digital Image Processing Lecture 7 5
Finding CoefficientsFinding Coefficients
How to find ark and brk, which are often unknown? Finding pairs of corresponding points (x,y),
(x’,y’) in both images,
Determining ark and brk by solving a set of linear equations.
More points than coefficients are usually used to get robustness. Least-squares fitting is often used.
How to find ark and brk, which are often unknown? Finding pairs of corresponding points (x,y),
(x’,y’) in both images,
Determining ark and brk by solving a set of linear equations.
More points than coefficients are usually used to get robustness. Least-squares fitting is often used.
Digital Image Processing Lecture 7 6
JacobianJacobian
A geometric transform applied to the whole image may change the co-ordinate system, and a Jacobian J provides information about how the co-ordinate system changes
The area of the image is invariant if and only if |J|=1 (|J| is the determinant of J).
What is the Jacobian of an affine transform?
A geometric transform applied to the whole image may change the co-ordinate system, and a Jacobian J provides information about how the co-ordinate system changes
The area of the image is invariant if and only if |J|=1 (|J| is the determinant of J).
What is the Jacobian of an affine transform?
yT
xT
yT
xT
yx
TTyxJ
yy
xx
yx
,
,,
Digital Image Processing Lecture 7 7
Variation of Affine (2D)Variation of Affine (2D)
Translation: displacement
Euclidean (rigid): translation + rotation
Similarity: Euclidean + scaling
(for isotropic scaling, sx = sy)
Translation: displacement
Euclidean (rigid): translation + rotation
Similarity: Euclidean + scaling
(for isotropic scaling, sx = sy)
0000 , , byxTayxT yx
cossin,
sincos,
00
00
yxbyxT
yxayxT
y
x
)cossin(,
)sincos(,
00
00
yxsbyxT
yxsayxT
yy
xx
Digital Image Processing Lecture 7 8
Types of transformations (2D)Types of transformations (2D)
Affine: Similarity + shearing
Some illustrations of shearing effects (courtesy of Luis Ibanez)
Affine: Similarity + shearing
Some illustrations of shearing effects (courtesy of Luis Ibanez)
ybxbbyxT
yaxaayxT
y
x
011000
011000
,
,
y
x
bb
aa
b
a
y
x
0110
0110
00
00
'
'
Digital Image Processing Lecture 7 9
Affine transformAffine transform
Shearing in x-direction Shearing in x-direction
y
xa
y
x
10
1
0
0
'
' 01
y
x
a01y
(x,y)( x + a01y , y )
y’
x
y
x’
Digital Image Processing Lecture 7 10
Affine transformAffine transform
Shearing in y-direction Shearing in y-direction
y
x
b10x
(x,y)
( x , y + b10x )
y
x
by
x
1
01
0
0
'
'
10
y
x
y’
x’
Digital Image Processing Lecture 7 11
Invariant PropertiesInvariant Properties
Translation:
Euclidean:
Similarity:
Affine:
Translation:
Euclidean:
Similarity:
Affine:
Digital Image Processing Lecture 7 12
Types of transformations (2D)Types of transformations (2D)
Bilinear: Affine + warping
Quadratic: Affine + warping
Bilinear: Affine + warping
Quadratic: Affine + warping
xybybxbbyxT
xy ayaxaayxT
y
x
11011000
11011000
,
,
2
022
2011011000
202
22011011000
,
,
ybxbxybybxbbyxT
yaxaxyayaxaayxT
y
x
Digital Image Processing Lecture 7 13
Least-Squares EstimationLeast-Squares Estimation
Because of noise in the images, if there are more correspondence pairs than minimally required, it is often not possible to find a transformation that satisfies all pairs.
Objective: To minimize the sum of the Euclidean distances of the correspondence set C={pi,qi}, where pi={xi,yi}, and qi is the corresponding point.
Because of noise in the images, if there are more correspondence pairs than minimally required, it is often not possible to find a transformation that satisfies all pairs.
Objective: To minimize the sum of the Euclidean distances of the correspondence set C={pi,qi}, where pi={xi,yi}, and qi is the corresponding point.
i
ii qpTCF2
;;
Digital Image Processing Lecture 7 14
Least-Squares Estimation - AffineLeast-Squares Estimation - Affine
For affine model, The Euclidean error , where
With the new notation,
To find which minimize we want
For affine model, The Euclidean error , where
With the new notation,
To find which minimize we want
Tbbaaba ],,,,,[ 011001100000
iii qXe
Ti
T
TTi
ipO
OpX
10
01
ii
T
iii qXqXCF ;
CF ;
0F
Digital Image Processing Lecture 7 15
Least-Squares Estimation - AffineLeast-Squares Estimation - Affine
This leads to
Therefore,
This leads to
Therefore,
022 i
iTii
Ti qXXX
ii
Ti
ii
Ti qXXX
1
Digital Image Processing Lecture 7 16
Some remarksSome remarks
So far, we only discussed global transformation (applied to entire image)
It is possible to approximate complex geometric transformations (distortion) by partitioning an image into smaller rectangular sub-images.
for each sub-image, a simple geometric transformation, such as the affine, is estimated using pairs of corresponding pixels.
geometric transformation (distortion) is then performed separately in each sub-image.
So far, we only discussed global transformation (applied to entire image)
It is possible to approximate complex geometric transformations (distortion) by partitioning an image into smaller rectangular sub-images.
for each sub-image, a simple geometric transformation, such as the affine, is estimated using pairs of corresponding pixels.
geometric transformation (distortion) is then performed separately in each sub-image.
Digital Image Processing Lecture 7 17
Real ApplicationReal Application
Registration of retinal images of different modalities Importance?
Registration of retinal images of different modalities Importance?
Color slide Fluorocein Angiogram
Digital Image Processing Lecture 7 18
MosaicingMosaicing
Mosaic
Digital Image Processing Lecture 7 19
Fusion of medical informationFusion of medical information
Digital Image Processing Lecture 7 20
Change DetectionChange Detection
Digital Image Processing Lecture 7 21
Computer-assisted surgeryComputer-assisted surgery
Digital Image Processing Lecture 7 22
What is needed?What is needed?
A known transformation, which is less likely
OR
Computing the transformation from features. Feature extraction Features in correspondence Transformation models Objective function
A known transformation, which is less likely
OR
Computing the transformation from features. Feature extraction Features in correspondence Transformation models Objective function
Digital Image Processing Lecture 7 23
What might be a good feature?What might be a good feature?
Color slide Fluorocein Angiogram
Digital Image Processing Lecture 7 24
Feature ExtractionFeature Extraction
Digital Image Processing Lecture 7 25
Features in Correspondence(crossover & branching points)
Features in Correspondence(crossover & branching points)
Digital Image Processing Lecture 7 26
Features in Correspondence(vessel centerline points)
Features in Correspondence(vessel centerline points)
Digital Image Processing Lecture 7 27
Transformation ModelsTransformation Models
)();(M pXpq qq θθ
121110987
654321
aaaaaa
aaaaaaθq
Tyxyxyx 1)( 22pXTyx ),(p
12-parameter quadratic transformation
p
q
q
Digital Image Processing Lecture 7 28
Transformation Model HierarchyTransformation Model Hierarchy
1
)M(p;q y
x
b a t
a -b tθ
y
xs
1
)M(p;q y
x
c d t
a b tθ
y
xa
1
)M(p;q
22
y
x
yx
td b a
tc a -b θ
y
xr
)M(p;q qθ
Similarity
Affine Reduced Quadratic
Quadratic
Digital Image Processing Lecture 7 29
Objective FunctionObjective Function
C
ii
ii
CE)q,p(
)/)q;p(M(d)(
Transformation parameters
Feature point in image pI
Feature point in image qI
Mapping of features from to based on
pI qI
Set of correspondences {}
Digital Image Processing Lecture 7 30
ResultResult