A Spectral Method for 3D Shape Reconstruction and Denoising Alfred Hero and Jia Li University of Michigan Ann Arbor, MI 48109 http://www.eecs.umich.edu/~hero/
A Spectral Method for 3D Shape Reconstruction and Denoising
Alfred Hero and Jia Li
University of MichiganAnn Arbor, MI 48109
http://www.eecs.umich.edu/~hero/
2
Outline
• Polar shape modeling• KLE representations for polar shape • Spectral methods for registration/reconstruction• Spectral methods for active contours
A. Hero and J. Li, SIAM Boston Mar. 2002
3
Motivation
• Why shape modeling?– Object recognition– Shape recovery– Medical image analysis
• Medical training• Clinical application
A. Hero and J. Li, SIAM Boston Sept 2002
4
3D Polar Surfaces• 3D surface
• Star-shaped (polar) surface
X
2321
,),(
)];,(),,(),,([),(
Rvuvuxvuxvuxvu
⊂ΩΩ∈
=X
),( φθr
A. Hero and J. Li, SIAM Boston Sept 2002
5
Spherical Coordinate Systemazimuthelevation
φθ
),( φθr
A. Hero and J. Li, MIA Paris Sept. 2002
6
3D Fourier Descriptors
• Orthonormal basis • Complete basis • Ordered in increasing spatial frequency• Common expansion in azimuth (longitude)
φθφθ imm
m eff )(),( ∑∞
−∞=
=
A. Hero and J. Li, MIA Paris Sept. 2002
7
Spherical Harmonics
• Angular solutions of Laplace equation on sphere• Karhunen-Loeve basis for isotropic random field• Requires computation of Legendre polynomials• Applied to shape approximation (Haigron&etal:98,
Danielsson&etal:ICPR98, Matheny&etal:PAMI95)• Implementation: SVD or FFT algorithm
φθπ
φθ immlmm
l ePmlmllY )cos(
)!()!(
412)1(),(
+−+−=
)(, θlmf
A. Hero and J. Li, MIA Paris Sept. 2002
8
First 10 Spherical Harmonics
A. Hero and J. Li, MIA Paris Sept. 2002
9
Computing Spherical Harmonic Coefficients• GaussElim/SVD method via optimized sampling
(Erturk&Dennis:ElectLett97, Matheny:PAMI95)(K+1) unknown coefficients C
• Spherical FFT(Driscoll&etal:AdvAppMath94,Healy&etal:ICASSP96)– orthogonal discrete approximation to SH – lower complexity than SVD method– requires equi-spaced samples in azimuth/elevation
),(),(),(0
φθφθφθ mlml
K
l
l
lm
ml
ml VBUAR ∑ ∑
= −=
+=
XCRVBUAR =⇒+=
2
A. Hero and J. Li, MIA Paris Sept. 2002
10
SH Numerical Comparisons
1)7
()9
()10
( 222 =++ zyx 1444 =++ zyx 1)9
()10
( 666 =++ zyx
3D Shapes used in the simulation
Liver Surface
A. Hero and J. Li, MIA Paris Sept. 2002
11
Shape modeling error vs. the highest order of spherical Harmonics for the four different shapes.
A. Hero and J. Li, MIA Paris Sept. 2002
12
Double Fourier Series• DFS are easier to manipulate than SH • DFS over the sphere is more difficult than over R • Boundary conditions for at spherical poles
• Standard 2D trigonometric series not admissible!• Are there interesting admissible bases besides SH?
=
≠=
=
mmfinite
fdd
mmfinite
f
m
m
even,0odd,
)(
0,00,
)(
θθ
θ
)(θmf
2
A. Hero and J. Li, MIA Paris Sept. 2002
13
Admissible DFS Expansion
.0even ,sinsin)(
, odd,sin)(
0,cos)(
1,
1,
1
00,
≠=
=
==
∑
∑
∑
=
=
−
=
mnff
mnff
mnff
J
nmnm
J
nmnm
J
nnm
θθθ
θθ
θθ
(Ref: Cheong:JournCompPhysics00)
Fourier coefficients can be computed using fast sin and cos transform
14
SH vs DFS: Comparison
SH
DFS
1 coeff 25 coeff 81 coeff
Metasphere surface
A. Hero and J. Li, MIA Paris Sept. 2002
15
Statistical Polar Shape Modeling
• Common procedure– Extract shape features or parameters
• Landmarks (Cootes CVIU95 &Frangi proc.Ipmi01)• SH coefficients (Staib PAMI96)• Distance map (Leventon proc. CVPR 00)
– Compute mean and covariance of shape parameters.– Principle component analysis– Incorporate into other image processing algorithms
• Random field model
A. Hero and J. Li, SIAM Boston Sept. 2002
16
Decomposition of Isotropic Random Fields Over Unit Sphere Using SH
• Isotropic polar field satisfies{ } )cos()(),(),( 22*11 γψγφθφθ == RXXE
A. Hero and J. Li, MIA Paris Sept. 2002
17
Karhunen-Loeve Expansion
),(),(),(0
φθφθ ∑ ∑∞
= −=
=l
l
lm
mlYmlAX
Ω= ∫ dYXmlA Sm
l ),(),(),( 2* φθφθ
{ } 0),( =mlAE{ } '' ,,''* ),(),( mmlllmlAmlAE δδλ=
∫−=1
1d)()(2 ttPt ll ψπλ
SH are eigenfunctions of isotropic r.f. on sphere (Yadrenko:76)
A. Hero and J. Li, MIA Paris Sept. 2002
18
Advantages of SH/DFS for Polar Shape Modeling
• A true frequency domain method over the unit sphere• Low order basis smooth-fit segmented boundary • Multiresolution structure can be used to iteratively
recover higher frequencies• Fast FFT methods can be implemented at each
resolution via Driscoll&Healy or Cheong algorithm• Orthogonal representation of isotropic random field
model
A. Hero and J. Li, MIA Paris Sept. 2002
19
Registration/Reconstruction• Rotation• Euler angles • SH representations:
)3(SOg ∈),,( γβα
)exp()()exp(),,(
),,(~
),(~),(~
),(),(
'
0
0
''
'
'
'
γβαγβα
γβα
φθφθ
φθφθ
imdimD
cDc
YcR
YcR
lmm
lmm
l
lm
ml
lmm
ml
K
l
l
lm
ml
ml
K
l
l
lm
ml
ml
−⋅⋅−=
=
=
=
∑
∑ ∑
∑ ∑
−=
= −=
= −=
(rotated reference)
(reference)
20
Joint Registration of 3D Rotation and Spherical Harmonic Coefficients
• Gaussian model
• Marginal coefficient likelihood functions
• ML estimator:
ml
nl
l
ln
lmn
ml
ml
ml
ml
aDc
ac
ηγβα
η
~),,(~ +=
+=
∑−=
( )
−−=
−−=
∑−=
22~
22
~2/)),,(~(exp
2/)(exp
lnl
l
ln
lmn
mlc
lml
mlc
aDcL
acL
ml
ml
σγβα
σ
{ } ∑ ∑∑
= −=
−=
−+−=
K
l
l
lm l
nl
l
ln
lmn
ml
l
ml
ml
a
ml
aDcaca
ml
12
2
2
2
}{,,,
)~
)),,(~()((}ˆ{,ˆ,ˆ,ˆ argmin σ
γβα
σγβα
γβα
A. Hero and J. Li, MIA Paris Sept. 2002
21
Bias Variance and CR bound
The performance of the maximum likelihood estimator which jointly estimates the Euler angles and the shape parameters.
A. Hero and J. Li, MIA Paris Sept. 2002
22
Bias and variance of radial reconstruction for Lowpass(red) vs Weiner(blue) filtering
A. Hero and J. Li, MIA Paris Sept. 2002
23
Application to Active Contour Surface Reconstruction
Ω∇+−= ∫ dfgffE S f ))(()(22
2α
o),( φθf
),( oog φθ
α : non-negative roughness parameter
Sept. 200224
Polar Euler-Lagrange Equation
0)1)((2 =∂
∂−−−∇
fg
gff ffα
2
2
22
sin1)sin(
sin1
φθθθ
θθ ∂∂+
∂∂
∂∂=∇
Minimizer f of E(f) must satisfy PDE over 2S
Front propagation consists of two steps:
At iteration n1.Update f: (solve Helmholtz)
2.Update g: (determine external force)1+
→nn ff
ggn
n
n fn
ffnnn gf
ggfff −
∂∂
−−=−∇ ++ )(112α
25
Methods of Solving Helmholtz
• Grid size• Computational complexity
– FEM/FDM Method– New Spectral Method
)(~)( 36 NONO)log( 2 NNO
NN ×
•Iterative FEM and FDM•Advantage: non-uniform sampling can be accommodated•Disadvantage: very high computational complexity
•Non-iterative spectral methods•Spherical harmonic expansions•Double Fourier series
Comparison:
0)(2 =−−∇ gff µ
A. Hero and J. Li, MIA ParisSept. 2002
26
Spectral Solution to Helmholtz
[ ])()()(sin
))(sin(sin
12
2
θθµθθ
θθ
θθθ mmmm
gffmfdd
dd −=−
1. Substitute Fourier descriptor into Helmholtz equation
)cos()( , /2+= ∑∞
−∞=
πθθ nnffn
nmm
2. Substitute Cheong’s expansion of )(θmf
φθφθ imm
m eff )(),( ∑∞
−∞=
=
)(θmf
Fourier Descriptor
Cheong’s trig series
A. Hero and J. Li, MIA Paris Sept. 2002
27
Helmholtz Recursions
oddor ,0 )41
21
41(
4)2)(1(
22
4)2)(1(
,2,,2
,2,
22
,2
=+−=
++++++−+−−
+−
+−
mggg
fnnfmnfnn
mnmnmn
mnmnmn
µ
µµµ
0 andeven )41
21
41(
4)1(
22
4)1(
,2,,2
,2,
22
,2
≠+−=
+++++−+−
+−
+−
mggg
fnnfmnfnn
mnmnmn
mnmnmn
µ
µµµ
A. Hero and J. Li, MIA ParisSept. 2002
28
1. 2. Obtain by solving matrix equations3.
−−−
−−−
=
−
−
−
−
−−
−−−
mJ
mJ
m
m
mJ
mJ
m
m
mJJ
JmJJ
m
m
gg
gg
ff
ff
bacba
cbacb
,1
,3
,3
,1
,1
,3
,3
,1
,11
3,33
3,33
1,1
21121
12112
MOOOMOOO
gf AB =
mngg ,),( →φθ
mnf ,),(, φθff mn →
After truncation obtain tridiagonal linear system
A. Hero and J. Li, MIA Paris Sept. 2002
29
Experiment 1: Ellipsoidal Surface Reconstruction Different Regularization Parameter
Original segmentation 410=µ
310=µ 210=µ
A. Hero and J. Li, MIA Paris Sept. 2002
30
Data Regularization Parameter
Different shapes Different noise levels
A. Hero and J. Li, MIA Paris Sept. 2002
31
Experiment 2: CT Thoracic Slices
A. Hero and J. Li, MIA Paris Sept. 2002
32
Edge-maps
A. Hero and J. Li, MIA Paris Sept. 2002
33
Active Contour Implementation
• Liver center estimated via ellipsoid fitting• Local edge detection gives coarse segmentation• 32 x 32 angular grid used for radial descriptor• Elliptic evolving contour computed via spectral
method• Convergent to within a pixel after 5 iterations• 3 values of investigated:
64 ,, −−−3 10=10=10= ααα
α
A. Hero and J. Li, SIAM Boston 34
Segmentation After 5 Iterations
−310=α 4−10=α 6−10=α
A. Hero and J. Li, MIA ParisSept. 2002
35
No regularization 410−=α
Reconstruction of liver surface from 2D CT slices
A. Hero and J. Li, MIA Paris Sept. 2002
36
Conclusions• SH lead to optimal shape filtering and ML
registration of polar 3D objects having isotropic random field models
• DFS/SH lead to spectral method for HelmholtzPDE introduced for 3D active balloons
• Application of recent advances in computational fluid dynamics to shape recovery problem
• Accelerated performance of parametric active contour on real and synthetic 3D images
Untitled