Wavelet-Based Denoising Using Hidden Markov Models
Post on 05-Jan-2016
18 Views
Preview:
DESCRIPTION
Transcript
Wavelet-Based Denoising Using Hidden Markov Models
ELEC 631 Course Project
Mohammad Jaber Borran
Some properties of DWT
• Primary– Locality Match more signals– Multiresolution
– Compression Sparse DWT’s
• Secondary– Clustering Dependency within scale
– Persistence Dependency across scale
Probabilistic Model for an Individual Wavelet Coefficient
• Compression many small coefficients
few large coefficients
S
W
pS(1)
fW|S(w|1)
pS(2)
fW|S(w|2)
fW (w)
Probabilistic Model for a Wavelet Transform
t
f
Ignoring the dependencies
Independent Mixture (IM) Model
t
f
Clustering
Hidden Markov Chain Model
t
f
Persistence
Hidden Markov Tree Model
Parameters of HMT Model
• pmf of the root node
• transition probability
• (parameters of the)
conditional pdfs
e.g. if Gaussian Mixture is used
)(1
mpS
rmii
,)(,
)|()( |, mwfwfii SWmi
2,, and mimi
: Model Parameter Vector
Dependency between Signs of Wavelet Coefficients
SignalWavelet
0T
T
w1T
w1
0T/2
TT/2
T/2
w2
T/2
w2
T
T/2 w2
T/2 w2
T
New Probabilistic Model for Individual Wavelet Coefficients
S
W
pS(1)
fW|S(w|1)
pS(2)
fW|S(w|2)
fW (w)pS(3)
fW|S(w|3)
pS(4)
fW|S(w|4)
• Use one-sided functions as conditional probability densities
Proposed Mixture PDF
• Use exponential distributions as components of the mixture distribution
0 0
0 )|(
,
,|
w
wemwf
wmi
SW
mi
ii
0 0
0 )|(
,
,|
w
wemwf
wmi
SW
mi
ii
If m is even:
If m is odd:
PDF of the Noisy Wavelet Coefficients
yQemyf mi
y
miSY
mimi
ii ,2
1
,|
22,,
)|(If m is even:
If m is odd:
),0(~ where, 2Nnnwy
yQemyf mi
y
miSY
mimi
ii ,2
1
,|
22,,
)|(
Wavelet transform is orthonormal, therefore if the additive noise is white and zero-mean Gaussian process with variance then we have
Noisy wavelet coefficient,
Training the HMT Model
• y: Observed noisy wavelet coefficients
• s: Vector of hidden states
• Model parameter vector
Maximum likelihood parameter estimation:
)|( maximize θyYθ
f
Intractable, because s is unobserved (hidden).
Model Training Using Expectation Maximization Algorithm
• and then,
N
i
ssiiS
N
iiiSY
ii
iispfsyff
ffff
2
,)(,1
1|
)(
1)()|( ,)|(),|(
)|(),|()|,()|(
θsθsy
θsθsyθsyθx
SY
SYYSX
• Define the set of complete data, x = (y,s)
ll fU θyθxθθ XSθ
,|)|(logE),( maximize
EM Algorithm (continued)
Nii
ii
N
M
N
iiiSY
N
i
ssiiS
l
M
ll
syfspf
ffU
,...,1 1|
2
,)(,1|
,...,1|
)|(loglog)(log),|(
)|,(log),|(),(
)(
1
sYS
sYSYS
θys
θsyθysθθ
• State a posteriori probabilities are calculated using Upward-Downward algorithm• Root state a priori pmf and the state transition probabilities are calculated using Lagrange multipliers for maximizing U.• Parameters of the conditional pdf may be calculated analytically or numerically, to maximize the function U.
Denoising
yQ
e
syfswyfswfsywf
wywy
SYWSYSWYSW
2
222
2
)()(
2
1
||||
2
1
)|(),|()|(),|(
• MAP estimate:
2| ),|(maxargˆ ysywfw YSW
ws
s
sYSWSs
ww
sywfsps
ˆ
|
ˆˆ
),|ˆ()(maxargˆ
Denoising (continued)
• Conditional mean estimate:
s
M
sS wspw ˆ)(ˆ
1
)(2
,|Eˆ 22
12
yy
Q
esyww
y
s
0 0.5 1-100
0
100
200
Orig
inal
0 0.5 1-100
0
100
200
Noi
sy
0 0.5 1-100
0
100
200
4 G
auss
ian,
Haa
r
0 0.5 1-100
0
100
200
4 G
auss
ian,
D8
0 0.5 1-100
0
100
200
4 E
xpon
entia
l, H
aar
0 0.5 1-100
0
100
200
4 E
xpon
entia
l, D
8
Init. MSE = 24.639723 4 mix, Haar 4 mix, D8
Gaussian Mixture 3.078267 7.020152Exponential Mixture 2.326472 7.030970
0 0.5 1-20
0
20
Orig
inal
0 0.5 1-20
0
20
Noi
sy
0 0.5 1-20
0
20
2 G
auss
ian,
D8
0 0.5 1-20
0
20
4 G
auss
ian,
D8
0 0.5 1-20
0
20
2 E
xpon
entia
l, D
8
0 0.5 1-20
0
20
4 E
xpon
entia
l, D
8
Init. MSE = 2.429741 2 mix, D8 4 mix, D8
Gaussian Mixture 0.471568 0.417795Exponential Mixture 0.426488 0.397808
0 0.5 1-200
-100
0
100
Orig
inal
0 0.5 1-200
0
200
Noi
sy
0 0.5 1-200
0
200
2 G
auss
ian,
D4
0 0.5 1-200
-100
0
100
4 G
auss
ian,
D8
0 0.5 1-200
-100
0
100
2 E
xpon
entia
l, D
4
0 0.5 1-200
-100
0
100
4 E
xpon
entia
l, D
8
Init. MSE = 92.907059 2 mix, D4 4 mix, D8
Gaussian Mixture 8.442306 7.873508Exponential Mixture 8.394187 7.862579
Conclusion
• Mixture distributions for individual wavelet coefficients can effectively model the non–Gaussian nature of the coefficients.
• Hidden Markov Models can serve as a powerful tool for wavelet-based statistical signal processing.
• One-sided exponential distributions for mixture components along with hidden Markov Tree model can achieve better performance in denoising.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-100
-50
0
50
100
150Noisy Signal
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-60
-40
-20
0
20
40
60
80
100
120
140Denoised Using 4 Gaussian Mixture and Haar Wavelet
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-60
-40
-20
0
20
40
60
80
100
120
140Denoised Using 4 Exponential Mixture and Haar Wavelet
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-15
-10
-5
0
5
10
15
20Noisy Signal
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-15
-10
-5
0
5
10
15Denoised Using 4 Gaussian Mixture and Daubechy Length 8
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-15
-10
-5
0
5
10
15Denoised Using 4 Exponential Mixture and Daubechy Length 8
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-200
-150
-100
-50
0
50
100
150Noisy Signal
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-150
-100
-50
0
50
100Denoised Using 4 Gaussian Mixture and Daubechy Length 8
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-150
-100
-50
0
50
100Denoised Using 4 Exponential Mixture and Daubechy Length 8
top related