Signal Processing, 2004/05 — 1. Overview S. Weiss (Digital) Signal Processing — Overview • In the first part, you have studied a good deal of background theory; • we want to explore some more (particularly digital) signal processing theory, applications, and architectures; • digital signal processing (DSP) is based on theory that goes back more than 200 years to Norbert Gauss and others; • DSP has seen a revolution due to the arrival of inexpensive computational devices in the 1990; • DSP has penetrated almost every aspect of life: Texas Instruments claims that every 10 minutes you use one of their dedicated digital signal processors. 1
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Signal Processing, 2004/05 — 1. Overview S. Weiss
(Digital) Signal Processing — Overview
• In the first part, you have studied a good deal of background theory;
• we want to explore some more (particularly digital) signal processing theory,applications, and architectures;
• digital signal processing (DSP) is based on theory that goes back more than 200years to Norbert Gauss and others;
• DSP has seen a revolution due to the arrival of inexpensive computational devicesin the 1990;
• DSP has penetrated almost every aspect of life: Texas Instruments claims thatevery 10 minutes you use one of their dedicated digital signal processors.
1
Signal Processing, 2004/05 — 1. Overview S. Weiss
DSP Applications
• Multimedia: compression and coding algorithms for audio, video, and still images;acoustic echo control for teleconferencing;
• Biomedical: signal detection; source separation for ECG or EEG; digital hearingaids;
• Defense: radar/sonar; detection and guidance systems;
• Automotive: GPS navigation; engine management;
2
Signal Processing, 2004/05 — 1. Overview S. Weiss
DSP Perspective: Software Defined Radio
• Reliability, repeatability and programmability of DSP has widely replaced analoguesystems in consumer and industrial markets;
• an ambitious future system relying on DSP is a software defined radio:
at RF or IF
ADC
DAC
DSP
etcdatavideoaudio
sampling
Sampling at an intermediate or even radio frequency (IF or RF) permits flexibilityof the modulation stage and use of advanced DSP techniques for synchronisation,equalisation, carrier offset recovery etc.
3
Signal Processing, 2004/05 — 1. Overview S. Weiss
Notation — Time Domain
• Continuous time signal: x(t) with t ∈ R; example: analogue speech signal or anyother voltage signal;
• discrete time signal x[n] withn ∈ Z; example: daily stockprice, video signal:
• vector quantity: x, if timedependent x[n] or xn
• matrix quantity: Tn+1
n+2
+3n
+4n
ff
ff
n
ftime
space
4
Signal Processing, 2004/05 — 1. Overview S. Weiss
Notation — Transform Domain
• Fourier transform:
x(t) —• X(jω) X(jω) =
∞∫−∞
x(t)e−jωt dt (1)
x[n] —• X(ejΩ) X(ejΩ) =∞∑
n=−∞x[n]e−jΩn (2)
with angular frequency ω = 2πf and normalised angular frequency Ω = 2πω/ωs
for sampling at ωs.
• note that X(ejΩ) is periodic with 2π.
• z-transform: H(z) •— h[n] such that H(z) =∞∑
n=−∞h[n] z−n
5
Signal Processing, 2004/05 — 1. Overview S. Weiss
Notation — Stochastic Signals
• A stochastic signal x[n] can be characterised by its probability density function(PDF);
• often we are interested in its autocorrelation function
rxx[τ ] = Ex[n]x∗[n− τ ] (3)
with lag τ , complex conjugation operator ·∗ and expectation operator E·;
• power of x[n] is given by rxx[0]; if x[n] is zero mean, then the variance σ2xx = rxx[0];
• power spectral density (PSD) of x[n]:
Rxx(ejΩ) •— rxx[τ ] such that Rxx(ejΩ) =∞∑
n=−∞rxx[τ ]e−jΩτ (4)
6
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Electrical (Voltage) Signals
• A signal is a variation over time of a measurable quanity that conveys informationof some form;
• a sensor (actuator) can convert a real world signal to (from) a convenient voltagesignal:
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Amplification and Noise
• Many real world signals yield small amplitudes (e.g. EEG order of 10−6v), andamplification is required prior to further processing;
• generally such signals are corrupted by noise; we assume an additive noise model:
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.5
0
0.5
time t / [s]
v(t)
n]v
[n]y[n]s
[
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−1.5
−1
−0.5
0
0.5
1
1.5
time t / [s]
x(t)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−1.5
−1
−0.5
0
0.5
1
1.5
time t / [s]
y(t)
8
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Additive Noise in the Frequency Domain
• Additive noise model in the frequency domain:
yn]
[n]
[n]
(e jΩ)
(e jΩ)
v
Ω
R
Ω
R e jΩ)(
ΩRvv
xx yy
x[
• x[n] is the signal, v[n] thecorrupting noise;
• we assume x[n] and v[n] tobe zero mean processes andmutually uncorrelated;
• measure signal quality: signal-to-noise ratio (SNR);
• signal and noise power can bedetermined from the PSDs.
• from assumptions −→ σ2yy = σ2
xx + σ2vv and Pyy(ejΩ) = Pxx(ejΩ) + Pvv(ejΩ).
9
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Signal-to-Noise Ratio
• The signal to noise ratio is a power ratio:
SNR =signal power
noise power(5)
• for zero-mean signals: SNR = σ2signal/σ2
noise;
• the range of values to be measured may span several orders of magnitude (such asthe human hearing); therefore a logarithmic scale has been introduced:
SNRdB = 10 · log10
σ2signal
σ2noise
= 20 · log10
σsignal
σnoise[deciBel, dB] (6)
• Examples: σ2signal = 1, 000 · σ2
noise −→ SNRdB = 30 dB;
σ2signal = 1, 000, 000 · σ2
noise −→ SNRdB = 60 dB.
10
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Analogue-to-Digital and Digital-to-Analogue Conversion
• In order to interface with the real world, we will need to convert between analogueand digital signals;
• this is accomplished by analogue-to-digital and digital-to-analogue converters(ADCs / DACs);
• analogue signals, such as a speech signal, may require a conversion into the digitaldomain for applying sophisticated source coding algorithms:
processingdigital
DACAD Canalogue
in out
analogue
• digital signals need to be converter to analogue quantities for transmission over anantenna link:
DAC AD Cdigital
in
digital
out
transmissionchannel
11
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Analogue-to-Digital Conversion
• An ADC requires conditioning of the analogue signal, sampling, and quantisation:
AAF
Ts
Qx[n]x t( )
Rsignal conditioning
• sampling: the selection of the sampling period Ts will affect the possiblerepresentation of the frequency content of x(t) by x[n];
• quantisation: the word length R will influence resolution, or — as derived in slide17 — the signal-to-quantisation noise ratio (SQNR);
• signal conditioning: contains an anti-alias filter (AAF) and a gain in order to adjustthe amplitude of x(t) to the input range of the quantiser Q.
12
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Sampling
• Sampling x[n] from a continuous x(t): xs(t) = x(t) ·∞∑
n=−∞δ(t− nTs);
• remembering the Fourier series theorem and the duality property of the Fouriertransform, it is clear that the frequency domain of the sampled signal xs(t) hasbeen periodised, w.r.t. ωs = 2π/Ts:
Xs(jω) =∞∑
n=−∞X(j(ω − nωs)) (7)
• hence
ωω ω ω
X(jω) (jω)sX
ω
ss s s0−2 − 2
,
13
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Nyquist Rate and Aliasing
• Nyquist sampling theorem: all information of a continuous time signal is retainedif spectral repetitions do not overlap; theoretically, the analogue signal can beperfectly reconstructed from the sampled version;
ωω ω ω
X(jω) (jω)sX
ω
ss s smaxω0−2 − 2
,aliasing
ωω ω ω
X(jω) (jω)sX
ω
ss s smaxω0−2 − 2
,no aliasing
• hence the requirement:
ωs > ωN = 2 · ωmax with Nyqust rate ωN (8)
14
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Signal Conditioning
• For a perfect signal representation, we have to sample a signal at least at theNyquist rate;
• in order to avoid aliasing, we must fulfil: ωmax < ωs/2;
• if the signal’s content is not exactly known, we can guarantee this by conditioningthe input signal to the ADC by an anti-alias filter:
(jω)H
ω ωω ωω
s ss− s− 0 /2/2
• this filter removes all frequency components above ωs/2, which would otherwisealias in the sampling stage.
15
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Quantisation
• An R-bit quantiser approximates a straightline with 2R levels; example with R = 3and asymmetric characteristic −→
• the input signal has to use the range[−V ; +V ] as best as possible, otherwiseclippling or coarse resolution result;
• quantisation noise: maximum q/2,uniformly distributed, additive;
-V +V
q
x
binaryoutput
000
011
010
001
100
101
110
111
• quantisation noise power (no clipping!)
σ2e =
∫ q/2
−q/2
e2 · p(e)de =1q
[13e3
]q/2
−q/2
=q2
12(9)
16
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Signal-to-Quantisation Noise Ratio
• Consider a uniformly distributed input signal x(t):
-V +V
( )p x
x1/2V
• therefore, the signal power is
σ2xx =
∫ V
−V
x2 · p(x)dx =1
2V
[13x3
]V
−V
=V 2
3(10)
• with (9) and q = 2V/2R, the signal-to-quantisation noise ratio (SQNR) is given by
SQNRdB = 10 · log10
V 2 · 12 · 22R−2
3 · V 2= 10 · log10 22R ≈ 6.02 ·R [dB] (11)
17
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Digital-to-Analogue Conversion
• For the onversion from the digital to the analogue domain, a DAC comprises of:
R
Qx[n]
Ts
x t( )
x t( ) x[n ]signal conditioning
RCF−1
Ts
t
,
• the voltage signal at the output of Q−1 contains high frequency components, whichneed to be removed:
ωω ω ω
X(jω) (jω)sX
ω
ss s s0−2 − 2
,spectral repetitions
desired spectrum
18
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Reconstruction Filter
• In order to reconstruct the analogue signal x(t), a lowpass filter is required toremove undesired high frequency components;
• this lowpass filter interpolates between adjacent sample values and is thereforetermed reconstruction filter; requirement: stopband edge at ωs/2:
(jω)H
ω ωω ωω
s ss− s− 0 /2/2
• with a perfect anti-alias filter and reconstruction filter, the concatenation of ADCand DAC imposes no loss on the signal (Nyquist);
• loss occurs due to (i) non-ideal anit-alias and reconstruction filters and (ii)quantisation.
19
Signal Processing, 2004/05 — 2. Conversion S. Weiss
Realistic ADC/DAC
• High quality analogue filters for anti-alias and reconstruction filtering require highorder and low tolerances on elements;
• Realistically, a sufficient roll-off has to be permitted, hence limiting the “ideal”reconstruction:
roll−off
| ( jω)|
ω
cut−offfrequency
gainH
• analogue components can be the most expensive part in a DSP system;
• we will study later how part of the analogue processing can be performed in thedigital domain by oversampling and Σ−∆ converters.
20
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Spectrum Estimation
• The discrete Fourier transform (DFT) has been discussed previously (see sections11 and 12 in Prof. Rogers’ notes);
• we want to consider techniques to estimate the spectra of stochastic signals;
• if x is an N -element vector taken from a tap delay line (TDL) and T an N -pointDFT matrix, then from the transform vector X
X = T · x (12)
a periodogram can be derived by squaring the Fourier coefficient; problem: if x israndom, then X will also be random;
• if we want to obtain a deterministic quantity for the spectrum of a random signal,clearly some form of expectation/averaging needs to be incorporated — either inthe time or the frequency domain.
21
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Recall: Stochastic Processes
• A stochastic process is characterised by some deterministic measures:
– its probability density function (PDF) (= normalised histogram);– its moments
∫xlp(x)dx of order l;
(note that l = 1 is mean, l = 2 is variance for zero-mean process);– its autocorrelation function;
• central limit theorem: the sum of arbitrarily distributed processes converges to aGaussian PDF;
• Gaussian (or normal) PDF of a process x[n] ∈ N (µx, σ2xx):
p(x) =1√
2πσxx
e−(x−µx)2
2σ2xx (13)
22
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Stationarity and Ergodicity
• Stationarity means that the statistical moments of a random process do not changeover time;
• a weaker condition is wide-sense stationarity (WSS), i.e. moments up to secondorder (mean and variance) are constant over time; this is sufficient unless higherorder statistics (HOS) algorithms are deployed;
• a stochastic process is ergodic if the expectation operation can be replaced by atemporal average,
σ2xx =
∞∫−∞
x2p(x)dx = Ex[n]x∗[n] = limN→∞
1N
N−1∑n=0
|x[n]|2 (14)
• remember: expectation is an average over an ensemble; a temporal average isperformed over a single ensemble probe!
23
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Moving Average (MA) Model / Signal
• The PDF does not contain any information on how “correlated” successive samplesare;
• consider the following scenario with x[n] ∈ N (0, σ2xx) being uncorrelated (successive
samples are entirely random):
- h[n] -
x[n] y[n] = x[n] ∗ b[n]
N (0, σ2xx) N (0, σ2
yy)
• y[n] is called a moving average process (and b[n] an MA model) of order N − 1 if
y[n] =N−1∑ν=0
b[ν]x[n− ν] is a weighted average over a window of N input samples.
24
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Autoregressive (AR) and ARMA Models / Signals• If the process y[n] has emerged from an uncorrelated process x[n] ∈ N (0, σ2
xx) viathe difference equation
y[n] = b[0]x[n]−N−1∑ν=1
a[ν]y[n− ν] ; (15)
it is called an N − 1 order auroregressive (“self-dependent”,[2]) signal;
• the system associated with (15), defined by its parameters b[0] and a[ν], ν =1 · · ·N − 1, is referred to as an autoregressive model.
• an MA model is often also called an all-zero model or FIR filter; an AR model is anall-pole filter; autoregressive moving average (ARMA) systems refer to a generalIIR filter;
• such a model can be very useful in characterising the stochastic process y[n] by aninnovations filter[3].
25
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Filtering a Random Signal
• Consider lowpass filtering an uncorrelated Gaussian signal x[n]:
- h[n] -
x[n] y[n] = x[n] ∗ h[n]
N (0, σ2x) N (0, σ2
y)
0 50 100 150−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
3
time n
x[n]
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
1.4
norm. angular freq. Ω/π
|H(e
jΩ)|
0 50 100 150−0.5
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5
time n
y[n]
• the output will have Gaussian distribution, but the signal only changes smoothly:neighbouring samples are “correlated”. We need a measure!
26
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Auto-Correlation Function I
• The correlation between a sample x[n] and a neighbouring value x[n− τ ] is givenby
rxx[τ ] = Ex[n] · x∗[n− τ ] = limN→∞
1N
N−1∑n=0
x[n] · x∗[n− τ ] (16)
• For two specific specific lags τ = 3 (left) and τ = 50 (right), consider:
0 50 100 150−0.6
−0.4
−0.2
0
0.2
0.4
0.6
time n
x[n]
, x[n
+3]
0 50 100 150−0.6
−0.4
−0.2
0
0.2
0.4
0.6
time n
x[n]
, x[n
+50
]• the curves on the left look “similar”, the ones on the right “dissimilar”.
27
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Auto-Correlation Function II
• For lag zero, note:
rxx[0] = limN→∞
1N
N−1∑n=0
x[n] · x∗[n] = σ2x + µ2
x (17)
• This value for τ = 0 is the maximum of the auto-correlation function rxx[τ ];xxr [τ]
τ
• large values in the ACF indicate strong correlation, small values weak correlation;
28
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Auto-Correlation Function III
• If a signal has no self-similarity, i.e. it is “completely random”, the ACF takes thefollowing form:
xxr [τ]
τ
• If we take the Fourier transform of rxx[τ ], we obtain a flat spectrum (or a lowpassspectrum for the ACF on slide 28);
• due to the presence of all frequency components in a flat spectrum, a completelyrandom signal is often referred to as “white noise”.
29
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Power Spectral Density
• The power spectral density (PSD), Rxx(ejΩ), defines the spectrum of a randomsignal:
Rxx(ejΩ) =∞∑
τ=−∞rxx[τ ] ejΩτ (18)
• PSD and ACF form a Fourier pair, rxx[τ ] —• Rxx(ejΩ), therefore
rxx[τ ] =12π
∫ π
−π
Rxx(ejΩ) e−jΩτ dΩ (19)
• note that the power of x[n] is (similar to Parseval)
rxx[0] =12π
∫ π
−π
Rxx(ejΩ) dΩ (= scaled area under PSD) (20)
30
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
PSD – Examples
• PSD for uncorrelated (“white”) zero mean noise:
σxxr [τ]
x2
τ
—• σe jΩ)
Ω0 π−π
x2
( xxR
• PSD for correlated zero mean noise:
σ xxr [τ]x2
τ—•
e jΩ)2π σx
2area:
0
Ωπ−π
( xxR
31
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Cross-Correlation
• The cross-correlation function between two signal x[n] and y[n] is defined analogousto (16):
rxy[τ ] = Ex[n] · y∗[n− τ ] = limN→∞
1N
N−1∑n=0
x[n] · y∗[n− τ ] (21)
• note: ryx[τ ] = r∗xy[−τ ] and especially rxx[τ ] = rxx[−τ ]
i.e. the auto-correlation function is symmetric, while the cross-correlation functionis not;
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Examples for Applying the Cross-Correlation
• Delay estimation. Imagine we send a random pulse x[n] and wait for the reflectedreturn signal y[n]:
x[n] n
y[n]n
T T
τ
r [τ]yx
• “Matched filtering”. Compare a received signal with an expected waveform; thecross-correlation will be maximum if the received signal matches up with a desiredsequence.
33
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
PSD and Innovations Filter
• Consider again filtering a random signal x[n] with a filter having an impulseresponse h[n] (either MA, AR, or ARMA):
- h[n] -
x[n] y[n] = x[n] ∗ h[n]
rxx[τ ] ryy[τ ]
Rxx(ejΩ) Ryy(ejΩ)
• relation between x[n] and y[n] is given by convolution: y[n] =∞∑
ν=−∞h[ν] x[n− ν];
• we are looking for the relations between rxx[τ ] and ryy[τ ] and between Rxx(ejΩ)and Ryy(ejΩ).
34
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
• The cross-correlation is:
ryx[τ ] = limN→∞
1N
N−1∑n=0
y[n] · x∗[n− τ ] (23)
= limN→∞
1N
N−1∑n=0
·
( ∞∑ν=−∞
h[ν] x[n− ν]
)x∗[n− τ ] (24)
=∞∑
ν=−∞h[ν] lim
N→∞
1N
N−1∑n=0
x[n− ν] x∗[n− τ ] (25)
=∞∑
ν=−∞h[ν] lim
N→∞
1N
N−1∑n=0
x[n] x∗[n− τ + ν] (26)
=∞∑
ν=−∞h[ν] rxx[τ − ν] = h[τ ] ∗ rxx[τ ] (27)
• note: rxy[τ ] = r∗yx[−τ ] = h∗[−τ ] ∗ rxx[τ ]
35
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
• Going further:
ryy[τ ] = limN→∞
1N
N−1∑n=0
y[n] · y∗[n− τ ] (28)
= limN→∞
1N
N−1∑n=0
y[n]∞∑
ν=−∞h∗[ν] x∗[n− ν] (29)
=∞∑
ν=−∞h∗[ν] lim
N→∞
1N
N−1∑n=0
y[n] x∗[ν − n + τ ] (30)
=∞∑
ν=−∞h∗[ν] ryx[ν + τ ] = h∗[−τ ] ∗ ryx[τ ] (31)
= h∗[−τ ] ∗ h[τ ] ∗ rxx[τ ] (32)
36
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
PSD and Innovations Filter II
• Hence, if a stochastic process x[n] is filtered:
- h[n] -
x[n] y[n] = x[n] ∗ h[n]
rxx[τ ] ryy[τ ]
Rxx(ejΩ) Ryy(ejΩ)
• we have
ryy[τ ] = h∗[−τ ] ∗ h[τ ] ∗ rxx[τ ] (33)
Ryy[τ ] = H∗(ejΩ) H(ejΩ) Rxx[τ ] (34)
= |H(ejΩ)|2 Rxx[τ ] (35)
• note for innovations filter (uncorrelated input x[n] ∈ N (0, 1)): Ryy[τ ] = |H(ejΩ)|2.
37
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Averaged Periodogram I
• The PSD is a deterministic quantity of a random process, while the periodogram israndom;
• averaging of the periodogram is required to eliminate (or at least reducerandomness);
• method 1: take N -point DFT of x[n], and average over a neighbourhood of squaredFourier coefficients;
π
jΩ)|2 |X(e jΩ)|2,
Ω
|X(e
• method 2: calculate a number of periodograms and average across the the frequencybins (Fourier coefficients).
38
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Averaged Periodogramme II
2
[n]
| . |2
DFT
| . |2
DFT
| . |2
|X(e jΩ)|2
n
DFT
averaging
jΩ)|2 jΩ)|2 jΩ)|2|X (e |X (e|X (e0 1
x
39
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Recall Spectral Leakage
• The DFT “periodises” the data, which is likely to create aberrations;
• N = 32, by chance the window ends fit:
0 20 40 60 80−1
0
1
x[n]
0 5 10 150
10
20
|X[k
]|
n k
N
• N = 27, discontinuties arise at the window edges, causing the main peak to “leak”:
0 20 40 60 80−1
0
1
x[n]
0 5 100
5
10
|X[k
]|N
n k
40
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Recall Windowing
• DFT applied to a segment of rectangularly windowed sinusoidal data:
10 20 30 40−20
0
20
40
timen
k
leakage
mag
nitu
de /
[dB
]
• DFT after application of a Hamming window to the same data:
10 20 30 40−20
0
20
40
timen
kmag
nitu
de /
[dB
]
• with windowing, the spectral leakage is reduced at the cost of a widened main lobe.
41
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Practical PSD Calculation
DFT
[n]
|X(e jΩ)|2
| . |2 | . |2 | . |2 | . |2 | . |2 | . |2
n
averaging
DFT DFT DFT DFT DFT
x
42
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Relation between Averaged Periodogram and PSD
• Note the differences and correspondences between the PSP and the averagedperiodogram:
E·x[n] −→ rxx[τ ]
DFT ↓ ↓ DFT
|X(ejΩ)|2 −→ Rxx(ejΩ)E·
(36)
• spectral estimation (approximations of the PSD) are usually based on the averagedperiodogram method outlined in slide 42.
43
Signal Processing, 2004/05 — 3. Spectrum Estimation S. Weiss
Summary• We have defined PSD theoretically and the averaged periodogram as a
computational technique to estimate it;
• usually the longer the data sequence x[n] available, the more confident (accurate)this estimate is;
• we will define a parametric method for spectrum estimation, trying to guess at theinnovations filter behind the signal at hand;
• for background and further reading:
[1] B. Girod, R. Rabenstein, and A. Stenger. Signals and Systems. J. Wiley & Sons, 2001.
[Good explanation of auto- and cross-correlation, and the PSD]
[2] T. K. Moon and W. C. Stirling. Mathematical Methods and Algorithms for Signal Processing.
Prentice Hall, 1999.
[3] A. Papoulis. Probability, Random Variables, and Stochastic Processes. McGraw-Hill, 3rd ed.,
1991.
[A classic text.]
44
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
Impulse Response
• For an FIR filter, the impulse response h[n] (earlier stated as zero-state response,i.e. the TDL is initialised to zero) is given by its filter coefficients:
n
n[ ]h
[n]y
n
N0
x
0
[n]
• the impulse response has finite support with h[n] = 0 ∀n ≥ N ;
• the impulse response can give direct clues about multipath propagation, or echosand reverberation.
46
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
Frequency Response
• Frequency response H(ejΩ): measures the gain of the system in the steady state:
• impulse and frequency response are a Fourier pair: h[n] —• H(ejΩ).
0 5 10 15 20 25−0.2
0
0.2
0.4
0.6
time index n
h[n]
—•
0 0.2 0.4 0.6 0.8 10
0.5
1
1.5
normalised angular freq. Ω / 2π
|H(e
jΩ)|
47
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
Magnitude and Phase Response
• The magnitude decides whether the system has e.g. lowpass, highpass, or bandpasscharacteristic;
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.5
1
1.5
normalised angular freq. Ω / 2π
|H(e
jΩ)|
• phase contains a number of exciting characteristics of a filter:
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−60
−40
−20
0
normalised angular freq. Ω / 2π
∠ H
(ejΩ
) /r
ad
48
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
Group Delay
• from H(ejΩ) = |H(ejΩ)|ejΦ(Ω), note for the group delay g(Ω):
g(ejΩ) = − d
dΩΦ(Ω) (37)
0 0.2 0.4 0.6 0.8 1−80
−60
−40
−20
0
normalised angular freq. Ω / 2π
∠ H
(ejΩ
) /r
ad
−→
0 0.2 0.4 0.6 0.8 110
11
12
13
14
normalised angular freq. Ω / 2π
g(ejΩ
) / s
ampl
es
• the group delay tells us by how long certain frequency components will be delayedwhen propagating through the filter;
• note the difference between linear phase and non-linear phase systems.
49
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
z-Transform
• Reconsider FIR filter:
• z-transform of difference equation:
Y (z) = b0X(z) + b1z−1X(z) + b2z
−2X(z) + · · ·+ bN−1z−N+1X(z) (38)
H(z) =Y (z)X(z)
=N−1∑i=0
biz−i (39)
• the z-transform with z = ejΩ+α can capture the transient behaviour of a system;
• note that H(ejΩ) = H(z)|z=ejΩ.
50
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
z-Transform — Some Useful Properties
• Assume a z-transform pair h[n] •— H(z) =∑∞
i=−∞ hi z−i
• time shift:h[n−∆] •— H(z) z−∆ (40)
• complex conjugation:
h∗[n] •— H∗(ejΩ) =∞∑
i=−∞h∗i z−i (41)
• time reverse:
h[−n] •— H(z−1) =∞∑
i=−∞hi zi (42)
51
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
Transfer Function
• We can factorise the transfer function H(z):
H(z) = b0 + b1z−1 + · · · bN−1z
−N+1 (43)
= b0(1− β0z−1)(1− β1z
−1) · · · (1− βN−2z−N+2) (44)
• the roots βi of H(z) are called the zeros of the transfer function;
• often a pole-zero plot is used for visulisation:
52
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
Real Valued Systems
• If the impulse response only contains real-valued coefficients, the zeros βi must beeither real valued, or occur as complex conjugate pairs (as seen in slide52;
• we could factorise the system into a sequence of first order sections:
X(z) - b0- 1− β0z
−1 · · · - 1− βN−2z−1 - Y (z)
• a complex multiplication take 4 real valued multiplications; therefore, forimplementational purposes, we would always prefer (44) over (44).
53
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
Minimum and Maximum Phase
• For a minimum phase system, all zeros lie inside the unit circle;
• a maximum phase system has all zeros outside the unit circle;
• interesting: if h[n] is minimum phase, then h[−n] is maximum phase;
• example for H1(z) = 1 + 12z−1 and H2(z) = 1
2 + z−1:
H1(z)H2(z)H3(z) = H1(z)H2(z)
•—
0 0.2 0.4 0.6 0.8 1−6
−4
−2
0
2
normalised angular freq. Ω / 2π∠
Hi(e
jΩ)
/rad
54
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
Linear Phase Filters
• recall: linear phase means constant group delay;
• a linear phase system must have a z-transform of the form (including a possibledelay for causality)
• any zero of a linear phase system must have a reciprocially matching partner (unlesson the unit circle);
• linear phase iff impulse response is symmetric: recall that hmin. phase[n] ∗hmin. phase[−n] will give an auto-correlation type result.
55
Signal Processing, 2004/05 — 4. Digital Filtering S. Weiss
Summary
• Note that certain properties of a system are best assessed in a specific domain:either impulse response, frequency domain, or transfer function;
• phase information is often vital, e.g. if we need to invert the system (for equalisationor control purposes);
• most FIR filter designs yield symmetric impulse response and hence are linear phase;
• we will consider such filter designs next ...
56
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
FIR and IIR Design
• Design characteristics: passband width, transition band width, stopband edge,stopband attenuation, and passband ripple:
0 dB
-3 dB
cΩ
e )Ωj(H
transition
ripple
stopband
Ωπ0
band width
attenuation
passband
frequencycutoff
passband
width
stopbandedge
• generally: high quality requires high number of coefficients.
57
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
Different Frequency Responses
• Four different basic filter types as defined by their magnitude response: (a) lowpass,(b) highpass, (c) bandpass, and (d) bandstop filter:
e )Ωj(H
Ω
e )Ωj(H
Ω
e )Ωj(H
Ω
e )Ωj(H
Ωπ π
π π
0 0
00
(a)
(c) (d)
(b)
58
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
FIR Filter Design: IDFT and Windowing
• We often have an idea what the magnitude response |H(ejΩ)| of the desired filtershould look like; additionally, we need to “invent” a phase response (e.g. linear);
• the frequency response is inversely Fourier transformed;
H(e jΩ) h[n]
Ωn
• the resulting time domain response is an approximation of the desired impulseresponse (holding the filter coefficients);
• some windowing may be required in order to enforce finite support in the timedomain.
• a variety of design approaches based on minimax or least squares methods exists.
59
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
FIR Design Examples
• Comparison of two filter designs with 16 and 48 coefficients, respectively:
0 5 10 15 20
0
0.1
0.2
h 1[n]
0 10 20 30 40 50
0
0.1
0.2
h 2[n]
time index n
0 0.1 0.2 0.3 0.4 0.5−80
−60
−40
−20
0
Ω / 2π
|H1(ejΩ)|
|H2(ejΩ)|
• both designs have identical passband edge at Ω = 0.1π.
60
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
IIR Filtering
• Various ways to represent the difference equation as a flow graph:
y[n] =L−1∑l=0
al x[n− l] +J−1∑j=1
bj y[n− j] (46)
[ ]1b [ ]2b[ ]0b
xx [ ] x [ -1 ] x [ ]-2 x [ - +1]
b[ -1 ]
n n n n L
L
a[J-1 ] a[ ]2 a[ ]1
y [ ]
y [ - J+1] y [ ]-2 y [ -1 ]n n n
n
adder
multiplier
delay element
61
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
IIR Stability
• transfer function:
H(z) =Y (z)X(z)
=b0 + b1z
−1 + · · · bL−1z−L+1
1− a1z−1 − a2z−2 − · · · − aJ−1z−J+1(47)
= b0(1− β0z
−1)(1− β1z−1) · · · (1− βL−2z
−1)(1− α0z−1)(1− α1z−1) · · · (1− αJ−2z−1)
(48)
• : note that H(z) can be spit into a cascade of FIR and recursive first-order sections;
• each recursive section 11−αz−1 must be stable; note geometric series
11− αz−1
=∞∑
i=0
αi z−i for |a| < 1 (49)
• poles must be inside the unit circle!
62
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
IIR Filter Design
• IIR filter design is based on analogue techniques (Butterworth, Chebychev etc);
• the analogue design is transformed from the Laplace- to the z-domain by means ofthe bilinear transform:
s =1Ts
1− z−1
1 + z−1(50)
• IIR filters generally achieve high quality filters with less coefficients than FIR.
• Appreciate the response length (again geometric series):
11− az−1
=∞∑
i=0
ai z−i for |a| < 1 (51)
63
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
FIR/IIR Comparison
• Comparison between FIR and IIR filters (both 24 coefficients):
0 0.1 0.2 0.3 0.4 0.5−80
−60
−40
−20
0
20FIR
|H(e
jΩ)|
/ [d
B]
0 0.1 0.2 0.3 0.4 0.5−80
−60
−40
−20
0
20FIR quantised, B=8 bit
norm. ang. freq. Ω/2π
|H(e
jΩ)|
/ [d
B]
0 0.1 0.2 0.3 0.4 0.5−80
−60
−40
−20
0
20IIR
0 0.1 0.2 0.3 0.4 0.5−80
−60
−40
−20
0
20IIR quantised, B=8 bit
norm. ang. freq. Ω/2π
0 0.1 0.2 0.3 0.4 0.511.5
11.5
11.5
11.5
11.5
11.5
11.5FIR
g(ejΩ
) / [
sam
ples
]
0 0.1 0.2 0.3 0.4 0.511.5
11.5
11.5
11.5
11.5FIR quantised, B=8 bit
g(ejΩ
) / [
sam
ples
]
norm. ang. freq. Ω/2π
0 0.1 0.2 0.3 0.4 0.5−20
0
20
40
60
80
100IIR
0 0.1 0.2 0.3 0.4 0.50
50
100
150IIR quantised, B=8 bit
norm. ang. freq. Ω/2π
• note that quantisation can affect an IIR filter badly (instability)!
64
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
Computational Complexity I
• Recall for general IIR filter:
y[n] =L−1∑l=0
blx[n− l] +J−1∑j=1
ajy[n− j] = bHxn + aHyn−1 (52)
• whereby vector notation:
xn = [x[n] x[n− 1] · · ·x[n− L + 1]]T (53)
yn−1 = [y[n− 1] y[n− 2] · · · y[n− J + 1]]T (54)
b = [b[0] b[1] · · · b[L− 1]]H (55)
a = [a[1] a[2] · · · a[J − 1]]H (56)
(57)
65
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
• Example 1. Sampling rate fs = 8 kHz; we want to model the impulse responseof an acoustic system with a T = .5 s duration, and therefore need T · fs = 4000coefficients. Total complexity: C = 4000fs = 32 MMAC/s;
• Example 2. We double the sampling rate to wideband audio with fs = 16 kHz;we require 8000 coefficients and yield a total complexity of C = 128 MMAC/s;
• Note: doubling the sampling rate means quadrupling the complexity!
• State-of-the-art digital signal processors perform approximately 1 GMAC/s.
66
Signal Processing, 2004/05 — 5. FIR and IIR Design S. Weiss
Summary on FIR/IIR Filtering
• Characterisation by impulse response, frequency response, or transfer function;
• Be aware of properties such as causality and minimum/maximum/non-minimumphase;
• Stability. FIR is strictly stable, IIR is not;
• Quality/Complexity. IIR filters need less coefficients than FIR.
• Linear Phase. For FIR filters a symmetric impulse response is necessary andsufficient for linear phase; IIR are never linear phase (although some yield anapproximation in the passband);
• Implementation. Both FIR and IIR are sensitive to fixed point implementation,but IIR has larger dynamics in its filter coefficients and may even become unstablewhen quantised.
67
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Adaptive Filtering
• Previously we have designed filters with specific characteristics; reconsider an FIRfilter with L coefficients w[n]:
ny[ ]
nd[ ]
e n[ ]
nx[ ]
L-10 1 2w w w w
• here the adjustment of the coefficients w[n] may be dependent on the environment,and the coefficients may be variable over time.
68
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Generic Adaptive Filtering
• The architecture of a generic adaptive filter comprises of an input signal x[n], adesired signal d[n], a filter output y[n], an error signal e[n], and the filter impulsresponse w[n];
ny[ ]
nd[ ]
e n[ ]
nx[ ]
L-10 1 2w w w w
• Aim of an adaptive filter: the filter w[n] forms an output y[n] from an inputx[n], such that, when subtracted from the desired signal d[n], the reslting error isminimised in a suitable sense.
69
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Adaptive Filter Architectures
• A generic adaptive filter can be applied in different architectures:
• system identification
ny[ ] e n[ ]
nd[ ]
nx[ ]w
c
• inverse system identification /equalisation
ny[ ] e n[ ]nx[ ]c
nd[ ]
w
• prediction
ny[ ] e n[ ]nx[ ]
nd[ ]
w
• noise cancellation
ny[ ] e n[ ]nx[ ]
nd[ ]
w
n[ ]v + ]nx[
70
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
System Identification
• We want to identify an unknown LTI system with impulse response c[n]:
ny[ ] e n[ ]
nd[ ]
nx[ ]w
c
• if the adaptive filter converges and e[n] is (ideally) zero, then the input-outputbehaviour of c[n] and w[n] is identical;
• if x[n] is sufficiently broadband, we have identified c[n] by means of a suitablemodel contained in w[n].
71
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Inverse System Identification / Equalisation
• We place the unknown system and the adaptive filter in series:
ny[ ] e n[ ]nx[ ]c
nd[ ]
w
• if the error signal is minised, then w[n] represents the inverse system of c[n];
• if c[n] is non-minimum phase, we need to permit a delay ∆, such that c[n]∗w[n] ≈δ[n−∆].
72
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Prediction
• Due to delaying the input signal x[n], the desired signal d[n] is a “future version”of the filter input:
ny[ ] e n[ ]nx[ ]
nd[ ]
w
• the adaptive filter should only be able to predict “periodic” or correlated componentswith d[n];
• the error signal e[n] would in this case contain any remaining uncorrelated signalcomponents.
73
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Adaptive Noise Cancellation
• consider a noise corrupted signal d[n] (e.g. helicopter pilot’s microphone signal):
ny[ ] e n[ ]nx[ ]
nd[ ]
w
n[ ]v + ]nx[
• if we can obtain a reference signa x[n] of the noise, the adaptive filter should beable to cancel the noise component from e[n], provided that the noise in x[n] andd[n] is linearly related.
74
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Acoustic Echo Cancellation
• hands-free telephony — full-duplex mode suffers from acoustic feedback:
[n]y
[n]e
[n]d
[n]x
room
echo
near endspeaker
speech+ echo
far endspeaker
roommodel replica
echo
adaptive digital filter – acoustic feedback is
very disturbing forthe far end speaker
– by identifying theroom acoustics (overthe frequency rangeof speech), theadaptive filter canproduce a replica ofthe echo;
• due to the reverberation times of a room, which the adaptive filter has to match,this is a very computationally demanding task.
75
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
How Can We Optimise the Filter Coefficients?
• In order to calulate an optimal filter wopt[n], the mean square error (MSE) Ee2[n]
is a good criterion to be minimised;
• Formulation of error signal e[n]:
e[n] = d[n]− y[n] = d[n]−L−1∑i=0
wi · x[n− i] = d[n]−wH · xn
• with definitions:w =
[w0 w1 · · · wL−1
]T;
xn =[x[n] x[n− 1] · · · x[n− L + 1]
]T;
• (we could do the derivation for complex valued signals, but for simplicity here stickwith the real valued case.)
76
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Optimal Filter — MSE Formulation
• Mean Square Error formulation (note that the optimum filter wopt is deterministic!):
Ee2[n]
= E
(d[n]−wT · xn)2
= E
d2[n]
− 2wT · Ed[n] · xn+ wT · E
xn · xT
n
·w
= σ2d − 2wT · p + wT ·R ·w
with covariance matrix R = Exn · xT
n
and correlation vector p = Ed[n] · xn;
• note: wT · xn is a scalar, hence wT · xn = (wT · xn)T = xTn ·w;
• important: the expression for Ee2[n]
is quadratic in the filter coefficients w;
hence optimisation should lead to a unique extremum!
77
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Covariance Matrix & Cross-Correlation Vector
• Given the auto-correlation sequence rxx[τ ] and the cross-correlation sequencerxd[τ ]:
rxx[τ ] =1N
N−1∑n=0
x[n] · x[n− τ ] rxd[τ ] =1N
N−1∑n=0
d[n] · x[n− τ ]
• the covariance matrix R and cross-correlation vector p can be written
R =
rxx[0] rxx[1] · rxx[L−1]rxx[1] rxx[0] · rxx[L−2]
... . . . ...rxx[L−1] rxx[L−2] · · · rxx[0]
p =
rxd[0]rxd[1]
...rxd[L−1]
• the Wiener-Hopf solution requires estimation of R and p, and a matrix inversion
(numerically costly and potentially unstable!).
78
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
MSE Surface
• Quadratic in the filter coefficients w, the surface of the MSE ξ = Ee2[n]
is a
hyperparabola in L + 1 dimensional hyperspace;
• example for L = 2 coefficients:
w0,opt
w1,optw0
w1
ξ
79
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Optimum Filter — Minimum Mean Squared Error
• Standard optimisation procedure to achieve minimum MSE:
∂
∂wEe2[n]
!= 0
• Inserting the previous expression for the MSE:
∂
∂wEe2[n]
= 0− 2p + 2R ·w
• if R is invertible, then the optimum filter coefficients wopt are given by (Wiener-Hopfequation):
wopt = R−1 · p
80
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Optimum Filter — Minimum Mean Squared Error
• Standard optimisation procedure to achieve minimum MSE:
∂
∂wEe2[n]
!= 0
• Inserting the previous expression for the MSE:
∂
∂wEe2[n]
= 0− 2p + 2R ·w
• if R is invertible, then the optimum filter coefficients wopt are given by (Wiener-Hopfequation):
wopt = R−1 · p
81
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Viability of Wiener-Hopf
• Confident estimation of R and p requires long time windows;
• inversion of R is of order O(L3) — very complex!
• Wiener-Hopf requires R to be invertible;
• even if R has full rank, it may be ill-conditioned.
82
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Iterative Solutions
• In order to bypass the problems of Wiener Hopf, we will utilise iterative approachesto converge towards the optimal solution;
– Starting from an initial guess w[0],gradient descent techniques followthe negative gradient of the MSEcost function:
w[n + 1] = w[n]− µ∇ξ[n]
– µ is called the “step size” of thealgorithm and controls convergence. . .
w0,opt
w1,optw0
w1
ξ
large µ
small µ
83
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Gradient Descent Techniques
• We require the gradient ∇ξ[n] = −2p + 2Rw[n] — see slide 81
• therefore:w[n + 1] = w[n] + 2µ(p−Rw[n]) (58)
• this is also know as the method of steepest descent;
• note I: matrix inversion is no longer required;
• note II: we still need to estimate R and p.
84
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Stochastic Gradient Techniques
• We omit expectations and minimise the instantaneous squared error:
we2[n] = −2e[n] · xn (59)
• this results in the least mean squares (LMS) algorithm:
w[n + 1] = w[n] + 2µe[n] · xn (60)
• the gradient estimate is noise, but through itertion, on average the gradient estimatepoints into the right direction.
85
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Least Mean Squares Algorithm
ny[ ]
nd[ ]
e n[ ]
nx[ ]
L-10 1 2w w w w
• Steps for the LMS adaptive filter:(1) update tap delay line vector x[n](2) calculate filter output: y[n] = wH[n]x[n](3) calculate error: e[n] = d[n]− y[n](4) LMS update: w[n + 1] = w[n] + 2µe[n] · xn
• Complexity: 2L multiply accumulate (L for scalar product, L for LMS update) persampling period.
86
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
LMS Convergence
• The step size µ controls the convergence of the LMS;
• intuitive from MSE surface plot: small (large) µ −→ slow (fast) convergence withhigh (low) final accuracy;
• Without proof: convergence can be guaranteed if
0 < µ <1
Lσ2xx
(61)
where L is the filter length and σ2xx the variance of the input signal.
87
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
LMS System Identification Example I
• Matlab code for LMS algorithm in system identification:
• For uncorrelated input, an adaptive filter of length L would identify the first Lelements of this geometric series.
90
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Channel Equalisation Example II
• Now assume a maximum phase channel C(z) = .5 + z−1;
• the inverse of this channel is unstable
W (z) =1
C(z)=
21 + 2z−1
= 2− 4z−1 + 8z−2 − · · · (64)
and the “geometric series” diverges; alternatively
W (z) =z
1 + 0.5z= z
∞∑n=0
(−.5)nzn = z − .5z1 + .25z2 − · · · (65)
converges but is anti-causal.
91
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Channel Equalisation Example II ctd.
• If we permit a delay of e.g. 3 samples, z−3, such that
W (z) =z−2
1 + 0.5z= z−2
∞∑n=0
(−.5)nz (66)
= z−2 − 0.5z−1 + 0.25− 0.125z1 + · · · (67)
an adaptive filter can identify the 3 causal coefficients of the geometric series.
• hence, for equalisation or inverse system identification, in general we have to permita delay for the system C(z)W (z);
• rule of thumb: not knowing anything about thye phase properties of C(z),practisioners choose a delay of L/2.
92
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Channel Equalisation Example III
• Now consider a channel C(z) = 1− z−1 — this is a highpass filter with a zero atz = 1 cancelling the dc (f = 0) frequency component;
• the inverse system would attempted to provide a very high (ideally infinite) gain atfs/2 — this is very dangerous!
• inspect the covariance matrix R; for uncorrelated input u[n] to the channel, theautocorrelation function of x[n] is given by
rxx[τ ] = c[τ ] ∗ c[−τ ] (68)
and is likely to be ill-conditioned.
• try to establish what filter response the LMS algorithm converges to.
93
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Channel Equalisation Example III ctd.
• In a realistic scenario, a channel suffers from additive white Gaussia noise v[n],such that
x[n] = c[n] ∗ u[n] + v[n] (69)
whereby v[n] is independent of u[n];
• therefore for the autocorrelation function due to independence:
rxx[τ ] = c[τ ] ∗ c[−τ ] + rvv[τ ] (70)
• for the covariance matrix R, we obtain
Rxx = Rcc + Rvv (71)
whereby Rcc is the covariance matrix of the noiseless channel output, and Rvv =σ2
vvI the covariance matrix of the AWGN.
94
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Channel Equalisation Example III ctd.
• Comparing Rcc and Rxx, the conditioning of the matrix is improved (
Rxx = QΛQH︸ ︷︷ ︸Rcc
+σ2vvI = Q
λc,0 + σ2
vv 0 00 λc,1 + σ2
vv 0 0... . . . ...0 0 λc,L−1 + σ2
vv
QH
(72)
• simulate the previous example with channel noise: in the presence of channelnoise, the adaptive filter strikes a balance between inversion of C(z) and noiseamplification;
• the adaptive filter provides a minimum mean square error (MMSE) solution ratherthan an inversion: the system inverse is “regularised” by the AWGN.
95
Signal Processing, 2004/05 — 6. Adaptive Filtering S. Weiss
Summary / Further Study
• Adaptive filters tuneable filter coefficients and can be applied in situation whereexactly desired response (highpass, lowpass, etc) is not known a priori;
• many other popular adaptive algorithms exist beyond the LMS;
• for further reading:
[1] B. Widrow and S.D. Stearns Adaptive Signal Processing. Prentice Hall, 1985.[Classical and easy to read text.]
[2] S. Haykin Adaptive Filter Theory. Prentice Hall, 1987, 1991, and 1996.[A good mathematical treatment — strangely the 2nd edition is consideredbetter than the third.]
96
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
7. Multirate Signal Processing
• In a multirate system, various components run at different sampling rates;
• Audio DSP: you may need to convert from professional audio (96kHz) down to HIFI(e.g. digital audio tape, 48kHz) or “wideband” audio (teleconferencing, 16kHz);
• Software defined radio (SDR): you may need to create a digital IF signal (sampledat, say, 75MHz) from a UMTS baseband signal (5MHz sampling rate);
Similarly, in an SDR receiver, you may have to extract a symbol stream at 5MHzfrom a 75MHz IF signal.
97
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Multirate Operations
• A multirate system usually comprises of digital filters, decimators, and expanders;
• anti-alias filtering followed by N -fold decimation (downsampling) to lower thesampling rate by a factor N :
- anti-alias filter -
&%'$↓ N -
• N -fold expansion or up-sampling followed by interpolation filtering to increase thesampling rate by a factor of N :
-
&%'$↑ N - interpolation filter -
98
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Decimation — Time Domain Desciption• To lower the sampling rate by a factor of N , a decimator or downsampler only
retains every N th sample — other samples are discarded; exampled for N = 4:
0 5 10 15 20 25 30 35 40−0.5
0
0.5
time index n
x[n]
0 1 2 3 4 5 6 7 8 9 10−0.5
0
0.5
time index m=n/4
x d[m]
• if the signal is suitably bandlimited, we loose no information in the original signal;also, the exact sampling point can be chosen arbitrarily.
99
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Decimation — Frequency Domain Description
• similar to sampling on slide 13, resampling “periodises” the spectrum with respectto the new sampling rate ω′s = ωs/N ; example for N = 4 (with unnormalisedangular frequencies):
X(jω)
ωω(jω)sX
ω ωωωω
ω
s s ss
s s
−4− 0 4
0−
• note: if the original signal is not suitably bandlimited, aliasing may occur.
100
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Anti-Alias Filtering
• A digital anti-alias filter can be employed to filter the signal prior to decimation:
H ejΩ( )
πΩ
ππN Nπ− // 0−
• it is difficult to create a brick-wall type filter; however, in the digital domain we canachieve a high quality filter design at a lower “cost” than in the analogue domain.
101
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Expansion — Time Domain Description• To increase the sampling rate by a factor of N , an expander or upsampler fill N −1
zero sample in between every other sample; exampled for N = 4:
0 1 2 3 4 5 6 7 8 9 10−0.5
0
0.5
time index m
x[m
]
0 5 10 15 20 25 30 35 40−0.5
0
0.5
time index n
x e[n]
• note that our signal is now impulsive . . . our signal can be expected to containsome high frequency components.
102
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Expansion — Frequency Domain Description
• Expansion simply re-scales the frequency axis:
ωωω
ωωω
jω)X (e
X(jω)
ωω
−
−4
s s
s s− 4 ss 0
0
• in the expanded signal, we see N − 1 spectral repetions in addition to the original(baseband) signal;
103
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Expansion — Interpolation Filter
• A digital interpolation or reconstruction filter can be employed to retireve thebaseband portion of the signal after expansion:
H ejΩ( )
πΩ
ππN Nπ− // 0−
• this is identical to the conditions required of the anti-alias filter.
104
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Oversampled ADC
• Consider sampling an analogue signal at fs = NfN rather than fs = fN :
]AAF
Ts
x t( )AAF N
x[n]
Rsignal conditioning
N
Qx[m
• the analogue anti-alias filter can now be relaxed:H(jω)
πN fN
ωπfN0
• the transition band of the anti-alias filter can be much wider than in the non-oversampled case; to compensate, we will postprocess the oversampled digital x[m]by (much cheaper) digital operations.
105
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Digital Postprocessing in Oversampled ADCs
• Requirement for the digital anti-alias filter is a tight cut-off at Ωc = π/N :
Ωπ
noise
signal
πN Nπ
H ej( Ω)
π 0− //−
• this is easier to achieve than a high quality analogue filter;
• additionally, the digital AAF reduces the quantisation noise power (q2/12) by afactor 1/N (recall that the area under PSD is a measure for the power!).
106
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Virtual Increase in Word Length
• Assume that the original signal was quantised with a word length R;
• the quantisation noise power after the digital anti-alias filter has removed theout-of-band noise for N = 4:
σ2q =
14
q2
12=
(q/2)2
12(73)
• if the quantisation step size is halved, we have gained an extra bit in resolution;
• for every quadrupling of the oversampling ratio N , we achieve an extra bit inresolution.
107
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Trade-off between Sampling and Wordlength
• We can oversample our analogue signal by 4∆R to achieve ∆R extra bits inresolution after digital postprocessing;
• to achieve 16 bit quality from a single bit quantiser, we would have to oversampledby a factor 415;
• single bit converters exist, but use an additional technique, whereby the quantisationnoise is “pushed” towards higher frequencies and can be better suppressed by adigital lowpass anti-alias filter.
108
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Sigma-Delta Converter
• A sigma-delta converter is a mixed analogue-digital device, which provides a low-bit(often 1 bit) output:
+ 1 bit outputinput
DAC
integrator comparator
−
• The input sigma is passed through an integrator (lowpass filter), and the output isconverted with single bit accuracy, i.e. only retaining the sign;
• the output bit-stream is converted into a voltage signal by a DAC and subtractedfrom the input.
•
109
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Discrete Time Model
• In order to analyse the sigma-delta circuit, we approximate it by a discrete timerepresentation, with input X(z), white additive noise N(z), and output Y (z):
X(z)
z−1
Y(z)
N(z)
H(z)−
1 bit ADC
1 bit DAC
• it is easy to verify that Y (z) = X(z) + (1− z−1)N(z);
• this means that the output contains the desired signal plus noise with a highpassPSD.
110
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Extracting High Bit Resolution
• The quantisation noise before and after noise shaping:
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−40
−30
−20
−10
0
10P
SD
/ [d
B]
normalised angular frequency Ω / π
AWGNnoise shaping
(PSD of N(z) is solid blue, noise component in Y (z) is dashed red)
• if we apply this in combination with oversampling, due to the reduced noise powerin the baseband, substantial resolution can be gained.
111
Signal Processing, 2004/05 — 7. Multirate Signal Porcessing S. Weiss
Summary
• We have considered multirate signal processing components, which permit tochange the sampling rate;
• oversampling ADCs have been considered as an application; although not consideredhere, oversampling can be similarly exploited to reduce the cost and increase theperformance of DACs;
• Additional benefit can be gained by noise shaping in combination with oversampling;ultimately, a singe bit converter can provide multi-bit quality (assuing that the digitalpostprocessing, e.g. anti-alias filtering) is performed at a higher bit resolution.