6.003 Signal Processing · Filtering: Implementing Convolution with DFT Let 𝐹 ... What is in the parenthesis is inverse DFT, remember iDFT gives the periodically extended signal

6.003 Signal Processing

Week 8, Lecture B:System Abstraction (IV): Filtering

6.003 Fall 2020

System AbstractionThree complete representations for linear, time-invariant systems.

The frequency response is Fourier transform of unit-sample/impulse response!

Difference/Differential Equations: relating output with input.

Unit-Sample/Impulse Response: responses across time for an impulse input.

Frequency Response: responses across frequencies for sinusoidal inputs.

Frequency Response

𝑥[𝑛] 𝑦[𝑛]

𝑌 Ω = 𝐻 Ω 𝑋 Ω

The frequency response provides insights regarding how a system modifies the frequency contents of input signals.

Examples: • Design a low-pass filter with certain cut off frequency Ω𝑐• Separating harmonics using band pass filters• Perform bass-boost• … Ω𝑐Ω𝑐

Today: Focusing on the filtering aspect of the systems

Filtering

We can view filtering in both the time and frequency domains:

Time domain:

Frequency domain:

Consider an example: perform low-pass filtering on a song (input 𝑥 𝑛 ), we can:

• Work in time domain: Obtain the unit-sample response ℎ 𝑛 of the filter system, perform convolution 𝑥 ∗ ℎ to obtain the filtered signal 𝑦 𝑛

• Work in frequency domain: Convert 𝑥 𝑛 to frequency domain 𝑋 Ω , perform frequency domain manipulation to obtain 𝑌 Ω and convert back to 𝑦 𝑛

Check yourself!Consider the system described by: 𝑦 𝑛 =

𝑥 𝑛 − 1 + 𝑥 𝑛 + 𝑥[𝑛 + 1]

3

Sketch this system’s response to the following input: 𝑥 𝑛 = 𝛿 𝑛 − 1 + 2𝛿 𝑛 − 2 + 3𝛿[𝑛 − 3]

First let us find the system’s unit-sample response:

when 𝑥 𝑛 = 𝛿 𝑛 , 𝑦 𝑛 = ℎ 𝑛

when 𝑥 𝑛 = 𝛿 𝑛 − 1 + 2𝛿 𝑛 − 2 + 3𝛿[𝑛 − 3]

𝑦 𝑛 =𝛿 𝑛 − 2 + 𝛿 𝑛 − 1 + 𝛿[𝑛]

3+ 2 ∙

𝛿 𝑛 − 3 + 𝛿 𝑛 − 2 + 𝛿[𝑛 − 1]

3+ 3 ∙

𝛿 𝑛 − 4 + 𝛿 𝑛 − 3 + 𝛿[𝑛 − 2]

3

𝑦 𝑛 =𝛿 𝑛 + 3𝛿 𝑛 − 1 + 6𝛿 𝑛 − 2 + 5𝛿 𝑛 − 3 + 3𝛿 𝑛 − 4

3

=𝛿 𝑛 − 1 + 𝛿 𝑛 + 𝛿[𝑛 + 1]

3

Check yourself!Consider the system described by: 𝑦 𝑛 =

𝑥 𝑛 − 1 + 𝑥 𝑛 + 𝑥[𝑛 + 1]

3

Sketch this system’s response to the following input: 𝑥 𝑛 = cos(2𝜋𝑛

100)

If the output signal is a scalar multiple of the input signal, we refer to the signal as an eigenfunction, and the multiplier as the eigenvalue.

Now that 𝑥 𝑛 is infinite long, not easy to perform 𝑥 ∗ ℎ.𝑦 𝑛 = 𝑥 ∗ ℎ 𝑛 =

𝑘=−∞

∞

𝑥[𝑘] ℎ 𝑛 − 𝑘

Check yourself!Consider the system described by: 𝑦 𝑛 =

𝑥 𝑛 − 1 + 𝑥 𝑛 + 𝑥[𝑛 + 1]

3

Sketch this system’s response to the following input: 𝑥 𝑛 = cos(2𝜋𝑛

100)

Now that 𝑥 𝑛 is infinite long, not easy to perform 𝑥 ∗ ℎ.

Complex exponentials are eigenfunctions of LTI systems!

𝐻(𝛺) 𝑒𝑗Ω𝑛𝑒𝑗Ω𝑛

Check yourself!Consider the system described by: 𝑦 𝑛 =

𝑥 𝑛 − 1 + 𝑥 𝑛 + 𝑥[𝑛 + 1]

3

Sketch this system’s response to the following input: 𝑥 𝑛 = cos(2𝜋𝑛

100)

Now that 𝑥 𝑛 is infinite long, not easy to perform 𝑥 ∗ ℎ.

𝐻(𝛺) 𝑒𝑗Ω𝑛𝑒𝑗Ω𝑛 ℎ 𝑛 =𝛿 𝑛 − 1 + 𝛿 𝑛 + 𝛿[𝑛 + 1]

3

𝐻 Ω =

𝑛=−∞

∞

ℎ 𝑛 𝑒−𝑗Ω𝑛 =𝑒𝑗Ω + 1 + 𝑒−𝑗Ω

3=1 + 2cos(Ω)

3

𝜋−𝜋 2𝜋

3−2𝜋

30

|𝐻 Ω |

𝑦 𝑛 = 𝐻2𝜋

100cos

2𝜋𝑛

100=

1+2cos(2𝜋

100)

3cos

2𝜋𝑛

100

0 < Ω <2𝜋

3: low pass

2𝜋

3< Ω < 𝜋: signal sign flip (phase delay by )

Since input is a single frequency within 0,2𝜋

3,

with 𝐻(2𝜋

100) just a constant

Check yourself!Consider the moving average filter from before, characterized by:

𝑦 𝑛 =𝑥 𝑛 − 1 + 𝑥 𝑛 + 𝑥[𝑛 + 1]

3

We have found this system’s frequency response:

Does this agree with the input/output relationship we see?

3-pt averager

𝑥[𝑛] 𝑦 𝑛

𝜋−𝜋 2𝜋

3−2𝜋

30

|𝐻 Ω |

Design a Low Pass FilterIf instead of the 3-pt averaging, we have the following unit-sample response:

What does the frequency response look like?ℎ1 𝑛 =1

2𝛿 𝑛 +

1

4𝛿 𝑛 − 1 +

1

4𝛿[𝑛 + 1]

𝐻1 Ω =

𝑛=−∞

∞

ℎ1 𝑛 𝑒−𝑗Ω𝑛 =

1

2+1

4(𝑒−𝑗Ω + 𝑒𝑗Ω)

𝐻1 Ω =1

2+

1

2cos(Ω)

𝜋−𝜋 0

Design a Low Pass FilterIf 𝐻2 Ω = (𝐻1 Ω )

2= (1

2+

1

2cos Ω )2, how does the shape of 𝐻2 Ω compare with 𝐻1 Ω ?

What does ℎ2[𝑛] look like? For what values of n is ℎ2[𝑛] nonzero?

ℎ2 𝑛 = ℎ1 ∗ ℎ1 𝑛 =1

16𝛿 𝑛 + 2 +

1

4𝛿 𝑛 + 1 +

3

8𝛿 𝑛 +

1

4𝛿 𝑛 − 1 +

1

16𝛿[𝑛 − 2]

This function is nonzero for n ∈ {-2, −1, 0, 1, 2}

𝜋−𝜋 0

𝐻1 Ω

𝐻2 Ω

Design a Low Pass FilterHow about 𝐻𝑁 Ω = (𝐻1 Ω )

𝑁= (1

2+

1

2cos Ω )𝑁 for N=3, 10, 100, 1000?

What does ℎ𝐿[𝑛] (N=1000) look like? How many nonzero values do we expect?

Many of the samples are small, but we have 2001 non-zero samples!

𝑯𝑳 𝜴

Plotting only a small portion of values

ℎ1 ∗ ℎ1 ∗∙∙∙∗ ℎ1 ∗ ℎ1

Bass BoostLet 𝑥[·] be our original input, and (𝑥 ∗ ℎ𝐿)[·] represent a low-passed version of 𝑥[·].

We want to use a single convolution to produce a new signal with the high frequencies still present, but the low frequencies amplified:

Find an expression for ℎ𝐵 𝑛 .

𝑥 ∗ ℎ𝐵 𝑛 = 𝑥 𝑛 + 𝐴 𝑥 ∗ ℎ𝐿 [𝑛]

𝑋 Ω 𝐻𝐵 Ω = 𝑋 Ω + A𝑋 Ω 𝐻𝐿 Ω

𝑋 Ω 𝐻𝐵 Ω = 𝑋 Ω (1 + A𝐻𝐿 Ω )

𝐻𝐵 Ω = 1 + A𝐻𝐿 Ω

In order to find ℎ𝐵 𝑛 , we consider perform the above operation in frequency domain:

ℎ𝐵 𝑛 = 𝛿[𝑛] + Aℎ𝐿[𝑛]

ℎ𝐵 𝑛 =1

2𝜋න2𝜋

𝐻𝐵(Ω) 𝑒𝑗Ω𝑛dΩ ℎ𝐿 𝑛 =

1

2𝜋න2𝜋

𝐻𝐿(Ω) 𝑒𝑗Ω𝑛dΩ 𝛿 𝑛 =

1

2𝜋න2𝜋

1 ∙ 𝑒𝑗Ω𝑛dΩ

Filtering: Comparison

Time domain:

Frequency domain:

Working in time domain: determine a desired frequency response H(·), compute the unit sample response h[·] of that filter, and then convolve an input signal with h[·].

• Required a frequency response for which h[·] is reasonably straightforward to calculate. • Long unit sample responses h[·] can mean that the convolution itself can take a long time.

• Integration with continuous frequency Ω hinders computation.

Working in frequency domain: from x[·] obtain X(·), then multiply X(·) by the desired frequency response H(·) to obtain Y(·), and then convert to y[·].

𝑦[𝑛] =1

2𝜋න2𝜋

𝑋 Ω 𝐻(Ω) ∙ 𝑒𝑗Ω𝑛 𝑑Ω

𝑋 Ω =

𝑛=−∞

∞

𝑥[𝑛] ∙ 𝑒−𝑗Ω𝑛

We can use DFT when working in frequency domain!

Filtering: Implementing Convolution

Time domain:

Frequency domain:

Let 𝐹 Ω = 𝐹𝑎 (Ω) ∙ 𝐹𝑏 (Ω), find 𝑓[𝑛].

multiplication of Fourier transforms equivalent to convolution in the time domain

𝑓[𝑛] =1

2𝜋න2𝜋

𝐹 Ω ∙ 𝑒𝑗Ω𝑛 𝑑Ω =1

2𝜋න2𝜋

𝐹𝑎 (Ω) ∙ 𝐹𝑏 (Ω) ∙ 𝑒𝑗Ω𝑛 𝑑Ω

=1

2𝜋න2𝜋

𝐹𝑎 (Ω) ∙

𝑚=−∞

∞

𝑓𝑏[𝑚] ∙ 𝑒−𝑗Ω𝑚 ∙ 𝑒𝑗Ω𝑛 𝑑Ω =

𝑚=−∞

∞

𝑓𝑏[𝑚]1

2𝜋න2𝜋

𝐹𝑎 (Ω) ∙ 𝑒𝑗Ω(𝑛−𝑚) 𝑑Ω

=

𝑚=−∞

∞

𝑓𝑏[𝑚] 𝑓𝑎[𝑛 − 𝑚] ≡ (𝑓𝑏∗ 𝑓𝑎)[𝑛]

𝑥 ∗ ℎ 𝑛DTFT

𝐻 Ω 𝑋 Ω

Filtering: Implementing Convolution with DFT

Let 𝐹[𝑘] = 𝐹𝑎[𝑘] ∙ 𝐹𝑏[𝑘], find 𝑓[𝑛].

The expression in the parenthesis looks like 𝑓𝑎[𝑛 − 𝑚] since

𝑓[𝑛] =

𝑘=0

𝑁−1

𝐹[𝑘]𝑒𝑗2𝜋𝑁 𝑘𝑛 =

𝑘=0

𝑁−1

𝐹𝑎[𝑘] ∙ 𝐹𝑏[𝑘]𝑒𝑗2𝜋𝑁 𝑘𝑛 =

𝑘=0

𝑁−1

𝐹𝑎[𝑘] ∙1

𝑁

𝑚=0

𝑁−1

𝑓𝑏[𝑚]𝑒−𝑗

2𝜋𝑁 𝑘𝑚 𝑒𝑗

2𝜋𝑁 𝑘𝑛

=1

𝑁

𝑚=0

𝑁−1

𝑓𝑏[𝑚]

𝑘=0

𝑁−1

𝐹𝑎[𝑘] ∙ 𝑒𝑗2𝜋𝑁 𝑘(𝑛−𝑚)

𝑓𝑎 [𝑛] =

𝑘=0

𝑁−1

𝐹𝑎[𝑘]𝑒𝑗2𝜋𝑁 𝑘𝑛

But 𝑓𝑎[𝑛] was only defined for 0 ≤ 𝑛 < 𝑁, and 𝑛 −𝑚 can fall outside that range.

How should we evaluate 𝑓𝑎[𝑛] when n is not between 0 and N−1?

Filtering: Implementing Convolution with DFTLet 𝐹[𝑘] = 𝐹𝑎[𝑘] ∙ 𝐹𝑏[𝑘], find 𝑓[𝑛].

The expression in the parenthesis looks like 𝑓𝑎 𝑛 −𝑚 , but 𝑓𝑎[𝑛] was only defined for 0 ≤𝑛 < 𝑁, and 𝑛 −𝑚 can fall outside that range. How should we evaluate 𝑓𝑎[𝑛] when n is not between 0 and N−1?

𝑓[𝑛] =

𝑘=0

𝑁−1

𝐹[𝑘]𝑒𝑗2𝜋𝑁𝑘𝑛

=1

𝑁

𝑚=0

𝑁−1

𝑓𝑏[𝑚]

𝑘=0

𝑁−1

𝐹𝑎[𝑘] ∙ 𝑒𝑗2𝜋𝑁 𝑘(𝑛−𝑚)

What is in the parenthesis is inverse DFT, remember iDFT gives the periodically extended signal (Please refer to slide #10 of Lec05A, slide #3 of Lec 05B).

So the expression in the parenthesis equal to 𝑓𝑎𝑝 𝑛 −𝑚 , where 𝑓𝑎𝑝 𝑛 is a periodically

extended version of 𝑓𝑎 𝑛 :

=1

𝑁

𝑚=0

𝑁−1

𝑓𝑏[𝑚] 𝑓𝑎𝑝[ 𝑛 − 𝑚 ]

≡1

𝑁(𝑓𝑏 ⊛𝑓𝑎)[𝑛]

𝑓𝑎𝑝 𝑛 =

𝑖=−∞

∞

𝑓𝑎[𝑛 + 𝑖𝑁]

𝑓[𝑛] =1

𝑁

𝑚=0

𝑁−1

𝑓𝑏[𝑚] 𝑓𝑎[ 𝑛 − 𝑚 𝑚𝑜𝑑 𝑁]

=1

𝑁(𝑓𝑏 ∗ 𝑓𝑎𝑝)[𝑛]

𝑓𝑎 𝑛

𝑓𝑎𝑝 𝑛

= 𝑓𝑎 𝑛 𝑚𝑜𝑑 𝑁

Circular Convolution:

Circular ConvolutionLet us use an example to illustrate the difference between circular convolution & conventional convolution.

(𝑓𝑏∗ 𝑓𝑎) 𝑛 =

𝑚=−∞

∞

𝑓𝑏 𝑚 𝑓𝑎 𝑛 − 𝑚 = 𝑓𝑏[0]𝑓𝑎 𝑛 + 𝑓𝑏[1]𝑓𝑎 𝑛 − 1 + 𝑓𝑏[2]𝑓𝑎 𝑛 − 2

(𝑓𝑏⊛𝑓𝑎) 𝑛 =

𝑚=0

𝑁−1

𝑓𝑏 𝑚 𝑓𝑎 𝑛 −𝑚 𝑚𝑜𝑑 𝑁

= 𝑓𝑏[0]𝑓𝑎 𝑛 𝑚𝑜𝑑 𝑁 + 𝑓𝑏[1]𝑓𝑎 𝑛 − 1 𝑚𝑜𝑑 𝑁+ 𝑓𝑏[2]𝑓𝑎 𝑛 − 2 𝑚𝑜𝑑 𝑁

𝑓𝑏[0]𝑓𝑎 𝑛

𝑓𝑏[1]𝑓𝑎 𝑛 − 1

𝑓𝑏[2]𝑓𝑎 𝑛 − 2

(𝑓𝑏∗ 𝑓𝑎) 𝑛

(𝑓𝑏∗ 𝑓𝑎)

𝑓𝑎𝑝 𝑛

(𝑓𝑏⊛𝑓𝑎)

𝑓𝑏[0]𝑓𝑎 𝑛 𝑚𝑜𝑑 𝑁

𝑓𝑏[1]𝑓𝑎 𝑛 − 1 𝑚𝑜𝑑 𝑁

𝑓𝑏[2]𝑓𝑎 𝑛 − 2 𝑚𝑜𝑑 𝑁

(𝑓𝑏⊛𝑓𝑎) 𝑛

= (𝑓𝑏∗ 𝑓𝑎𝑝)[𝑛]

The parts of the conventional convolution that would fall outside the DFT window “alias” to points inside the DFT window.

aliasing in time!

Different Ways to Consider Circular Convolution

The result of circular convolution is equivalent to:

(𝑓𝑏⊛𝑓𝑎) 𝑛 =

𝑚=−∞

∞

𝑓𝑏 𝑚 𝑓𝑎 𝑛 − 𝑚 𝑚𝑜𝑑 𝑁 =

𝑚=−∞

∞

𝑓𝑏 𝑚 𝑓𝑎𝑝 𝑛 − 𝑚 = (𝑓𝑏∗ 𝑓𝑎𝑝)[𝑛]

𝑓𝑎𝑝 𝑛

(𝑓𝑏⊛𝑓𝑎)(𝑓𝑏∗ 𝑓𝑎)

• A periodically-extended version (periodic in N) of the result of convolving the two signals:

(𝑓𝑏⊛𝑓𝑎) 𝑛 =

𝑚=−∞

∞

(𝑓𝑏∗ 𝑓𝑎)[𝑛 − 𝑚𝑁]

• A convolution of one of the signals with a periodically-extended version of the other:

(𝑓𝑏⊛𝑓𝑎) 𝑛 = (𝑓𝑏∗ 𝑓𝑎𝑝)[𝑛], 𝑓𝑎𝑝 𝑛 = σ𝑚=−∞∞ 𝑓𝑎[𝑛 − 𝑚𝑁]

Implementing Convolution

With DTFT:

If

𝑥 𝑛𝐷𝑇𝐹𝑇

𝑋(Ω)

and

ℎ 𝑛𝐷𝑇𝐹𝑇

𝐻(Ω)

then

(𝑥 ∗ ℎ) 𝑛𝐷𝑇𝐹𝑇

𝐻(Ω)𝑋(Ω)

With DFT:

If

𝑥 𝑛𝐷𝐹𝑇

𝑋[𝑘]

and

ℎ 𝑛𝐷𝐹𝑇

𝐻[𝑘]

then

1

𝑁(𝑥 ⊛ ℎ) 𝑛

𝐷𝐹𝑇𝐻 𝑘 𝑋[𝑘]

SummaryWe can perform filtering either from time or frequency domain:

Time domain:

Frequency domain:

𝑥 ∗ ℎ 𝑛𝐷𝑇𝐹𝑇

𝐻 Ω 𝑋 Ω

Freq. domain multiplication correspond to time domain (circular) convolution:

1

𝑁(𝑥 ⊛ ℎ) 𝑛

𝐷𝐹𝑇𝐻 𝑘 𝑋[𝑘]

The concept of circular convolution:

(𝑓𝑏⊛𝑓𝑎) 𝑛 = (𝑓𝑏∗ 𝑓𝑎𝑝)[𝑛], 𝑓𝑎𝑝 𝑛 = σ𝑚=−∞∞ 𝑓𝑎[𝑛 − 𝑚𝑁]

(𝑓𝑏⊛𝑓𝑎) 𝑛 =

𝑚=−∞

∞

(𝑓𝑏∗ 𝑓𝑎)[𝑛 − 𝑚𝑁]