Faculty - C1. Systems and Filtersfaculty.nps.edu/rcristi/EO3404/C-Filters/text/1-Systems.pdfC1. Systems and Filters Objectives • Define discrete time systems and their characteristics

C1. Systems and Filters Objectives

• Define discrete time systems and their characteristics • Introduce Finite Impulse Response (FIR) Filters and the convolution operation • Introduce the Frequency Response of Filters

1. Introduction The goal of this unit is to present a number of operations we do on signals. For example, we want to remove a disturbance from a signal, or at least attenuate its effects. This is a typical example when we collect data from a transducer (acoustic data, for example) and we preprocess it to eliminate as much of the disturbances as possible. This is a typical filtering problem which we will be addressing in the next Chapter. In some other cases we want to detect a pulse of a sonar or radar so that we can determine the range of a target. In this case we need design what is called a matched filter which will be the subject of the last chapter in this unit. All these problems are addressed using the concept of “system”, which is a generic term for processing a given signal (the input) into an other signal (the output). In this chapter we present different classes of signals, and in particular what we call LTI (for Linear Time Invariant) systems. This class of systems is not only very simple to implement, but it lands itself to a frequency response description which is very intuitive and also very close to the physical formulation of the problem. In what follows, we present the properties of systems in the next section, followed by the systems we will be implementing, the Finite Impulse Response (FIR) Filters. In order to analyze and design these filters, we need the concept of the Discrete Time Fourier Transform (DTFT) presented in section 4 and the Frequency Response in section 5.

2. Systems: Definitions Video: Systems, Definitions (4:35)

http://faculty.nps.edu/rcristi/eo3404/c-filters/videos/c1_02_systemsDefinition.mp4

A system is a transformation of an input signal ][nx into an output signal ][ny . Every time there is a relationship of cause between an input signal (the cause) and an output signal (the effect) we can define a system. This is represented in the figure below.

A discrete time system

Typical example in this section is the problem of filtering where the input signal ][nx is an observation of a signal with noise. The goal of the filter is to eliminate or at least attenuate the effect of noise. This is shown in the figure below.

Filtering Problem

Although we will be concentrating exclusively on a subset of systems, called Linear Time Invariant (LTI) we want to spend a bit of effort in defining the main classes of systems. Since there are techniques which can be applied only to LTI systems (such as the convolution and the frequency response to be defined later) it is important to understand when a system is LTI and when it is not. To simplify the notation, we indicate the input output relation of a system S as

{ }][][ nxSny = Video: Linearity (12:14)

http://faculty.nps.edu/rcristi/eo3404/c-filters/videos/c1_03_linearity.mp4

Linearity. Consider a system S and any two input output pairs

{ }{ }][][

][][

22

11

nxSnynxSny

==

Then the system is linear if, for any two constants 21 ,aa we can write { }][][][][ 22112211 nxanxaSnyanya +=+

In other words, the output is the superposition of the respective outputs. This is shown in the figure below.

Linearity of a System.

Example. Consider the system defined as

]1[][][ −+= nxnxny Let the input be ][][][ 2211 nxanxanx += then the output is given by (grouping some terms)

( ) ( )]1[][]1[][][ 222111 −++−+= nxnxanxnxany which is clearly ][][ 2211 nyanya + . So the system is linear. Example. Consider the system defined as

( )2][2][ nxny = Then again let the input be ][][][ 2211 nxanxanx += and the output becomes

( ) ( ) ( )222

211

22211 ][2][2][][2][ nxanxanxanxany +≠+=

So the system is non linear.

Video: Time Invariance (8:24)

http://faculty.nps.edu/rcristi/eo3404/c-filters/videos/c1_04_timeInvariance.mp4

Time Invariance. Consider a system S and any input output pairs

{ }][][ nxSny = Then the system is Time Invariant if and only if

{ }][][ DnxSDny −=− with D any time delay. In other words the response to a delayed input corresponds to a delayed output.

This is shown in the figure below.

Time Invariance

See an example. Example. Again consider the system

]1[][][ −+= nxnxny Let the input now be ][ Dnx − . Then the output is

]1[][][1 −−+−= DnxDnxny Since ][][1 Dnyny −= the system is time invariant. Example. Consider the system

][2][ nnxny = Now input a delayed signal and the output becomes

][2][1 Dnnxny −= Since ][)(2][ DnxDnDny −−=− is not equal to ][1 ny the system is not Time Invariant.

Video: Stability (9:58) http://faculty.nps.edu/rcristi/eo3404/c-filters/videos/c1_05_stability.mp4

Stability. A system is stable provided the output is bounded whenever the input is bounded. This is shown in the figure below.

Stability

Example. Consider again the system

]1[][][ −+= nxnxny If the input is bounded as Anx ≤|][| for all n then the output is bounded as

Anxnxny 2|]1[||][||][| ≤−+≤ So the output is bounded and the system is stable. Example. Consider the system

][][ nnxny = Take the input 1][ =nx for all n , clearly bounded. The output is nny =][ clearly unbounded so the system is not stable.

Video: Causality (4:11) http://faculty.nps.edu/rcristi/eo3404/c-filters/videos/c1_06_causality.mp4

Next important property is causality. This has to do with the fact that, in real time implementation, we can use only past an present data, not future data. In particular we can say the following: Causality. A system is causal if and only if the output at any time n does not depend on any future input value. In other words, in a causal system the effect comes after the cause. See some examples. Example. Consider the system defined as follows

]3[4]2[2]1[3][ −+−−−= nxnxnxny In this case we see that the output ][ny at time n depends only on past inputs at times

3,2,1 −−− nnn . Then the system is causal. Example. Consider the system

]3[4]2[2]1[3][ −+−−+= nxnxnxny In this case the output ][ny , at time n depends on a future value of the input at time

1+n . So this system is non causal. As we can see from the figure below, for a causal system is at rest for 0<n , the output comes after the input.

A causal system

3. Finite Impulse Response (FIR) Filters.

Video: FIR Filters (3:26) http://faculty.nps.edu/rcristi/eo3404/c-filters/videos/c1_07_firFilters.mp4

A Finite Impulse Response (FIR) Filter is a Linear, Time Invariant, Stable system defined as follows

][][...]1[]1[][]0[][ NnxNhnxhnxhny −++−+=

where the terms ][],...,1[],0[ Nhhh are the coefficients of the filter and N is the filter order. The above expression can be written in a more compact form as

][*][][][][0

nxnhnxhnyN

=−= ∑=

and it is called the convolution of the two sequences ][nh and ][nx . The “star” in the rightmist expression means “convolution” of the two sequences (not to be confused with the star of complex conjugate). The sequence of coefficients Nnnh ,...,0],[ − characterizes the filter and it is called the Impulse Response.

Video: Convolution (12:34) http://faculty.nps.edu/rcristi/eo3404/c-filters/videos/c1_08_convolution.mp4

Example. Take a simple averaging filter defined as

( )]9[...]1[][101][ −++−+= nxnxnxny

Then the output signal ][ny is obtained by averaging the 10 latest samples of the input sequence. The effect is to attenuate the noise by smoothing the signal. In this example, the filter coefficients are given by

9,...,0,101][ == nnh

A fundamental property of the FIR filter, and of all Linear Time Invariant systems, is that the response to a complex exponential is, per se, a complex exponential.

Video: Response to a Complex Exponential (20:42) http://faculty.nps.edu/rcristi/eo3404/c-filters/videos/c1_09_firResponse.mp4

Property: Response to a Complex Exponential. Consider an FIR filter with impulse response Nnnh ,...,0],[ = and let the input be njenx 0][ ω= . Then the output is given by

njeHny 0)(][ 0ωω=

where )( 0ωH is a complex coefficient computed as

∑=

−=N

jehH0

00][)(

ωω

Proof. Just by replacing the input ][nx with nje 0ω , so that we obtain

njN

jN

nj eehehny 000

0

)

0

)( ][][][ ωωω

== ∑∑

=

−

=

−

The term in bracket is )( 0ωH .

This is shown in the figure below.

Response of a LTI System to a Complex exponential

See an example: Example. Consider the FIR filter defined by the following relation:

( )]9[...]1[][101][ −++−+= nxnxnxny

In this filter all the coefficients are given by

9,...,0,101][ == nnh

Let the input be njenx π1.0][ = , a complex exponential with frequency πω 1.00 = . Then we compute the constant

( ) 4137.11.0

101.09

0

1.0 6392.01

1101

1011.0 j

j

jj e

eeeH −

−

×−

=

− =−−

=

= ∑ π

πππ

and the response becomes ( )

)4137.11.0(

1.04137.1

6392.0 6392.0][

−

−

=

=nj

njj

eeeny

π

π

If the input the signal is a sum of complex exponentials as

∑=k

njk

keXnx ω][

then the output can be computed using superposition as

∑=k

njk

keYny ω][

where kkk XHY )(ω= . In other words every frequency component is treated separately and its coefficient is multiplied by the value of the frequency response )(ωH at the corresponding frequency kω . This can be seen in the following example. Example. Consider the Filter in the figure below.

Filter for the Example

Let the input be

)7.03.0cos(2)2.01.0cos(3][ −++= nnnx ππ This can be written in terms of complex exponentials usig Euler formulas as

( ) ( )( ) ( ) njjnjj

njjnjj

eeeeeeeenx

ππ

ππ

3.07.03.07.0

1.02.01.02.0

0.10.1 5.15.1][

−−

−−

++

++=

Each complex exponential can be treated independently using the Frequency Response of the system, which can be computed for every frequency as

( )

−−

=+++= −

−−−

ω

ωωωω j

jjj

eeeeH

11

51...1

51)(

54

Now we need to compute it for the frequencies πω 1.0±= and πω 2.0±= . We obtain the following values

0.6283 0.6283

1.885 1.885

(0.1 ) 0.904 , ( 0.1 ) 0.904(0.3 ) 0.3115 , ( 0.3 ) 0.3115

j j

j j

H e H eH e H e

π π

π π

−

−

= − =

= − =

The output signal can now be computed as ( )( ) ( )( )( )( ) ( )( )

0.2 0.1 0.2 0.1

0.7 0.3 0.7 0.3

[ ] 0.1 1.5 0.1 1.5

0.3 1.0 0.3 1.0

j j n j j n

j j n j j n

y n H e e H e e

H e e H e e

π π

π π

π π

π π

− −

− −

= + − +

+ + −

Combining the complex exponentials we can now write it in terms of sinusoidal signals as

[ ] 2.712cos(0.1 0.428) 0.6230cos(0.3 2.585)y n n nπ π= − + − In the next section we see that any signal can be expanded in terms of complex exponentials using the Discrete Time Fourier Transform (DTFT).

4. The Discrete Time Fourier Transform (DTFT).

Video: Discrete Time Fourier Transform (8:14) http://faculty.nps.edu/rcristi/eo3404/c-filters/videos/c1_10_DTFT.mp4

In order to introduce the DTFT, let ],[nx with +∞<<∞− n be a sequence of infinite length and define

{ } ∑+∞

−∞=

−==n

njenxnxDTFTX ωω ][][)(

with ω being the digital frequency in radians. Given )(ωX we dermine ][nx as follows

{ } ∫−

==π

π

ω ωωπ

ω deXXIDTFTnx nj)(21)(][

The reason of this formula can be easily seen by substituting for )(ωX and obtain

...00][00...21][)(

21 )( ++++== ∫∑∫

+

−

−+∞

−∞=−

nxdexdeX njnjπ

π

ωπ

π

ω ωπ

ωωπ

The integral on the right is zero if ≠n , and it is equal to π2 when =n which yields the result. There are two important properties: 1. Periodicity. The DTFT )(ωX is periodic with period π2 . Therefore all the information is in one period, say )(ωX with πωπ +<≤− . Proof. This comes directly from the definition

( ) )(][][)2( 2 ωπω ωπω XenxenxXn

nj

n

nj ===+ ∑∑+∞

−∞=

−+∞

−∞=

+−

2. Symmetry. Let ][nx be a real sequence. Then { }][)( nxDTFTX =ω is symmetric in the sense that

)()( * ωω XX =− Proof. Just apply the definition to obtain

)(][][)( **

ωω ωω XenxenxXn

nj

n

nj =

==− ∑∑

+∞

−∞=

−+∞

−∞=

See an example. Let ][nx be a square pulse as shown in the figure below, defined as

−≤≤

=otherwise 0

10 if 1][

Nnnx

Square pulse.

Notice that this signal has an infinite length, since it is defined for all n . Even if it is zero up to infinity, it is still defined and it has an infinite length. Then, applying the definition of the DTFT we obtain

)(1

1)(1

0ωω ω

ωω

Nj

NjN

n

nj We

eeX =−−

== −

−−

=

−∑

where ( )( )2/sin

2/sin )( 2/)1(

ωωω ω NeW Nj

N−−= . Its magnitude is shown in the figure below.

Plot of |)(| ωNW . See another example. Let ][nx be given by

<≥

=0 if 00 if

][nna

nxn

With a such that 1|| <a . Then its DTFT is computed as

ωωω j

n

njn

aeeaX −

+∞

=

−

−== ∑ 1

1)(0

5. Frequency Response of an LTI System

Video: Frequency Response of an LTI System (10:27) http://faculty.nps.edu/rcristi/eo3404/c-filters/videos/c1_11_dtftExample.mp4

The function

πωπω ω +<≤−= ∑=

− ,][)(0

N

n

njenhH

is called the frequency response of the FIR filter described by

∑=

−=N

nxhny0

][][][

In particular the response to a sinusoidal signal can be determined using linearity and superposition. Property: Response to a Sinusoid. Consider an FIR filter with impulse response

Nnnh ,...,0],[ = and let the input be )cos(][ 0 αω += nAnx . Then the output is given by

( ))(cos)(][ 000 ωαωω HnAHny ∠++=

where )( 0ωH and )( 0ωH∠ are magnitude and phase the complex coefficient

∑=

−=N

jehH0

00][)(

ωω

Proof. Recall that a sinusoid can be written in terms of complex exponentials as

njjnjj eeAeeAnAnx 00

22)cos(][ 0

ωαωααω −−

+

=+=

Since the system is linear, the total response is the sum of the two responses to the complex exponentials as

njjnjj eHeAeHeAny 00 )(2

)(2

][ 00ωαωα ωω −− −

+

=

Since the filter coefficients ][nh are real, the frequency response )(ωH is symmetric, which yields

)()( 0

*0 ωω HH =−

Then the output can be written as

( ) ( ) njHjjnjHjj eeHeAeeHeAny 0000 )(0

)(0 )(

2)(

2][ ωωαωωα ωω −∠−−∠

+

=

The two terms can be combined together in a cosine, using Euler’s formulas. See an example. Consider the same FIR filter as in the previous example, and the input

)3.01.0cos(5.1][ += nnx π

Then, again, ( ) 4137.19

0

1.0 6392.01011.0 jj eeH −

=

− =

= ∑

ππ as computed above. This leads

to the response

)1137.11.0cos(9588.0 )4137.13.01.0cos(5.16392.0][

−=−+×=

nnny

ππ

The particular property of the response to the exponential is very important. In fact in general, the numerical computation of the reponse by convolution is a very involved operation. However, in the particular case when the input is an exponential, like we have seen njenx 0][ ω= , the output is itself an exponential, just multiplied by a coefficient as njeHny 0)(][ 0

ωω= . This can be generalized to any signal, using the Discrete Time Fourier Transform (DTFT) introduced at the beginning of this chapter. In particular we have the following property: Property: General Response of an FIR Filter. Given an FIR filter with impulse response

Nnnh ,...,0],[ = and any input ][nx with { }][)( nxDTFTX =ω , then the output ][ny is such that

)()()( ωωω XHY = where { }][)( nyDTFTY =ω and )(ωH is the frequency response of the system. Proof. Recall that, by the DTFT you can write the input ][nx as

∫+

−

=π

π

ω ωωπ

deXnx nj)(21][

This shows that any signal ][nx can be represented as sum of complex exponentials. Since the

dependence on time is in nje ω only, we can use the frequency response for each exponential as

∫+

−

=π

π

ω ωωωπ

deHXny nj)()(21][

Comparing with the expression of the IDFT

∫+

−

=π

π

ω ωωπ

deYny nj)(21][

we see easily that )()()( ωωω XHY = .

Since, for every ω the frequency response )(ωH is a complex number, we can plot it in terms of magnitude and phase. In matlab, first we define a vector h with all filter coefficients, and then we use the command freqz to compute and plot the frequency response. Let’s see and example. Example. Take the same FIR filter defined by the following relation:

( )]9[...]1[][101][ −++−+= nxnxnxny

Then if we want to plot the frequency response, define the vector of impulse response coefficients

h=[0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1]; and then compute and plot the frequency response

freqz(h,1); The result is shown in the figure below.

Frequency Response of the FIR Filter in the example.

From the plots, notice the following:

• The magnitude is in dB’s, as ( ))(log20)( 10 ωω HHdB= ;

• The horizontal axis is scaled as πω / .