ANALYSIS OF ADAPTIVE FILTERS AND LMS ALGORITHMshodhganga.inflibnet.ac.in/bitstream/10603/103068/10/10... · 2018-07-07 · 28 3. ANALYSIS OF ADAPTIVE FILTERS AND LMS ALGORITHM 3.1

26

CHAPTER – 3

ANALYSIS OF ADAPTIVE

FILTERS AND LMS

ALGORITHM

27

CHAPTER – 3

CHAPTER – 3: ANALYSIS OF ADAPTIVE FILTERS AND

LMS ALGORITHM

S.No. Name of the Sub-Title Page No.

3.1 Introduction 28

3.2 Adaptive Filtering Problem 29

3.3 Filter Structures 30

3.3.1 Finite Impulse Response Filter 31

3.3.2 Infinite Impulse Response Filter 34

3.3.3 Properties of FIR Filter 36

3.4 Task of an Adaptive Filter 37

3.5 Introduction to LMS Algorithm 38

3.6 Overview of Adaptive LMS Algorithm 40

3.7 DA based Approach for Inner Product Computation 41

28

3. ANALYSIS OF ADAPTIVE FILTERS AND LMS

ALGORITHM

3.1 INTRODUCTION

Basically, an adaptive filter is a computational device which is

used to realize a relation between the two signals in an iterative

manner in real time. Adaptive filters are often realized by running the

set of program instructions in devices such as DSP chip or as a

microprocessor or set of logic operations implemented in the field

programmable gate array (FPGA) or in a semicustom VLSI integrated

circuit or custom VLSI integrated circuit. By neglecting the errors due

to numerical precision effects in the implementation, the adaptive

filter’s primary job is characterized and physical realization is takes

place. That’s why the realization of the mathematical form of the

adaptive filter in software and hardware is opposed.

We can define the adaptive filter with the help of four aspects

which are explained below

1. The type of signals that a filter can process.

2. The architecture that defines the relationship between the

input signal and the output signal of the filter.

3. The parameters of this architecture that can be changed

iteratively to influence the filter’s input-output relationship.

4. Finally, the adaptive algorithm followed that decides how the

parameters are adjusted from one time instant to the next.

29

The parameters that can be adjusted in adaptive filters are

specified by the selection of the adaptive filter structure. The

algorithm that is used for the updating of the parameter values may

have innumerable of forms. Among this, optimization procedure is

used which minimizes the error creation that is useful for the

particular task at hand.

The general mathematical representation of the adaptive

filtering problem explains the operation of the adaptive filter. With an

overview of the advantages of adaptive filtering, they are used

successfully and widely in several applications. The simple

mathematical derivation of the least-mean- square algorithm which is

most popularly used method for the adjustment of the coefficients of

the adaptive filter is explained in this chapter.

3.2 THE ADAPTIVE FILTERING PROBLEM

The figure 3.1 shows the problem occurred in the adaptive filter

diagram. A input sample x(n) from the digital input is fed to the device,

called an adaptive filter which produces an output signal y(n) based

on the input output relationship which is nothing but the physical

architecture of the filter. Adaptive filters contains some adjustable

parameters and these values decide how y(n) is computed. An error

signal e(n) is generated by taking the difference between the desired

signal d(n) and the actual obtained output signal.

30

e(n) = d(n) - y(n) (3.1)

By using the error signal, we will produce adapts or alters that

are used to update the parameters of the filter from n time to (n+1)

time. As the time index is increased from n, it is hoping that the

output of the filter is better matched with the desired signal by the

adaption process. So that the error between the two signals is

decreased over the time.

Figure 3.1 Block Diagram of Adaptive Filter

3.3 FILTER STRUCTURES

In adaptive filters y(n) is computed from the input x(n) will be

affected by number of parameters. These parameters are defined as

coefficient vector w(n) (this is also called as the weight vector).

W(n) = [w0(n), w1(n),……., wN-1(n)] (3.2)

31

Where {wi(n)}, 0 ≤ I ≤ L-1 contains the L parameters of the

system at n time, by this definition, general I/O relation of the

adaptive filter is as follows:

y(n) = f(w(n), y(n-1), y(n-2),…….y(n-N), x(n), x(n-1),…….,x(n-M+1)) (3.3)

Where the equation 3.3 represents linear or non linear function

with M and N are positive integers. Implicitly this definition realizes a

causal filter, in which there is no necessity for the future values of x(n)

to compute the output y(n).

The general equation of the adaptive filter is given by the above

equation. In this structure main intention is to make the linear

relationship between the input and desired signal. This type of

relationship is obtained by finite impulse response (FIR) filter

structures or infinite impulse-response (IIR) filter. The direct form FIR

filters also known as transversal filter or a tapped-delay-line filter.

3.3.1 Finite Impulse Response Filter

A finite impulse response (FIR) filter is used for the

implementation of any kind of frequency response by digital means.

Usually, a series of multipliers, delays and adders are used for the

implementation of FIR filters to obtain the filter output.

The simple block diagram of an FIR filter of length N is shown in

figure.3.2., where the delay results because of the operations on prior

32

input samples. The wi values are the coefficients used for

multiplication, so that the output at time n is the summation of all the

delayed samples multiplied by the appropriate coefficients.

The general equation that defines the output of a normal FIR

filter in terms of its input vector is given by:

y[n] = w0 x[n] + w1 x[n-1] +.......+ wNx[n-N] (3.4)

● x[n] is the input signal,

● y[n] is the obtained output signal,

● wi are the filter weights and

● N is the order of the filter.

Generally, an Nth-order filter will have (N + 1) terms on the right-hand

side which are commonly referred as taps.

The above equation 3.4 can also be expressed as a convolution

result of the coefficient sequence wi with given the input vector:

y n = x n-i

L-1

i=0

wi n

= WT(n) X(n) (3.5)

33

That is, the filter output is a weighted sum of the current and a

finite number of previous values of the input.

Figure: 3.2 Structure of an FIR filter

The process of selecting the filter's length and coefficients is

called filter design. The aim is to set those parameters properly such

that certain desired stop band and pass band parameters will result

from running the filter.

The direct form configuration of the FIR filter contains a long

critical path. Why, Because of the inner product present in the filter

output path. The construction of the multiplier needs more hardware

and area occupied by this is more. In order to increase the throughput

of the filter it is necessary to reduce the critical path of the filter.

When the input sampling rate is high by making the critical path less

than the sampling period we can increase the throughput of the filter.

34

3.3.2 Infinite Impulse Response Filter

The direct form IIR filter structure is shown below. The output

of the this filter can be represented as

y n = y n-i ai

N

i=1

n + x n-j bj n

N

j=0

(3.6)

The block diagram will not be represented in this form. That can

be represented in vector form as

y(n) = WT(n) U(n)

W(n) = [ a1(n), a2(n) …… aN(n) b0(n), b1(n),…….bN(N) ]T

U(n) = [ y(n-1), y(n-2) …. y(n-N) x(n), x(n-1) …… x(n-N) ]T

To compute the output y(n) of the IIR filter involves fixed

number of adders, multipliers and delay elements unlike from the

direct form FIR structure.

This structure of the IIR filter is called as the lattice filter, which

is the third order structure of the IIR filter. This is a FIR filter

structure which contains L-1 preprocessing stages used to compute

the set of auxiliary values known as backward prediction error values

represented as {bi(n)}, 0≤ I ≤L-1. These are uncorrelated signals, which

represent the parameters of x(n) through the linear transformation.

So, the backward prediction error values are used in place of the

35

delayed inputs in a structure similar to that of FIR filter. The

convergence performance is improved because of the uncorrelated

nature of the prediction errors to the adaptive filter coefficients, when

the algorithm selection is appropriate.

Figure: 3.3 Structure of the IIR filter

The computational complexity is the sensitive issue regarding

the selection of the adaptive filter structure. In the real time the

operations of the adaptive filter typically occur. All the calculations of

the filter occur during in the one sample period tine. The structures

discussed above are useful, because the output signal y(n) can be

computed by using the simple arithmetic operations and finite

amount of memory.

When the parameter values are fixed, then we can use nonlinear

systems in addition to the linear systems discussed above for which

36

the principle of the superposition does not hold. Such systems may be

useful even when the relationship between the d(n) and x(n) is not

linear. In the field of neural networks many nonlinear models are

developed.

3.3.3 Properties of FIR filter

The reason for our interest on adaptive FIR filter structure is

because of the properties discussed below. These lines prove the

advantage of FIR filter compared to the IIR filter in the adaptive

filtering design. An FIR filter has a number of useful properties which

sometimes make it preferable to an infinite impulse response (IIR)

which are discussed below. FIR filters:

● Are stable by nature. Because, all the poles are located at the

origin and hence are located within the unit circle, which is the

reason for their inherent stability.

● Require no feedback. This is nothing but, any rounding errors

are not compounded by summed iteration and the same relative

error will occur in each calculation which makes its

implementation simpler.

● Can be designed to be linear in phase. Linearity in phase is

nothing but the phase change will be proportional to the

frequency which is usually a desired characteristic for phase-

37

sensitive applications such as crossover filters, and mastering,

where transparent filtering is adequate.

The main drawback of FIR filters is that it requires considerably

more computation power compared to a similar IIR filter. This is

especially true in the case of low frequency applications where the low

frequencies are affected by the filter.

3.4 TASK OF AN ADAPTIVE FILTER

While discussing the adaptive filters, the common doubt that

raises in most of the minds is nothing but ”If we had already known

the desired signal, then what is the need for finding out the output

signal that matches exactly with the desired signal?” The process of

equating the actual output y(n) to desired output d(n) with some

system yields the adaptive filtering task. By considering the following

issues relate to many adaptive filtering problems.

Generally, the point of interest in all the cases may not be the

same. i.e., we may not be interested in the desired signal always

and our interest may be to represent output signal y(n) in terms

of d(n) that is contains the input signal x(n). In some cases we

may be interested in the parameter values W(n) exclusively.

In some situations where d(n) is not available at all times. When

d(n) is unavailable, we will try to estimate the desired response

38

signal d(n), we will use our most-recent error based parameter

to compute y(n).

There may be some cases in which d(n) is never available. In

such cases, one can use a “hypothetical” d(n), based on its

amplitude characteristics and its predicted statistical behavior.

There is a method called blind adaptation algorithms in which

suitable estimates of d(n) are formed from the signals available

to the adaptive filter. Such algorithms even work which could

be considered as a tribute both to the developers who initiated

the algorithms and to the technological maturity that has

improved in the field of the adaptive filtering.

In general, the relationship between x(n) and d(n) will vary from

time to time. In those situations, with the help of adaptive filtering

algorithms, we will try to change or update the parameter values to

follow the changes in the relationship as “encoded” by the two

sequences of the signal. This behavior is commonly referred as

tracking.

3.5 INTRODUCTION TO LMS ALGORITHM

In high performance systems such as digital signal processors

(DSP), microprocessors, FIR filters etc. uses barrel shifters,

multipliers, multiplexers, delay elements as key components.

Performance and sequentially of the multiplier determines the

system’s performance. Multipliers are very slowest and most area

39

consuming elements in the system. Hence, optimization of speed and

area are major constrains in the design of multiplier. But these two

are conflicting constrains so improving in speed mostly increases the

area.

Adaptive digital filters are used widely in many digital signal

processing (DSP) applications like signal de-noising, sonar signal

processing, system identification, channel equalization for

communications, clutter rejection in radars, acoustic echo

cancellation and networking systems interference etc., because of

their linearity and adaptability to the changes in the signals they

process. In order to satisfy this requirement, we can make use of a

finite-impulse-response (FIR) filter because of its stable nature whose

coefficients are updated by the well known Widrow-Hoff least mean

square (LMS) algorithm, which is well known for its simplicity and

better convergence performance.

In direct form FIR filters an inner product computation present

in the forward path gives long critical path to obtain a filter output.

Therefore, it’s necessary to reduce the critical path if the input signal

has a high sampling rate by this critical path never exceed the

sampling period.

In most of the cases, we need a high performance adaptive filter

in order to reduce the system clock rate which eventually results in

low power consumption because the sampling frequencies for digital

40

processing of these signals are close to the system clock frequencies

[27].

Distributed arithmetic (DA) is one of the efficient techniques for

realization of higher order filters as it can achieve high throughput

without the help of hardware multiplier. It was first introduced by

Croisier and later developed by Peled and Liu [31].

3.6 OVERVIEW OF ADAPTIVE LMS ALGORITHM

This section gives a brief review of the LMS adaptive filter

algorithm and followed by the description of the proposing DA-based

technique for adaptive filter [7]. The least mean square (LMS)

algorithm computes an error and a filter output in each and every

cycle which is equal to the difference between the desired response

and the current filter output. The filter weights are updated in every

training cycle, for this the estimated error value is used. In the LMS

adaptive filters the weights of the nth iteration are updated by the

following equations:

w(n+1) = w(n) + µ.e(n).x(n) (3.7)

Where

e(n)=d(n)-y(n)

y(n)=wqT(n).x(n)

x(n) is the input vector and w(n) is weight vector at the nth

training iteration they are given by as follows:

41

x(n)=[x(n),x(n-1),………, x(n-N+1)]T

w(n)=[w0(n), w1(n),……., wN-1(n)]

d(n) represents the desired response, N is the filter length

y(n) represents the filter output at the nth iteration

e(n) is the error computed at the nth iteration, and this error signal

e(n) is used to update the weights, μ is the convergence factor

Generally, the pipelined architectures uses the delayed error

signal e(n-m) for updating the current weight instead of using the

recent error value. The feedback error signal e(n) may be available

after certain number of clock cycles, which is called the “adaptation

delay”, here m is considered as adaptation delay. The weight-update

equation of the pipelined LMS adaptive filter is given as follows:

w(n+1) = w(n) + µ.e(n-m).x(n-m) (3.8)

3.7 DA-BASED APPROACH FOR INNER-PRODUCT COMPUTATION

DA is basically a bit-serial computational operation that

performs an inner product computation. The advantage of DA is its

efficiency of mechanization. The LMS adaptive filter, needs to perform

an inner product computation in each cycle, the most of the critical

path is contributed by this. DA inner product generation considers the

calculation of the following sum of products:

y = wkxk

N-1

k=0

(3.9)

42

Figure: 3.4 Multiply and Accumulation Unit

Figure 3.4 shows the normal multiply and accumulation

diagram without using the distributed arithmetic concept. Where A, B,

C, D are considered as inputs and C0, C1, C2, C3 are considered as

weights.

In the equation wk denotes the weights of the filter and xk

denotes the inputs of the filter. If we Consider bit width of the weight

is L, each component in the weight vector may be represented by two’s

complementation form, such that │wk│<1, then we may express wk as

wk=-wk+ 𝑤𝑘𝑙2-l

L-1

l=1

(3.10)

Where the wkn are the bits, 0 or 1, wk0 is the sign bit, and wk,N-1

is the least significant bit (LSB). Now by combining the equations 3.9

and 3.10 the expression y in terms of the bits of wkl are shown below:

𝑦 = 𝑥𝑘 −𝑤𝑘0 + 𝑤𝑘𝑖 2−𝑙

𝐿−1

𝑙=1

𝑁−1

𝑘=0

(3.11)

43

Equation 3.11 is the conventional form of expressing the inner

product. Direct form of mechanization of this equation defines a

“lumped” arithmetic computation. Let us instead interchange the

order of the summations, which gives us:

y=- xk .

N-1

k=0

wk0 + xk

N-1

k=0

. wkl

N-1

l=1

2-l (3.12)

Figure: 3.5 MAC circuit for DA

Figure: 3.6 ROM construction in the conventional four point inner

product block

44

After the bit level rearrangement of the multiply and

accumulation the circuit design is modified as above. Equation 3.12

defines a distributed arithmetic computation. Consider the bracketed

term in the above equation:

yl = xk .

N-1

k=0

wkl (3.13)

Because each wkn may take on the values of 0 and 1 only, we

may pre-compute the values and store in a ROM. The input data can

be used to directly access the address of the memory and result can

be dropped into accumulator. After N such cycles, the memory will

contain the result. In the diagram where the ROM portion comes is

shown in figure 3.7.

Figure: 3.7 Conventional DA Based Four Point Inner Product Block