Wnn Slides

7/28/2019 Wnn Slides

1/60


2/60

Wavelet Network

Wavelet + Neural Network

Wavelet decomposition as universal

approximation- like neural network

An alternative to neural network- having poor

convergence

Wavelet Transform as a neural network


3/60

Fourier Analysis-

Which breaks down a signal into constituent sinusoids of

different frequencies

A mathematical technique for transforming our view of the

signal from time-based to frequency-based

in transforming to the frequency domain, time

information is lost.

From Fourier transform of a signal-impossible to tellwhen a particular event took place !


4/60

Signals contain different non-stationary or transitory

characteristics: abrupt changes-often the most

important part of the signal

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

Time (s)

V

-And Fourier analysis is not suitable


5/60

Wavelet Analysis-

Wavelet- a small wave on the surface of a liquid

Performs local analysis

--to analyze a localized area of a larger signal.

-capable of revealing aspects like trends, breakdown points,

discontinuities in higher derivatives of data set


6/60

A wavelet -- a waveform of effectively limited duration that

has an average value of zero.

Sinusoids do not have limited duration they extend from

minus to plus infinity. Sinusoids are smooth and predictable,

wavelets tend to be irregular and asymmetric.

Wavelet Analysis-


7/60

Fourier analysis - sine waves of various frequencies.

Wavelet analysis --breaks a signal into shifted and scaled

versions of the original (ormother) wavelet

just as some foods are better handled with a fork than a spoon

It also makes sense that local features can be described better

with wavelets that have local extent


8/60

Mathematically, the process of Fourier analysis is represented by theFourier transform:

( ) ( )j t

F f t e dt

=


9/60

= ''2'' )]()([),( dtettgtxftSTFT ftjx

Gabors : Short Time Fourier Transform


10/60


11/60

Fourier Wavelet


12/60


13/60

Continuous Wavelet Transform (CWT)

Multiplying each coefficient by the appropriately scaled and shifted wavelet

yields the constituent wavelets of the original signal:

( ) ( )C scale,position f t (scale,position,t)dt

=


14/60

Scaling a wavelet -stretching (or compressing) it.

Scaling


15/60

Shifting means---delaying (or hastening) its onset.

delaying a function f(t) by k ; f(t-k)

Shifting


16/60

Take a wavelet and compare it to a section at the start of the original signal.For a C value (scale)

Shift the wavelet to the right and repeat steps 1 until youve coveredthe whole signal.

Scale (stretch) the wavelet and repeat above steps

Repeat all above steps for all scales.

Processing


17/60

the CWT can operate at every scale, from that of the original signal

up to some maximum scale that you determine by trading off your

need for detailed analysis with available computational capability

The CWT is also continuous in terms of shifting: during

computation, the analyzing wavelet is shifted smoothly over the full

domain of the analyzed

function.

Continuous Wavelet Transform?


18/60

Calculation at a subset of scales and positions at which to

make our calculations.

if we choose scales and positions based on power of two

so-called dyadic scales and positions then our

analysis will be much more efficient and just as accurate.

Discrete Wavelet Transform


19/60

For many signals, the low-frequency content is the most

important part. It is what gives the signal its identity. The high-

frequency content, on the other hand, imparts flavor.

Human voice- remove the high-frequency components, the voice

sounds different, but we can still tell whats being said.

remove enough of the low-frequency components, we hear

gibberish.

In wavelet analysis, we often speak of:

approximations :the high-scale, low-frequency components of thesignal

details are the low-scale, high-frequency components.

Approximations and Details


20/60

The filtering process, at its most basic level, looks like this:


21/60

By looking carefully at the computation, we may keep only one

point out of two in each of the two 2000-length samples to get the

complete information. This is the notion ofdownsampling.


22/60

Multiple-Level Decomposition


23/60

Assembled back into the original signal without loss of

information. This process is called reconstruction, or

synthesis.

The mathematical manipulation that effects

synthesis is called the inverse discrete wavelet transform

(IDWT).


24/60

Reconstruction


25/60

S=A1+D1

=A2+D2+D1

=A3+D3+D2+D1

Reconstructed

Signal

Components


26/60

Daubechies family of wavelets (dbN)

Db1 is the Haar wavelet

Haar wavelet


27/60

Morletwavelet no scaling function


28/60

cA3 cD3

cD2 cD1

Signal Analysissignal


29/60

Signal Reconstruction


30/60

X= the input vector, T= the translation parameter, d= the dilation parameter,

= the wavelet function, w= weight, = the addition parameter in the

network to take care nonzero mean function

( ) ( )2

2 21X

X X e

=

( )X d X T =

translated by a vector T and then dilated by d (scaling)

Wavelet Network

=Mexican Hat

-TJ

d1 w1

d2 w2

dJ wJ

-T1

-T2X y

+

+

+

+ +


31/60

x2

(dh, Th)

w1

v1

w2

wh

1

2

h

x1

v2

vd

Hidden layer

Linear output

xn

neuron

Input

(d1, T1)

+1

( )1

output vectorinput vector

the output layer weight vector

the wavelet function

the translation vector

the dialation vectorthe bias at the output

hn

j j jj

w d

=

= + + T

y x t v x

y

x

w

t

d


32/60

The updating equation is,

n n-1 n nV V K e= +

where the parameter vector V for single output case consists of

[ ]1 J 1 J 1 JV ,w .......w ,T ...T ,d ...d

=

and n stands for iteration index.

The Kalman gain is obtained by1T

n n 1 n n n n 1 nK P h R h P h

= +

The covariance has the relation,

Tn n n n 1P I K h I QI = +

and the Jecobian hn

is given by,

1n n

n

V V

yhV

==

[ ]1 1 1 1 11, ... , ... , ...n J J J J Jh wF w F wG w F =

Kalman Filtering based WN training


33/60

Wavelet Network for Signal Processing Applications

(i) channel equalization and (ii) system identification

Wavelet decomposition as universal approximation- like

neural networkAn alternative to neural network- having poor

convergence


34/60

Channel Equalizer

Trainingalgorithm

Z-N

Input Output

e

_+

Noise

Basic channel equalization scheme

_-

++

Channel equalization--

Telephone linesMicrowave links

Satellite channels

Underwater acoustic channels

Intersymbol interference


35/60

(i) A linear channel of the type

y (n) = 0.2602 x (n) + 0.9298 x (n-1) + 0.2602 x (n-2)

(ii) A non-linear channel

y (n) = x(n) + 0.1x2 (n) + 0.05 x3 (n)


36/60

2 2

y(n) y(n-1)[y(n) 2.5]y((n 1) u(n)

1 y ( ) y ( -1)n n

+ ++ = +

+ +

y((n 1) (y(n),y(n-1),u(n))f+ =

with u(n) = sin (2n/25)

A nonlinear dynamic plant described as


37/60

Mackey-Glass system described by

)-(n(y1

)-y(nby(n)a)-(11)y(n10 +

+=+

Where a = 0.1, b= 0.2, = 17 and y(0) = 12.The identification considers the models as:

y (n+6) = (y (n), (n-6), (n-12), (n-18))where is an unknown nonlinear function

The networks are trained with 1000 sets.


38/60

Convergence characteristic of MLANN and WN based equalizers for linear channel at(a) 20dB and (b) 30dB NSR

0 200 400 600 800 1000

-35

-30

-25

-20

-15

-10

-5

0

MLANN

WN

iteration

MSEindB

0 200 400 600 800 1000-40

-35

-30

-25

-20

-15

-10

-5

0

MLANN

WN

iteration

MSEindB

(a) (b)


39/60

Bit error rate characteristic of the MLANNand WN based channel equalizer for linear

channel

0 2 4 6 8 10

MLANN

WN

SNR in dB

BitErrorRate

10-2

10-3

10-1

100


40/60

Convergence characteristic of the MLANN and WN based channelequalizer for non-linear channel at (a) 20 and (b) 30 dB NSR

0 200 400 600 800 1000-40

-35

-30

-25

-20

-15

-10

-5

0

MLANN

WN

iteration

MSEindB

0 200 400 600 800 1000-40

-35

-30

-25

-20

-15

-10

-5

0

MLANN

WN

iteration

MSEindB


41/60

0 2 4 6 8 10

100

Bit error rate characteristic ofthe MLANN and WN based channel equalizer for non-

linear channel

MLANN

WN

SNR in dB

BitErrorRate

10-1

10-2


42/60

0 20 0 4 00 60 0 8 00 1 0 000

0 .2

0 .4

0 .6

0 .8

1

1 .2

1 .4

Test result [original () and predicted( ) data]


43/60

Comparison of the system responseand the network output for MLP


44/60

Comparison of the system response and thenetwork output for the WN


45/60

Error square of the WN for the system


46/60

Convergence characteristic of the wavelet for the system


47/60

Wavelet Network forPower Disturbance Classification

Power Quality (PQ)?

Electrical Disturbances

Shunt Capacitor switching

Wh PQb i t t?


48/60

computerized equipments -highly sensitive

Semiconductor industry Drive systems or robots

Programmable logic controllers

Deregulation of power industry

Why PQ becomes important?


49/60

Impact to Silicon Valley

One cycle interruption makes a silicon deviceworthless

Five minutes shut down of a chip fabricationplant causes delay from a day to a week

One second of power outage makes e-commerce sites lose millions of dollars worthof business

Why PQ becomes important?


50/60

Power Quality Problems Different PQ problems

Need of Diagnosis Remedy

Classification of disturbances- essential


51/60

Several typical PQ disturbances

Voltage SagVoltage Swell

Impulse

Oscillatory Transient Interruption

Notch

Voltage Sag


52/60

g g

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

Voltage Swell

Impulse


53/60

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

Oscillatory Transient

Interruption


54/60

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

Notch

Interruption


55/60

Conventional Disturbance Classifier

Wavelet Decomposition

Feature Extraction

Neural Network as Classifier

Signal

T


56/60

-TJ

d1 w1

d2 w2

dJ wJ

-T1

-T2X y

+

+

+

+ +

-TJ

d1 w1

d2 w2

dJ wJ

-T1

-T2X y

+

+

+

+ +

-TJ

d1 w1

d2 w2

dJ wJ

-T1

-T2X y

+

+

+

+ +

Inputfeatu

revector

C2

C6

C1


57/60

50 Hz System

1-cycle data-input vector (60 samples)Training sets 15 only for each module

Testing sets 25 for each moduleStructure 60x12x1- for module-1

Network Design

Testresults:


58/60

97.0%Total

38/40Notch

39/40Oscillatory Transient

39/40Impulse

40/40Outage

39/40Voltage swell

38/40Voltage sag

Classification rate ofProposed method

Different power qualitydisturbance

Test results:

Classification rate for 6-class PQ disturbances


59/60

Solution to PQ classification problem

9 Wavelet network

9 EKF training

9 Classification accuracy

9 On-line application


60/60

RBF network the basis functions are not orthogonal

Which implies that the RBF NN representation is not

unique

In WN- the basis function can be non radial-symmetric

Wnn Slides

Documents