Wavelets & Wavelet Algorithms: Harmonic Function Form, Orthogonal Functions, Fourier Coefficients & Series

Wavelets & Wavelet Algorithms

Vladimir Kulyukin

www.vkedco.blogspot.comwww.vkedco.blogspot.com

Harmonic Function Form, Orthogonal Functions, Fourier Coefficients & Series

Outline

● Review● Harmonic Function Form● Trigonometric Polynomials & Series● Orthogonal Functions● Function Multiplication & Definite Integrals● Computation of Definite Integrals● Fourier Coefficients & Series

Review

Longitudinal Waves

● Ideal longitudinal waves can be viewed as a time series of medium compression (peaks) and decompression (valleys)

● Such series are mathematically represented with sinusoids

Fourier's Discovery

Complex waves can be effectively decomposed into simple waves

Jean-Baptiste Joseph Fourier (1768 - 1830)

Steps of Fourier's Analysis: Step 01: Take Complex Wave

Steps of Fourier's Analysis: Step 02: Decompose Wave into Its Constituents

Steps of Fourier's Analysis: Step 03: Compute Frequency Spectrum

Definition

Reference: J. O. Smith III, Mathematics of the Discrete Fourier Transform with Audio Applications, 2nd Edition (https://ccrma.stanford.edu/~jos/st/).

Rotational Velocity & Period

versa. viceand position, same reach the point to theit takes

longer theorigin, thearound rotatespoint aslower thes,other wordIn

seconds.in of valuelonger the the,rad/secin of valueesmaller th The P

Rotational Velocity & Frequency

versa. viceand time,of units 2every hasit nsoscillatiofewer

theorigin, thearound rotatespoint aslower thes,other wordIn

.frequency esmaller th the, velocity rotational esmaller th The

f

Obtaining Sinusoids from y(t)=sin(t)

.by each valuemultiply 4)

; 1by axis- thealong curve shift the 3)

axis);-( axis- thealong xpandcompress/e 2)

; 2 as of period thecompute 1)

:follows as

sin from onbtained becan sin sinusoidAny

A

t

xt

ty

ttxtAty

Function Synthesis & Analysis● Like numbers, new functions can be constructed

(synthesized) from existing functions via addition, subtraction, multiplication, and division

● All these function operations are pointwise: the values of functions at specific points are added, subtracted, multiplied, or divided (division by 0 is still not allowed!)

● To analyze a complex function is to obtain the list of functions and function operations through which the complex function was synthesized

Curve Synthesis Example

+

ttx 5sin1

ttx 4sin2

ttx 3sin3

ttttx 3sin34sin25sin

Harmonic Function Form

Review: Sine of Angle Sum

sincoscossinsin

Common Harmonic Function Form

a=y

b=x

A=r

Source: http://en.wikipedia.org/wiki/Sine_wave#/media/File:ComplexSinInATimeAxe.gif

;tan

cos

sintan;cossin1

;coscos and sinsin;cos;sin

1222

2222

b

a

b

abaA

A

ba

rAbrAar

x

A

b

r

y

A

a

Common Harmonic Function Form

.sincosThen .cos and sinLet

.sincoscossin

sincoscossinsin

:have weformula, Sum Angle of Sine theUsing

.sinLet

tbtatxAbAa

tAtA

ttAtA

tAtx

Common Harmonic Function Form

.tansin Thus,

. and tan:follows as and get can we, and

given are weSince .sin that show toneed We:Proof

.harmonic is sincosfunction Every :Claim

122

221

b

atbatx

baAb

aAba

tAtx

tbtatx

.sincos

sincoscossincossinsincos

sintansin :onVerificati 122

tbta

tAtAttA

tAb

atbatx

Harmonic Form Example 01

.3sin13cos33

3sin2 So,

.12

12

3cos2cos ;3

2

32

3sin2sin

Then .3

,3,2 have We

.sincos i.e., form, harmonicin 3

3sin2 Write

ttttx

AbAa

A

tbtatxttx

Harmonic Form Example 01

labOctave/Matt = 0:0.001:2*pi;figure;plot(t, 2*sin(3*t + pi/3));xlabel('x');ylabel('y');xlim([0 7]);ylim([-3 3]);title('2sin(3t + pi/3)');

3/3sin2 ofGraph ty

Harmonic Form Example 01

labOctave/Matt = 0:0.001:2*pi;sqrt_of_3 = sqrt(3);figure;plot(t, sqrt_of_3*cos(3*t));xlabel('x');ylabel('y');xlim([0 7]);ylim([-3 3]);title('sqrt(3)*cos(3t)');

ty 3cos3 ofGraph

Harmonic Form Example 01

labOctave/Matt = 0:0.001:2*pi;figure;plot(t, sin(3*t));xlabel('x');ylabel('y');xlim([0 7]);ylim([-3 3]);title('sin(3t)');

ty 3sin ofGraph

Harmonic Form Example 01

labOctave/Matt = 0:0.001:2*pi;figure;plot(t, sqrt(3)*cos(3*t) + sin(3*t));xlabel('x');ylabel('y');xlim([0 7]);ylim([-3 3]);title('sqrt(3)*cos(3*t) + sin(3*t)');

Graph

Harmonic Form Example 01

/33t2sin tt 3sin3cos3

Common Harmonic Function Form with Periods

sinusoid. a still is hat remember t

Also, .definition theinto period theintroduces

explicitlyit because convenient more is form This

.sincossincos

.2 with harmonic a be Let

..222

tx

l

tb

l

tatbtatx

lPtxl

lP

Trigonometric Polynomials & Series

Periodic Harmonics

;4

sin4

cos

;3

sin3

cos

;2

sin2

cos

;sincos

:Examples

,...3,2,1 ,sincos

form theof harmonics heConsider t .2Let

444

333

222

111

l

tb

l

tatx

l

tb

l

tatx

l

tb

l

tatx

l

tb

l

tatx

kl

ktb

l

ktatx

lP

kkk

Periodic Harmonics

period. a also is period a of multiple

integralany because, of period a is 2,,2 Since

.2222

why.is Here .2 period a has

,...3,2,1 ,sincos form theof harmonicAny

txlPZkkTl

lkTk

l

lk

Tl

k

lP

kl

ktb

l

ktatx

kk

kk

kk

kkk

Trigonometric Polynomials of Order n

.order of polynomial tric trigonomea is

period. a isnumber

any for which function a isconstant a because , of period a is 2

constant. a is where,sincos

:sum following heConsider t

1

nts

tsl

Al

ktb

l

ktaAts

n

n

n

kkkn

Infinite Trigonometric Series

constant. a is where,sincos1

Al

ktb

l

ktaAts

kkk

Important Question

series?

tric trigonomea of sum theas drepresente beit

Can .2 of period a has that Suppose ltf

Fundamental Question Phrased Differently

motions?y oscillator simple of sum a as

drepresente bemotion y oscillatorcomplex aCan

Reducing 2L to 2Pi

.2 of period a has which ,sincos

sincos

Then .Then .Let

.sincos

Then series. tric trigonomea of sum a

is 2 period of function a that assume a usLet

1

1

1

kkk

kkk

kkk

ktbktaA

tl

l

kb

tl

l

kaA

tlf

tlx

l

xt

l

kxb

l

kxaAxf

lxf

Reducing 2L to 2Pi

.2 of period a has

sincosThen

.2 of period a has if ,Conversely .2 of period

a has sincos

Then series. tric trigonomea of sum the

is 2 period of function a that assume a usLet

1

1

l

l

xkb

l

xkaA

l

xxf

t

ktbktaAtl

ft

lxf

kkk

kkk

A practical implication of this is that if we know how to solve a trigonometric series problem on an interval of length of 2PI, we know how to solve it on an interval of length of 2L, and vice versa

Orthogonal Functions

Why We Need Orthogonality

● Any formal system must have its primitives (e.g., a point in geometry, 0 and 1 in natural number theory, key notes in music)

● What is a primitive? A primitive is something that cannot be expressed through something else

● In 2D geometry, it is impossible to express the x-axis through the y-axis

● In music, it is impossible to express DO through RE

Why We Need Orthogonality

● We need the same kind of conceptual framework of primitives in sinusoid analysis

● As we will soon learn, orthogonal trigonometric functions are the primitives of sinusoid analysis

● To define function orthogonality, we have to review function multiplication & definite integrals

Function Multiplication Example 01

xxy 2 2

Function Multiplication: Example 01

labOctave/Mat

x = -20:.2:20; figure;plot(x, x);xlabel('x');ylabel('y');xlim([-8 8]);ylim([-8 8]);title('y=x');

xy ofGraph

Function Multiplication: Example 01

labOctave/Mat

x = -20:.2:20; figure;plot(x, x.^2-2);xlabel('x');ylabel('y');xlim([-8 8]);ylim([-8 8]);title('y = x^2 - 2');

2 ofGraph 2 xy

Function Multiplication: Example 01

labOctave/Mat

x = -20:.2:20; figure;plot(x, (x.^2-2).*x); xlabel('x');ylabel('y');xlim([-8 8]);ylim([-8 8]);title('y=(x^2-2)*x');

xx 2 ofGraph 2

Function Multiplication: Example 01

*

xxy 22

xy

22 xy

Function Multiplication Example 02

xxy cossin

Function Multiplication: Example 02

labOctave/Mat

t = 0:0.001:2*pi;

figure;plot(t, sin(t));xlabel('x');ylabel('y');title('y=sin(x)');

xy sin ofGraph

t = 0:0.001:2*pi;

figure;plot(t, sin(t));xlabel('x');ylabel('y');title('y=sin(x)');

Function Multiplication: Example 02

labOctave/Mat

t = 0:0.001:2*pi;

figure;plot(t, sin(t));xlabel('x');ylabel('y');title('y=sin(x)');

xy cos ofGraph

t = 0:0.001:2*pi;

figure;plot(t, cos(t));xlabel('x');ylabel('y');title('y=cos(x)');

Function Multiplication: Example 02

xxxxx

xxxxxx

2sin2

1cossincossin2

sincoscossinsin

Function Multiplication: Example 02

labOctave/Mat

t = 0:0.001:2*pi;

figure;plot(t, sin(t));xlabel('x');ylabel('y');title('y=sin(x)');

xxy cossin ofGraph

t = 0:0.001:2*pi;

figure;plot(t, sin(t).*cos(t));xlabel('x');ylabel('y');title('y=sin(x)cos(x)');

Function Multiplication: Example 02

*

xxy cossin xy sin

xy cos

Function Multiplication: Example 02

xxy cossin xy 2sin5.0

Definite Integrals

Integration Example 01

?22

1

2 xdxx

Integration Example 01

.4

3

4

12153

4

15

2

1

2

42

4

1

4

16

22

42

222

:math thedo usLet

2

1

22

1

42

1

2

1

3

2

1

2

1

32

1

32

1

2

xxxdxdxx

xdxdxxdxxxxdxx

Integration Example 02

?

2

2sin

cossin

2

0

2

0

dxx

dxxx

Integration: Example 02

.0114

10cos4cos

4

1

2cos4

12sin

2

1

2

2sin

:math thedo usLet

20

2

0

2

0

xdxxdxx

Definition of Function Orthogonality

b

a

dxxgxfxgxf 0 if orthogonal are , Functions

Orthogonality of SIN(X) & COS(X)

0cossinbecause ,orthogonal are cos,sin2

0

dxxxxx

lar.perpendicuremain always velocity rotational same the

with circle thearound rotating ,cos and sin arrows, Two xx

Orthogonality of SIN(X) & COS(X)

xsin

xcos

Review: Products of Sines & Cosines

coscos2

1sinsin

coscos2

1coscos

Examples of Orthogonal Functions

Function Orthogonality Example 01

?2sinsin2

0

dxxx

2

Function Orthogonality Example 01

0. is lakes, blue theminus hillsgreen thearea, combined theSo, lakes.

blue two theof area the toequal is hillsgreen two theof area The

22

Function Orthogonality Example 01

.00sin6sin6

10sin2sin

2

1

3sin6

1sin

2

13cos

2

1cos

2

13coscos

2

1

2cos2cos2

12sinsin

:math thedo usLet

20

20

2

0

2

0

2

0

2

0

2

0

xxdxxdxxdxxx

dxxxxxdxxx

Function Orthogonality Example 02

?2coscos2

0

dxxx

2

Function Orthogonality Example 03

0. is lakes, blue theminus hillsgreen thearea, combined theSo, lakes.

blue three theof area the toequal is hillsgreen three theof area The

2

Function Orthogonality Example 03

.00sin2sin2

10sin6sin

6

1

sin2

13sin

6

1cos

2

13cos

2

1cos3cos

2

1

2cos2cos2

12coscos

:math thedo usLet

20

20

2

0

2

0

2

0

2

0

2

0

xxdxxdxxdxxx

dxxxxxdxxx

Function Orthogonality Example 04

?sin2

0

2 dxx

2

Function Orthogonality Example 04

. is area e that whitus sgraph tell the theSo equal. are area

green theand area whiteThe .21 is rectangle theof area The

2

Function Orthogonality Example 04

.0sin4sin4

102

2

1

2sin4

1

2

12cos

2

11

2

12cos1

2

1

coscos2

1sinsinsin

:math thedo usLet

20

20

2

0

2

0

2

0

2

0

2

0

2

0

2

xxdxxdxdxx

dxxxxxdxxxdxx

Function Orthogonality Example 05

?cos2

0

2 dxx

2

Function Orthogonality Example 05

. is hillsgreen theof area that the

us sgraph tell the theSo hills.green two theof area the toequal is

lakes blue two theof area The .21 is rectangle theof area The

2

Function Orthogonality Example 05

.022

100

4

1

2

12sin

4

10cos2cos

2

1

coscos2

1coscoscos

:math thedo usLet

20

20

2

0

2

0

2

0

2

0

2

xxdxx

dxxxxxdxxxdxx

Function Orthogonality Results

.cos

.sin

.02coscos

.02sinsin

.0cossin

:far so achieved have weresultsn integratio theare Below

2

0

2

2

0

2

2

0

2

0

2

0

dxx

dxx

dxxx

dxxx

dxxx

Computing Definite Integrals

Motivation

● Integrating functions by hand is fun but a) error-prone and b) difficult (unless you are a math major :-))

● However, which is great for CS majors, integration of many sinusoids and many other useful functions can be approximated with summations, i.e., for-loops

● Computing summations makes doing integrals by hand less important than it used to be

Back to Integration Example 02

0cossin2

0

dxxx

t = 0:0.001:2*pi;figure;plot(t, sin(t).*cos(t));xlabel('x');ylabel('y');xlim([0 7]);ylim([-2 2]);title('sin(x)cos(x)');sum01 = sum(sin(t).*cos(t));display(strcat('SUM01 = sin(x)cos(x) = ', num2str(sum01)));

Output: SUM01 = sin(x)cos(x) on [0, 2pi] =-7.5484e-05

Back to Integration Example 02

?cossin:, to2,0 from interval thechange weifWhat

dxxx

t = -pi:0.001:pi;figure;plot(t, sin(t).*cos(t));xlabel('x');ylabel('y');xlim([0 7]);ylim([-2 2]);title('sin(x)cos(x)');sum01 = sum(sin(t).*cos(t));display(strcat('SUM01 = sin(x)cos(x) = ', num2str(sum01)));

Output: SUM01 = sin(x)cos(x) on [-pi, pi] =-7.5484e-05

Back to Integration Example 02

?3cos2sin:periodsmodify totscoefficien add weifWhat 2

0

dxxx

t = 0:0.001:2*pi;figure;plot(t, sin(2*t).*cos(3*t));xlabel('x');ylabel('y');xlim([0 7]);ylim([-2 2]);title('sin(x)cos(x)');sum01 = sum(sin(2*t).*cos(3*t));display(strcat('SUM01 = sin(x)cos(x) = ', num2str(sum01)));

Output: SUM01 = sin(2x)cos(3x) on [0,2pi] =-0.00015097

Back to Integration Example 03

t = 0:0.001:2*pi;figure;plot(t, sin(1*t).*sin(2*t));xlabel('x');ylabel('y');xlim([0 7]);ylim([-4 4]);title('sin(ax)sin(bx)');sum01 = sum(sin(1*t).*sin(2*t));display(strcat('SUM01 = sin(ax)sin(bx) on [0, 2pi] = ', num2str(sum01)));

Output: SUM01 = sin(ax)sin(bx) on [0, 2pi] =-3.9592e-06

02sinsin2

0

dxxx

Basic Trigonometric System

Definition

?orthogonal pairwise they are

s,other wordIn ?primitives asfunction functions Can these

.2 of periodcommon thehave functions theseAll

,...sin,cos,...,2sin,2cos,sin,cos,1

functions ofset infinite theis system tric trigonomebasic The

nxnxxxxx

Integration Formulas

0

cos1sin

0sin

cos

, real,0integer any For

22

22

a

a

a

a

a

a

a

a

n

nxdxnx

n

nxdxnx

an

Integration Formulas

222

2 22

2

2cos1sin

2

2cos1cos

, real,0integer any For

a

a

a

a

a

a

a

a

dxnx

dxnx

dxnx

dxnx

an

Integration Formulas

22

22

22

0sinsin2

1cossin

0coscos2

1sinsin

0coscos2

1coscos

, real,, integersany For

a

a

a

a

a

a

a

a

a

a

a

a

dxxmnxmndxmxnx

dxxmnxmndxmxnx

dxxmnxmndxmxnx

amn

Orthogonality of Basic Trigonometric System

.2,over 0 is

system tric trigonomebasic theof functionsdifferent two

any of integral that theshow formulasn integratio The

,...sin,cos,...,2sin,2cos,sin,cos,1

functions ofset infinite theis system tric trigonomebasic The

aa

nxnxxxxx

Fourier Series

Trigonometric Series for Functions of Period 2PI

integrals. theof sum the toequal is sum theof integral thei.e.,

by term, termintegrable is series theand integrable be toassumed is

where,sincos2

:expansion following thehas and 2 period offunction a is Suppose

1

0

xf

kxbkxaa

xf

xf

kkk

Integration of Trigonometric Series

.22

0012

sincos12

:,over integrate usLet

.sincos2

:by term termintegrableexpansion series rictrigonomet

has and 2 period offunction integrablean is Let

000

1

0

1

0

1

0

aa

xa

badxa

dxkxbdxkxadxa

dxxf

xf

kxbkxaa

xf

xf

kkk

kkk

kkk

Computing Coefficients

integrals. theof sum the toequal is sum theof integral thei.e.,

by term, termintegrable is series theand integrable be toassumed is

where,sincos2

:expansion following thehas and 2 period offunction a is Suppose

1

0

xf

kxbkxaa

xf

xf

kkk

? from and tscoefficien thecompute topossibleit Is xfba kk

Computing Cosine Coefficients

.coscoscos

cossincoscoscos2

cos

:,over integrate and cosby sidesboth multiply usLet

.sincos2

:by term termintegrable

expansion series tric trigonomea has and 2 period offunction integrablean is Let

2

1

0

1

0

nnn

kkk

kkk

adxnxadxnxnxa

dxnxkxbdxnxkxadxnxa

dxnxxf

nx

kxbkxaa

xf

xf

,...3,2,1,cos

1ndxnxxfan

Computing Sine Coefficients

.sinsinsin

sinsinsincossin2

sin

:,over integrate and sinby sidesboth multiply usLet

.sincos2

:by term termintegrableexpansion

series tric trigonomea has and 2 period offunction integrablean is Let

2

1

0

1

0

nnn

kkk

kkk

bdxnxbdxnxnxb

dxnxkxbdxnxkxadxnxa

dxnxxf

nx

kxbkxaa

xf

xf

,...3,2,1,sin

1ndxnxxfbn

Fourier Coefficients

,...3,2,1,sin

1ndxnxxfbn

,...3,2,1,cos

1ndxnxxfan

Fourier Series

. of seriesFourier thecalled

is sincos2

series tric trigonomeThe

,...3,2,1,sin1

,cos1

where,sincos2

:expansion series

tric trigonomefollowing thehas and 2 period offunction a is If

1

0

1

0

xf

kxbkxaa

nnxxfbnxxfa

kxbkxaa

xf

xf

kkk

nn

kkk

References● J. O. Smith III, Mathematics of the Discrete Fourier Tranform with

Audio Applications, 2nd Edition.

● G. P. Tolstov. Fourier Series.