Dissertation New

8/11/2019 Dissertation New

1/97

ADAPTIVE FOURIER ANALYSIS FOR UNEQUALLY-

SPACED TIME SERIES DATA

By

Hong Liang

Dissertation submitted to the faculty of the

Virginia Polytechnic Institute and State University

in partial fulfillment of the requirements for the degree of

Doctor of Philosophy

in

Statistics

Robert V. Foutz, Chair

Marion R. Reynolds, Jr.

Donald R. Jensen

George R. Terrell

Christine Anderson-Cook

April 16, 2002Blacksburg, Virginia

Keywords: Adaptive Fourier Analysis, Unequally-Spaced, Time Series, Walsh-Fourier,Wavelet


2/97

Adaptive Fourier Analysis For Unequally-Spaced Time SeriesData

by

Hong Liang

Robert V. Foutz, Chairman

Statistics

(ABSTRACT)

Fourier analysis, Walsh-Fourier analysis, and wavelet analysis have often been used in

time series analysis. Fourier analysis can be used to detect periodic components that have

sinusoidal shape; however, it might be misleading when the periodic components are not

sinusoidal. Walsh-Fourier analysis is suitable for revealing the rectangular trends of time

series. The flaw of the Walsh-Fourier analysis is that Walsh functions are not periodic.

The resulting Walsh-Fourier analysis is more difficult to interpret than classical Fourier

analysis. Wavelet analysis is very useful in analyzing and describing time series with

gradual frequency changes. Wavelet analysis also has a shortcoming by giving no exact

meaning to the concept of frequency because wavelets are not periodic functions. In addi-

tion, all three analysis methods above require equally-spaced time series observations.

In this dissertation, by using a sequence of periodic step functions, a new analysis

method, adaptive Fourier analysis, and its theory are developed. These can be applied to

time series data where patterns may take general periodic shapes that include sinusoids as

special cases. Most importantly, the resulting adaptive Fourier analysis does not require

equally-spaced time series observations.


3/97

iii

Acknowledgements

I would like to sincerely express my appreciation to Dr. Robert V. Foutz, my aca-

demic advisor, who introduced me to time series analysis. His excellent guidance, cour-

ses, support, wisdom, patience, and enthusiasm were essential throughout my academic

career at Virginia Tech.

I would like to thank all of my committee members: Dr. Reynolds, Dr. Jenson,

Dr. Terrell, and Dr. Anderson-Cook. I want to thank Dr. Reynolds for his wonderful

courses helping me intuitively understand applied statistics and academic research me-

thods. I would like to send thanks to Dr. Jensen for his careful reading of my dissertation.

To Dr. Terrell, I thank him for his comments and suggestions. Lastly, I would also like to

thank Dr. Anderson-Cook for her many suggestions of modifications in my dissertation.

I want to give my thanks to all faculties at Virginia Tech in the Statistics Depart-

ment, Graduate School, and International Student Center. In the Statistics Department, I

would especially like to thank Dr. Good who offered me a reference paper for my disser-

tation. In Graduate School, I would especially like to thank Dr. McKeon, for his outstand-

ing service for international students.

Finally, I would like to thank my wife, Haiming and my son, Sixing for their love

and support.


4/97

iv

Table of Contents

1 Introduction 1

2 Review of data analysis methods 4

2.1 Fourier Analysis . 4

2.2 Walsh-Fourier Analysis 7

2.3 Wavelet Analysis . 12

2.4 The spectral analysis methods for unequally-spaced time series data 17

2.4.1 Interpolation methods ..... 17

2.4.2 Least-Squares methods 19

2.4.3 Linear algebra method and the others .. 21

3 Adaptive Fourier Analysis 23

3.1 Motivation 23

3.2 The periodic step functions 24

3.3 The properties of the periodic step functions 27

3.4 Multiresolution analysis 28

3.5 The adaptive Fourier analysis of the digital time series .. 29

3.6 Methodology .. 32

4

Examples 36

Example 4.1 .. 36

Example 4.2 . 45

Example 4.3 . 49


5/97

v

Example 4.4 . 52

Example 4.5 54

5 Conclusion and future research 58

5.1 Summary and conclusion . 58

5.2 Proposed future research .. 59

Bibliography 61

Appendix 64

Appendix 1. Proof of Theory 3.1 and its Corollary 3.1 .. 64

Appendix 2. The construction of x e (t) . 73

Appendix 3. Proof of x(t) =n

limm

lim s mn, (t) 75

Appendix 4. A nonstandard inner product < y, z> 4 77

Appendix 5. Proof of (3.12) . 77

Appendix 6. Background .. 79

A6.1 Vector space . 79

A6.2 Inner product space, Hilbert space and their properties 82

A6.3 Orthogonal projection and orthonormal bases 87

Vita 90


6/97

vi

List of Figures

Figure 2.1 The first eight Walsh functions . 10

Figure 2.2 Some basis functions for Fourier transform .. 11

Figure 2.3 Four different mother wavelets ..... 13

Figure 2.4 The signal ss = 10*cos(pi*t/15)+3*cos(pi*t/10) ... 15

Figure 2.5 Decomposition of SS into the sum of 16 Wavelet functions . 15

Figure 2.6 Doppler(t) =((t*(t-1))**0.5)*sin(2.1*pi/(t+0.5)) 16

Figure 3.1 The periodic step function in the complex plan 25

Figure 3.2 The step function f )(,6,2 tj ..... 26

Figure 4.1 Monthly accidental deaths in the U.S.A., 1973-1978 .. 36

Figure 4.2 Frequency component, k=6 in Ex. 4.1 ... 38

Figure 4.3 Adaptive Fourier line spectrum for Ex. 4.1 ... 39

Figure 4.4 Fourier line spectrum in Ex. 4.1 39

Figure 4.5 Monthly accidental deaths without Mar. in Ex. 4.1 40

Figure 4.6 Frequency component, k=6 in Ex.4.1 without Mar. ... 40

Figure 4.7 Monthly accidental deaths without Jan., , Jun., respectively .. 42

Figure 4.8 Frequency component, k=6, without Jan., , Jun., respectively ... 42

Figure 4.9 Monthly accidental deaths by deleting randomly 10 points, first time 43

Figure 4.10 Frequency component, k=6, deleting randomly 10 points, first time... 43

Figure 4.11 Monthly accidental deaths by deleting randomly to points, second time..44


7/97

vii

Figure 4.12 Frequency component, k=6, deleting randomly 10 points, second time.. 44

Figure 4.2(1) Original time series in Ex.4.2 . 46

Figure 4.2(2) k = 2 freq. Component in Adaptive Fourier analysis 46

Figure 4.2(3) k = 2 freq. component(-) in Fourier analysis & original signal(.) 47

Figure 4.2(4) Multiresolution decomposition of the signal 48

Figure 4.2(5) Time-scale plot for the signal .. 48

Figure 4.3(1) Original time series in Ex.4.3 .50

Figure 4.3(2) Reconstruction of the time series in Ex.4.3 .50

Figure 4.3(3) Adaptive Fourier ANOVA in Ex.4.3 ...51

Figure 4.3(4) Fourier ANOVA in Ex.4.3 .. 51

Figure 4.3(5) Frequency component k=3 in Ex.4.3 51


Figure 4.4(1) Original time series in Ex.4.4 ...53

Figure 4.4(2) Reconstruction of the time series in Ex.4.4 .. 53

Figure 4.4(3) Adaptive Fourier line spectrum in Ex.4.4 53



Figure 4.5(1) Original time series in Ex.4.5 56

Figure 4.5(2) Reconstruction of the time series in Ex.4.5 ... 56

Figure 4.5(3) Adaptive Fourier line spectrum in Ex.4.5 . 57

Figure 4.5(4) Fourier line spectrum in Ex.4.5. 57

Figure 4.5(5) Frequency component k=2 in Ex.4.5.57

Figure 4.5(6) Frequency component k=1 in Ex.4.5 . 57


8/97

1

1 Introduction

In the statistical analysis of time series, Fourier analysis has provided a general method for

discovering or analyzing the periodicity and examining the global energy-frequency dis-

tribution in time series data. It has been a valuable and powerful tool of time series analy-

sis, but it still has some crucial limitations : the data must consist of equally-spaced obser-

vations; and periodic components in the data must be sinusoidal . Otherwise, the analysis

might give misleading results.

For non-sinusoidal waveforms, such as square-wave and rectangular waveform, Fourier

analysis needs too many additional harmonic components to decompose approximately

the waveform functions, and therefore it spreads the energy over a wide frequency range,

which might cause Fourier ANOVA to be misunderstood. Beauchamp (1975,1984) empiri-

cally demonstrated that when a time series is based upon a sinusoidal waveform, Fourier

analysis is more efficient, and when a time series is rectangular or with sharp discontinui-

ties, Walsh-Fourier analysis is more efficient. However, the Walsh functions are not perio-

dic, and therefore the Walsh-Fourier decomposition is more difficult to interpret than the

Fourier harmonic decomposition.

In actual time series analysis, given a sequence of N values x =(x1, x2, , xN), by using the

discrete Fourier Transform (DFT), x can be evaluated only at a special set of N/2 evenly

spaced frequencies . When x comes from a non-sinusoidal function containing only less

than N/2 frequencies, DFT may cause energy spreading.


9/97

2

For the time series of unequally-spaced data, Fourier analysis, the Discrete Fourier Trans-

form, cant be used directly. A variety of methods have been suggested to overcome this

limitation of the DFT due to unequally-spaced data. Among them are the interpolation me-

thod, the Least-squares method, and the algebra method. But these methods have some

shortcomings, and they also suffer the shortcoming of Fourier spectral analysis , since

they are all Fourier based.

In wavelet analysis, a given function x(t) is decomposed into a sum of wavelet functions.

These wavelet functions are derived from a single mother wavelet function (t) by apply-

ing varying translations and dilations to (t). Given a mother wavelet function (t), for

real a>0, and real b, a sequence of wavelet functions can be derived as

ba, (t) = a21 (

abt ),

where arepresents the scale parameter and brepresents the translation parameter. In a

loose sense, we may identify the wavelet parametersa, b

with frequency and time ,

respectively. See Priestley(1996). Wavelet analysis is very useful in analyzing and describ-

ing time series data with gradual frequency changes. However, wavelets are not periodic

functions, and the concepts of frequency and of periodicity have no precise meaning in the

resulting wavelet analysis. In addition, different mother wavelet may have different analy-

sis results.

In this research, we develop a new analysis method, adaptive Fourier analysis and its

theory. The advantage of adaptive Fourier analysis and its theory is that it can be applied


10/97

3

to time series data where patterns may take general periodic shapes that include sinusoids

as special cases. Most importantly, the resulting adaptive Fourier analysis does not require

equally-spaced time series observations.

Chapter 2 reviews the present available time series data analysis methods. In Chapter 3,

adaptive Fourier analysis and the definition of ADFT are presented. Chapter 4 gives some

examples. Chapter 5 makes conclusions and suggestions for future research. Proofs are

contained in the Appendix1-5. Appendix6 provides the background for theoretical results

and their proofs in this research. All conceptions and their properties used in Chapter3 and

Appendix1-5, such as vector space, direct sum, inner product, Hilbert space, weight inner

product, orthogonal projection, and Gram-Schmidt procedure, are presented in Appendix6.


11/97

4

2. Review of data analysis methods

2.1 Fourier Analysis

2.1.1 Fourier Series

The basic idea of a Fourier series is that any function x(t) L2[0,T] can be decomposed

into an infinite sum of cosine and sine functions:

x(t)= ]2

sin2

cos[0 T

ktb

T

kta

k

kk

=

+ , for all t (2.1)

where ak= T

txT 0

)(1

cosT

kt2dt , (2.2)

bk= T

txT

0

)(1

sinT

kt2dt. (2.3)

This is due to the fact that {1, cosT

kt2, sin

T

kt2, k=1, 2, 3, ...} form a basis for the space

L

2

[0,T]. The summation in (2.1) is up to infinity, but x(t) can be well approximated in

the L2sense by a finite sum with K cosine and sine functions:

X(t) ]2

sin2

cos[0 T

ktb

T

kta

K

k

kk

=

+ . (2.4)

This decomposition shows that x(t) can be approximated by a sum of sinusoidal shapes at

frequencies k = 2k/T, k = 0,1, , K. In addition, the variability inx(t) as measured by

dttx

T

0

2|)(| can be approximately partitioned into the sum of the variability of the sinusoi-

dal shapes:


12/97

5

dttxT

0

2|)(| = T

0

[ ]2

sin2

cos[0 T

ktb

T

kta

K

k

kk

=

+ 2dt 2

0

||=

K

k

k . (2.5)

A standard technique of time series analysis is to treat the partition (2.5) as an analysis of

variance (ANOVA) for identifying sinusoidal periodicities in a time series data set {x(t),

0< t T}. When x(t) has sharp discontinuities or a non-sinusoidal waveform, such as a

rectangular waveform, then we would require a very large number, K, of terms in its

Fourier series in order to get an adequate approximation.

2.1.2 Discrete Fourier Transform (DFT)

For an arbitrary time series data set,x = (x(t 1 ),x(t 2 ), ,x(tN ) ), if the observation times

are equally spaced, at interval t, then the data set x can be simply written asx= (x(1),

x(2), ,x(N) ) by taking t =1 and t j =j; and there is an orthogonal system { ekitw : - 2

N +

1 k 2N

if n is even and - 21N

k 21N

if n is odd }, so that the Discrete Fourier

Transform ofxcan be defined by

X*(k) =

N

1

=

N

t

itwketx1

)( , (2.6)

where the frequencies wk=2k/N, k=0,1, , 2/N , are called the Fourier frequencies;

and r is the largest integer no larger than r. The Fourier series ofx(t) can be written as

x(t) =

=

2/

2/)1(

* )(N

Nk

kX e kitw . (2.7)

Corresponding to (2.5), we have the ANOVA


13/97

6

NtxN

t

==

)(1

2

=

2/

2/)1(

2* |)(|N

Nk

kX . (2.8)

This representation provides an ANOVA for revealing how well the periodicities in x

may be described by the sinusoidal shapes X*(-k)e kitw + X*(k)e kitw . The ANOVA

decomposition in (2.8) holds only if the DFT, X* (t), is evaluated only at a fixed set of

N/2 equally-spaced frequencies wk, and the data set must be equally-spaced.

2.1.3. Spectrum ANOVA for Equally-Spaced Time Series Data

Let L 2be the set of all continuous-time, complex-valued functions. L 2 ={y(t), 0 t N}

for which the Lebesque integral N

ty0

2|)(| dt is finite. The set L 2 is a Hilbert space with

inner product

1 =N

1

N

tzty0

)()( dt, (2.9)

where )(tz is the complex conjugate of z(t). Let A k be the subspace of L2 that contains

all periodic functions that have the form c*exp(i k t) for complex-valued scalars c and for

k = Nk2 . Also, let r be the largest integer no larger than r. Classical analysis shows

that corresponding to every equally-spaced digital times series data set x = (x(1), x(2),,

x(N) ), there is a continuous-time function s in the N-dimensional subspace,

L NA, = A ][2

1 N+ + A

][2

N , (2.10)


14/97

7

with the property that s(t) = x(t) at t = 1, 2, , N. See Koopmans ( 1974, P. 17). The

function is s = P 2 ( x | L NA, ), the projection of x onto L NA, with respect to the inner

product

2 =N

1

=

N

t

tzty1

)(*)( . (2.11)

Because the subspaces A k in (2.10) are orthogonal with respect to the inner product < y,

z> 2 , the function s = P 2 ( x | L NA, ), has the Fourier series representation,

s(t) =

)|(

2/

2/)1(2

=

N

Nk

kAxP (t), (2.12)

where P 2 (x | A k ) is the projection of x onto A k with respect to < y, z > 2 . The corres-

ponding partition of |s| 22 = 2 is

|s| 22 =

2

2/

2/)1(2 |)|(|

=

N

Nk

kAxP . (2.13)

In the Fourier series (2.12), each frequency component P 2 ( x | A k ) + P 2 ( x | A k ) is a

sinusoid with frequency k . The corresponding partition (2.13) provides an ANOVA for

revealing how well the periodicities in s ( and in x ) may be described by the sinusoidal

shapes P 2 ( x | A k ) + P 2 ( x | A k ).

2.2 Walsh-Fourier Analysis

Fourier series can be used to partition functions into sinusoidal waves. Unlike Fourier

analysis, Walsh-Fourier analysis deals with the decomposition of functions into the


15/97

8

Walsh functions, which are rectangular waves. These waves have been used in signal

transmission, speech analysis and pattern recognition, etc. Walsh-Fourier analysis is

suited to the analysis of discrete valued and categorical valued time series. See Stoffer

(1991).

The Walsh functions are defined as products of the Rademacher functions and the Rade-

macher functions are defined as below.

Definition ( Rademacher functions) Consider the function defined on the half open

interval [0,1) by

0 (t) =

.)1,2/1[,1

)2/1,0[,1

tfor

tfor (2.14)

Extend it to the real line by periodicity of period 1 and set k(t) = 0 (2kt) for k = 0,1,

and real t. The functions k(t) are called the Rademacher functions. The Walsh system

{ W[0, t], W[1, t], W[2, t], }is obtained by taking all possible products of Rademacher

functions , where W[0, t] is defined as W[0, t] =1 and the other W[n, t] , n 1 is defined

by

W[n, t] = =

k

i

iit

0

))(( , (2.15)

where n= =

k

i

i

i

0

2 , k=1 and i=0 or 1 for i= 0, 1, ,k-1.

There are a number of definitions of the Walsh functions. Another definition of the

Walsh functions { W[0, t], W[1, t], W[2, t], }is by induction as follows:


16/97

9

Initialize the induction by defining

W[0, t] = 1, t [0, 1),

W[1, t] =

).1,2/1[,1

)2/1,0[,1

t

t (2.16)

Then proceed recursively for n =1, 2, , through

W[2n, t] =


17/97

10

Figure 2.1 The first eight Walsh functions

0 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 0 . 8 0 . 9 1

-1

0

1

W[0,t

]

0 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 0 . 8 0 . 9 1

-1

0

1

W[2,t

]

0 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 0 . 8 0 . 9 1

-1

0

1

W[4,t

]

0 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 0 . 8 0 . 9 1

-1

0

1

W[6,t

]

0 0 .1 0 .2 0 .3 0 .4 0 .5 0 .6 0 .7 0 .8 0 .9 1

-1

0

1

W[1,t

]

0 0 .1 0 .2 0 .3 0 .4 0 .5 0 .6 0 .7 0 .8 0 .9 1

-1

0

1

W[3,t]

0 0 .1 0 .2 0 .3 0 .4 0 .5 0 .6 0 .7 0 .8 0 .9 1

-1

0

1

W[5,t

]

0 0 .1 0 .2 0 .3 0 .4 0 .5 0 .6 0 .7 0 .8 0 .9 1

-1

0

1

W[7,t

]


18/97

11

-4 -3 -2 -1 0 1 2 3 4-1

0

1

sin(t)

Figure 2.2 Som e basis functions for Fourier transform

-4 -3 -2 -1 0 1 2 3 4-1

0

1

sin(2

t)

-4 -3 -2 -1 0 1 2 3 4-1

0

1

sin(3t)

-4 -3 -2 -1 0 1 2 3 4-1

0

1

cos(t)

-4 -3 -2 -1 0 1 2 3 4-1

0

1

c

os(2t)

-4 -3 -2 -1 0 1 2 3 4-1

0

1

cos(3t)

The Walsh functions, W[n, t] , n1, form a complete orthogonal sequence on the interval

[ 0, 1 ) and they have square wave shapes in which each Walsh function W[ n, t ] takes on

only the value 1 and +1. For any functionx(t) which has period 1 and is Lebesgue inte-

grable on [0,1), it can be decomposed into an infinite sum of the Walsh functions.

x(t)

=0i

aiW[i, t], (2.19)

where ai = 1

0

x(t) W[i, t] dt , i=0,1,2, (2.20)

For equally spaced time series data x= (x(0),x(1), ,x(N-1)) and N is a power of 2, the


19/97


20/97

13

Another commonly used wavelet is Morlet wavelet defined as

(t) =e2t cos( t 2ln/2 ) e

2t cos(2.885 t). (2.24)

Four different mother wavelets: Haar, Daublet, Symmlet, and Coiflet are shown in Figure

2.3, where the first letter of the wavelet indicates the name: d for Daublet, s for Symmlet,

and c for Coiflet; the number of the wavelet indicates its width and smoothness. See

Bruce and Gao (1996, p.8).

Figure 2.3. Four different mother wavelets

`haar' mother, psi(0,0)

0.0 0.2 0.4 0.6 0.8 1.0

-1.0

-0.5

0.0

0.5

1.0

`d4' mother, psi(0,0)

-1.0 -0.5 0.0 0.5 1.0 1.5 2.0

-1.0

0.0

1.0

`s12' mother, psi(0,0)

-4 -2 0 2 4 6

-1.0

-0.5

0.0

0.5

1.0

1.5

`c12' mother, psi(0,0)

-4 -2 0 2 4 6

-0.5

0.0

0.5

1.0

1.5

Given a mother wavelet (t), an infinite sequence of wavelets can be constructed by

varying translations band dilations aas below

a,b(t) = |a|-1/2

( abt

). (2.25)

By defining the continuous wavelet transform W(a,b) as

W(a,b) = < x(t), a,b(t) > =

x(t)a,b(t)dt , (2.26)


21/97

14

we can represent x(t) as

x(t) =1

1

C

0

a-2W(a,b)a,b(t)dadb (2.27)

where C1 = dt

2|)(| and ()=

(t)e-itdt .

When aand btake on discrete sets of values, we can similarly obtain the discrete wavelet

transform as

W(m,n) = =

x(t)m,n(t)dt, (2.28)

and x(t) =

=m

=n

Wm,nm,n(t). (2.29)

For an equally spaced time series data x =(x(1), x(2), , x(N) ), we can take approximate

wavelet transforms by replacing the (2.28) by an estimate such as:

W(m,n) =

x(t)m,n(t)dt

=

N

l

lx1

)( m,n( l ). (2.30)

It follows that a class of discrete wavelet transform (DWT) for equally spaced time series

data can be implemented by using an efficient computational algorithm. See Bruce and

Gao (1996, p37-39).

An example of wavelet approximation is given in Figure 2.5. In the example, the signal

is ss =10*cos(*t/15)+3*cos(*t/10), which is plotted in Figure 2.4. Its wavelet appro-

ximation is plotted in Figure 2.5.


22/97

15

Figure 2.4 The signal ss=10*cos(*t/15)+3*cos(*t/10)

Index

ss

0 10 20 30 40 50 60

-10

-5

0

5

10

Figure 2.5 Decomposition of ss into the sum of 16 wavelet functions

D3.3

D3.5

D1.1

D3.4

D3.1

D2.1

D3.8

S5.2

D3.6

D3.2

D5.2

D4.3

D4.2

D5.1

D4.1

D4.4

Approx

0 10 20 30 40 50 60

Another example is a wavelet decomposition for the Doppler signal as shown in Figure

2.6. See Bruce and Gao (p.28). In Figure 2.6, the high frequency oscillations at the begin-

ning of the signal are captured mainly by the fine scale detail components D1 and D2,

while the lower frequency oscillations are captured mainly by the coarse scale compon-

ents D6 and S6.


23/97

16

Wavelet analysis is very powerful and efficient in the analysis of data or functions, x(t)

with gradual frequency changes. However, wavelets are not periodic functions. For

example, the Morlet wavelet is Fourier based but its oscillations are dampened by the

exponential factor e2t . In addition, the concepts of frequency and periodicity have no

precise meaning in wavelet analysis. See Priestley (1996).

Figure 2.6 Doppler(t)= ((t*(t-1))**0.5)*sin(2.1* /(t+0.05))

S6

D6

D5

D4

D3

D2

D1

Data

0 200 400 600 800 1000


24/97


25/97

18

(2.31) can be expressed as a matrix form by

X= Dx , (2.32)

where

X=

)(

)(

)(

1

1

0

NzX

zX

zX

, x =

]1[

]1[

]0[

Nx

x

x

, (2.33)

and

D =

)1(1

21

11

)1(1

21

11

)1(0

20

10

...1

...1

...1

N

NNN

N

N

zzz

zzz

zzz

. (2.34)

If the N sampling points, z 0 , z 1 , , z 1N are distinct, then D is nonsingular, and thus the

inverse of NDFT can be determined by

x = D 1 X. (2.35)

If the N sampling points, z 0 , z 1 , , z 1N are equally-spaced angles on the unit circle in

the z-plane, then ( 2.35 ) corresponds to the classical DFT. X(z) can be expressed as the

Lagrange polynomial of order N-1,

X(z) = ],[

)(

)(1

0k

N

k kk

k zX

zL

zL

=

(2.36)

whereL 0 (z),L 1 (z), ,L 1N (z) are the fundamental polynomials, defined by

L k (z) =

ki

izz ),1(1 k = 0, 1, , N-1. (2.37)

X(z) can also be expressed as the Newton interpolation,


26/97

19

X(z) =c 0 +c1 (1-z 0 z1 )+c 2 (1-z 0 z

1 )(1-z 1 z1 )+ +c 1N

=

2

0

1 )1(N

k

kzz , (2.38)

where c 0 =X[z 0 ],

c1 = 110

01

z-1

][

z

czX,

c 2 =)1)(1(

)1(][1

211

20

120102

zzzz

zzcczX,

(2.39)

Note that in (2.39), each jc depends only onX[z 0 ] ,X[z1 ], ,X[z j ] and z 0 , z1 , , z j .

While interpolation methods may be satisfactory in some applications, they all produce

some distortion and loss of information, see Scargle ( 1989 ), and they may cause some

distortion in the spectrum, especially for the data with high frequency components. In

addition, these interpolation methods cant yield an orthogonal and additive spectrum

ANOVA decomposition for the original time series data.

2.4.2 Least-Squares Methods

The least squares method is used on Fourier expansion or the inverse transform to find

the period which minimizes the unexplained variance of the series. This method is very

similar to multivariate regression analysis when multiple periods are present. In the least-

squares sense, a periodogram analysis is based on the trigonometric regression model

x f (t) ==

K

k

PkA

1

( cos(2 t/P k ) + B kP sin(2 t/P k )) + , (2.40)


27/97

20

where K is the smallest integer less than or equal to (N-1)/2 and P k = N/k, k = 1, 2, ..., K.

To minimize the mean square difference between (2.40) and the data, one seeks to mini-

mize =

N

j 1 [x(t j )-x f (t j )]

2

, where t 1 , ..., tN are not necessarily equally-spaced, and t N =

N. Thus AkPand B

kPmay be determined by standard linear least-squares techniques.

Some other least-squares methods can be found. The classical DFT power spectrum,

periodogram is defined by (See Brockwell and Davis 1993, p332)

I(k) =N

1| kit

N

t

teX

=

1

|2

=N

1[ 2

1

)sin( tX t

N

t

t =

+ 2

1

)cos( tX t

N

t

t =

]. (2.41)

For unequally-spaced data, t1, t2, , tN are not equally-spaced points. Scargle (1982, 1989),

and Lomb(1976) defined a modified periodogram by

I*() =2

1{

=

=

N

j

j

N

j

jj

t

tX

1

2

1

2

)(cos

)](cos[

+

=

=

N

j

j

N

j

jj

t

tX

1

2

1

2

)(sin

)](sin[

}, (2.42)

where is defined by

tan(2) = j

N

j

t=1

2sin / j

N

j

t=1

2cos . (2.43)

They showed that their modified periodogram and least squares fitting of sinusoidal

waves to the data are exactly equivalent. This method has been called as Lomb-Scargle

method , which has recently been used in biomedical sciences. See Schimmel ( 2001 ),

Van Olofsen, VanHartevelt, and Kruyt (1999), Ruf(1999), Schluz and Stattegger (1997).


28/97

21

The least-squares method can detect some frequency components in relatively simple

sinusoidal spectral situations, but for complicated spectra, it encounters more difficulties.

See Swan ( 1982 ). When time series data contain fractions of non-Gaussian noise or con-

sist of periodic signals with non-sinusoidal patterns, Lomb-Scargle method makes more

difficult the interpretation of analysis results, and can lead to misleading estimates of fre-

quency components. See Schimmel ( 2001 ).

2.4.3 Linear Algebra Method

From the classical DFT definition , X

*

(k), the Fourier transform of X(t) is defined by

X*(k) = =

N

t

NktietX

1

/2)( , (2.44)

where k = -(N-1)/2, , (N-1)/2, if N is odd ,

k = -N/2, , N/2, if N is even.

(2.44) can be written as in matrix form by

*X = WNX (2.45)

and X =W1

N

*X . (2.46)

For unequally spaced data uX , Kuhn (1982 ) and Swan (1982 ) defined the DFT expan-

sion similar to (2.46) by

uX = W u1

N

*X , (2.47)

where W u1

N is a matrix function of the unequally spaced data uX and*X is the DFT of

certain unknown equally spaced data X .Substituting (2.45) into (2.46) gives

uX = W u1

NWNX . (2.48)


29/97

22

Solving (2.48) gives X , and doing DFT on X gives*X , then , Kuhn(1982) and Paul R.

Swan ( 1982 ) used *X to do spectrum analysis of the data. However, the method is

limited in that the deterministic component of noisy signals must be band limited to less

than the usual Nyquist limit. See Swan (1982).

In addition, the other methods include the string length methods, phase dispersion

minimization method, and the CLEAN algorithm; see Rao, Priestley, and Lessi (1997,

p275-286 ). Regarding the CLEAN algorithm, also see Baisch and Bokelmann. ( 1999).

Other discussions and methods for unequally-spaced time series are in Barthes ( 1995 ),

Engle and Russell (1998), and Good and Doog (1958).

The methods above in Section 2.4.1 through 2.4.3 are Fourier based; they also suffer the

shortcoming of Fourier spectral analysis, i.e., they are not efficient for non-sinusoidal

waveforms, and they might lead to misunderstandings of the spectral ANOVA .


30/97

23

3. Adaptive Discrete Fourier Analysis

3.1 Motivation

The purpose of the research is to generalize the Fourier analysis of the digital data x. The

generalization begins by replacing each space A k in the representation of L NA, in ( 2.10)

with a larger space B k ; and replacing an equally-spaced digital time series data set x =

(x(1),, x(N)) in (2.10) with an arbitrary digital time series data set x =(x(t 1 ), , x(tN)),

where t 1 , , tN are not necessarily equally-spaced and tN= N. While each function s in

A k has real and imaginary parts that are sinusoids, the real and imaginary parts of func-

tions in B k have more general periodic shapes. Like sinusoids, these shapes satisfy (3.1)

below.

s(t) = - s(t-kN2 ). (3.1)

This means that the second half of the periodic shape of s(t) for kN2 t kN is the reflec-

tion of its first half about the origin. For the general shapes in B k , the first half of the

periodic shape, s(t) , can be any shape at all, provided k

N

ts2

0

2|)(| dt . This genera-

lization of Fourier analysis, called adaptive discrete Fourier analysis, is accomplished

via the periodic step functions f jnk ,, defined by Foutz and Lee ( 2000 ) instead of sinusoid

functions and via their properties as follows.


31/97

24

3.2 The periodic step functions f jnk ,,

When k > 0, the periodic step function sin( 2 t2 ) jumps to 0, 1, and 1 at the times t

where sin( t )=0, 1, and 1 respectively. It follows that the complex valued step function

exp(i2

t2) = cos(

2

t2) + i*sin(

2

t2) (3.2)

jumps to 1, i, -1 and i with frequency , See Figure 3.1 below. The function exp( i2

+ u

t

2) is a version of (3.2) that is shifted backwards by

2u time periods. For k > 0

and k = Nk2 , let particular time-shifted versions of (3.2) be denoted by

f jnk ,, (t) = exp(i2

+

+ 1

122

n

jtk

),

f jnk ,, (t) = exp(i2

+

n

jtk 122

),

f jn,,0 (t) = 1, for k = 0; where j = 1, 2, ,n. (3.3)

The functions f jnk ,, (t) and f jnk ,, (t) are complex conjugates. The appearance of some of

these functions is shown in Figure 3.2.


32/97

25

Figure 3.1. The periodic step function f jnk ,, in the complexplan

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

The periodic step function in the complex plane


33/97

26

Figure 3.2. the step function f j,6,2 (t)

-3 -2 -1 0 1 2 3

-1

0

1

the s tep func t ion w i th k=2 , n= 6

j=3,im

ag

-3 -2 -1 0 1 2 3

-1

0

1

j=6,real

-3 -2 -1 0 1 2 3

-1

0

1

j=6,imag

-3 -2 -1 0 1 2 3

-1

0

1

the step function with k=2, n= 6

j=2,r

eal

-3 -2 -1 0 1 2 3

-1

0

1

j=2,imag

-3 -2 -1 0 1 2 3

-1

0

1

j=3,real


34/97


35/97

28

The proof of the Theorem 4.1 is given in Appendix 1. This theorem is the first and most

important result in this research. This is due to the fact that the following Corollary 3.1,

and the property of s mn, are based on this theorem, and the adaptive Fourier analysis of x

proceeds by investigating the properties of s mn, (t).

3.2.5. The decomposition of L 2 .

Corollary 3.1. L 2 has another decomposition as

L 2 = + B 1 + B 0 + B1 + ,

where B k is the subspace of L2 that contains all periodic functions that have the

form s ][k (t), in which s ][k (t) = - s ][k (t-k ), for k 0; and B 0 ={c, c is complex-valued

scale}.

The proof of the Corollary3.1 is also given in Appendix 1.

3.4. Multiresolution analysis

The sequence of spaces { L nKB ,, } ZK represents a ladder of subspaces of increasing

resolution as K increases, and it has the following properties:

1. L nB ,1, L nB ,2, ;

2. ( hZK LnKB ,, )= L nB ,0, = { 1 };

3.n

lim( ZK

L nKB ,, ) = L2 .


36/97


37/97

30

3.5.2. The digital data set, x = (x(t1 ), , x(tN )) is replaced by a continuous-time function

x e (t ) in B 1 +B 0 +B1 . It is required that x e (t j ) = x (t j ), at j = 1, 2,, N.

Theorem 3.2 For digital data set x = ( x(t1

), x(t2

), , x(tN ) ) , where t1

, , tN are not

necessarily equally-spaced and tN = N, there always exists a step function x e (t) L 1,B =

B 1 +B 0 +B1 such that

x e (t j ) = x (t j ) at j = 1, 2,, N.

The proof of this theorem is in Appendix 2.

3.5.3 x e (t ) is projected onto L nKB ,, by using a weight inner product m,3 .

3.5.3.1. Construct a weight inner product

The weight function

w m (t) =

+

+

+

otherwise

tttifm

tttifm

tttifm

m

mNmN

mm

mm

,

,

,

,

1

21

21

21

221

2

21

121

1

(3.6)

steps up to the value m near t = t 1 , , t N and approaches 0 elsewhere. w m (t) approaches

a Dirac -function, (t), as m . It is used to construct the inner product

m,3 =N

1

N

m dttztytw0

* )()()(

with the properties:

mlim

N

1dttztytw

N

m 0

2|)()(|)( =N

1 2

1

|)()(|=

N

j

jj tzty , (3.7)


38/97

31

andm

lim m,3 =N

1

=

N

j

jty1

2 )( , (3.8)

whenever y and z are in L nKB ,, .

3.5.3.2 The projection

The inner product m,3 is used to project x e (t) onto L nKB ,, for any K 1. The projec-

tion, s mn, = P m,3 ( x e | L nKB ,, ), has the property below

x(t) =n

limm

lim s mn, (t) (3.9)

at each t = t1 , , tN . This is proven in Appendix 3. The adaptive Fourier analysis of

x proceeds by investigating the properties of s mn, (t) for increasing n and for large m.

3.5.4 Adaptive Fourier series representation and ANOVA

The spaces B nk, are not orthogonal with respect to the usual inner products < y,z>1 , 2 and < y, z > m,3 . However, the spaces are orthogonal with respect to a nonstandard

inner product 4 that is defined in Appendix 4. It follows that s mn, has an adaptive

Fourier series representation

s mn, (t) = =

K

Kk

mnks ,, (t), (3.10)

where s mnk ,, (t) = P 4 ( s mn, | B nk, ) is the projection of s mn, onto B nk, with respect to 4 . The corresponding adaptive Fourier series representation for x is

x(t) =n

limm

lim =

K

Kk

mnks ,, (t), (3.11)


39/97


40/97


41/97

34

s mn, (t) = =

D

r

rr twd1

)( , (3.15)

where dr= < x e , w r> m,3 . (3.16)

Let a sr, be the element of MD in the rth row and s th column, and let a*,sr be its complex

conjugate. Substituting MD v for w in (3.15) and (3.16) gives

s mn, (t) = = =

D

r

ssrr

r

s

tvad1

,1

)( = = =

D

s

ssrr

D

sr

tvad1

, )( , (3.17)

and dr= mr

s

sesr vxa ,31

*, , > 4 , it follows from ( 3.17 ) that the projection

P 4 ( s mn, | B nk, ) is given by

s mnk ,, (t) = P 4 ( s mn, | B nk, ) =

++

++= =

nnKk

nKksssrr

D

rtvad

)(

1)(,

1 )( . (3.19)

Definition. Fix a positive integer n. The adaptive discrete Fourier transform (ADFT ) of

the continuous-time extension x e (t) of the digital time series x is the set of n N scalars

jnk ,, = < x e , f jnk ,, > m,3

for j = 1, 2, , n and K k K.

Property. ADFT jnk ,, = < x e , f jnk ,, > m,3 are relative robust to the construction of x e (t),

they only require that x e (t i ) = x(t i ) for i = 1, 2, ... , N.


42/97


43/97

36

4. Examples

Example 4.1 The time series data set x(1), x(2), , x(72) in Table 4.1 and Figure 4.1 is

taken from Brockwell and Davis (1991, p7), It contains the monthly accidental deaths in

the U.S.A. from January, 1973 through December, 1978.

Table 4.1Monthly Accidental Deaths in the U.S.A., 1973-1978

1973 1974 1975 1976 1977 1978Jan. 9007 7750 8162 7717 7792 7836Feb. 8106 6981 7306 7461 6957 6892Mar. 8928 8038 8124 7776 7726 7791

Apr. 9137 8422 7870 7925 8106 8129May 10017 8714 9387 8634 8890 9115Jun. 10826 9512 9556 8945 9299 9434Jul. 11317 10120 10093 10078 10625 10484Aug. 10744 9823 9620 9179 9302 9827Sep. 9713 8743 8285 8037 8314 9110Oct. 9938 9129 8433 8488 8850 9070Nov. 9161 8710 8160 7874 8265 8633Dec. 8927 8680 8034 8647 8796 9240

0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 06 5 0 0

7 0 0 0

7 5 0 0

8 0 0 0

8 5 0 0

9 0 0 0

9 5 0 0

1 0 0 0 0

1 0 5 0 0

1 1 0 0 0

1 1 5 0 0

F i g u re 4 . 1 M o n t h ly a c c i d e n t a l d e a th s i n U . S . A . , 1 9 7 3 -1 9 7 8


44/97


45/97

38

Table 4.2 *10^5

K Period

kN

ANOVA

|P 2 (x|A k )|22 +|P 2 (x|A k )|

22

Adaptive

|s mnk ,, |24 +|s mnk ,, |

24

k=1 72 1.2080 1.0852

k=2 36 0.1747 0.3414k=3 24 0.3197 0.1907k=4 18 0.1009 0.6844k=5 72/5 0.0492 0.1493k=6 12 5.2262 6.0817

Total 7.0787 8.5327

Figure 4.2 Frequency component with k=6 in Example 4.1


46/97

39

1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 60

2

4

6

8x 10

5 Figure 4.3: Adaptive Fourier line spect rum for Exam ple 4.1

1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 60

2

4

6x 10

5 Figure 4.4 Fourier line spectrum in Example 4.1

(2). To illustrate that adaptive Fourier analysis can be applied to unequally-spaced time

series data, we delete the Marchs records, then the data is reduced to an unequally-spa-

ced time series with 66 sample points, x = ( x(t1 ), x(t 2 ), ..., x(t 66 )). In this case, classical

Fourier ANOVA cant be used. However, adaptive Fourier ANOVA can be used to re-

veal the frequency component shapes and discovering periodicity. Similar to the above,

we pick K = 6, n =2 and m=1000 respectively. The resulting adaptive Fourier ANOVA is

given in Table 4.3, it also reveals that this unequally-spaced time series has the same

major frequency component with k = 6. The time series x = ( x(t1 ), x(t 2 ), ... , x(t 66 ) ) is

plotted on Figure 4.5 and the major frequency component s mn,,6 + s mn,,6 is plotted on

Figure 4.6.


47/97

40

Table 4.3 *10^5

K Period

kN

Adaptive ANOVA

|s mnk ,, |24 +|s mnk ,, |

24

k=1 72 1.1436

k=2 36 0.7401k=3 24 0.1260k=4 18 0.4762k=5 72/5 0.1795k=6 12 6.5532

Total 9.2187

(3). If we take Januarys record out from the first year and Februarys record out from

the second year, , Junes record out from the last year, then the data will be reduced to

an another unequally-spaced time series with 66 sample points, x = ( x( t1 ), x( t 2 ), ...


48/97

41

, x(t 66 ) ). In this case, adaptive Fourier ANOVA can be also used to reveal the frequency

component shapes and discovering periodicity. Similar to the above, we pick K= 6, n = 2

and m =1000 respectively. The resulting Adaptive Fourier ANOVA is given in Table 4.4.

It also reveals that this unequally-spaced time series has the same major frequency com-

ponent with k=6. The time series x = ( x(t 1 ), x(t 2 ), ..., x(t 66 )) is plotted on Figure 4.7 and

the major frequency component s mn,,6 + s mn,,6 is plotted on Figure 4.8.

Table 4.4 *10^5

K Periodk

N Adaptive ANOVA|s mnk ,, |24 +|s mnk ,, |

24

k=1 72 1.1122k=2 36 0.5772k=3 24 0.1494k=4 18 0.5806k=5 72/5 0.1614k=6 12 5.8413

Total 8.4222


49/97

42

(4). If we take randomly 10 records out from the original data set, then the data will be

reduced to an unequally-spaced time series with 62 sample points, x = ( x(t1 ), x(t 2 ), ... ,

x( t 62 ) ). The resulting adaptive Fourier ANOVA is given in Table 4.5, it also reveals

that this unequally-spaced time series has the same major frequency component with k=6.

The time series x = ( x( t1 ), x( t 2 ), ... , x(t 62 ) ) is plotted on Figure 4.9 and the major fre-

quency component s mn,,6 + s mn,,6 is plotted on Figure 4.10.


50/97

43

Table 4.5 *10^5K Period

kN

Adaptive ANOVA

|s mnk ,, |24 +|s mnk ,, |

24

k=1 72 1.3207k=2 36 0.3167

k=3 24 0.1788k=4 18 0.6656k=5 72/5 0.4468k=6 12 6.5826

Total 9.5111

(5). We repeat (4) above again, and obtain similar results shown in Table 4.6 and Figure

4.11 and 4.12.


51/97

44

Table 4.6 *10^5K Period

kN

Adaptive ANOVA

|s mnk ,, |24 +|s mnk ,, |

24

k=1 72 1.1929k=2 36 0.4377

k=3 24 0.1794k=4 18 0.5834k=5 72/5 0.1726k=6 12 6.8078

Total 9.3739

This example implies that when a small fraction of the original data, 10/72 14% in this

example, is removed or missed, the adaptive Fourier analysis can still detect the main

frequency components in the original time series data.


52/97

45

Example 4.2. To illustrate that for some signals with jumps, adaptive Fourier analysis

outperforms Fourier analysis, we take an equally-spaced sample s = (s1, s2, , s192 ) from

the signal s(t) with sample size N =192 to do adaptive Fourier analysis and Fourier analy-

sis. Here s(t) is given by

s(t) =


53/97

46

component with k = 2 and thus in Fourier ANOVA, 20.5% of the sample energy has

leaked onto a wide frequency range, which implies that in this case adaptive Fourier

analysis outperforms Fourier analysis.

Table 4.2(1)

K Period

kN

ANOVA

|P 2 (x|A k )|22 +|P 2 (x|A k )|

22

Adaptive ANOVA

|s mnk ,, |24 +|s mnk ,, |

24

k=1 192 0.0000 0.0008

k=2 96 2.2004 2.7639k=3 62 0.0000 0.0008

Total 2.2004 2.7647


54/97

47

0 1 2 3 4 5 6-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

Figure 4.2(3): k=2 freq. component(-) in Fourier Analysis & original signal(.)

Fourier freq. comp.(k=2)

originial signal

To illustrate that adaptive Fourier analysis is different from wavelet analysis, we apply

wavelet analysis to the sample data s = ( s1, s2, , s192 ) in Figure 4.2(1). The multiresolu-

tion decomposition of the signal is plotted in Figure 4.2(4) and the time-scale plot for the

signal is plotted on Figure 4.2(5). In wavelet analysis, fine scale wavelet functions usu-

all reveal where or when the signal has high frequency components in a loose sense.

However, in this case, all the fine scale wavelet functions focus on the discontinuous

points of the signal shown in Figure 4.2(4) andespecially in Figure 4.2(5). Therefore, in

this case, wavelet analysis cant provide any information about the frequency component.


55/97

48

S5

D5

D4

D3

D2

D1

Data

0 50 100 150

Figure 4.2(4): Multiresolution decomposition of the signal

Figure 4.2(5). Time-scale plot for the signal

Time

1/Scale

0 50 100 150

0.0

0.2

0.4

0.6

0.8

1.0


56/97

49

Example 4.3. To illustrate that for sinusoidal signal, with equally-spaced data, Fourier

analysis and adaptive Fourier analysis are almost the same, we take an equally-spaced

sample s = ( s 1 , s 2 , ..., s 60 ) from the signal s( t ) = 10*cos(2 t/15) + 3cos(2 t/10) over

[1, 30], sample size N = 60. The sample series, s = ( s 1 , s 2 , ... , s 60 ) is plotted on Figure

4.3(1). The Fourier ANOVA in Table 4.3(1) reveals that the sample series contains a

major component k=2 and a minor one k = 3. In adaptive Fourier ANOVA, the frequency

number K = 3, the dimensions n = 5 and m = 500 are used, and the ANOVA result is also

given in Table 4.3(1), again showing that the adaptive frequency component for k = 2,

namely s mn,,2 + s mn,,2 , is the major component ; for k = 3, s mn,,3 + s mn,,3 is the minor

component. The power distribution for the first three frequencies of adaptive Fourier

analysis is plotted on Figure 4.3(3); the one for Fourier Analysis is plotted on Figure

4.3(4); s mn,,3 + s mn,,2 + s mn ,,1 + s mn,,1 + s mn,,2 + s mn ,,3 is plotted on Figure 4.3(2); s mn,,3

+s mn,,3 is plotted on Figure 4.3(5); and s mn,,2 + s mn,,2 is plotted on Figure 4.3(6). Table

4.3(1) shows that Fourier and adaptive Fourier ANOVA are very close; Figure 4.3(5) and

Figure 4.3(6) show that major and minor adaptive frequency components both have a

sinusoidal shape. The results together show that for equally-spaced sampled series s =

(s1 , s 2 , ..., s 60 ) from a sinusoidal signal, Fourier ANOVA and adaptive ANOVA are very

close.


57/97

50

Table 4.3(1)

K period

kN

ANOVA

|P 2 (x|A k )|22 +|P 2 (x|A k )|

22

Adaptive ANOVA

|s mnk ,, |24 +|s mnk ,, |

24

k=1 60 0.162 0.1744k=2 30 47.206 46.8258k=3 20 5.2238 5.2992Total 52.5918 52.2994


58/97

51

1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 30

10

20

30

40

50

Figure 4.3(4): Fourier line s pectrum

1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 30

10

20

30

40

50

Figure 4.3(3): A daptive Fourier line spect rum


59/97

52

Example 4.4. To illustrate that adaptive Fourier analysis can be applied to unequally-

spaced data from a sinusoidal signal, we use the uniform distribution to take a random

sample s = ( s(t 1 ), s(t 2 ), ..., s(t 60 )) from the signal s(t) in Example 4.3 above. The sample

size is N = 60. The sample series s = ( s(t 1 ), s(t 2 ), ... , s(t 60 ) ) is plotted on Figure 4.4(1).

Similarly to Example 4.3, we pick K = 3, n = 7 and m = 500 respectively. The resulting

adaptive Fourier ANOVA is given in Table 4.4(1), and plotted on Figure 4.4(1); s mn,,3 +

s mn,,2 + s mn ,,1 + s mn,,1 + s mn,,2 + s mn,,3 is plotted on Figure 4.4(2); s mn,,2 + s mn,,2 is plotted

on Figure 4.4(3); and s mn,,3 + s mn,,3 is plotted on Figure 4.4(4). Table 4.4(1) shows that

adaptive Fourier analysis can reveal the two frequency components with k = 2 and k = 3.

Figure 4.4(4) and Figure 4.4(5) show that these two frequency components are sinu-

soidal .

Table 4.4(1)

K period

kN

Adaptive ANOVA

|s mnk ,, |24 +|s mnk ,, |

24

k=1 60 0.1624k=2 30 49.4853k=3 20 3.5624Total 53.2102


60/97


61/97

54

Example 4.4 implies that for an unequally-spaced time series where the observation times

follow uniform distribution , the adaptive Fourier analysis can still detect the main fre-

quency components in the original time series data.

Example 4.5. To illustrate that for non-sinusoidal signal, adaptive Fourier analysis out-

performs Fourier analysis, we take an equally-spaced sample s = ( s1 , s 2 , ..., s 60 ) from

the signal s(t) with sample size N = 60 to do adaptive Fourier analysis. The signal s(t) is

given by


62/97

55

s(t) =


63/97

56


64/97

57

1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 20

1

2

3

Figure 4.5(3): A daptive Fourier line spect rum

1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 30

0.5

1

1.5

2

Figure 4.5(4): Fourier line spec trum


65/97

58

5. Conclusion and Future Research

5.1 Summary and Conclusion

The goal of this dissertation is to develop theory and methods that can be applied to

equally and unequally-spaced time series in which the frequency components of time

series may take general periodic shapes that include sinusoids as special cases. The re-

sults of the research are summarized as follows:

(1) Theorem 3.1 Any function s(t) in L 2 that satisfies s(t) = -s(t-k ) is the limit of

step functions s n (t) in B nk, as n , in the L2 sense.

(2) Corollary 3.1. L 2 has another decomposition as

L 2 = + B 1 + B 0 + B1 +

where B k is the subspace of L2 that contains all periodic functions that have

the form s ][k (t), in which s ][k (t) = - s ][k ( t-k ), for k 0; and B 0 = {c, c is

complex-valued scale}.

(3) Through a weight inner product, < y, z > m,3 , a new method of projection has

been developed, which can be applied to project x e (t) onto L nKB ,, as s mn, (t) =

P m,3 ( x e (t) | L nKB ,, ), where x e (t) is the continuous- time extension of equally

or unequally-spaced time series. The adaptive Fourier analysis of time series

x proceeds by investigating the properties of the projection, s mn, (t).


66/97

59

(4) By theorem 3.1 an important property of s mn, (t) can be proved as

x(t) =n

limm

lim s mn, (t),

at each t = t1 , , tN .

(5) A multiresolution analysis (MRA) of L 2 has been presented, which showed

that the step functions used in this research can generate a MRA of L 2 .

(6) Through a nonstandard inner product 4 , adaptive Fourier ANOVA has

been developed, which can be applied to equally and unequally-spaced time

series.

(7) Examples: Example 4.1 gives an application of adaptive Fourier analysis to a

real data set. Example 4.2 illustrates that for some signals with jumps, adap-

tive Fourier analysis outperforms Fourier analysis and is different from wave-

let analysis. Example 4.3 illustrates that adaptive Fourier analysis can be

applied to the time series with general patterns that include sinusoids as

special cases. Example 4.4 illustrates that adaptive Fourier analysis can direct-

ly deal with unequally-spaced time series with sinusoidal frequency compo-

nents. Example 4.5 illustrates that adaptive Fourier analysis outperforms

Fourier analysis for a non-sinusoidal signal.

5.2 Proposed Future Research

(1) A better algorithm is needed for computing the adaptive Fourier trans-

form. Before a better algorithm is developed, use of adaptive Fourier


67/97

60

analysis in analyzing time series with large sample size would be time-

consuming.

(2) To do statistics tests for hidden periodic components, relevant theory

and methods for adaptive Fourier analysis are needed.

(3) The orthogonal series approach to nonparametric regression has be-

come popular lately. People can approximate a function by polynomi-

als, sinusoids, step functions, and wavelets and apply these approxima-

tions to relevant nonparametric regressions. Nonparametric regression

by the step functions used in this research seems promising and needs

to be developed. We expect that this kind of nonparametric regression

would be applied to unequally-spaced data observations and would

have fewer terms in the regression models for some cases, i.e. would

use fewer degrees of freedom.


68/97

61

Bibliography

Anderson, J.A. (1969),Real Analysis. Gordon and Breach Science Publishers, New

York.

Bruce, Andrew and Gao, Hong-Ye (1996),Applied Wavelet Analysis With S-Plus.Springer-Verlag Inc, New York.

Barthes, D. (1995), Cleaning Power Spectra of Short, Unevenly Spaced Time-SeriesApplication to LPVS,Astronomy & Astrophysics Supplement Series. 111(2):373-386 Jun 1995.

Beauchamp, K.G. (1984),Applications of Walsh and Related Functions. London:Academic Press.

Brockwell, P.J. and Davis, R.A. (1990), Time Series: Theory and Methods(SecondEdition). Springer-Verlag Inc, New York.

Bowen, Ray M. and Wang, C. C. (1976),Introduction to Vectors and Tensors Linear andMultilinear Algebra. Plenum Press, New York.

Baisch, S. and Bokelman, GHR (1999), Spectral Analysis with Incomplete Time Series:An Example from Seismology, Computers & Geosciences. 25(7): 739-750 Aug.1999.

Carothers, N.L. (2000),Real Analysis,Cambridge University Press.

Dorny, C. Nelson (1980),A Vector Space Approach to Models and Optimization. RobertE. Krieger Publishing Company.

Edelson, R.A. and Krolik, J.H. (1988), The Discrete Correlation Function: A NewMethod For Analyzing Unevenly Sampled Variability Data, The AstrophysicalJ., 333: 646-659.

Engle, RF and Russell JR (1998), Autoregressive Conditional Duration: A New Modelfor Irregularly Spaced Transaction Data,Econometrica.66(5): 1127-1162.

Foutz, R.V. and Lee, H. (2000), Adaptive Fourier Series and The Analysis OfPeriodicities in Time Series Data, J. Time Ser. Anal. Vol. 21, No. 6 p649-662.

Frazier, Michael W. (1999),An Introduction to Wavelets Through Linear Algebra,Springer-Verlag Inc, New York.


69/97

62

Golubov, B., Efimov, A. and Skvorstsov, V. (1991), Walsh Series and Transforms:Theory and Applications, Kluwer Academic Publishers.

Good, I. J. and Doog, K. C. (1958), A Paradox Concerning Rate of Information,Information and Control, Volume 1, 113-126.

Halmos, P. R. (1957),Introduction to Hilbert Space and the Theory of SpectralMultiplicity. New York: Chelsea.

Koopmans, L.H. (1974), The Spectral Analysis of Time Series. New York: Academic Press.

Kuhn, J.R. (1982), Recovering Spectral Information From Unevenly Sampled Data:Two Machine-efficient Solutions, The Astronomical J.,Vol. 87, No. 1, pp 196-202.

Lomb, N.R. (1976), Least-Squares Frequency Analysis of Unequally Spaced Data,Astrophysics and Space ScienceVol. 39, pp447-462.

Marvasti, Farokh (2001),Nonuniform Sampling Theory and Practice. KluwerAcademic/Plenum Publishers.

Meisel, D.D. (1978), Fourier Transforms of Data Sampled at Unequal Observational Intervals, The Astronomical J., Vol. 83, No. 5, P538-545.

Priestley, M.B. (1996), Wavelets and Time-dependent Spectral Analysis, J. Time Ser. Anal. 17, 85-103.

Rao, T. Subba, Priestley, M.B. and Lessi, O. (1997),Applications of Time SeriesAnalysis in Astronomy and Meteorology. Chapman & Hall.

Ruf T. (1999), The Lomb-Scargle Periodogram in Biological Rhythm Research:Analysis of Incomplete and Unequally Spaced Time-Series,Biological RhythmResearch30(2): 178-201 APR 1999.

Scargle, J.D. (1982), Studies in Astonomical Time Series Analysis. II. StatisticalAspects of Spectral Analysis of Unevenly Spaced Data, The Astronomical J.,263: 835-853.

Scargle, J.D. (1989), Studies in Astronomical Time Series Analysis. III. FourierTransforms, Autocorrelation Functions, and Cross-Correlation Functions ofUnevenly Spaced Data, The Astronomical J., Vol. 343, pp874-877.

Schimmel, M. (2001), Emphasizing Difficulties in The Detection of Rhythms withLomb-Scargle Periodograms,Biological Rhythm Research.32(3): 341-345 Jul.2001.


70/97

63

Schulz, M. and Stattegger, K. (1997), Spectrum: Spectral Analysis of Unevenly Spaced

Paleoclimatic Time Series, Computers & Geosciences23(9): 929-945. 11/1997.

Stoffer, D.S. (1991), Walsh-Fourier Analysis and Its Statistical Applications, J. Am.Stat. Assoc. 86, 461-85.

Swan, P.R. (1982), Discrete Fourier Transforms of Nonuniformly Spaced Data, TheAstronomical J.,Vol.87, No.11 p1608-1615.

Van, Dugen HPA, Olofsen, E. VanHartevelt, JH, Kruyt, EW (1999), A Procedure ofMultiple Period Searching in Unequally Spaced Time-Series with The Lomb-Scargel Method,Biological Rhythm Research, 30(2): 149-177 APR 1999.


71/97

64

Appendix 1

Theorem 3.1. Any function s(t) in L 2 that satisfies s(t) = - s(t -k ) for all t R , is the

limit of step functions s n (t) in B nk, as n , in the L 2 sense.

Proof: Let k be a positive integer, and n = 2 m for some positive integer m, G j and H j are

any real numbers, for j = 1, 2, , n. The function

x n (t) = )(

1

=

+n

j

jj HiG f jnk ,, (t) is in B nk, ,

since each f jnk ,, (t) is in B nk, . We pick the constants G j and H j symmetrically such that

G1 = - H1 ,

(G 2 , H 2 ) = (G n , - H n ),

(G 3 , H 3 ) = (G 1n , - H 1n ),

(G2n , H

2n ) = (G 2

2+n

, - H2

2+n

),

H1

2+n

= 0.

We denote the imaginary part of a function x(t) as I m {x(t)} and divide the interval - k1

t < -k1 +

k2 into n equally spaced small intervals as I1 = [- k

1 , -k1 +

kn2 ), I 2 = [- k

1 +kn2 , -

k1 +

kn22 ), , I l = [- k

1 +kn

l

2

)1( ,-k1 +

knl2 ), , and I n = [- k

1 +kn

n

2

)1( , -k1 +

knn2 ). Now note that


72/97


73/97

66

3 0 . When t I 3 = [ - k1 +

kn22 , -

k1 +

kn23 ),

I m {x n (t)} = I m { )(3

1

=

+j

jj HiG f jnk ,, (t)} + I m { )(4

=

+n

j

jj HiG f jnk ,, (t)}

= (G1 + G 2 +G 3 )+ (H 4 + + H n )

= G1 + G 2 +G 3 - H 2 - H 3

= s 2 + G 3 - H 3 ,

Let s 3 = s 2 + G 3 - H 3 , then

I m {x n (t)} = s 3 = s 2 + G 3 - H 3 , = G1 , for t I 3 = [ - k1 +

kn22 , -

k1 +

kn23 ).

4 0 . Similarly, for l 2n , we can obtain

I m {x n (t)} = s l

= s 1l +G l - H l for t I l .

and s l = (G1 ++ G l ) (H 2 ++ H l ) for 2 l 2n .

5 0 . For l=2

n +1, and t I12+

n,

I m {x n (t)} = I m { )(1

1

2

+

=

+

n

j

jj HiG f jnk ,, (t)} + I m { )(2

2

+=

+n

j

jjn

HiG f jnk ,, (t)}

= (G1 ++ G 12

+n)+ (H

22

+n++ H n )

= s2n + G

12+n

.

6 0 . Similarly, for l= 2n +2 and t I 22+n

, we have

I m {x n (t)} = s 22

+n

= s1

2+n

+G2n +H

2n .


74/97

67

7 0 For 2n +2 l n and t I l , similarly we have

I m {x n (t)} = s l

= s1

l +G2+

ln +H2+

ln .

Based on 1 0 through 7 0 above, we have

I m {x n (t)} =

++

++

.,

,

,

,

,

,

22

11

33

22

11

22

22

nn Itfors

Itfors

Itfors

Itfors

Itfors

Itfors

nn

nn

(A1)

where

++=

++=+=

+=

+=

+=

=

++++

++

+

.221

1212

11

3323

2212

11

222222

22

222

HGss

HGsHGss

Gss

HGss

HGss

Gs

nn

nnnnn

nn

nnn

(A2)

The equation (A2) above can be represented by matrix form as


75/97


76/97

69

Set s n = 1 and s j = 0 if 1 j < n, then the equation (A3 ) has an unique solution since

rank( C nn ) = n. Corresponding to the solution, I m { x n ( t ) }, is denoted as I m x nn, ( t ),

where

I m x ,n n ( t ) =

==

=

=

+

++

+

++

)...(,0

,1,1

)...(,0

,1

,1

)...(,0

423

1313

33

132

11

11

nn

nn

nn

nn

nn

nn

n

IItfor

ItforsItfors

IItfor

Itfors

Itfors

IItfor

,

For each l , let the step function I m x nln , ( t ) = I m x nn, ( t + knl2 ) be a location shifted

version of I m x nn, ( t ), then we obtain 2n-1 step functions, I m x nln , ( t ) in B nk, , for l=

0, 1, 2, , (n-1).

For any > 0, there is a step function H(t) with || I m {s(t)}IA (t)-H(t)|| 2 < k8 , where I A (t)

= 1 if t A = [-k1 , -

k1 +

k ) and I A (t) = 0 if t A = [- k

1 , -k1 +

k ), See Carothers (2000, p

350 ). If we define H(t) = - H( t-k ) for t [-

k1 +

k , -

k1 +

k2 ), and let H(t) repeat itself

periodically over interval of lengthk2 , and let A j = [ - k

1 +k

j )1( , -k1 +

k

j ), for j = 1, 2, ,

, 2k. Then, over C[0, 2 ], we have


77/97

70

|| I m {s(t)}-H(t)||22 =

=

k

j

2

1

|| IjA

(t) (I m {s(t)}-H(t)) ||22

< 2k(k8

) 2

= 42 , (A4)

where IjA(t) = 1 if t A j and I jA (t) = 0 if t A j .

Furthermore, since H( t ) is a step function, H( t ) must be constant on each of the open

intervals (t j , t 1+j ), where - k1 = t 0 < t1 < < t L = - k

1 +k2 , and j = 0, 1, 2, , L-1. Note

that step function is bounded, we can set M =),[ 211

maxkkk

t +| H(t)|. Considering the subinter-

vals I1 I 2 , I 3 I 4 , , I 14 n I n4 , where I1 I 2 I 3 I 4 I 14 n I n4 = [- k1 ,

-k1 +

k2 ), we define 2n constant values c1 , c 2 , , c n2 by

c i =

=

LjjsomeforIItandIItifM

LjforIItandIItiftH

iijii

iijii

0,,,

1...,,2,1,0,),(

212212

212212 (A5)

for i = 1, 2, , 2n.

Set h(t) = =

n

l

nlml txIc1

),12( )( , (A6)


78/97

71

then, h(t) is a step function in B nk, , and satisfies that h(t)= -h( t- k ) and |h(t)-H(t)| 2M.

Thus, over C[k1 , -

k1 +

k2 ), we have

|| h(t)-H(t)|| 22 L(2M)2 *

nk

=

nk

LM 24. (A7)

When n > 2216

LM , based on (A7), over C[0, 2 ], we obtain

|| h(t)-H(t)|| 22 42 . (A8)

(A4) and (A8) together show

|| I m {s(t)}-h(t)|| 2 || I m {s(t)}-H(t)|| 2 + || H(t)-h(t)|| 2

2 + 2

=

Similarly, the real part of s(t) has the same conclusion , therefore conclusions together

prove the Theorem 3.1.

Corollary 3.1 L 2 has another decomposition as below

L 2 = + S 1 +S 0 + S1 +


79/97


80/97

73

Appendix 2

Construct x e (t) as follows.

Case 1. Assume there are not t i = t j 2N for any i , j {1, 2, , N} where t N = N. Set

t =Njj ',1

min {|t j -(t 'j 2N )|. Define x e (t) as below

.

x e (t) =

+


81/97

74

x e (t) =

+


82/97

75

Appendix 3

We need to prove

x(t) =n

limm

lim s mn, (t) , for t = t 1 , , tN . (3.9)

Note the construction of x e (t) in Appendix 2, we know that x e (t) is continuous at all t =

t j , j = 1, 2, , N. By Theorem 3.1, x e (t) is a limit of step functions e n,1 (t) in L nB ,1, for

increasing n. In addition, x e (t) is continuous at t = t 1 , , t N , based on its construction in

Appendix 2. Thus, given any

> 0, when n is large enough, we have

| x e (t) - e n,1 (t) | < 21

, for t = t1 , , tN . (A3.1)

which implies that

nlim

Nttt 1

sup | x e (t) - e n,1 (t) | = 0. (A3.2)

Now note x(t) = x e (t) , for t = t 1 , , tN . It follows from (A3.1) or (A3.2) that

nlim | x(t) - e n,1 (t) | = 0 , for t = t 1 , , tN . (A3.3)

The function e n,1 (t) is in L nKB ,, when K 1, and the projection s mn, (t) = P m,3 ( x e |

L nKB ,, ) minimizes | x e (t)- s mn, (t) | m,3 over functions in L nKB ,, . Thus,

| x e - s mn, | m,3 | x e (t) - e n,1 (t) | m,3 . (A3.4)

It follows from (A3.4) that

N

1dttstxtw mne

N

m

2,

0|)()(|)( N

1dttetxtw ne

N

m

2,1

0|)()(|)( . (A3.5)

Taking limits on both sides of (A3.5) for increasing m, we have


84/97

77

Appendix 4

A nonstandard inner product < y, z > 4 may be defined on L nKB ,, = B nK, + + B n,1 +

+ B n,1 + + B nK, as follows: Theorem 1 of Foutz and Lee (2000) shows that each

y in L nKB ,, has an unique representation,

y(t) = =

K

Kk

ky (t) , (A4.1)

where y k (t) is a unique function in B nk, for each k. Shift each frequency component in

y k (t) in (A4.1) by the fraction u of its period to obtain

y u (t) = =

K

Kk

ky (t + kuN ). (A4.2)

Define the inner product between y and z in L nKB ,, to be

< y, z > 4 = 1 du . (A4.3)

In Foutz and Lee (2000) it is proven that the spaces B nk, for k = -K, , K are orthogonal

with respect to < y, z > 4 .

Appendix 5.

It remains to verify (3.12). The left side of (3.12) is


85/97

78

| =

K

Kk

mnk ts )(,, |24 ,

This equals

=

K

Kk

mnk ts24,, |)(| ,

because the frequency component s mnk ,, ( t ) is in B nk, and because the spaces B nk, are

orthogonal with respect to the inner product < y, z > 4 . This sum in turn equals the right

side of (3.12), because s mnk ,, (t) is periodic with period kN and therefore

| s mnk ,, (t) | 24 = 1

0 1N2

0 ,, |)(| +N

mnkk

uNts dtdu

= 1

0

1

N

2

0,, |)(|

N

mnk ts dtdu

=N

1 20

,, |)(|N

mnk ts dt.


86/97

79

Appendix 6. Background

The concepts and results listed in this Appendix will provide an adequate background for

theoretical results and their proofs in Chapter 3 and the Appendix1-5. All conceptions and

their properties used in Chapter3 and Appendix1-5, such as vector space, direct sum, inner

product, Hilbert space, weight inner product, orthogonal projection, and Gram-Schmidt

procedure, are presented in this Appendix.

A6.1 Vector Spaces

Definition 6.1. (Vector space) Let F be a field, and V a set with operations addition and

scale multiplication. V is a vector space if it satisfies the following properties:

1) For operation addition +, V has properties (1) through (5) as below.

for anyx, y V, x +yV (closure for addition).

(1)(x + y) +z=x + (y + z) (Associativity for addition).

(2)x + y = y + x(commutativity for addition).

(3)There is an element 0 V such that x + 0 = x for anyx V (Existence of

additive identity).

(4)For anyx V, there is an element xsuch thatx + (-x) = 0.

2) For operation scale multiplication, V has properties 6 through 10 as below for any ,

F and anyx, yV.

(6) x V (Closure for scale multiplication).

(7) (x)= ()x (Associativity for scale multiplication).

(8) (+)x= x+ x (First distributive).


87/97

80

(9) (x + y) = x +y (Second distributive).

(10) 1x = x.

Remark. If the field F is real numbers R, the space V is called a real vector space;

similarly if the field F is complex numbers C, the space V is called a complex vector

space.

Definition 6.2. ( Linear independent ) A set of vectors { x 1, x 2 , , x n } in a vector

space V is said to be linearly dependent if there is a set of scales { 1 , 2 , , n } not all

zero, such that

1 x1 + 2 x 2 + + n x n = 0. (6.1)

If (6.1) holds only when i

= 0 for any i = 1, 2, , n, we say that {x

1, x

2, , x

n } are

linearly independent.

Definition 6.3. (Basis and dimension) Let {x 1, x 2, , x n } be linearly independent set

in vector space V, if for any vectorxV , {x1, x 2, , x n, x} are linearly dependent,

we say that { x 1 , x 2, , x n } is a basis of V and that V is finite dimensional and the

dimension of V is n, written dim V, namely n=dim V.


88/97

81

Definition 6.4.(Span)Let V be a vector space, and U V. The span of U is the set of all

linear combinations of elements of U.

Definition 6.5. ( Subspace ) A nonempty subset U of a vector space V over F is a sub-

space if:

x, y U x+ y U, for anyx, y U, and any , F.

Definition 6.6. (Sum of spaces)Let U and W be subspaces of a vector space V. The sum

of U and W, written U+W, is the set

U+W= {x + y|xU,yW}.

The intersection of U and W, written UW , is the set

UW = {x|xU andx W}.

The union of U and W, written UW, is the set

UW = {x|x U or x W }.

Some properties of these operations are stated in the following theorems. ( Theorem 6.1

through Theorem 6.4), see Bowen and Wang(1976, P53).

Theorem 6.1. If U and W are subspaces of V, then U+W is a subspace of V.


89/97

82

Theorem 6.2. If U and W are subspaces of V, then U and W are also subspaces of U+W.

Theorem 6.3. If U and W are subspaces of V, then the intersection UW is a subspace

of V.

Theorem 6.4. Let xbe a vector in U + W, where U and W are subspaces of V. The

decomposition ofx U+W into the formx = y + z, where y U andz W, is unique if

and only if UW = { 0 }.

Definition 6.7.(Direct sum)The sum of two subspaces U and W in V is called the direct

sum of U and W and written by

U W, if UW = { 0 }.

A6.2 Inner product spaces , Hilbert spaces and their properties

Definition 6.8. (Inner product space) A complex vector space V is said to be an inner

product space if there exists an operation called inner product by which any ordered pair

of vectorsyandzin V determines an element of C, written by such that:

(a) =_______

, >< yz , where_______

, >< yz is the complex conjugate of ;

(b) = + ;

(c) = < y, z>;


90/97

83

(d) 0 and = 0 if and only if y= 0;

for allx, y, zV and C.

Definition 6.9. (Norm).The norm of an elementxof an inner product space is an opera-

tion, written by || ||, defined by

||x|| = >< xx, .

Theorem 6.5(The Schwarz inequality)

| | ||y || ||z||

is valid for any two vectorsy, zin an inner product space.

For Theorem 6.5 and its proof, see Bowen and Wang (1976, p60).

Theorem 6.6( The triangle inequality)

||y + z|| ||y || + ||z||

is valid for any vectors y, z in an inner product space.

For Theorem 6.6 and its proof, see Bowen and Wang (1976, p60).

Proposition 6.1. (Properties of the norm). A norm on an inner product space V over C

satisfies the following properties:

(a) ||y || = | | ||y ||;


91/97


92/97

85

Definition 6.11. (Cauchy sequence) A sequence {x n , n = 1, 2, } of elements of an

inner product space is said to be a Cauchy sequence if

||x n -x m || 0 as n, m .

For Definition 6.11, see Brockwell and Davis (1994, p. 45).

Definition 6.12. (Hilbert space) A Hilbert space H is an inner product space which is

complete, i.e. an inner product space in which every Cauchy sequence {x n }converges in

norm to some elementx H.

For Definitions 6.12 , see Brockwell and Davis (1994, p. 45).

Example 6.1. ( The standard inner product for C n ). The standard inner product for

C

n

is defined by

= =

n

i

ii yx1

_

,

wherex= (x1, x 2, , x n )andy= (y 1, y 2, , y n ).

Remark. C n is definedas the set of all n-tuples of complex numbers, that is

z =

nz

z

z

2

1

,

where z 1 , z 2 , , z n C.


93/97

86

For Examples 6.1, see Dorny (1980, p240).

Example 6.2. (The standard inner product for function spaces). The standard inner

product for a function space C(a, b) is defined by

= b

a

tgtf )()( dt,

where f and g C(a, b). Where C(a, b) is the set of continuous, complex-valued func-

tions on [a, b].

For Examples 6.2 , see Dorny (1980, p240-241).

Example 6.3 (A weighted Inner product for function spaces). An inner product of the

following form is often appropriate for C(a, b):

= b

a

tgtftw )()()( dt, 0 < w(t)< .

For Examples 6.3, see Dorny (1980, p242).


94/97

87

A6.3 Orthogonal Projection and Orthonormal Bases

Definition 6.13. ( Orthogonal ) Let V be a complex inner product space. Forx, yV,

we sayx andyare orthogonal, writtenx y, if = 0.

For Definition 6.13, see Frazier (1999, p84).

Definition 6.14. (Orthogonal and orthonormal set) Let V be a complex inner product

space and X be a set in V. X is called an orthogonal set if any two different vectors of X

are orthogonal; X is called an orthonormal set if X is an orthogonal set and ||x|| = 1 for

allx X.

For Definition 6.14, see Frazier (1999, p84).

Theorem 6.7. Let V be a complex inner product space, and let X = {x1, x 2, , x n } be

an orthogonal set of vectors in V and 0 X. Then X is a linearly independent set.

For Theorem 6.7 and its proof, see Frazier (1999, p84).

Theorem 6.8. Let V be a complex inner product space, and let X = {x 1, x 2, , x n } be

an orthogonal set in V withx j 0 for allj. Ifx span X, then

x= jn

j j

jx

x

xx

=

>

Dissertation New

Documents