lyapunov exponents and chaos theory

8/9/2019 lyapunov exponents and chaos theory

1/33

P h y s i ca 1 6 D ( 1 9 8 5 ) 2 8 5 - 3 1 7

N o r t h - H o l l a n d , A m s t e r d a m

D E T E R M I N I N G L Y A P U N O V E X P O N E N T S F R O M A T IM E S E R IE S

A l a n W O L F ~ - , J a c k B . S W I F T , H a r r y L . S W I N N E Y a n d J o h n A . V A S T A N O

Department of Physics, University of Texas, Austin, Texas 78712, USA

Re c e i v e d 1 8 Oc t o b e r 1 9 8 4

W e p r e s e n t t h e f i rs t a l g o r i t h m s t h a t a l lo w t h e e s t i m a t i o n o f n o n - n e g a t i v e Ly a p u n o v e x p o n e n t s f r o m a n experimental t i m e

s e r ie s . Ly a p u n o v e x p o n e n t s , w h i c h p r o v i d e a q u a l i t a t iv e a n d q u a n t i t a t i v e c h a r a c t e r i z a t i o n o f d y n a m i c a l b e h a v i o r , a r e r e l a t e d t o

t h e e x p o n e n t i a l l y f a s t d i v e rg e n c e o r c o n v e r g e n c e o f n e a r b y o r b i t s i n p h a s e s p a c e . A s y s t e m w i t h o n e o r m o r e p o s i t i v e Ly a p u n o v

e x p o n e n t s i s d e f i n e d t o b e c h a o t i c. O u r m e t h o d i s r o o t e d c o n c e p t u a l l y i n a p r e v i o u s l y d e v e l o p e d t e c h n i q u e t h a t c o u l d o n l y b e

a p p l i e d t o a n a l y t i c a l l y d e f i n e d m o d e l s y s t e m s : w e m o n i t o r t h e long-term g r o w t h r a t e o f small v o l u m e e l e m e n t s in a n a t t r a c t o r .

T h e m e t h o d i s t es t ed o n m o d e l s y s t em s w i t h k n o w n L y a p u n o v s p e c tr a , a n d a p p l ie d t o d a t a f o r t h e B e l o u s o v - Z h a b o t i n s k i i

r e a c t i o n a n d C o u e t t e - Ta y l o r f l o w .

Contents

1 . I n t r o d u c t i o n

2 . T h e L y a p u n o v s p e c t ru m d e f in e d

3 . C a l c u l a t i o n o f Ly a p u n o v s p e c t r a f r o m d i f f e r e n ti a l e q u a t i o n s

4 . A n a p p r o a c h t o s p e c t r a l e s t i m a t i o n f o r e x p e r i m e n t a l d a t a

5 . Sp e c t r a l a l g o r i t h m i m p l e m e n t a t i o n *

6 . I m p l e m e n t a t i o n d e ta i ls *

7 . D a t a r e q u i r e m e n t s a n d n o i s e*

8 . Resul t s

9 . C o n c l u s i o n s

Appendices

A . Ly a p u n o v s p e c t r u m p r o g r a m f o r s y s t e m s o f d i ff e r e n t ia l

e q u a t i o n s

B . F i x e d e v o l u t i o n t i m e p r o g r a m f o r ~ '1

1 I n t r o d u c t i o n

Co n v i n c i n g ev i d en ce fo r d e t e rmi n i s t i c ch ao s h as

c o m e f r o m a v a r i e t y o f r e c e n t e x p e r i m e n t s [ 1 - 6 ]

o n d i s s i p a t i v e n o n l i n ea r sy s t ems ; t h e re fo re , t h e

q u e s t i o n o f d e t e c t i n g a n d q u a n t i f y i n g c h a o s h a s

b e c o m e a n i m p o r t a n t o n e . H e r e w e c o n s i d e r t h e

s p e c t r u m o f L y a p u n o v e x p o n e n t s [ 7 - 1 0 ] , w h i c h

h a s p r o v e n t o b e t h e m o s t u s e f u l d y n a m i c a l d i -

a g n o s t i c f o r c h a o t i c s y s te m s . L y a p u n o v e x p o n e n t s

a re t h e av e rag e ex p o n en t i a l r a t e s o f d i v e rg en ce o r

t P r e s e n t a d d r e s s : T h e C o o p e r U n i o n , S c h o o l o f E n g i n e er i ng ,

N . Y . , N Y 1 0 0 03 , U S A .

* Th e r e a d e r m a y w i s h t o s k i p t h e s t a r r e d s e c t i o n s a t a f i r s t

r e a d i n g .

co n v e rg en ce o f n ea rb y o rb i t s i n p h ase sp ace . S i n ce

n ea rb y o rb i t s co r re sp o n d t o n ea r l y i d en t i ca l s t a t e s ,

ex p o n en t i a l o rb i t a l d i v e rg en ce mean s t h a t sy s t ems

wh o se i n i t i a l d i f fe ren ces we may n o t b e ab l e t o

r e s o l v e w i ll s o o n b e h a v e q u i t e d i f f e r e n t l y - p r e d i c -

t i v e ab i l i t y i s r ap i d l y l o s t . An y sy s t em co n t a i n i n g

a t l e a s t o n e p o s i t i v e L y a p u n o v e x p o n e n t i s de f in e d

t o b e c h a o t ic , w i t h t h e m a g n i t u d e o f t h e e x p o n e n t

re f l ec ti n g t h e t i me sca l e o n w h i ch sy s t em d y n am i cs

b e c o m e u n p r e d i c ta b l e [ 1 0 ] .

F o r s y s t e m s w h o s e e q u a t i o n s o f m o t i o n a r e e x -

p l i c i t l y kn o wn t h e re i s a s t ra i g h t fo rward t ech n i q u e

[ 8 , 9 ] f o r c o m p u t i n g a c o m p l e t e L y a p u n o v s p e c -

t r u m . T h i s m e t h o d c a n n o t b e a p p l i e d d i r e c t l y t o

ex p e r i men t a l d a t a fo r rea so n s t h a t wi l l b e d i s -

cu ssed l a t e r . We wi l l d e sc r i b e a t e ch n i q u e wh i ch

fo r t h e f i r s t t i me y i e l d s e s t i ma t e s o f t h e n o n -n eg a -

t i v e L y a p u n o v e x p o n e n t s f r o m f i n i t e a m o u n t s o f

e x p e r i m e n t a l d a t a .

A l e s s g en e ra l p ro ced u re [ 6 , 1 1 -1 4 ] fo r e s t i ma t -

i n g o n l y t h e d o m i n a n t L y a p u n o v e x p o n e n t in e x -

p e r i m e n t a l s y s t e m s h a s b e e n u s e d f o r s o m e t i m e .

Th i s t e ch n i q u e i s l i mi t ed t o sy s t ems wh e re a we l l -

d e f i n e d o n e - d i m e n s i o n a l ( l - D ) m a p c a n b e r e -

co v e red . Th e t ech n i q u e i s n u mer i ca l l y u n s t ab l e

an d t h e l i t e ra t u re co n t a i n s sev e ra l ex amp l es o f i t s

i mp ro p e r ap p l i ca t i o n t o ex p e r i men t a l d a t a . A d i s -

c u s s i o n o f t h e 1 - D m a p c a l c u l a t i o n m a y b e f o u n d

0 1 6 7 - 2 7 8 9 / 8 5 / $ 0 3 . 3 0 E l se v ie r S ci e nc e P u b li s h er s

(No r t h -H01 an d Ph y s i c s Pu b l i sh i n g Di v i s i o n )


2/33

286 A. Wolf et al. / Determining Lyapunov exponents ro m a time series

i n r e f . 1 3. I n r e f. 2 w e p r e s e n t e d a n u n u s u a l l y

r o b u s t 1 - D m a p e x p o n e n t c a l c u l a t i o n f o r e x p e r i -

m e n t a l d a t a o b t a i n e d f r o m a c h e m i c a l r e a c ti o n .

E x p e r i m e n t a l d a t a i n e v i t a b l y c o n t a i n e x t e r n a l

n o i s e d u e t o e n v i r o n m e n t a l f lu c t u a t io n s a n d l i m i t e d

e x p e r i m e n t a l r e s o l u t i o n . I n t h e l i m i t o f a n i n f i n i t e

a m o u n t o f n o i s e - f r e e d a t a o u r a p p r o a c h w o u l d

y i e l d L y a p u n o v e x p o n e n t s b y d e f i n i ti o n . O u r a b il -

i t y t o o b t a i n g o o d s p e c t r a l e s t i m a t e s f r o m e x p e r i -

m e n t a l d a t a d e p e n d s o n t h e q u a n t i t y a n d q u a l i t y

o f t h e d a t a a s w e l l a s o n t h e c o m p l e x i ty o f t h e

d y n a m i c a l s y s t e m . W e h a v e t e s t e d o u r m e t h o d o n

m o d e l d y n a m i c a l s y s te m s w i t h k n o w n s p e c t r a a n d

a p p l i e d i t t o e x p e r i m e n t a l d a t a f o r c h e m i c a l [2 , 1 3]

a n d h y d r o d y n a m i c [3] s t ra n g e a t t r ac t o r s .

A l t h o u g h t h e w o r k o f c h a r a c te r i zi n g c h a o ti c d a t a

i s s t i l l i n i t s i n f a n c y , t h e r e h a v e b e e n m a n y a p -

p r o a c h e s t o q u a n t i f y i n g c h a o s , e .g ., f r a c t a l p o w e r

s p e c t r a [ 15 ], e n t r o p y [ 1 6 - 1 8 , 3 ], a n d f r a c t a l d i m e n -

s io n [p r o p o sed in r e f . 1 9 , u sed in r e f . 3 - 5 , 2 0 , 2 1 ] .

W e h a v e t e s t e d m a n y o f t h e s e a l g o r i t h m s o n b o t h

m o d e l a n d e x p e r i m e n t a l d a t a , a n d d e s p i t e t h e

c l a i m s o f t h e i r p r o p o n e n t s w e h a v e f o u n d t h a t

t h e s e a p p r o a c h e s o f t e n f a i l t o c h a r a c t e r i z e c h a o t i c

d a t a . I n p a r t i c u l a r , p a r a m e t e r i n d e p e n d e n c e , t h e

a m o u n t o f d a t a r e q u i r ed , a n d t h e s t a b il i ty o f r e -

s u i t s w i t h r e s p e c t t o e x t e r n a l n o i s e h a v e r a r e l y

b e e n e x a m i n e d t h o r o u g h l y .

T h e s p e c t r u m o f L y a p u n o v e x p o n e n t s w il l b e

d e f i n e d a n d d i s c u s s e d i n s e c t i o n 2 . T h i s s e c t i o n

i n c l u d e s t a b l e I w h i c h s u m m a r i z e s t h e m o d e l s y s -

t e m s t h a t a r e u s e d i n t h i s p a p e r . S e c t i o n 3 i s a

r e v ie w o f t h e c a l c u l a t io n o f t h e c o m p l e t e s p e c t r u m

o f e x p o n e n t s f o r s y s t e m s i n w h i c h t h e d e f i n i n g

d i f f e re n t i a l e q u a t i o n s a r e k n o w n . A p p e n d i x A c o n -

t a i n s F o r t r a n c o d e f o r t h i s c a l c u l a t i o n , w h i c h t o

o u r k n o w l e d g e h a s n o t b e e n p u b l i s h e d e l s e w h e r e .

I n s e c t i o n 4 , a n o u t l i n e o f o u r a p p r o a c h t o e s t i m a t -

i n g t h e n o n - n e g a t i v e p o r t i o n o f t h e L y a p u n o v

e x p o n e n t s p e c t r u m i s p r e s e n te d . I n s e c t io n 5 w e

d e s c r i b e t h e a l g o r i th m s f o r e s t i m a t i n g t h e t w o

l a r g e s t e x p o n e n t s . A F o r t r a n p r o g r a m f o r d e -

t e r m i n i n g t h e l a r g e s t e x p o n e n t i s c o n t a i n e d i n

a p p e n d i x B . O u r a l g o r it h m r eq u i re s i n p u t p a r a m e -

t e r s w h o s e s e l e c t i o n i s d is c u s s e d i n s e c t i o n 6 . S e c -

t i o n 7 c o n c e r n s s o u r c e s o f e rr o r i n t h e c a l c u l a t i o n s

a n d t h e q u a l i t y a n d q u a n t i t y o f d a t a r e q u i re d f o r

a c c u r a t e e x p o n e n t e s t i m a t i o n . O u r m e t h o d i s a p -

p l i e d t o m o d e l s y s t e m s a n d e x p e r i m e n t a l d a t a i n

s e c t i o n 8 , a n d t h e c o n c l u s i o n s a r e g i v e n i n

sec t io n 9 .

2 T h e L y a p u n o v s p e c t ru m d e f in e d

W e n o w d e f i n e [8 , 9 ] t h e s p e c t ru m o f L y a p u n o v

e x p o n e n t s i n t h e m a n n e r m o s t r e l e v a n t to s p e c t ra l

c a l c u l a t i o n s . G i v e n a c o n t i n u o u s d y n a m i c a l s y s -

t e m i n a n n - d i m e n s i o n a l p h a s e s p a ce , w e m o n i t o r

t h e l o n g - t e r m e v o l u t i o n o f a n i n f i n i t e s i m a l n - s p h e r e

o f i n i t i a l c o n d i t i o n s ; t h e s p h e r e w i l l b e c o m e a n

n - e l l i p s o i d d u e t o t h e l o c a l l y d e f o r m i n g n a t u r e o f

t h e f l ow . T h e i t h o n e - d i m e n s i o n a l L y a p u n o v e x p o -

n e n t i s t h e n d e f i n e d i n t e r m s o f t h e l e n g t h o f th e

e l l i p s o i d a l p r i n c i p a l a x i s p i t ) :

h ~ = l im 1 lo g 2

p c t )

t -- ,o o t p c ( O ) '

( 1 )

w h e r e t h e ) h a r e o r d e r e d f r o m l a rg e s t t o s m a l l e s t t .

T h u s t h e L y a p u n o v e x p o n e n t s a r e re l a te d t o t he

e x p a n d i n g o r c o n t r a c t i n g n a t u r e o f d i f fe r e n t d i re c -

t i o n s i n p h a s e s p a c e . S i n c e t h e o r i e n t a t i o n o f t h e

e l l i p s o i d c h a n g e s c o n t i n u o u s l y a s i t e v ol v es , t h e

d i r e c t i o n s a s s o c i a t e d w i t h a g i v e n e x p o n e n t v a r y i n

a c o m p l i c a t e d w a y t h r o u g h t h e a t tr a c t o r. O n e c a n -

n o t , t h e r e f o r e , s p e a k o f a w e l l - d e fi n e d d i r e c t i o n

a s s o c i a t e d w i t h a g i v e n e x p o n e n t .

N o t i c e t h a t t h e l i n e a r e x t e n t o f t h e e l l i p s o i d

g r o w s a s 2 h tt , t h e a r ea d e f in ed b y th e f i r s t two

p r in c ip a l ax es g r o w s a s 2 (x ~* x2 )t, t h e v o lu m e d e -

f i n e d b y t h e f i r s t t h r e e p r i n c i p a l a x e s g r o w s a s

2 ( x'+x 2+x ~) t, an d so o n . Th i s p r o p e r ty y i e ld s

a n o t h e r d e f i n i t io n o f t h e s p e c t ru m o f e x p o n e n t s :

tWhile the existence of this limit has been questioned [8, 9,

22], the fact is that the orbital divergenceof any data set

may

be quantified.Even if the limit does not exist for the underlying

system, or cannot be approached due to having finite amounts

of noisy data, Lyapun ovexponent estimates could still provide

a useful characterizationof a given data set. (See section 7.1.)


3/33

A Wo l f e t aL / De termin ing Lyapunov expo nen ts f rom a t ime ser ies

287

the sum of the first j exponents is defined by the

long term exponential growth rate of a j-volume

element. This alternate definition will provide the

basis of our spectral technique for experimental

data.

Any continuous time-dependent dynamical sys-

tem without a fixed point will have at least one

zero exponent [22], corresponding to the slowly

changing magnitude of a principal axis tangent to

the flow. Axes that are on the average expanding

(contracting) correspond to positive (negative) ex-

ponents. The sum of the Lyapunov exponents is

the time-averaged divergence of the phase space

velocity; hence any dissipative dynamical system

will have at least one negative exponent, the sum

of all of the exponents is negative, and the post-

transient motion of trajectories will occur on a

zero volume limit set, an attractor.

The exponential expansion indicated by a posi-

tive Lyapunov exponent is incompatible with mo-

tion on a bounded attractor unless some sort of

fo ld ing

process merges widely separated trajecto-

ries. Each positive exponent reflects a direct ion

in which the system experiences the repeated

stretching and folding that decorrelates nearby

states on the attractor. Therefore, the long-term

behavior of an initial condition that is specified

with

any

uncertainty cannot be predicted; this is

chaos. An attractor for a dissipatiVe system with

one or more positive Lyapunov exponents is said

to be str ange or chaotic .

The signs of the Lyapunov exponents provide a

qualitative picture of a system's dynamics. One-

dimensional maps are characterized by a single

Lyapunov exponent which is positive for chaos,

zero for a marginal ly stable orbit, and negative for

a periodic orbit. In a three-dimensional continuous

dissipative dynamical system the only possible

spectra, and the attractors they describe, are as

follows: ( + , 0 , - ) , a strange attractor; (0 ,0 ,- ), a

two-toms; (0, - , -) , a limit cycle; and ( - , - , - ) ,

a fixed point. Fig. 1 illustrates the expanding,

slower than exponential, and contracting char-

acter of the flow for a three,dimensional system,

the Lorenz model [23]. (All of the model systems

that we will discuss are defined in table I.) Since

Lyapunov exponents involve long-time averaged

behavior, the short segments of the trajectories

shown in the figure cannot be expected to accu-

rately characterize the positive, zero, and negative

exponents; nevertheless, the three distinct types of

behavior are clear. In a continuous four-dimen-

sional dissipative system there are three possible

types of strange attractors: their Lyapunov spectra

are (+ , + , 0 , - ) , (+ , 0 , 0 , - ) , and (+ , 0 , - , - ) .

An example of the first type is Rossler's hyper-

chaos attractor [24] (see table I). For a given

system a change in parameters will generally

change the Lyapunov spectrum and may also

change both the type of spectrum and type of

attractor.

The magnitudes of the Lyapunov exponents

quantify

an attractor's dynamics in information

theoretic terms. The exponents measure the rate at

which system processes create or destroy informa-

tion [10]; thus the exponents are expressed in bits

of information/s or bits/orbit for a continuous

system and bits/iteration for a discrete system.

For example, in the Lorenz attractor the positive

exponent has a magnitude of 2.16 bits/s (for the

parameter values shown in table I). Hence if an

initial point were specified with an accuracy of one

part per million (20 bits), the future behavior

could not be predicted after about 9 s [20 bits/(2.16

bits/s)], corresponding to about 20 orbits. After

this time the small initial uncertainty will essen-

tially cover the entire attractor, reflecting 20 bits of

new information that can be gained from an ad:

ditional measurement of the system. This new

information arises from scales smaller than our

initial uncertainty and results in an inability to

specify the state o f the system except to say that it

is somewhere on the attractor. This process is

sometimes called an information gai n- reflecting

new information from the heat bath, and some-

times is called an information loss-bits shifted

out of a phase space variable

register

when bits

from the heat bath are shifted in.

The average rate at which information con-

tained in transients is lost can be determined from


4/33

288 A. W ol f e t a l. / De termin ing Lyapun ov expon en ts f rom a t ime series

o

. . t . : . . . . " .

. . . . - . . . . . . . . . . . . . , : . ~ ' . ' . . . .

. . . . - - . : - : . : : - : . . . . . . . . .

. . , : ' . . ~ . . - - . . : ~ : : . - . : . . : ' . . : . . : . . . . . . , ~ . , . : . . : ' . .

- . . . . . s t a r t . ' ~ . . -

: ' : : : ' N ~ ~" ""

~ '" " " " "

. ' : ' : ' ( ( , ~ ' . ~ , m ~ : ' ' '

. - ' . : . . % V 4 ; ' : '

" ' - " . ' . . . " . . .

~ .

. . , . . : / ~ . : . . . . . . . . . . . . . . . . .

i l [ l l l

t i m e - ~

- . . o , - : . . . - . .

. . . . . . . . , : , , . : :. . . .

b ) . . . ' , . < ' . ~ : : . : . ' : . . . . . - . . , . . . ~ . ' ~ : ' - : , ~ : . . ~ r . . : - :

. ~ . . . . . . : ' ~ .? ' - , ', ; ~ " x - ~ i l ~ I - " . . .

" " ~ , ~ ' . : ~ ' ~ " , - ~ , ~ ' ~ x x x ~ . ~ ' - . '. . :

" : : : , . . . . " : " " " - . . . -

. : : : , . f , ~ , _ , , , ~ . ~ - , . - , , . , . - . . . : . . . . . .

$ : L ~ . . . . . ~ - ~ . ' . . . _ _ ' ~ . . . . : - . : . . . . . . . : : . : . . :

. - ' . ~ . . . ~ . ; . ; . , . . . . . . . . . . . . . . . . . . . . . . .

, . : , v ' : . : ~ . : ~ . ' . ' . . . . . :

~ s t a r t

[ l , , m , , , , l , , , , , l l l i , , l l l l i , , d l l

t i m e

~

. . . - . - . . . . . : ; . .- - - - ~ . .- : . .

. . . ; ; . ~ . . . . . ~ . : : . . . . . . . . . . . s ~ . ~ , . ~ ; , . . ' : . . : : . ~ : ~ - : . v

: ' "

, ' , . . . : ~ . ' . - 2 ~ ' W ~ ' . ~ - . . . . . '. . ' : : . . . ' : ' . :

- - " " . " -

:

. , , ~, ' .

. . . . . . . . .

t i m e

F i g . 1 . T h e s h o r t t e r m e v o l u t i o n o f t h e s e p a r a t i o n v e c t o r b e t w e e n t h r e e c a r e f u ll y c h o s e n p a i r s o f n e a r b y p o i n t s i s s h o w n f o r t h e

L o r e n z a t t r a c t o r , a ) A n e x p a n d i n g d i r e c t i o n ( ~1 > 0 ) ; b ) a " s l o w e r t h a n e x p o n e n t i a l " d i r e c t i o n ( ~ '2 = 0 ); C ) a c o n t r a c t i n g d i r e c t i o n

(X3 < 0) .

t he ne ga t i ve e xpone n ts T he a s ym pt o t i c de c a y o f a

pe r t u r ba t i on t o t he a t t r a c t o r is gove r ne d by t he

l e a s t ne ga t i ve e xpone n t , w h i c h s hou l d t he r e f o r e be

t he e a s ie s t o f t he ne ga t i ve e xpone n t s t o e s t i m a t e t .

t W e h a v e b e e n q u i t e s u c c e s s fu l w i t h a n a l g o r i t h m f o r d e -

t e r m i u i n g t h e d o m i n a n t ( s m a l l e s t m a g n i t u d e ) n e g a t i v e e x p o -

n e n t f r o m p s e u d o - e x p e r i m e n t a l d a t a ( a s i n g l e t im e s e r i es e x -

t r a c t e d f r o m t h e s o l u t i o n o f a m o d e l s y s t e m a n d t r e a t e d a s a n

e x p e r i m e n t a l o b s e r v a b l e ) f o r s y s t e m s t h a t a r e n e a r l y i n t e g e r -

d i m e n s i o n a l . U n f o r t u n a t e l y , o u r a p p r o a c h , w h i c h i nv o l v e s m e a -

s u r i n g t h e m e a n d e c a y r a t e o f m a n y i n d u c e d p e r t u r b a t io n s o f

t h e d y n a m i c a l s y s t e m , i s u n l i k e l y t o w o r k o n m a n y e x p e r i m e n -

t a l s y s t e m s . T h e r e a r e s e v e ra l f u n d a m e n t a l p r o b l e m s w i t h t h e

c a l c u l a t i o n o f n e g a t i v e e x p o n e n t s f r o m e x p e r i m e n t a l d a t a , b u t

F o r t he L o r e nz a t t r a c to r t he ne ga t ive e xpon e n t is

s o l a r ge t ha t a pe r t u r be d o r b i t t yp i c a l ly be c om e s

i nd i s t i ngu i s ha b l e f r om t he a t t r a c t o r , by " e ye " , i n

le ss th an on e m ean orb i t a l pe r iod ( see fig. 1 ).

o f g r e a t e s t i m p o r t a n c e i s t h a t pos t t rans ien t d a t a m a y n o t

c o n t a i n r e s o l v a b l e n e g a t i v e e x p o n e n t i n f o r m a t i o n a n d p e r

t u r b e d d a t a m u s t r e f l ~ t p r o p e r t ie s o f th e u n p e r t u r b e d s y s t e m ,

t h a t i s , p e r t u r b a t i o n s m u s t o n l y c h a n g e t h e s t a t e o f t h e s y s t e m

( c u r r e n t v a l u e s o f t h e d y n a m i c a l v a ri a b le s ) . T h e r e s p o n s e o f a

p h y s i c a l s y s t e m t o a n o n - d e l t a f u n c t i o n p e r t u r b a t i o n i s d i f fi c u lt

t o i n t e r p r e t , a s a n o r b i t s e p a r a t i n g f r o m t h e a t t r a c to r m a y

r e f l e c t e i th e r a l o c a l l y r e p e l li n g re g i o n o f t h e a t t r a c t o r ( a

p o s i t i v e c o n t r i b u t i o n t o t h e n e g a t i v e e x p o n e n t ) o r t h e f i n i te r i se

t i m e o f t h e p e r t u r b a t i o n .


5/33


6/33

290

4 .

14/olfet aL / Determining Lyapunov exponents from a time series

exponents appears to be satisfied for some model

systems [30]. The calculation of dimension from

this equation requires knowledge of all but the

most negative Lyapunov exponents.

3 Calculation of Lyapu nov spectra from differential

equations

Our algorithms for computing a non-negative

Lyapunov spectrum from experimental data are

inspired by the technique developed indepen-

dently by Bennetin et al. [8] and by Shimada and

Nagashima [9] for determining a complete spec-

trum from a set of differential equations. There-

fore, we describe their calculation (for brevity, the

ODE approach) in some detail.

We recall that Lyapunov exponents are defined

by the long-term evolution of the axes of an infini-

tesimal sphere of states. This procedure could be

implemented by defining the principal axes with

initial conditions whose separations are as small as

computer limitations allow and evolving these with

the nonlinear equations of motion. One problem

with this approach is that in a chaotic system we

cannot guarantee the condition of small sep-

arations for times on the order of hundreds of

orbital periodst, needed for convergence of the

spectrum.

This problem may be avoided with the use of a

phase space plus tangent space approach. A fidu -

cial trajectory (the center of the sphere) is defined

by the action of the nonlinear equations of motion

on some initial condition. Trajectories.of points on

the surface of the sphere are defined by the action

of the linearized equations of motion on points

infinites imally separated from the fiducial trajec-

tory. In particular, the principal axes are defined

by the evolution via the linearized equations of an

initially orthonormal vector frame anchored to the

fiducial trajectory. By definition, p r i n c i p a l a x e s

de f i ned by t he l i near s ys t em ar e a l w ays i n f i n i t e s i ma l

r e l a t i ve t o t he a t t r ac t o r Even in the linear system,

principal axis vectors diverge in magnitude, but

this is a problem only because computers have a

limited dynamic range for storing numbers. This

divergence is easily circumvented. What has been

avoided is the serious problem of principal axes

finding the global fo ld when we really only want

them to probe the local stretch.

To implement this procedure the fiducial trajec-

tory is created by integrating the nonlinear equa-

tions of motion for some post-transient initial

condition. Simultaneously, the linearized equa-

tions of motion are integrated for n different ini-

tial conditions defining an arbitrarily oriented

frame of n orthonormal vectors. We have already

pointed out that each vector will diverge in magni-

tude, but there is an additional singularity-in a

chaotic system, each vector tends to fall along the

local direction of most rapid growth. Due to the

finite precision of computer calculations, the col-

lapse toward a common direction causes the tan-

gent space orientation of all axis vectors to become

indistinguishable. These two problems can be

overcome by the repeated use of the Gram-

Schmidt reorthonormalization (GSR) procedure on

the vector frame:

Let the linearized equations of motion act on

the initial frame of orthonormal vectors to give a

set of v e c t o r s { v 1 . . . . V n . (The desire of each

vector to align itself along the ~1 direction, and

the orientation-preserving properties of GSR mean

that the initial labeling of the vectors may be done

arbitrarily.) Then GSR provides the following or-

thonormal set { ~ . . .. . v,' }:

1D1

v ~ = I I v , l l

v2- ~

v~=

tlv~ - ~l l

tSh oul d the mean orbital period not be well-defined, a

characteristic time can be either the mean time between inter-

sections of a Poincar6 section or the time corresponding to a

domi nant power s pectral feature.

v. - ~ ._ , . . . . . ~

4 )


7/33

A. Wol f e t al . / D etermining Lyapunoo exponents f rom a t ime ser ies

291

w h e r e ( , ) s ig n i fi e s t h e i n n e r p r o d u c t . T h e

f r e q u e n c y o f r e o r t h o n o r m a l i z a t i o n i s n o t c r i t i c a l ,

s o l o n g a s n e i t h e r t h e m a g n i t u d e n o r t h e o r i e n t a -

t i o n d i v e r g e n c e s h a v e e x c e e d e d c o m p u t e r l i m i t a -

t i o ns . A s a r u l e o f th u m b , G S R i s p e r f o r m e d o n

t h e o r d e r o f o n c e p e r o r b i t a l p e ri o d .

W e s e e t h a t G S R n e v e r af fe c ts t h e d i re c t i o n o f

t h e f i r s t v e c t o r i n a s y s t e m , s o t h i s v e c t o r t e n d s t o

s e e k o u t t h e d i r e c t i o n i n t a n g e n t s p a c e w h i c h i s

m o s t r a p i d l y g r o w i n g ( c o m p o n e n t s a l o n g o t h e r

d i r e c t i o n s a r e e i t h e r g r o w i n g l e s s r a p i d l y o r a r e

s h r in k i n g ) . T h e s e c o n d v e c t o r h a s i ts c o m p o n e n t

a l o n g t h e d i r e c t i o n o f t h e f i r s t v e c t o r r e m o v e d , a n d

i s t h e n n o r m a l i z e d . B e c a u s e w e a r e c h a n g i n g i ts

d i r e c t i o n , v e c t o r v 2 i s n o t f r e e to s e e k o u t t h e m o s t

r a p i d l y g r o w i n g d i re c t io n . B e c a u s e o f th e m a n n e r

i n w h i c h w e a r e c h a n g i n g i t, it a l s o i s n o t f r e e to

s e e k o u t t h e s e c o n d m o s t r a p i d l y g r o w i n g d i r e c -

t i o n t . N o t e h o w e v e r t h a t th e v e c t o r s ~ a n d if2

s p a n t h e s a m e t w o - d i m e n s i o n a l s u b s p a c e as th e

v e c t o r s v x a n d v 2 . I n s p i t e o f r e p e a t e d v e c t o r

r e p l a c e m e n t s t h e s p a c e t h e s e v e c t o r s d e f i n e c o n t i n u -

a l l y s e e k s o u t t h e tw o - d i m e n s i o n a l s u b s p a c e t h a t i s

m o s t r a p i d l y g r o w i n g .

T h e a r e a d e f i n e d b y t h e s e

v e c t o r s i s p r o p o r t i o n a l t o 2 (x ~+ x2 )t [ 8] . T h e l e n g t h

o f v e c t o r v t i s p r o p o r t i o n a l t o 2 x~t s o t h a t m o n i t o r -

i n g l e n g t h a n d a r e a g r o w t h a l lo w s us t o d e t e r m i n e

b o t h e x p o n e n t s . I n p r a c t ic e , as ~ a n d if2 a r e

o r t h o g o n a l , w e m a y d e t e r m i n e h 2 d i r e ct l y f r o m

t h e m e a n r a t e o f g r o w t h o f t h e p r o j e c ti o n o f v e c t o r

v 2 o n v e c t o r 4 . I n g e n e r a l, t h e s u b s p a c e s p a n n e d

b y t h e f ir s t k v e c t o r s is u n a f f e c t e d b y G S R s o t h a t

t h e l o n g - t e r m e v o l u t io n o f t h e k - v o l u m e d e f in e d

b y t h e s e v e c t o r s i s p r o p o r t i o n a l t o 2 ~ w h e r e # =

~.ki_ 1 ~ i t

P r o j e c t i o n o f t h e e v o l v e d v e c t o r s o n t o t h e

n e w o r t h o n o r m a l f r a m e c o r r e c tl y u p d a t e s t h e r a te s

o f g r o w t h o f e a c h o f th e f i rs t k - p r i n c i p a l a x e s i n

tTh is is clear when we consider that we may obtain different

directions of vector 02 at some specified ime if we exercise our

freedom to choose the intermediate tim es at wh ich GSR is

performed. T hat is, beginning with a specified v1 and 02 at

time ti, we may perform replacemen ts at times t~+ x and ti+2,

obtaining the vectors ~ , t~ and th en v~' , v~' or we may

propagate directly to tim e ti+ 2, obtaining vl*, v~. t~' and v~

are not

par alle l; therefore, the d etails of propaga tion and

replacemen t determine the orientation of 02

t u r n , p r o v i d i n g e s t i m a t e s o f t h e k l a r ge s t L y a p u n o v

e x p o n e n t s . T h u s G S R a l lo w s th e i n te g r a t i o n o f t h e

v e c t o r f r a m e f o r a s l o n g a s i s r e q u i r e d f o r s p e c t r a l

c o n v e r g e n c e .

F o r t r a n c o d e f o r th e O D E p r o c e d u r e a p p e a r s i n

a p p e n d i x A . W e i l l u s t r a t e t h e u s e o f t h i s p r o c e d u r e

f o r t h e R o s s l e r a t t r a c t o r [ 26 ]. T h e s p e c t r a l c a l c u la -

t i o n r e q u i r e s t h e i n t e g r a t i o n o f th e 3 e q u a t i o n s o f

m o t i o n a n d 9 li n e a ri z e d e q u a t io n s f o r o n t h e o r d e r

o f 1 0 0 o r b i t s o f m o d e l t i m e (a f e w c p u m i n u t e s o n

a V A X 1 1 / 7 8 0 ) t o o b t a i n e a c h e x p o n e n t t o w i t h i n

a f e w p e r c e n t o f it s a s y m p t o t i c v a lu e . I n p r a c t i c e

w e c o n s i d e r t h e a s y m p t o t i c v a l u e to b e a t t a i n e d

w h e n t h e m a n d a t o r y z e r o e x p o n e n t (s ) a re a f e w

o r d e r s o f m a g n i t u d e s m a l l e r t h a n t h e s m a l le s t

p o s i t i v e e x p o n e n t . T h e c o n v e r g e n c e r a t e o f z e r o

a n d p o s i t i v e e x p o n e n t s i s a b o u t t h e s a m e , a n d i s

m u c h s l o w e r t h a n t h e c o n v e rg e n c e r a te o f n e g a ti v e

e x p o n e n t s . N e g a t i v e e x p o n e n t s a r i s e f r o m t h e

n e a r l y u n i f o r m a t t ra c t i v e n e ss o f t he a t t r a c t o r w h i c h

c a n o f t e n b e w e l l e s t i m a t e d f r o m a f e w p a s se s

a r o u n d a n a t t r a c t o r , n o n - n e g a t i v e e x p o n e n t s a r is e

f r o m a o n c e - p e r - o r b i t s tr e t ch a n d f o ld p r o c e s s t h a t

m u s t b e s a m p l e d o n t h e o r d e r o f h u n d r e d s o f

t i m e s ( o r m o r e ) f o r r e a s o n a b l e c o n v er g e n c e .

T h e m e t h o d w e h a v e d e s c r i b e d f o r f i n d i n g

L y a p u n o v e x p o n e n t s is p e r h a p s m o r e e a si ly u n d e r-

s t o o d f o r a d i s c r e t e d y n a m i c a l s y s t e m . H e r e w e

c o n s i d e r t h e H 6 n o n m a p [2 5] ( s e e t a b l e I ). T h e

l i n e a r i z a t i o n o f t h i s m a p i s

[ , s x .

= L / B y . ,

5 )

w h e r e

1 0 ] 6

a n d X ~ i s t h e ( n - 1 ) st i t e r a t e o f a n a r b i t r a r y

i n i t i a l c o n d i t i o n X 1.

A n o r t h o n o r m a l f r a m e o f p r i n c ip a l a x is v e c t o r s

s u c h a s ( ( 0 ,1 ) , ( 1 , 0 ) ) i s e v o l v e d b y a p p l y i n g t h e

p r o d u c t J a c o b i a n t o e a c h v e ct o r. F o r e i t he r v e c to r


8/33

292

A. Wolf et al./ Determin ing Lyapunov exponents

from u tune

eries

the operation may be written in two different

ways. For example, for the vector 0,l) we have

or, by regrouping the terms,

In eq. 7) the latest Jacobi matrix multiplies

each current axis vector, which is the initial vector

multiplied by all previous Jacobi matrices. The

magnitude of each current axis vector diverges,

and the angular separation between the two vec-

tors goes to zero. Fig. 2 shows that divergent

behavior is visible within a few iterations. .GSR

corresponds to the replacement of each current

axis vector. Lyapunov exponents are computed

Fig. 2. The action of the product Jacobian on an initially

orthonorma l vector frame is illustrated for the H non map: (1)

initia l frame; (2) first iterate; and (3) second iterate. By the

second iteration the divergence in vector magnitude and the

angular collapse of the frame are quite apparent. Initial condi-

tions were chosen so that the angular collapse of the vectors

was uncommon ly slow.

from the growth rate of the length of the first

vector and the growth rate of the area defined by

both vectors.

In eq. 8) the product Jacobian acts on each of

the initial axis vectors. The columns of the product

matrix converge to large multiples of the eigenvec-

tor of the biggest eigenvalue, so that elements of

the matrix diverge and the matrix becomes singu-

lar. Here GSR corresponds to factoring out a large

scalar multip lier of the matrix to prevent the mag-

nitude divergence, and doing row reduction with

pivoting to retain the linear independence of the

columns. Lyapunov exponents are computed from

the eigenvalues of the long-time product matrix?.

We emphasize that Lyapunov exponents are not

local quantities in either the spatial or temporal

sense. Each exponent arises from the average, with

respect to the dynamical motion, of the loca l de-

formation of various phase space directions. Each

is determined by the long-time evolution of a

singZe volume element. Attempts to estimate expo-

nents by averaging local contraction and expan-

sion rates of phase space are likely to fail at the

point where these contributions to the exponents

are combined. In fig. 3a we show vector vi at each

renormalization step for the Lorenz attractor over

the course of several hundred orbits [32]. The

apparent multivaluedness of the most rapid ly

growing direction in some regions of the attrac-

tor) shows that this direction is not simply a

function of position on the attractor. While this

direction is often nearly paralle l to the flow on the

Lorenz attractor see fig. 3b) it is usually nearly

transverse to the flow for the Rossler attractor. We

conclude that exponent calcula tion by averaging

local divergence estimates is a dangerous proce-

dure.

+We are aware o f an attempt to estimate Lyapunov spectra

from experimen tal data through direct estimation of local

Jacobian matrices and formation of the long time product

matrix [31]. This calculation is essentially the same as ours (we

avoid m atrix notation by diagon alizing the system at each step)

and has the same problems of sensitivity to external noise, and

to the amoun t and resolution of data required for accurate

estimates.


9/33

4 . Wo l f e t a l. / De termin ing Lyap unoo exponen ts f rom a t im e ser ies 293

.

b )

_ - . _ _ , , , , . - - _ - -

J l e o 1

Fi g . 3 . A m o d i f i c a t i o n t o t h e OD E s p e c t r a l c o d e ( s ee a p p e n d i x A ) a l lo w s u s t o p l o t t h e r u n n i n g d i r e c t i o n o f g r e a t e s t g r o w t h ( v e c t o r

v ~ ) i n t h e L o r e n z a t t r a c t o r . I n ( a ) , in f r e q u e n t r e n o r m a l i z a t i o n s c o n f i r m t h a t t h i s d i r e c t i o n i s n o t s i n g l e - v a lu e d o n t h e a t t r a c t o r . I n ( b ) ,

f r e q u e n t r e n o r m a l i z a t i o n s s h o w u s t h a t t h i s d i r e c t i o n is u s u a l l y n e a r l y p a r a l l e l t o t h e f lo w . I n t h e R o s s l e r a t t r a c t o r , t h i s d i r e c t io n i s

u s u a l l y n e a r l y o r t h o g o n a l t o t h e f l ow .

4 A n a p p r o a c h to sp e c tr a l e s t i m a t i o n f o r

e x p e r i m e n t a l d a t a

Experimental data typically consist of discrete

measurements of a single observable. The well-

known technique of phase space reconstruction

with delay coordinates [2, 33, 34] makes it possible

to obtain from such a time series an attractor

whose Lyapunov spectrum is identical to that of

the original attractor. We have designed a method,

conceptually similar to the ODE approach, which

can be used to estimate non-negative Lyapunov

exponents from a reconstructed attractor. To un-

derstand our method it is useful to summarize

what we have discussed thus far about exponent

calculation.

Lyapunov exponents may be defined by the

ph se sp ce evolution of a sphere of states. At-

tempts to apply this definition numerically to

equations of motion fail since computer limita-

tions do not allow the initial sphere to be con-

structed sufficiently small. In the ODE approach

one avoids this problem by working in the t ngent

sp ce

of a fiducial trajectory so as to obtain always

infinitesimal principal axis vectors. The remaining

divergences are easily eliminated with Gram-

Schmidt reorthonormalization.

The ODE approach is not directly applicable to

experimental data as the linear system is not avail-

able. All is not lost provided that the linear ap-

proximation holds on the smallest length scales

defined by our data. Our approach involves

working in a reconstructed attractor, examining

orbital divergence on length scales that are always

as small as possible, using an approximate GSR

procedure in the reconstructed

ph se sp ce

as


10/33

294 A. Wolfet al. Determining Lyapunov exponents rom a tim e series

n e c e s s a r y . T o s i m p l i f y t h e e n s u i n g d i s c u s s i o n w e

w i ll a s s u m e t h a t t h e s y st e m s u n d e r c o n s i d e r a t i o n

p o s s e s s a t l e a s t o n e p o s i ti v e e x p o n e n t .

T o e s t i m a t e X1 w e i n e f f ec t m o n i t o r t h e l o n g - t e r m

e v o l u t i o n o f a s in g le p a i r o f n e a r b y o r b it s . O u r

r e c o n s t r u c t e d a t t r a c t o r , t h o u g h d e f i n e d b y a s i ng le

t r a j e c t o r y , c a n p r o v i d e p o i n t s t h a t m a y b e c o n s i d -

e r e d t o l i e o n d i f f e r e n t t r a j e c t o r i e s . W e c h o o s e

p o i n t s w h o s e t e m p o r a l s e p a r a t i o n i n t h e o r i g i n a l

t i m e s e r i e s i s a t l e a s t o n e m e a n o r b i t a l p e r i o d ,

b e c a u s e a p a i r o f p o i n ts w i t h a m u c h s m a l le r

t e m p o r a l s e p a r a t i o n i s c h a r a c t e r i z e d b y a z e r o

L y a p u n o v e x p o n e n t. T w o d a t a p o i n ts m a y b e c o n -

s i d e r e d t o d e f i n e t h e e a r l y s t a t e o f t h e f i r s t p r i n -

c i p a l a x i s s o l o n g a s t h e i r s p a t i a l s e p a r a t i o n i s

s m a l l. W h e n t h e i r s e p a r a t i o n b e c o m e s l a rg e w e

w o u l d l ik e t o p e r f o r m G S R o n t h e v e c t o r t h e y

d e f i n e ( s i m p l y n o r m a l i z a t i o n f o r t h i s s i n gl e v e c t o r) ,

w h i c h w o u l d i n v o l v e r e p la c i n g t h e n o n - f i d u c ia l

d a t a p o i n t w i t h a p o i n t c l o s e r to t h e f i d u c ia l p o i n t ,

i n t h e s a m e d i r e c t i o n a s t h e o r i g i n a l v e c t o r . W i t h

f i n i t e a m o u n t s o f d a t a , w e c a n n o t h o p e t o f i n d a

r e p l a c e m e n t p o i n t w h i c h f a ll s e x a c t l y a l o n g a

s p e c i f i e d l i n e s e g m e n t i n t h e r e c o n s t r u c t e d p h a s e

s p a c e , b u t w e c a n l o o k f o r a p o i n t t h a t c o m e s

c l o s e . I n e f f e c t t h ro u g h a s i m p l e r e p l a c e m e n t p r o -

c e d u r e t h a t a t t e m p t s t o p r e s e r v e o r i e n ta t io n a n d

m i n i m i z e t h e s iz e o f r e p la c e m e n t v e ct o rs w e h a v e

m o n i t o r e d t h e l o n g - t e r m b e h a v i o r o f a s i n g le p r i n -

c i p a l a x i s v e c t o r . E a c h r e p l a c e m e n t v e c t o r m a y b e

e v o l v e d u n t i l a p r o b l e m a r is e s, a n d s o on . T h i s

l ead s u s t o an e s t i ma t e o f X1 . (S ee f i g . 4 a . )

T h e u s e o f a f i n it e a m o u n t o f e x p e r i m e n t a l d a t a

d o e s n o t a l l o w u s t o p r o b e t h e d e s i r e d in f i n it e si m a l

l e n g t h s c a l e s o f a n a t t r a c t o r . T h e s e s c a l e s a r e a l s o

i n a c c e s s i b le d u e t o t h e p r e s e n c e o f n o i s e o n f in i te

l e n g t h s c a l e s a n d s o m e t i m e s b e c a u s e t h e c h a o s -

p r o d u c i n g s t r u c t u r e o f t h e a t t r a c t o r i s o f n e g li g ib l e

s p a t i a l e x t e n t . A d i s c u s s i o n o f t h e s e p o i n t s i s d e -

f e r r e d u n t i l s e c t i o n 7 . 1 .

A n e s t i m a t e o f t h e s u m o f t h e t w o la r g e s t e x p o -

n e n t s X1 + X 2 i s s i m i l a r l y o b t a i n e d . I n t h e O D E

p r o c e d u r e t h i s i n v o l v e s t h e l o n g - t e r m e v o l u t i o n o f

a f i d u c i a l t r a j e c t o r y a n d a p a i r o f t a n g e n t s p a c e

v e c t o r s . I n o u r p r o c e d u r e a t r i p l e o f p o i n t s i s

e v o l v e d i n t h e r e c o n s t r u c t e d a t t r a c t o r . B e f o r e th e

a r e a e l e m e n t d e f i n e d b y t h e t r i p l e b e c o m e s c o m -

p a r a b l e t o t h e e x t e n t o f t h e a t t r a c t o r w e m i m i c

G S R b y k e e p i n g t h e f id u c ia l p o i n t , r e p l a c in g t h e

r e m a i n d e r o f t h e t r i p l e w i t h p o i n t s t h a t d e f i n e a

s m a l l e r a r e a e l e m e n t a n d t h a t b e s t p r e s e r v e t h e

e l e m e n t ' s p h a s e s p a c e o r i e n t a t i o n . R e n o r m a l i z a -

t i o n s a r e n e c e s s a r y s o l el y b e c a u s e v e c t o r s g r o w t o o

l a rg e , n o t b e c a u s e v e c t o r s w i l l c o l l a p s e t o i n d i s -

t i n g u i s h a b l e d i r e c t i o n s i n p h a s e s p a c e ( t h i s i s u n -

l i k e ly w i t h t h e l i m i t e d a m o u n t s o f d a t a u s u a l l y

a v a i l a b l e i n e x p e r i m e n t s ) . T h e e x p o n e n t i a l g r o w t h

r a t e o f a r e a e l e m e n t s p r o v i d e s a n e s t i m a t e o f X 1

+ X 2 . (See f ig . 4b . )

O u r a p p r o a c h c a n b e e x te n d e d t o a s m a n y n o n -

n e g a t i v e e x p o n e n t s a s w e c a r e t o e s t im a t e : k + 1

p o i n t s i n t h e r e c o n s t r u c t e d a t t r a c t o r d e fi n e a k -

v o l u m e e l e m e n t w h o s e l o n g - t e r m e v o l u t i o n i s p o s -

s i b le t h r o u g h a d a t a r e p l a c e m e n t p r o c e d u r e t h a t

a t t e m p t s t o p r e s e r v e p h a s e s p a c e o r i e n t a t i o n a n d

p r o b e o n l y t h e s m a l l s c a l e s t r u c t u r e o f t h e a t t r a c -

t o r . T h e g r o w t h r a t e o f a k - v o l u m e e l e m e n t p r o -

v i d e s a n e s t i m a t e o f t h e s u m o f t h e f i r s t k

L y a p u n o v e x p o ne n t s.

I n p r i n c i p l e w e m i g h t a t t e m p t t h e e s t i m a t i o n o f

n e g a t i v e e x p o n e n t s b y g o i n g to h i g h e r - d i m e n s i o n a l

v o l u m e e l e m e n t s, b u t i n f o r m a t i o n a b o u t c o n t r a c t -

i n g p h a s e s p a c e d i r e c t i o n s i s o f t e n i m p o s s i b l e t o

r e s o lv e . I n a s y s t e m w h e r e f r a c t a l s t ru c t u r e c a n b e

r e s o l v e d , t h e r e i s t h e d i f f ic u l t y t h a t t h e v o l u m e

e l e m e n t s i n v o l v i n g n e g a t i v e e x p o n e n t d i r e c t i o n s

c o l l a p s e e x p o n e n t i a l l y f a s t , a n d a r e t h e r e f o r e

n u m e r i c a l l y u n s t a b l e f o r e x p e r i m e n t a l d a t a ( s e e

s e c t i o n 7 . 1 ).

5 Spec tnd algorithm implementation

W e h a v e i m p l e m e n t e d s e v e r a l v e r s i o n s o f o u r

a l g o r i t h m s i n c l u d i n g s i m p l e " f i x e d e v o l u t i o n t i m e "

p r o g r a m s f o r ~'1 a n d X1 h E , " v a r i a b l e e v o l u t i o n

t i m e " p r o g r a m s f o r X I + ~ : , a n d " in t e r ac t iv e "

p r o g r a m s t h a t a r e u s e d o n a g ra p h ic s m a c h i n e t .

tT he interactive progra m avoids the profusion of input

parame ters required for our increasingly sophisticated expo-


11/33

A Wolf et al / Determining Lyapunov exponents from a time series 2 9 5

I n a p p e n d i x B w e i n c l u d e F o r t r a n c o d e a n d

d o c u m e n t a t i o n f o r t h e h 1 f i xe d e v o l u t io n t i m e

p r o g r a m . T h i s p r o g r a m i s n o t s o p h i s t ic a t e d , b u t i t

i s c o n c i s e , e a s i l y u n d e r s t o o d , a n d u s e f u l f o r l e a r n -

i n g a b o u t o u r t e c h n i q u e . W e d o n o t i n c l u d e t h e

f ix ed ev o lu t io n t im e co d e f o r )~x + )~2 ( th o u g h i t i s

b r i e f ly d i s c u s s e d a t t h e e n d o f a p p e n d i x B ) o r o u r

o t h e r p r o g r a m s , b u t w e w i ll s u p p ly t h e m t o i n te r -

e s t e d p a r t i e s . W e c a n a l s o p r o v i d e a h i g h l y e ff i-

c i e n t d a t a b a s e m a n a g e m e n t a l g o r i t h m t h a t c a n b e

u s e d i n a n y o f o u r p r o g r a m s t o e l i m i n a t e t h e

e x p e n s i v e p r o c e s s o f e x h a u s t i v e s e a r c h f o r n e a r e s t

n e i g h b o r s . W e n o w d i s c u s s t h e f i x e d e v o l u t i o n

t i m e p r o g r a m f o r A a n d t h e v a r ia b l e e v o l u t io n

t i m e p r o g r a m f o r ~ x

+ h 2

i n s o m e d e t a i l.

5 .1 .

F i x e d e v o l u t i o n t i m e p r o g r a m f o r )~1

G i v e n t h e t i m e s e r i e s x ( t ) , a n m - d i m e n s i o n a l

p h a s e p o r t r a i t i s r e c o n s tr u c t e d w i t h d e l a y co o r d i -

n a t e s [2 , 3 3 , 3 4 ] , i . e . , a p o in t o n th e a t t r ac to r i s

g i v e n b y

{ x ( t ) , x ( t + ~ ) . . . . x ( t + [ m -

1]~')}

w h e r e z i s t h e a l m o s t a r b i t r a r i l y c h o s e n

d e l a y t i m e . W e l o c a t e t h e n e a r e s t n e i g h b o r ( i n

t h e E u c l i d e a n s e n s e ) t o t h e i n i t i a l p o i n t

{ x ( t o ) . . . . . X ( t o + [ m -

1 ]~ ) } an d d en o te t h e d i s -

t a n c e b e t w e e n t h e s e t w o p o i n t s

L ( t o ) .

A t a l a t e r

t i m e t t , t h e i n i t i a l l e n g t h w i l l h a v e e v o l v e d t o

l e n g t h L ' ( t x ) . T h e l e n g t h e l e m e n t i s p r o p a g a t e d

t h r o u g h t h e a t t r a c t o r f o r a t i m e s h o r t e n o u g h s o

t h a t o n l y s m a l l s c a l e a t t r a c t o r s t r u c t u r e i s l ik e l y t o

b e e x a m i n e d . I f t h e e v o l u t io n t i m e i s t o o l a r ge w e

n e n t p r o g r a m s . T h i s p r o g r a m a l l o w s t h e o p e r a t o r t o o b s e r v e :

t h e a t t r a c t o r , a l e n g t h o r a r e a e l e m e n t e v o l v i n g o v e r a r a n g e o f

t i m e s , t h e b e s t r e p l a c e m e n t p o i n t s a v a i l a b l e o v e r a r a n g e o f

t i m e s , a n d s o f o r t h . E a c h o f t h e s e is s e e n i n a t w o o r t h r e e -

d i m e n s i o n a l p r o j e c t i o n ( d e p e n d i n g o n t h e g r a p h i c a l o u t p u t

d e v i c e ) w i t h t e r m i n a l o u t p u t p r o v i d i n g s u p p l e m e n t a r y i n f o r m a -

t i o n a b o u t v e c t o r m a g n i t u d e s a n d a n g l e s i n t h e d i m e n s i o n o f

t h e a t t r a c t o r r e c o n s t r u c t i o n . U s i n g t h is i n f o r m a t i o n t h e o p e r -

a t o r c h o o s e s a p p r o p r i a t e e v o l u t i o n ti m e s a n d r e p l a c e m e n t

p o i n t s . T h e p r o g r a m i s c u r r e n t l y w r i t te n f o r a V e c t o r G e n e r a l

3 4 0 5 b u t m a y e a s i l y b e m o d i f i e d f o r u se o n o t h e r g r a p h i c s

m a c h i n e s . A 1 6 m m m o v i e su m m a r i z i n g o u r a l g o r it h m a n d

s h o w i n g t h e o p e r a t i o n o f t h e p r o g r a m o n t h e L o r e n z a t t r a c t o r

h a s b e e n m a d e b y o n e o f t h e a u t h o r s (A . W .) .

m a y s e e L ' s h r i n k a s t h e t w o t r a j e c t o r i e s w h i c h

d e f i n e i t p a s s t h r o u g h a f o l d i n g r e g i o n o f t h e

a t t r a c t o r . T h i s w o u l d l e a d t o a n u n d e r e s t i m a t i o n

o f h i - W e n o w l o o k f o r a n e w d a t a p o i n t t h a t

sa t i s f i e s two c r i t e r i a r ea so n ab ly we l l : i t s sep -

a r a t i o n , L ( t l ) , f r o m t h e e v o l v e d f i d u c i a l p o i n t i s

s m a l l , a n d t h e a n g u l a r s e p a r a t i o n b e t w e e n t h e

e v o l v e d a n d r e p l a c e m e n t e l e m e n t s i s s m a l l ( s ee f ig .

4 a ) . I f a n a d e q u a t e r e p l a c e m e n t p o i n t c a n n o t b e

f o u n d , w e r e t a i n t h e p o i n t s t h a t w e r e b e i n g u s e d .

T h i s p r o c e d u r e i s r e p e a t e d u n t i l t h e f i d u c i a l t r a je c -

t o r y h a s t r a v e r s e d t h e e n t i r e d a t a f i l e , a t w h i c h

p o i n t w e e s t i m a t e

M L , ( t k )

Y'~ lo g 2 , (9)

k I =

t M _ t o L ( t t , _ x )

k=l

w h e r e M i s t h e t o t a l n u m b e r o f r e p l a c e m e n t s te p s .

I n t h e f i x e d e v o l u ti o n t i m e p r o g r a m t h e t i m e s t ep

A = t k + 1 - - t k

( E V O L V i n t h e F o r t r a n p r o g r a m )

b e t w e e n r e p l a c e m e n t s i s he l d c o n s t a n t . I n t h e l i m i t

o f a n i n f i n it e a m o u n t o f n o i s e- f re e d a t a o u r p r o c e -

d u r e a l w a y s p r o v i d e s r e p l a c e m e n t v e c t o rs o f i n f in i -

t e s i m a l m a g n i t u d e w i t h n o o r i e n t a t i o n e r r o r , a n d

)k 1 i s o b t a i n e d b y d e f i n i t io n . In s e c t i o n s 6 a n d 7 w e

d i s c u s s t h e s e v e r i t y o f e r r o r s o f o r i e n t a t i o n a n d

f i n i t e v e c t o r s i ze f o r f i n it e a m o u n t s o f n o i s y e x p e r i -

m e n t a l d a t a .

5.2.

V a r i a b l e e v o l u t i o n t i m e p r o g r a m f o r )~1 + )~ 2

Th e a lg o r i t h m f o r e s t im a t in g h x + 1~2 i s s im i l a r

i n s p i r i t t o t h e p r e c e e d i n g a l g o r i t h m , b u t i s m o r e

c o m p l i c a t e d i n i m p l e m e n t a t i o n . A t r io o f d a t a

p o i n t s i s c h o s e n , c o n s i s t i n g o f t h e i n i t i a l f i d u c i a l

p o i n t a n d i t s t w o n e a r e s t n e i g h b o r s . T h e a r e a

A ( t o )

d e f i n e d b y t h e s e p o i n t s i s m o n i t o r e d u n -

t i l a r e p l a c e m e n t s t e p i s b o t h d e s i r a b l e a n d p o s s i -

b l e -

t h e e v o l u t i o n t i m e i s v a r i a b l e . T h i s m a n d a t e s

t h e u s e o f s e v e r a l a d d i t i o n a l i n p u t p a r a m e t e r s : a

m i n i m u m n u m b e r o f e v o l u t i o n s t e p s b e t w e e n r e -

p l a c e m e n t s ( J U M P M N ) , t he n u m b e r o f st ep s t o

e v o lv e b a c k w a r d s ( H O P B A K ) w h e n a r e p l a ce m e n t

s i t e p r o v e s i n a d e q u a t e , a n d a m a x i m u m l e n g t h o r

a r e a b e f o r e r e p l a c e m e n t i s a t te m p t e d .


12/33

296

A. Wolf et aL Determining Lyapunov exponents rom a t ime series

a ) ~

t L I m i

s %/

L l t ~ t t2 t i q t u l o l t

b , .

" t i I

M t o ) ~ r t - - t2 ~it luci*l- - ~ I f . .~ 'tec '

o

Fig. 4. A schem atic representation o f the evolution and replacement procedure u sed to es tima te Lyapunov exponents from

experimental data. a) The largest Lyapunov exponent is computed from the growth of length elements. When the length of the vector

between two poin ts becomes large, a new point is chosen near the reference trajectory, minimizing both the replacement length L and

the orientation change ~. b) A similar procedure is followed to calculate the sum of the two largest Lyapunov exponents from the

growth of area elements. When an area element becomes too large or too skewed, two new points are chosen near the reference

trajectory, minimizing the replacement area A and the change in phase space orientation between the original and replacement area

elements.

E v o l u t i o n c o n t i n u e s u n t il a " p r o b l e m " a ri se s. I n

o u r i m p l e m e n t a t i o n t h e p r o b l e m l i st i n c lu d e s : a

p r i n c i p a l a x is v e c t o r g r o w s t o o l a rg e o r t o o r a p id l y ,

t h e a r e a g r o w s t o o r a p id l y , a n d t h e s k e w n e s s o f

t h e a r e a e l e m e n t e x c e e d s a t h r e s h o l d v a l u e .

W h e n e v e r a n y o f th e s e c r it e r ia a r e m e t , th e t r ip l e

is e vo l v e d b a c k w a r d s H O P B A K s te p s an d a re -

p l a c e m e n t i s a t t e m p t e d . I f r e p la c e m e n t f ai ls , w e

w i l l p u l l t h e t r i p l e b a c k a n o t h e r H O P B A K s t e p s ,

a n d t r y a g a i n . T h i s p r o c e s s i s r e p e a t e d , i f n e c e s -

s a r y , u n t i l t h e t r i p l e is g e t t i n g u n c o m f o r t a b l y c l o s e

t o t h e p r e v i o u s r e p l a c e m e n t s i t e . A t t h i s p o i n t w e

t a k e t h e b e s t a v a i l a b l e r e p l a c e m e n t p o i n t , a n d

j u m p f o r w a r d a t le as t J U M P M N s te p s t o s ta r t t he

n e x t e v o l u t i o n . A t t h e f ir s t r e p l a c e m e n t t i m e , t l ,

t h e t w o p o i n t s n o t o n t h e f id u c ia l t r a je c t o r y a re

r e p l a c e d w i t h t w o n e w p o i n t s t o o b t a i n a s m a l le r

a r e a A ( t t ) w h o s e o r i e n t a ti o n i n p h a s e s p a c e is

m o s t n e a r l y t h e s a m e a s th a t o f t h e ev o l v e d a re a

A ( t l ) .

D e t e r m i n i n g t h e se t o f re p l a c e m e n t p o i n t s

t h a t b e s t p r e s e r v e s a r e a o r i e n t a t i o n p r e s e n t s n o

f u n d a m e n t a l d i f f i c u l t i e s .

P r o p a g a t i o n a n d r e p l a c e m e n t s te p s a r e r e p e a t e d

( s e e f i g . 4 b ) u n t i l t h e f i d u c i a l t r a j e c t o r y h a s

t r a v e r s e d t h e e n t i r e d a t a f il e a t w h i c h p o i n t w e

e s t i m a t e

1 _ _ E l o g 2 - ( 1 0 )

~1 ~2 = tM -- t o A ( t k , x ) ,

k = l

w h e r e t k i s t h e t i m e o f t h e k t h r e p l a c e m e n t s t e p .

I t i s o f t e n p o s s i b l e t o v e r i f y o u r r e s u l ts f o r X~

t h r o u g h t h e u s e o f t h e h 1 + h 2 c a l c u la t io n . F o r


13/33

A W ol f e t aL De term in ing Lyapuno v exponen ts f ro m a t ime series 297

a t t r a c t o r s t h a t a r e v e r y n e a r l y tw o d i m e n s i o n a l

t h e r e i s n o n e e d t o w o r r y a b o u t p r e s e rv i n g o ri e n t a-

t i o n w h e n w e r e p l a c e t r i p l e s o f p o i n t s . T h e s e e l e -

m e n t s m a y r o t a t e a n d d e f o r m w i t h in t h e p la n e o f

t h e a t t r a c t o r , b u t r e p l a c e m e n t t ri p l e s a l w a y s l i e

w i t h i n t h i s s a m e p l a n e . S i n c e X 2 f o r t h e s e a t t r a c -

t o r s i s z e r o , a r e a e v o l u t i o n p r o v i d e s a d i r e c t e s t i -

m a t e f o r h 1. W i t h e x p e r i m e n t a l d a t a t h a t a p p e a r

t o d e f i n e a n a p p r o x i m a t e l y t w o - d i m e n s i o n a l a t-

t r a c t o r , a n i n d e p e n d e n t c a l c u l a t io n o f d f f r o m i ts

d e f i n i t i o n ( f e a s i b l e f o r a t t r a c t o r s o f d i m e n s i o n l e s s

t h a n t h r e e [ 3 5] ) m a y j u s t i f y t h i s a p p r o a c h t o e s ti -

m a t i n g h x .

6 Im plem entat ion deta ils

6.1 .

election of embedding dimension and delay

t ime

I n p r i n c ip l e , w h e n u s i n g d e l a y c o o r d i n a te s t o

r e c o n s t r u c t a n a t t ra c t o r , a n e m b e d d i n g [ 3 4 ] o f t h e

o r i g i n a l a t t r a c t o r i s o b t a i n e d f o r a n y s u f f i c i e n t l y

l a r g e m a n d a l m o s t a n y c h o i c e o f t i m e d e l a y ~-, b u t

i n p r a c t i c e a c c u r a t e e x p o n e n t e s t i m a t i o n r e q u i r e s

s o m e c a r e i n c h o o s i n g t h e s e t w o p a r a m e t e r s . W e

s h o u l d o b t a i n a n e m b e d d i n g i f m i s c h o s e n to b e

g r e a t e r t h a n t w i c e th e d i m e n s i o n o f t h e u n d e r l y i n g

a t t r a c t o r [ 3 4 ] . H o w e v e r , w e f i n d t h a t a t t r a c t o r s

r e c o n s t r u c t e d u s i n g s m a l l e r v a l u es o f m o f t e n

y i e l d r e l ia b l e L y a p u n o v e x p o n e n t s . F o r e x a m p l e ,

i n r e c o n s t r u c t i n g t h e L o r e n z a t t r a c t o r f r o m i t s

x - c o o r d i n a t e t i m e s e r i e s a n e m b e d d i n g d i m e n s i o n

o f 3 i s a d e q u a t e f o r a c c u r a te e x p o n e n t e s t i m a t i o n ,

w e l l b e l o w t h e s u f f ic i e n t d i m e n s i o n o f 7 g i v e n b y

r e f . [ 34 11 ". W h e n a t t r a c t o r r e c o n s t r u c t i o n i s p e r -

f o r m e d i n a s p a c e w h o s e d i m e n s i o n is t o o l ow ,

" c a t a s t r o p h e s " t h a t i n t e rl e a v e d i s t in c t p a rt s o f t h e

a t t r a c t o r a r e l i k e l y t o r e s t f l t . F o r e x a m p l e , p o i n t s

f W e h a v e f o u n d t h a t i t i s o f t e n p o s s i b l e t o i g n o r e s e v e r a l

c o m p o n e n t s o f e v o l v i n g v e c t or s i n c o m p u t i n g t h e i r a v e r a ge

e x p o n e n t i a l r a t e o f g r o w t h : k e e p i n g t w o o r m o r e c o m p o n e n t s

o f t h e v e c t o r o f t e n s uf fi ce s f o r t h i s p u r p o s e . A s o u r d i s c u s s i o n

o f " c a t a s t r o p h e s " w i l l s o o n m a ke c l e a r , t h e s e a r c h f o r r e p l a c e -

m e n t p o i n t s m o s t o f t e n r e q u i r e s t h a t a l l o f t h e d e l a y c o o r d i -

n a t e s b e u s e d .

o n s e p a r a t e l o b e s o f t h e L o r e n z a t t r a c t o r m a y b e

c o i n c i d e n t i n a t w o - d im e n s i o n a l r e c o n s t r u c ti o n o f

t h e a t t r a c t o r . W h e n t h i s o c c u r s , r e p l a c e m e n t e l e -

m e n t s m a y c o n t a i n p o i n t s w h o s e s e p a r a t i o n i n t h e

o r i g i n a l a t t r a c t o r i s v e r y l a r g e ; s u c h e l e m e n t s a r e

l i a b l e t o g r o w a t a d r a m a t i c r a t e i n o u r r e c o n -

s t r u c t e d a t t r a c t o r i n t h e s h o r t t e r m , p r o v i d i n g a n

e n o r m o u s c o n t r i b u t i o n t o t h e e s t i m a t e d e x p o n e n t .

A s t h e s e e l e m e n t s t e n d t o b l o w u p a l m o s t i m -

m e d i a t e l y , t h e y a r e a l s o q u i t e t r o u b l e s o m e t o r e -

p l a c e , .

I f m i s c h o s e n t o o l a r ge w e c a n e x p ec t , a m o n g

o t h e r p r o b l e m s , t h a t n o i s e in t h e d a t a w i ll t e n d t o

d e c r e a s e t h e d e n s i t y o f p o i n t s d e f i n i n g t h e a t t r a c -

t o r , m a k i n g i t h a r d e r t o f i n d r e p l a c e m e n t p o i n t s .

N o i s e i s a n i n f i n i t e d i m e n s i o n a l p r o c e s s t h a t , u n -

l i k e t h e d e t e r m i n i s t i c c o m p o n e n t o f t h e d a t a , f l U s

e a c h a v a i l a b l e p h a s e s p a c e d i m e n s i o n i n a re -

c o n s t r u c t e d a t t r a c t o r ( s e e s e c t i o n 7 . 2 ) . I n c r e a s i n g

m p a s t w h a t i s m i n i m a l l y re q u i re d h a s t h e e f f e c t o f

u n n e c e s s a r i l y i n c r e a si n g t h e l ev e l o f c o n t a m i n a t i o n

o f t h e d a t a .

A n o t h e r p r o b l e m i s s e e n i n a t h r e e - d i m e n s i o n a l

r e c o n s t r u c t i o n o f t h e H t n o n a t t r a c t o r . T h e r e c o n -

s t r u c t e d a t t r a c t o r l o o k s m u c h l i k e t h e o r i g i n a l

a t t r a c t o r s i t ti n g o n a t w o - d i m e n s i o n a l sh e e t, w i t h

t h i s s h e e t s h o w i n g a s i m p l e t w i s t i n t h r e e - s p a c e .

W e e x p e c t t h a t t h i s b e h a v i o r i s t y p i c a l ; w h e n m i s

i n c r e a s e d , s u r f a c e c u r v a t u r e i n c r e a se s ~ . I n c r e a s i n g

m t h e r e f o r e m a k e s i t i n c r e a s i n g l y d i f fi c u l t t o s a t i s f y

o r i e n t a t i o n c o n s t r a i n t s a t r e p l a c e m e n t t i m e , a s t h e

a t t r a c t o r i s n o t s u f f ic i e n t ly f l a t o n t h e s m a l l e s t

l e n g t h s c a l e s f i l l e d o u t b y t h e f i x e d q u a n t i t y o f

d a t a . I t i s a d v i s a b l e t o c h e c k t h e s t a t i o n a r i t y o f

* I f t w o p o i n t s l i e a t o p p o s i t e e n d s o f a n a t t r a c t o r , i t i s

p o s s i b l e t h a t t h e i r s e p a r a t i o n v e c t o r l i es e n t i re l y o u t s id e o f t h e

a t t r a c t o r s o t h a t n o o r i e n t a t io n p r e s er v i n g r e p l a ce m e n t c a n b e

f o u n d . I f t h i s g o e s u n d e t e c t e d , t h e c u r r e n t p a i r o f p o i n t s i s

l i ke l y t o b e r e t a i n e d f o r a n o r b i t a l p e r i o d o r l o n g e r , u n t i l t h e s e

p o i n t s a r e a c c i d e n t a l l y t h r o w n c lo s e t o g e t h e r.

* A s i m p l e s t u d y f o r t h e H t n o n s y s t e m s h o w e d t h a t f o r

r e c o n s t r u c t i o n s o f i n c r e as i n g d im e n s i o n t h e m e a n d i s t an c e b e -

t w e e n t h e p o i n t s d e f i n i n g t h e a t t r a c t o r r a p i d l y c o n v e r g e d t o a n

a t t r a c t o r i n d e p e n d e n t v a l u e . T h e f o l d p u t i n e a c h n e w p h a s e

s p a c e d i r e c t i o n b y t h e r e c o n s t r u c ti o n p r o c e s s t e n d e d t o m a k e

t h e c o n c e p t o f " n e a r b y p o i n t i n p h a s e s p a c e " m e a n in g l e s s f o r

t h i s f i n i t e d a t a s e t .


14/33

298 A W o l f e t a l / D e t e r m in i n g L y a p u n o v e x po n e n t s f r o m a t i m e s er ie s

Fi g . 5 . Th e s t r a n g e a t t r a c t o r i n t h e Be l o u s o v - Z h a b o t i n s k i i r e a c t i o n is r e c o n s t r u c t e d b y t h e u s e o f d e l a y c o o r d i n a t e s f r o m t h e b r o m i d e

i o n c o n c e n t r a t i o n t i m e s e r i e s [ 2] . Th e d e l a y s s h o w n a r e a ) ~ ; b ) ; a n d c ) ~ o f a m e a n o r b i t a l p e r i o d . N o t i c e h o w t h e f o l d i n g re g i o n o f

t h e a t t r a c t o r e v o l v e s f r o m a f e a t u re l e s s " p e n c i l " t o a l a r g e s c a l e tw i s t.

r e s u l t s w i th m t o e n s u r e r o b u s t e x p o n e n t es ti -

m a t e s .

C h o i c e o f d e l a y t i m e i s a l s o g o v e r n e d b y t h e

n e c e s s i t y o f a v o i d i n g c a t a s t r o p h e s . I n o u r d a t a [2 ]

f o r t h e B e l o u s o v - Z h a b o t i n s k i i c h e m i c a l r e a c t i o n

( s e e f i g. 5 ) w e s e e a d r a m a t i c d i f f e r e n c e i n t h e

r e c o n s t r u c t e d a t t r a c t o r s f o r t h e c h o i c e s T = 1 / 1 2 ,

~" - - 1 / 2 a n d I" = 3 / 4 o f t h e m e a n o r b i t a l p e r i o d .

I n t h e f i rs t c a s e w e o b t a i n a " p e n c i l - l i k e " r e g i o n

w h i c h o b s c u r e s t h e f o l d in g r e g i o n o f t h e a t tr a c t o r .

T h i s s t r u c t u r e o p e n s u p and grows larger r e l a t i v e

t o t h e t o t a l e x t e n t o f t h e a t t r a c t o r f o r t h e l a r g e r

v a l u e s o f ~', w h i c h i s c l e a r l y d e s i r a b l e f o r o u r

a l g o r i t h m s . W e c h o o s e n e i t h e r so s m a ll t h a t t h e

a t t r a c t o r s t r et c h e s o u t a lo n g t he h n e x = y - - z =

. . . , n o r s o l a r g e t h a t m z i s m u c h l a r g e r t h a n t h e

o r b i t a l p e r i o d . A c h e c k o f t h e s t a ti o n a r i t y o f e x p o -

n e n t e s t i m a t e s w i t h t- i s a g a i n r e c o m m e n d e d .

6 .2 . volution t imes between replacements

D e c i s i o n s a b o u t p r o p a g a t i o n t i m e s a n d r e p l a c e -

m e n t s t e p s i n t h e s e c a l c u l a t i o n s d e p e n d o n a d -

d i t i o n a l i n p u t p a r a m e t e r s , o r in t h e c a s e o f t h e

i n t e ra c t i v e p r o g r a m , o n t h e o p e r a t o r ' s j u d g e m e n t .

( T h e s t a t i o n a r i t y o f )~l v a l u e s o v e r r a n g e s o f a l l

a l g o r i t h m p a r a m e t e r s i s i l l u s t r a t e d f o r t h e R o s s l e r

a t t r a c t o r i n f ig s. 6 a - 6 d . ) A c c u r a t e e x p o n e n t c a lc u -

l a t i o n t h e r e f o r e r e q u i r e s t h e c o n s i d e r a t i o n o f t h e

f o l l o w i n g i n t e r r e l a t e d p o i n t s : t h e d e s i r a b i l i t y o f

m a x i m i z i n g e v o l u t i o n t im e s , t h e t r a d e o f f b e t w e e n

m i n i m i z i n g r e p l a c e m e n t v e c t o r s iz e a n d m i n i m i z -

i n g t h e c o n c o m i t a n t o r i e n t a t i o n e r r o r , a n d t h e

m a n n e r i n w h i c h o r i e n t a t i o n e r r o r s c a n b e e x -

p e c t e d t o a c c u m u l a t e . W e n o w d i sc u s s th e s e p o in t s

i n t u r n .

M a x i m i z i n g t h e p r o p a g a t i o n t i m e o f v o l u m e e l e-

m e n t s i s h i g h l y d e s i r a b l e a s i t b o t h r e d u c e s t h e

f r e q u e n c y w i t h w h i ch o r i e n t a t io n e r r o r s a r e m a d e

a n d r e d u c e s t h e c o s t o f t h e c a l c u l a ti o n c o n s i d er -

a b l y ( e l e m e n t p r o p a g a t i o n i n v o l v e s m u c h l e s s

c o m p u t a t i o n t h a n e l e m e n t r ep l a ce m e n t ) . I n o u r

v a r i a b l e e v o l u t i o n t i m e p r o g r a m t h i s i s n o t m u c h

o f a p r o b l e m , a s r e p l a c e m e n t s ar e p e r f o r m e d o n l y

w h e n d e e m e d n e c e s s a r y ( t h o u g h th e p r o g r a m h a s

b e e n m a d e c o n s e r v a t i v e i n s u c h j u d g m e n t s ) . I n t h e

i n t e r a c t i v e a l g o r i t h m t h i s i s e v e n l e ss o f a p r o b l e m ,

a s a n e x p e r i e n c e d o p e r a t o r c a n o f t e n p r o c e ss a

l a r g e f il e w i t h a v e r y sm a l l n u m b e r o f r e p l a c e-

m e n t s . T h e p r o b l e m i s s e v e r e , h o w e v e r , i n o u r

f i x e d e v o l u t i o n t i m e p r o g r a m , w h i c h i s o t h e r w i s e

d e s i r a b l e f o r i t s e x t r e m e s i m p l i c i t y . I n t h i s p r o -

g r a m r e p l a c e m e n t s a r e a t t e m p t e d a t f i x e d t i m e

s te p s, i n d e p e n d e n t o f t h e b e h a v i o r o f th e v o l u m e

e l e m e n t .


15/33

A W o l f e t a l l D e t e r m i n in g L y a p u n ov e x p o n e n ts f r o m a t i m e se ri es 299

o 2 1

I a )

~0. t

)..

.._i

o. 8

o : o : :

8 1 2

T A U (ORBITS)

I.B

. /~^

l I l I l l l I I I I I I I I I I I

2 3

EVOLUTION TIN (ORglT~

0,2

( c )

~ - ~ ~" k

2 f ( d )

-

~ q ~ ' ~ . . . . t b . . . . . . . ~ ' ' ~ S ' ' ' ' ' . . . . , b . . . .

5 25 5 5

WAXINUN LEN ~ CUTOFF NINIB LENGTH

( OF HORIZONThL EXTENT) ( OF HORIZONTAL BCI'ENT)

Fig. 6. Stationarity of ~t for Rossler attractor data (8192 points spanning 135 orbits) for the fixed evolution ime program is shown

for the input param eters: a) Tau (delay time); b) evo lution ime between replacementsteps; c) maximum ength of replacementvector

length allowed; and d) m inimum length of replacementvector allowed. The c orrect value of the positive exponent s 0.13 bits /s and is

show n by the h orizonta l line in these figures.

O u r n u m e r i c a l r e s u l ts o n n o i s e - f r e e m o d e l s y s-

t e m s h a v e p r o d u c e d t h e e x p e c t e d r e s ul ts : t o o f r e -

q u e n t r e p l a c e m e n t s c a u s e a d r a m a t i c lo s s o f p h a s e

s p a c e o r i e n t a ti o n , a n d t o o i n f r e q u e n t r e p la c e m e n t s

a l l o w v o l u m e e l e m e n t s t o g r o w o v e r l y l a rg e a n d

e x h i b i t f o l d i n g . F o r t h e R o s s l e r , L o r e n z , a n d t h e

B e l o u s o v - Z h a b o t i n s k i i a t tr a c to r s , e a c h o f w h i c h

h a s a o n c e - p e r - o r b i t c h a o s g e n e r a t i n g m e c h a n i s m ,

w e f i n d t h a t v a r y i n g t h e e v o l u t io n t im e i n t h e

r a n g e t o 1 o r b i t s a l m o s t a l w a y s p r o v i d e s s t a b le

e x p o n e n t e s t i m a t e s . I n s y s t e m s w h e r e t h e m e c h a -

n i s m f o r c h a o s is u n k n o w n , o n e m u s t c h e e k f o r

e x p o n e n t s t a b i l it y o v e r a w i d e r a n g e o f e v o l u t io n

t i m e s . F o r s u c h s y s t e m s i t i s p e r h a p s w i s e t o

e m p l o y o n l y t h e v a r i a b l e e v o l u t i o n t i m e p r o g r a m

o r t h e i n t e r a c t i v e p r o g r a m .

T h e r e a r e o t h e r c r i t e r i a t h a t m a y a f f e ct r e p l a c e -

m e n t t i m e s f o r v a r i a b l e e v o l u t i o n t i m e p r o g r a m s

s u c h a s a v o i d i n g r e g i o n s o f h i g h p h a s e s p a c e v e lo c -

i ty , w h e r e t h e d e n s i t y o f r e p l a c e m e n t p o i n t s i s

l i k e l y t o b e s m a l l . S u c h f e a t u r e s a r e e a s i l y i n -

t e g r a t e d i n t o o u r p r o g r a m s .

I n t h e L o r e n z a t t r a c t o r , t h e s e p a r a t r i x b e t w e e n

t h e t w o l o b e s o f t h e a t t r a c t o r i s n o t a g o o d p l a c e

t o f in d a r e p l a c e m e n t d e m e n t . A n e l e m e n t c h o se n

h e r e i s l i k e l y t o c o n t a i n p o i n t s t h a t w i l l a l m o s t


16/33

300 A Wol f et aL / Determining Lyapunov exponents from a time series

immediately fly to opposite lobes, providing an

enormous contribution to an exponent estimate.

This effect is certainly related to the chaotic nature

of the attractor, but is not directly related to the

values of the Lyapunov exponents. This has the

same effect as the catastrophes that can arise from

too low a value of embedding dimension as dis-

cussed in section 6.1. While we are not aware of

any foolproof approach to detecting troublesome

regions of attractors it may be possible for an

exponent program to avoid catastrophic replace-

ments. For example, we may monitor the

f u t u r e

behavior of potential replacement points and re-

ject those whose separation from the fiducial

trajectory is atypical of their neighbors.

6.3. hor ter l eng ths versus or ien ta t ion errors

With a given set of potential replacement points

some compromise will be necessary between the

goals of minimizing the length of replacement

vectors and minimizing changes in phase space

orientat ion. On the one hand, short vectors may in

general be propaga ted further in time, resulting in

less frequent orientation errors. On the other hand,

we may wish to minimize orientation errors di-

rectly. We must also consider that short vectors

are likely to contain relatively large amounts of

noise.

In the fixed evolution time program the search

for replacements involves looking at successively

larger length scales for a minimal orientation

change. In the variable evolution time program,

points satisfying minimum length and orientation

standards are assigned scores based on a linear

weighting (with heuristically chosen weighting fac-

tors) of their lengths and orientation changes. We

have also performed numerical studies by search-

ing successively larger angular displacements while

attempting to satisfy a minimum length criterion.

Fortunately, we find that these different ap-

proaches perform about equally well. Attempts to

solve the tradeoff problem analytically have sug-

gested opt ima l choices of initial vector magni-

tude, but due to the system dependent nature of

these calculations, we cannot be confident that our

results are of general validity.

The problem of considering the magnitude of

evolved or replacement vectors is complicated by

the fact that at a given point in an attractor, the

orientation of a vector can determine whether or

not it is too large. If we consider a system with an

underlying 1-D map such as the Rossler attractor,

it is the magnitude of the vector's component

transverse to the attractor that is relevant. In this

case our algorithm is closely related to obtaining

the Lyapunov exponent of the map through a

determination of its local slope profile [13]. The

transverse vector component plays the role of the

chord whose image under the map provides a

slope estimate. This chord should obviously be no

longer than the smallest resolvable structure in the

1-D map, a highly system-dependent quantity.

Since the underlying maps of commonly studied

model and physical systems have not had much

detailed structure on small length scales (consider

the logistic equation, cusp maps, and the Be-

lousov-Zhabotinskii map [2]) we have somewhat

arbitrar ily decided to consider 5-10% of the trans-

verse attractor extent as the maximum acceptable

magnitude of a vector's transverse component.

6.4.

Th e a c c u m u l a t i o n o f o r ie n t a ti o n e r r or s

The problem of the accumulation of orientation

errors is reasonably well understood. Consider for

simplicity a very nearly two-dimensional system

with a ( + , 0 , - ) spectrum, such as the Lorenz

attractor . Post-transient data traverse the subspace

characterized by the positive and zero exponents.

Length propagation with replacement on the at-

tractor is clearly susceptible to orientation error

that will mix contributions from the positive and

zero exponents in some complex, system depen-

dent manner . Now consider the n th replacement

step (see fig. 4a) with an orientation change within

the plane of the attractor of 0~. Further, let the

angle the replacement vector makes with respect to

the vector t be ~n- We make the crucial assump-

tion that vectors are propagated for a time t that


17/33

A. Wolfet aL Determining Lyapunov exponents rom a tim e series

301

i s l o n g e n o u g h t h a t g r o w t h a l o n g d i r e c t i o n s d 1 a n d

d 2 a r e r e a s o n a b l y w e l l c h a r a c t e r i z e d b y t h e e x p o -

n e n t s h 1 a n d h 2 r e s pe c t iv e l y . T h e n f o r t h e n e w

r e p l a c e m e n t v e c t o r

L ( t . )

= L ( ~ c o s # . + t2 s i n # . ) ( 1 1 )

a n d a t t h e n e x t r e p l a c e m e n t

L ' ( t n + l ) = L ( t C x ( c o s ~ . ) 2 x a , + ~(sin ~n)2X=tr) ,

( 1 2 )

w h e r e t r i s t h e t i m e b e t w e e n s u c c e s s i v e r e p l a c e -

m e n t s t e p s ( t n + 1 - t . ) . T h e c o n t r i b u t i o n t o e q . ( 9 )

f r o m t h i s e v o l u t i o n i s t h e n

og2 [COS2 7~n22h'tr + s i n E 7 ~ . 2 2 a 2 ' , ]

( 1 3 )

a n d t h e a n g l e t h e n e x t r e p l a c e m e n t v e c t o r L ( t . + 1 )

m a k e s w i t h ~ i s

~ n + l = a r c t a n ( b " t a n # . ) + 1 9.+ 1, ( 1 4 )

w h e r e

b = 2 (a2-a*)t r. (1 5)

I f w e a s s u m e a l l a n g l e s a r e s m a l l c o m p a r e d t o

u n i t y a n d s e t # 0 = ~ 90, e q . ( 1 4 ) im p l i e s t h a t

~n = ~ ~n m bm~

(16)

m=O

I f t h e o r i e n t a t i o n c h a n g e s h av e z e r o m e a n a n d a r e

u n c o r r e l a t e d f r o m r e p l a c e m e n t to r e p l a c e m e n t t h e n

a n a v e r a g e o v e r t h e c h a n g e s g i ve s

t o b e

a ~ . l - # 2 [ b E ( I - b E N ' ) ]

h i 2 ( l n 2 ) N t A l t r N t 1 b E '

( 1 8 )

w h e r e N t is t h e t o t a l n u m b e r o f r e p l a c e m e n t s t e ps .

I f t h e r e i s n o d eg en e racy , i .e . , b E , l - # 2

~'1 = 2 ( l n 2 ) ~ . l t r " ( 1 9 )

F o r t h e L o r e n z a t t r a c t o r ,

b 2

h a s a v a l u e o f a b o u t

0 . 3 3 f o r a n e v o l u t i o n t i m e o f o n e o r b i t , s o a n

o r i e n t a t i o n e r r o r o f a b o u t 1 9 d e g r e e s r e s u l t s i n a

1 0 % e r r o r i n X 1. I f w e c a n m a n a g e t o e v o l v e t h e

v e c t o r f o r t w o o r b i t s , t h e p e r m i s s i b l e i n i t i a l o r i e n -

t a t i o n e r r o r i s a b o u t 2 7 d e g re e s , a n d s o o n . W e s e e

t h a t a g i v e n o r i e n t a t i o n e r r o r a t r e p l a c e m e n t t i m e

s h r i n k s t o a v a l u e n e g l ig i b le c o m p a r e d t o t h e n e x t

o r i e n t a t i o n e r r o r , p r o v i d e d t h a t p r o p a g a t i o n t i m e s

a r e l o n g e n o u g h . O r i e n t a t i o n e r r o r s d o n o t a c c u -

m u l a t e b e c a u s e t h e r e i s n o m e m o r y o f p r e v i o u s

e r r o r s .

T h i s c a l c u l a t i o n m a y b e g e n e r a l i z e d t o a n a t -

t r a c t o r w i t h a n a r b i tr a r y L y a p u n o v s p e c t ru m a n d

a s i m i l a r r e s u l t i s o b t a i n e d . T h e e a s e o f e s t i m a t i n g

t h e i t h e x p o n e n t d e p e n d s o n h o w s m a l l th e q u a n -

t i t y

2 (x'*~-x,)tr

i s . P r o b l e m s a r i s e w h e n s u c c e s s i v e

e x p o n e n t s a r e v e r y c lo s e o r i d e n ti c a l. H y p e r c h a o s ,

w i t h a s p e c t r u m o f [ 0 .1 6 , 0 . 03 , 0 . 0 0, = - 4 0 ] b i t s / s

a n d a n o r b i t a l p e r i o d o f a b o u t 5 . 16 s, h as a n e a s i ly

d e t e r m i n a b l e f ir s t e x p o n e n t ,

lyapunov exponents and chaos theory

Documents