Self-Organised Criticality Order, Chaos, andarpwhite/courses/5900H/notes/SI Lecture 24a.… · Order, Chaos, and SOC Why is the Feigenbaum constant useful? Scenario • a chaotic

Order, Chaos, andSelf-Organised Criticality

Thiemo Krink, Dept. of Computer Science, University of Aarhus

IntroductionOrder, Chaos, and SOC

Dynamic systems

Loose definitionAny system, where components have motion

Analysis requires to determine the• affected components• initial state of the components• set of rules that describe the state transitions

IntroductionOrder, Chaos, and SOC

Types of behaviour in dynamic systems

(a) Fixed points

(b) Simple periodic orbits

(c) Period-n orbit

(d) Chaos

Quasi-periodic

Order

OrderOrder, Chaos, and SOC

OrderOrder, Chaos, and SOC

When do we find order in a system?E

nerg

y

When a system is in balance at a stable equilibrium

Main characteristics

• little disturbances have no consequences • the system response is proportional to the impact• dramatic disturbances can cause state transitions

OrderOrder, Chaos, and SOC

Which kind of systems are stable?

• ice crystals• a sand grain on the floor • according to economic theory: economic systems (General Equilibrium Theory)

Chaos

ChaosOrder, Chaos, and SOC

ChaosOrder, Chaos, and SOC

What is chaos?

A type of dynamic behaviour, which is not predictablethough deterministic

ChaosOrder, Chaos, and SOC

What is chaos?

Properties of chaos

• chaos is deterministic• causes highly complicated non-linear motion• can only be predicted on a short time scale

A type of dynamic behaviour, which is not predictablethough deterministic

ChaosOrder, Chaos, and SOC

Where do we find chaos?

Almost everywhere, for instance in• turbulence of water and air • wobble of planets • global weather patterns• electric-chemical activity in the human brain

ChaosOrder, Chaos, and SOC

A simple chaotic system in physics

F

ChaosOrder, Chaos, and SOC

Chaos versus stochasticity

Chaos is• deterministic• only predictable for a very short while

Stochasticity is • unpredictable for single events• but allows valuable prediction of the overall future outcome

ChaosOrder, Chaos, and SOC

The logistic map - a model for chaos

A simple population growth model (Bob May / Feigenbaum)

xt+1 = 4rxt (1− xt )123

r reproduction rate of the population; r ∈ [0...1]

xt number of individuals at time t; xt ∈ [0...1]

t time

decrease due to overpopulation

For the population will die out

For the population will reach a fixed size > 0

ChaosOrder, Chaos, and SOC

The logistic map - fixed points

Logistic map with r = 7/10

r ≤ 14

⇒

14

< r ≤ 34

⇒

xt

xt+1

stablefixed point

unstable fixed point

ChaosOrder, Chaos, and SOC

The logistic map - simple cycles

Logistic map with r = 8/10

ChaosOrder, Chaos, and SOC

The logistic map - n-period cycles

Logistic map with r = 88/100

ChaosOrder, Chaos, and SOC

Analyzing stability and instability

MethodExamine the local behaviour of the map near afixed point.

′f (xF ) <1 attracting and stable

′f (xF ) = 0 super - stable

′f (xF ) >1 repelling and unstable

′f (xF ) =1 neutral

f (x) = 4rx(1− x)

′f (x) = 4r(1− 2x)

xF fixed point

Let

ChaosOrder, Chaos, and SOC

The logistic map - chaos

Logistic map with r = 1

ChaosOrder, Chaos, and SOC

Chaos and Bifurcation

What is bifurcation?A point at which a system moves from a period-nlimit cycle to a period-2n cycle (here: by increase of parameter r)

Observationsthe increase of r gets smaller and smaller for higherbifurcationsat a critical value, the system will fall into an infinitecycle, i.e., chaos

•

•

ChaosOrder, Chaos, and SOC

Bifurcation and self-similarity

r

xt

Observations• transitions between cycles become shorter and shorter• self-similarities

ChaosOrder, Chaos, and SOC

How fast will the next bifurcation occur?

r

xt

ak value of r where the logistic map bifurcates into a period - 2k cycle

ChaosOrder, Chaos, and SOC

Universality of the Feigenbaum constant

How fast will the next bifurcation occur?

ak value of r where the logistic map bifurcates into a period - 2k cycle

d∞ = 4.669202… for all one-dimensional systems!

da a

a akk k

k k

= −−

≥−

+

1

1

with k 2

Feigenbaum constant

ChaosOrder, Chaos, and SOC

Why is the Feigenbaum constant useful?

Scenario• a chaotic process with a single tuneable parameter r• first few bifurcations can empirically be determinede.g. a4 and a5

Benefit of Feigenbaum constant• estimation of when the system will become chaotici.e. a∞ -> value of r for chaos

ChaosOrder, Chaos, and SOC

Why is chaos non-predictable by IT?

the computational complexity for an accurate predictionis NP space completemeasurement errorsan initial state involving an irrational number cannot beaccurately represented (finiteness problem)

•

••

ChaosOrder, Chaos, and SOC

Non-predictability - an example

ChaosOrder, Chaos, and SOC

Summary of chaos theory

Chaotic systems aredeterministicextremely sensitive to initial conditionshave no memory and cannot evolvenot complex

••••

Swarm Intelligence Self-organization

Dynamics in agent-based vs. classical analytical models

Swarm Intelligence Self-organization

Lotka-Volterra Systems vs. Individual-based models

Swarm Intelligence Self-organization

Prey-predator systems - an example

Analogies in biology and computing

Prey-Predator <=> Producer-Consumer

Co-evolution <=> Adaptive Producer-Consumer

Common mechanisms

• coupled states and feedback dynamics

Swarm Intelligence Self-organization

Lotka-Volterra systems

DefinitionMathematical models for the cyclic nature of population dynamics arising from the interactionsof agents.

Small fish population Shark population

requires &reduces

Swarm Intelligence Self-organization

A prey-predator system in Lotka-Volterra

dF

dtF a bS

dS

dtS cF d

F

a

b

S

c

d

t

= −

= −

( )

( )

growth rate of small fish

growth rate of sharks

number of small fish

reproduction rate of small fish

number of small fish that a shark can eat

number of sharks

amount of energy that a shark gains from a small fish

death rate of sharks

time

Swarm Intelligence Self-organization

Generalized Lotka-Volterra systems

dx

dtx A xi

i ijj

n

j= −=∑

1

1( )

Differential equations for a n-species predator-prey system

xi

Aij

number of individuals of the ith species

effect of species j on species i

Swarm Intelligence Self-organization

The four classes of feedback behaviour

(a) Fixed points

(b) Simple periodic orbits

(c) Period-n orbit

(d) Chaos

Swarm Intelligence Self-organization

Dynamics of the prey-predator system

A simple Lotka-Volterra attractor

infinite many cyclesdetermined by the startpopulation sizes F (fish population)

S (s

hark

pop

ulat

ion)

Swarm Intelligence Self-organization

Do Lotka-Volterra syst. have fixed points?

dF

dt

dS

dtF

d

cS

a

b= = = =0 for and

Yes, they have a fixed point, i.e., stability at:

Swarm Intelligence Self-organization

Can Lotka-Volterra systems be chaotic?

Yes, but only in more than two dimensions, i.e.,for more than two interacting agents.

A

A A A

A A A

A A A

=

= − −

11 12 13

21 22 23

31 32 33

0 5 0 5 0 1

0 5 0 1 0 1

0 1 0 1

. . .

. . .

. .α

dx

dtx A xi

i ijj

n

j= −=∑

1

1( )

with α = 1 5.

Example:

x1 x2

x3

Swarm Intelligence Self-organization

Dynamics of Individual-Based Models (IBM)

2d world view

A very simple model of three interacting species...

Swarm Intelligence Self-organization

Dynamics of Individual-Based Models (IBM)

Population dynamics

…the complex result on the population level

Swarm Intelligence Self-organization

Individual-based models vs. Lotka-Volterra

Lotka-Volterra IBMs

World view top-down bottom-up

State space small astronomically huge

Validity large scale all scales

Prerequisites knowledge about knowledge about global mechanisms indiv. mechanisms

Self-organised CriticalityOrder, Chaos, and SOC

What is complexity?

• Complexity is the study of how large collections ofsimple units produce a wide variety of behaviour.

• Complex behaviour is neither linear (stable systems)nor uncorrelated (chaos).

degree of disorder

com

plex

ity

stable systems

complex systems

chaotic systems

Where do we find complexity?

Self-organised CriticalityOrder, Chaos, and SOC

E

E

E

EE

E E E E E E E E E E E E E E E0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2 4 6 8 10 12 14 16 18 20

E

E

E

E

EE

EEEEEEEEEEEEEE-3

-2.5

-2

-1.5

-1

-0.5

0

0 0.5 1 1.5 2 2.5 3

Log/Log transform

f (x) = x−τ f (x) = x−1 = 1x

In general: here:

Fre

quen

cy

Impactlo

g (F

requ

ency

)log (Impact)

Self-organised CriticalityOrder, Chaos, and SOC

Observation in Complex Systems• the frequency of an event regarding its impact

follows a power-law distribution

Order, Chaos, and SOC

Earthquakes and power laws

Self-organised Criticality

The frequency is correlated with the impact strength

Order, Chaos, and SOC

Cities and population sizes

Self-organised Criticality

The Zipf law is a power law

Order, Chaos, and SOC

The game of life revisited

Self-organised Criticality

The game of life follows a power law

Order, Chaos, and SOC

Fractals revisited

Self-organised Criticality

Fractal structures cause power law phenomena

δ area of a measure square

D = 1.52

Order, Chaos, and SOC

Fractals in time - “One-over-f” signals

Self-organised Criticality

The strength of the signal is inversely proportional to thefrequency f - here: light emitted from a quasar

Order, Chaos, and SOC

How does a non-complex signal look like?

Self-organised Criticality

time

ampl

itude

White noise - a typical example for uncorrelated behaviour

Order, Chaos, and SOC Self-organised Criticality

What is the mechanism behind power laws?

Ideascooperative phenomenon of many components, sincesystems with few degrees of freedom cannot do itmust be an open system - closed systems reach anequilibriumcould be related to spatial structure

•

•

•

Order, Chaos, and SOC

What is self-organised criticality (SOC)?

Self-organised Criticality

A critical state at the edge between stability and chaos,where complexity is formed by self-organisation

Important characteristic

The process of self-organisation takes place over a longtransient period

Order, Chaos, and SOC

The Sandpile Model

Self-organised Criticality

Z(x, y) → Z(x, y) +1

Select a random x and y

If Z(x,y) = 4 then

Repeat until termination

Let

Z(x, y) → Z(x, y) − 4

Z(x ±1, y) → Z(x ±1, y) +1

Z(x, y ±1) → Z(x, y ±1) +1

and

Z(x, y) no. of grains at position x,y

For all (x,y)

Repeat until no Z(x,y) = 41 2 0 2 3

2 3 2 3 0

1 2 3 3 2

3 1 3 2 1

0 2 2 1 2

Z(2,1) = 2

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

1 2 0 2 3

2 3 2 3 0

1 2 3 3 2

3 1 3 2 1

0 2 2 1 2

adding of one sand grain: Z(3,3) -> Z(3,3) + 1

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

1 2 0 2 3

2 3 2 3 0

1 2 4 3 2

3 1 3 2 1

0 2 2 1 2

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

1 2 0 2 3

2 3 2 3 0

1 2 4 3 2

3 1 3 2 1

0 2 2 1 2

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

1 2 0 2 3

2 3 3 3 0

1 3 0 4 2

3 1 4 2 1

0 2 2 1 2

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

1 2 0 2 3

2 3 3 3 0

1 3 0 4 2

3 1 4 2 1

0 2 2 1 2

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

1 2 0 2 3

2 3 3 4 0

1 3 2 0 3

3 2 0 4 1

0 2 3 1 2

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

1 2 0 2 3

2 3 3 4 0

1 3 2 0 3

3 2 0 4 1

0 2 3 1 2

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

1 2 0 3 3

2 3 4 0 1

1 3 2 2 3

3 2 1 0 2

0 2 3 2 2

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

1 2 0 3 3

2 3 4 0 1

1 3 2 2 3

3 2 1 0 2

0 2 3 2 2

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

1 2 1 3 3

2 4 0 1 1

1 3 3 2 3

3 2 1 0 2

0 2 3 2 2

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

1 2 1 3 3

2 4 0 1 1

1 3 3 2 3

3 2 1 0 2

0 2 3 2 2

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

1 3 1 3 3

3 0 1 1 1

1 4 3 2 3

3 2 1 0 2

0 2 3 2 2

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

1 3 1 3 3

3 1 1 1 1

2 0 4 2 3

3 3 1 0 2

0 2 3 2 2

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

1 3 1 3 3

3 1 1 1 1

2 0 4 2 3

3 3 1 0 2

0 2 3 2 2

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

1 3 1 3 3

3 1 2 1 1

2 1 0 3 3

3 3 2 0 2

0 2 3 2 2

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

Order, Chaos, and SOC Self-organised Criticality

The Sandpile Model

Geological time (million years)

Per

cent

ext

inct

ion

(Sepkoski, 1993)

96% (Raup, 1986)lo

g (f

requ

ency

)log (percent extinction)

Order, Chaos, and SOC Self-organised Criticality

Mass Extinction follows a power-law

Gradualist Model Punctuated Equilibrium

Charles Darwin, John Maynard Smith, Richard Dawkins

Stephen J. Gould, Stuart Kauffman, Per Bak

Tim

e

Change Change

Order, Chaos, and SOC Self-organised Criticality

Gradualist Model vs. Punctuated Equilibrium

Order, Chaos, and SOC

The evolution/extinction model

Self-organised Criticality

Topologya ring model of neighboured species

Algorithm1. Initialize all species with random fitness2. Repeat

Substitute the species with the worst fitnessand its direct neighbours

species 1 species 2 species 3 species 4 species 5

species 1 species 2 species 3 species 4 species 5

Order, Chaos, and SOC

The evolution/extinction model

Self-organised Criticality

F=0.23 F=0.47 F=0.05 F=0.31 F=0.54

currently worst species

block of species, which getsubstituted

Order, Chaos, and SOC

The evolution/extinction model

Self-organised Criticality

Fitn

ess

Time

Order, Chaos, and SOC

The evolution/extinction model

Self-organised Criticality

Species index

Fitn

ess

Order, Chaos, and SOC

Consequence of self-organised criticality

Self-organised Criticality

avalanches always occur in complex systemscatastrophes in complex systems are inevitable!This includes man-made complex systems, such asaeroplanes, power plants, and stock exchange

••

Order, Chaos, and SOC Self-organised Criticality

Application to Evolutionary AlgorithmsChallenges in EAs• parameter control• premature loss of diversity

SOC sandpile model

controlscreates

Mass extinction model

Outbreeding model

Mutation modelcontrols

controls

Order, Chaos, and SOC Self-organised Criticality

Application to Evolutionary Algorithms1

2

3

Challenges in EAs• parameter control• premature loss of diversity

size

of

aval

anch

e

time steps

Powerlaw distribution

Summary

Summary

• Chaos is unpredictable though deterministic• Chaos is extremely sensitive to initial conditions• Chaos has no memory and cannot evolve• The transition into a chaotic state can be estimated

Order, Chaos, and SOC

Chaos

Summary

Summary

• Chaos is unpredictable though deterministic• Chaos is extremely sensitive to initial conditions• Chaos has no memory and cannot evolve• The transition into a chaotic state can be estimated

• Chaos Complexity• Complex behaviour is neither linear nor uncorrelated• Complexity is found at the border of stability and chaos• The common mechanism of complex systems can be

simulated by simple models (SOC)

Order, Chaos, and SOC

≠Complexity

Chaos