Design principles for adaptive self-organizing systems Finding Fluid Form Symposium University of Brighton December 9-10, 2005 Peter Cariani .

Design principles for

adaptive self-organizing systems

Finding Fluid Form SymposiumUniversity of Brighton

December 9-10, 2005

Peter Cariani

www.cariani.comDepartment of Physiology

Tufts Medical School

Boston

Organismic biology (undergrad @ MIT mid 1970s)

Biological cybernetics & epistemology (1980s)Biological alternatives to symbolic AI

Howard Pattee, Systems Science, SUNY-Binghamton

Temporal coding of pitch & timbre (1990s)

Auditory neurophysiology, neurocomputationHow is information represented in brains?

Commonalities of coding across modality & phyla Neural timing nets for temporal processing

Auditory scene analysisPossibilities inherent in time codesTemporal alternatives to connectionism

signal multiplexing; adaptive signal creation broadcast

My trajectory

Elaboration of structures & functions over time in biological, social, and technological realms,

What makes new functions possible (functional emergence)? Can we put these principles to work for us?Is structural complexification by itself sufficient? (No)Notions of function & functional emergence are needed.What kinds of functions? Sensing, effecting, coordinatingIs pure computation on symbols sufficient? (No)How are brains/minds capable of open-ended creativity?Neural codes, temporal codes, timing nets Neural coding of pitch in the auditory systemRethinking the architecture of the brain:

Temporal alternatives to connectionismAdaptive signal creation & multiplexing, Broadcast coordinative strategies

Evolution of ideas

D

F1

5

1-2-F

M

D-F-1

2-F-M

M-D

1-M

Processes forcombining primitives

Combinatoric emergence:New combinations

of pre-existing primitives

Creative emergence:De novo creation of new primitives

DF 1

5

1-2-F

M

D-F-1

2-F-*

M-D

a-M

a *

Process forconstructing

newprimitives

Sets ofpossible

combinations of primitives 2- -F M

Sets of primitives( , , )axioms atoms states

Add *, a

Combinatoric vs. creative emergence

An example

Exhaustive description Limited description

All permutations of

6 arbitrarily defined objects

All permutations of

single digits

0 1 2 3 4 5 6 7 8 9

consisting of 6 tokens

One well-defined set

having 610 permutations

BOUNDED

Ill-defined number of sets, each w. 610 permutations

UNBOUNDED

Describing the world: Two perspectives

Omniscent

“God’s eye view”

Postulational,

ontological

analytical mode

Perspective of

the limited observer

epistemological

empirical mode

Appearance of newstructures over time

Violations of expectations“Surprise”

Well-defined vs. ill-defined realms

Exhaustive description

God’s eye view

Limited description

Limited observer

Description of

all-possible

organism-environment

relations

Environment as

ill-defined realm

Description is

dependent on set of observables

(environment has as many properties as one can measure)

System-environment as

well-defined realm

No fundamental novelty is possibleAll novelty is combinatoric

Combinatoric andCreative emergence

CLOSED WORLD ASSUMPTION OPEN WORLD ASSUMPTION

A

F e a t u r e 1

A

A

B

B

B

T h r e e f e a t u r e sl i n e a r l y

s e p a r a b l e

A

B

B

A

F e a t u r e 1

A

A

A

A

B

B

B

B

A

B

T w o f e a t u r e sn o t l i n e a r l y

s e p a r a b l e

Effect of adding a new observable

New features

CREATING A NEW OBSERVABLE ADDS A NEW PRIMITIVETHAT INCREASES THE EFFECTIVE DIMENSIONALITY OF THE SYSTEM

OntologyAristotelian hylomorphism

Material substrate that exists independently of us,

yet whose form is largely ill-defined, incompletely known

Organization is embedded in material system

(e.g. mind is the organization of the nervous system)

Conscious awareness requires a particular kind of

regenerative informational organization

embedded in a material system (cybernetic

functionalism)

Aristotle's Causes: Multiple complementary modes of

explanation that answer different kinds of questions

Philosophy

These are different modesof explanation that answerdifferent kinds of questionsabout the behavior of thesystem.

Functional organizationand form are embedded inthe material substrate;consequently, they changewhen the substrate changes.

Mind is the functionalorganization of the brain.

The structure of conscious experiencemirrors the structure of the mind.

Epistemology

Pragmatism (truth of a model related to its purpose)

Perspective of the limited observer

Relativism: different observational frames & purposes

Analytical, empirical and pragmatic truths

Analytic: truths of convention (non-material truths, finist mathematics)

Empirical: truths of measurement, observation (science)

Pragmatic: truths of efficacy & aesthetics (engineering, art)

Constructivism & epistemic autonomy:

by semi-freely choosing our own observables & concepts,

we construct ourselves (for better or worse)

Philosophy

We are interested in

designing & fabricating systems that

autonomously organize themselves

to elaborate structures & improve functions

in response to challenges of their environments

in ways that are meaningful and useful

to us and/or them

Design principles for adaptive, self-organizing systems

Richness of material possibility (e.g. polymeric combinatorics)

+ Ability to steer & stabilize structure(feedback to structure: sensors, coordination mechanisms, effectors)

+ Means to interact w. material world

(sensing, action = "situatedness", semantics)

+ Means to evaluate actions re: purposes(goal-laden representations, "intentionality")

---------------------------------------------------------------------------------------

-

=> Material system capable of adaptive,

elaboration & improvement of informational

functions


Richness of material possibility (need polymers, replicated aperiodic structure, Schrodinger's aperiodic crystal,

analog dynamics, ill-defined interactions)

Ability to steer & stabilize structure(need controls on self-production of internal structure, enzymes)

Means to interact w. material world

(Need sensors, effectors, neural nets)

Means to evaluate actions re: purposes

(Need natural selection or internal goal states, limbic system)


QuickTime™ and aUSBVision decompressor

are needed to see this picture.

Richness of material possibility

Complexity is easy

Steerable complexity is hard



Vibratory dynamics of matter

Cymatics:

Bringing Matter to Life

with Sound

Hans Jenny

Material possibility+ Steer, stabilize, specify, inherit

+ Sensorimotor interaction + Evaluation => ASOS


VARIATION + SELECTION + INHERITANCE => ADAPTATION

Two phases in creative learning processes

Expansive phase: generation of possibilityRealm of free & open creation

e.g. scientific imagination and hypothesis creation Contractive phase: selection of best possibilitiesRealm of clarity & rigorous evaluation

e.g. hypothesis testing (clarity, removal of ambiguity)

Analog dynamics and discrete symbols

We will also argue that one almost inevitably needs

mixed analog-digital systems for complex systems:

i.e. systems w. analog dynamics constrained by digital states ("symbols")

for reliable replication of function

for inheritability of adaptive improvements

Analog and digital are complementary modes of description

analog descriptions - continuous differential equations

digital descriptions - discrete states & ST rules/probabilities

Digital states or discrete symbols are ultrastable basins of attraction

Different theoretical approaches tounderstanding brains and their functions

Dynamicalsystems

approachesSymbol-

processing

Neuralinformationprocessing

differential growthhomeostasisanalog

states & switchesbranchingdiscrete

representationsprocessing

Requisite: sensorimotor loopsInner and outer loops

metabolism: self-productionsteering: percept-action coordinations

action

perception

interaction w.environment

Von Uexküll’s umwelts

McCulloch’sinternal andexternal loops

.

sensors

alterstructure

evaluateselect

Pragmatics

Syntactics

environment

coordinationPerceptstates

Decisionstates

effectors

Action

Performance

Predicted result

Observed result

World2

measure

Sp

So

World1

InitialConditions Predictive model

Physical laws

Formal rules

Encoding(Semantic)

(Syntactic)

Observer'schoices:

what to predictwhat to measure

(Pragmatic)

measure

Realm ofsymbolic

description

Realm ofmaterialaction

Si

A

Self-conscious description of the modeling process:Hertzian modeling relation: measurement & computation

The choice of observables

Finding the variablesThe would-be model maker is now in the extremely common situation of facing some incompletely defined "system," that he proposes to study through a study of "its variables." Then comes the problem: of the infinity of variables available in this universe, which subset shall he take? What methods can he use for selecting them?

W. Ross Ashby, "Analysis of the system to be modeled" in: The Process of Model-Building in the Behavioral Sciences, Ohio State Press, pp. 94-114; reprinted in Conant, ed. Mechanisms of Intelligence

The choice of observables - analogous problems

1. Choice of primitive features for classifiers

2. Evolution of sensory organs in organisms

3. Choice of sensors for robots

A

F e a t u r e 1

A

A

B

B

B

T h r e e f e a t u r e sl i n e a r l y

s e p a r a b l e

A

B

B

A

F e a t u r e 1

A

A

A

A

B

B

B

B

A

B

T w o f e a t u r e sn o t l i n e a r l y

s e p a r a b l e

Effect of adding a new observable

alterstructure

measure evaluate

Syntactic Axis

Pragmatic Axis

control

Computation

S i Sf

environment

Semioticsof adaptivedevices

Feedback to state

Feedback to structure alters functionalities

Semiotic relations (Charles Morris)

SYMBOL(Functional state)

SYSTEM-GOALS EXTERNALWORLD

OTHERSYMBOLS

Semanticspercept-action linkagesPragmatics

valuations

Syntacticsrules on symbol - types

PURPOSE

MEANING

Sensory inputsMotor outputs

WORLD

BRAIN

Externalsemanticlinkages

Syntacticlinkages

Pragmaticlinkages

DESIRESDRIVES THOUGHTS

SENSATIONS

B

Evaluatere: goalsFrontal & limbic systems

Internallygeneratedpatternsequences

sensorysystems

motorsystems

Adaptivity in

percept-action

loops (Cariani)

∆ = alter structure → alter function

evaluate

Syntactic Axis

Si S

f

∆∆control

environment

SemanticAxis

PragmaticAxis

measure

∆

coordination

C

symbolic

nonsymbolic

environment

user-specified initial state

Computation

final state

state trans. rules

Pure computation (state-determined system, no independent informational transactions w. environment)

m e a s u r e c o n t r o l

sensorarray effector

array

a c t i o n

v e c t o r

f e a t u r e

v e c t o r

performance

e n v i r o n m e n t

computation

Fixed robotic device

Fixed sensors,coordinators, and effectors;

Purely reactiveand driven by its inputs;

Incapable oflearning

Computationallyadaptivedevice

computation

a c t i o n

v e c t o r

f e a t u r e

v e c t o r

∆

performance

training

c o n t r o lm e a s u r e

e n v i r o n m e n t

t e s t

Trainable machinesNeural networksAdaptive classifiersGenetic algorithmsRobots w. adaptive programs

Capable of learningnew percept-actionmappings (classifications)

Some observations about adaptability

Whatever functionalities are fixed, the designer must specify

works for well-defined problems & solutions

advantage: predictable, reliable behavior

drawback: problems of specification

Whatever is made adaptive must undergo a learning phase

needed for ill-defined problems & solutions

some unpredictability of solutions found

creative behavior!

the more autonomy, the more potentially creative

Consequently, there are tradeoffs between

adaptability & efficiency

autonomy/creativity & control/predictability

coordinationfeature vector

test

performance

adaptiveconstruction

of a new sensor∆sensors

action vector

environment

Evolution/adaptive construction of new sensors

sensory evolutionimmune systemsperceptual learning

capable of learningnew perceptual categories

new feature primitives

(new observables)

• When a system can choose its own categories – through which it perceives and acts on the world – that system achieves some limited degree of epistemic autonomy.

• A rudimentary electrochemical device was built by cyberneticist Gordon Pask in 1958 that grew its own sensors to create its own “relevance criteria.”

Epistemic autonomy

"With this ability to make or select proper filters on its inputs, such a device explains the central problem of epistemology. The riddles of stimulus equivalence or of local circuit action in the brain remain only as parochial problems." .

Warren McCulloch, preface ,Gordon Pask (1961) . An Approach to Cybernetics.

From: "Physical analogues to the growth of a concept", Symposium onthe Mechanization of Thought Processes, National Physical Laboratories,November 24-28, 1958, H.M.S.O., London, Volume II, p.919.

Principles of action/use

1. Front-ends for trainable classifiers

Useful in ill-defined situations where one does not a priori know what features are adequate to effect a classification

2. Adaptive, self-organizing sensors

Grow structures over analog-VLSI electrode arrays in order to sense new aspects of the world. Use biochemical and/or biological systems coupled to an electrode array

3. Materially-based generator of new behaviors (adaptive pattern-generators)

Similar steerable, ill-defined systems could be used to generate new patterns (sound, images) in an open-ended way that is not at all obvious to the observer/controller

4. Epistemic autonomy

Device chooses how it will be connected to the outside world; what aspects of the material world (categories) are relevant to it. (Symbol grounding, frame problem)

Feedback to state vs. feedback to structure

A thermostat is limited in the information that it can gain from its environment by the fixed nature of its sensors. It has feedback to state, but not feedback to structure. The amount of information that such a system can extract from its environment is finite at any time, and bounded by its fixed structure.

A system capable of sensory evolution or perceptual learning has the ability to change its relation to its environs. Such a system has an open-ended set of observational primitives. It has both feedback to state and feedback to structure. The amount of information that such a system can extract from its environment is finite at any time, but unbounded. Such a system is open-ended.

Analog dynamics without inheritable constraint (Hans Jenny)



von Neumann's kinematic (robotic) self-reproducing automaton (1948)

Inheritable constructionanalog dynamics constrained & selected by discrete symbols

∆

constructionpossibilitiesA B C Dα β χ δ1 2 3 4

computationfeature vector

testperformance

∆∆

controlmeasure

action vector

environment

constructionlanguage

constructall parts ofthe device

select fromexisting alternatives

physicalconstruction

Aδ3 (mutation)

Symbolically-encodedmemory permits results ofan optimization process tobe passed to subsequentgenerations

Purely analog adaptivesystem must be trainedeach generation

Genetic algorithm +Pattern grammar forguiding constructionconstrained search

The homeostat

Relation to Ashby's homeostat

∆



testperformance

∆∆

controlmeasure

action vector

environment





Aδ3 (mutation)

Evaluation of ability to control inputs

Analog sensor/controller

Uniselector25x25x25x25 = 390k construction possibilities -> variety of the control system, unconstrained search

Relation to Ashby's homeostat

∆



testperformance

∆∆

controlmeasure

action vector

environment





Aδ3 (mutation)

Evaluation of ability to control inputs

Analog sensor/controller

Uniselector25x25x25x25 = 390k construction possibilities -> variety of the control system, unconstrained search

Ashby's homeostat

Environment

Analog controller

(ill-defined structure)

Uniselector

evaluate(in bounds?)

Adaptive analogcontroller

Structure of particular controllersis unknown to designer

Requisite variety forcontrol is the number ofalternative controllersavailable25x25x25 = 390,625

The homeostat & the brainA few cybernetics-inspired accounts of brain function

Sommerhoff (1974) Logic of the Living Brain

Klopf, The Selfish Neuron

Arbib, The Metaphorical Brain

Most successful neuroscientific application of cybernetics:

W.Reichardt's analysis of fly optomotor loop

The homeostat never caught on as a brain metaphor

Some possible reasons:• Homeostats never were cast in terms of neural nets• No obvious digital uniselector function in the brain• Predominance of problems of pattern recognition and

formulation of coherent action over simple problems of internal regulation

The brain as an adaptive self-organizing system

Ideas that flow from cybernetics and theoretical biology:

1) Brains as signal self-production systems

related to reverberant loops (a la Lorente, Lashley, Hebb, McCulloch, Pitts & many others)

2) Brains as pattern-resonance systems

related to Lashley, Hebb, many others

3) Brains as multiplexed signaling and storage systems

holographic paradigms, Longuet-Higgins, Pribram,John

4) Brains as mass-dynamics, broadcast systems

5) Brains as communications nets that create new signals

6) Brains as temporally-coded pulse pattern systems

I believe all this is possible using temporal pattern codes.

Regeneration of parts

F(A)F(B)F(C)

A+B+CF(D)

geneticconstruction

plans

geneticexpressionapparatus(universal

constructor)

replication of plans

replicationof constructor

plans

byproduct D

effect on survivalof whole system

A

byproductsrawmaterials

metabolicloops

B


geneticcontrol

geneticexpression &reproduction

genetic plans(symbolic memory)

set boundaryconditions

C

Von Neumann’s kinematic self-reproducing automaton

F(A)F(B)F(C)

A+B+CF(D)

geneticconstruction

plans

geneticexpressionapparatus(universal

constructor)

replication of plans

replicationof constructor

plans

byproduct D

effect on survivalof whole system

A

Autopoiesis and autocatalysis


metabolicloops

B

Symbolically-guided self-production


geneticcontrol

geneticexpression &reproduction

genetic plans(symbolic memory)

set boundaryconditions

C

Autopoiesis and autocatalysisLife is built upon cycles of self-production


metabolicloops

B

Brain function may be based on self-productions of spike patternsHebbian reverberant eigenstates and regenerative temporal patterns

A B

Resonantstate | A

Resonantstate | B

Contingentsensory input

Epistemiccut

S2

S3

S1

R2

R3

R1

CNS

Potentialstimuli

Potentialresponses

A BStimulus-driven switching betweenreverberant circuits (after Hebb, 1965)

Reverberant patternsas switchable eigenstates

Motorresponse

for A

Motorresponse

for B

McCulloch & Pitts (1943) Nets with circles render activity independentof time and semi-autonomous re: the environmentvon Foerster (1948) brain eigenstates as a form of ST memory

Why the mind is in the headWarren McCullochL.A. Jeffress, ed. Cerebral Mechanisms of Behavior(The Hixon Symposium, Wiley, 1951, reprinted inEmbodiments of Mind, MIT, 1965, concluding lines)

This brings us back to what I believe is the answer to the question: Why is the mind in the head?

Because there, and only there, are hosts of possible connections to be formed as time and circumstance demand. Each new connection serves to set the stage for others yet to come and better fitted to adapt us to the world, for through the cortex pass the greatest inverse feedbacks whose function is the purposive life of the human intellect.

The joy of creating ideals, new and eternal, in and of a world, old and temporal, robots have it not.

For this my Mother bore me.

The brain as aself-regeneratingpattern-resonancesystem

The same [resonance] is true of all bodies which can yield notes. Tumblers resound when a piano is played, on the striking of certain notes, and so do window panes. Nor is the phenomenon without analogy in different provinces. Take a dog that answers to the name "Nero." He lies under your table. You speak of Domitian, Vespasian, and Marcus Aurelius Antonius, you call upon all the Roman Emperors that occur to you, but the dog does not stir, although a slight tremor of his ear tells you of a faint response of his consciousness. But the moment you call "Nero" he jumps joyfully towards you. The tuning fork is like your dog. It answers to the name A.

Ernst Mach, Popular Lectures, “The fibers of Corti” c. 1865

Tuning in nervous systemsMinds as pattern-resonances

Pattern resonances: neural assemblies emitting annotative tag signals that elaborate a regenerating signal pattern

Primaryinteractions

Secondaryinteractions

Higher-order, morecomplex interactions

Figure 7. Time-coded broadcast schema for asynchronous, heterarchical global integration.

Incoming sensory signals

Creation of new primitive time patterns

Higher-orderinteractions

Miller, Ratliff, and Hartline (1961) How cells receive stimuli.Scientific American 215(3):222-238.

Phase-locking in visual neurons(Horseshoe crab ommatidium, 5-15 Hz flashes)

Phase-locking in auditoryl neuronsCat auditory nerve fibers, 250 Hz tone

Javel,

Higher-order interval pattern

Multiple intervals in same spike train

Interspike interval code

Temporalpatterncodes

Temporal pattern codes

250 Hz tone

Javel E, McGee JA, Horst W, Farley GR, Temporal mechanisms in auditory stimulus coding.In: G. M. Edelman, W. E. Gall and W. M. Cowan, ed, Auditory Function: NeurobiologicalBases of Hearing, Wiley: New York 1988; p. 518.

Phase-locking in auditory nerve fibers

Frequency and time in the auditory nerve

.

0

50

100

.1 1 10

Cochlea

Frequency(kHz)

Thresholdsound pressurelevel (dB SPL)

Threshold tuning curvesfor discharge rate

1

Peristimulus time (ms)

0 5 10 15 20 25

Stimulus waveform

Peristimulus time histograms

(100 presentations at 60 dB SPL)

Fundamental period 1/F0Formant period 1/F1

Single-formant vowel

10

.


10

1

0 5010 20 30 40

Phase-locking of discharges in the auditory nerveCat, 100x @ 60 dB SPL

Temporal codingin the auditory nerve

Work with Bertrand DelgutteCariani & Delgutte (1996)

Dial-anesthetized cats.100 presentations/fiber60 dB SPL

Population-interval distributions are compiled by summing together intervals from all auditory nerve fibers.

The most common intervals present in the auditory nerve are invariably related to the pitches heard at the fundamentals of harmonic complexes.

Est. Error: 1.9%Est. Error: 0.4%

1000 50 10050

Interspike interval (ms) Interspike interval (ms)

16 Hz5743 intervals

32 Hz467 intervals

Temporal period estimation for an LGN unit(sinusoidal luminance modulation)

20

40

60Rate tuning curve

Frequency (Hz)10 10010.1

Rate(spikes/s)

Raw spike train data courtesy of Andrzej Przybyszewski & Dan Pollen

Phase-locking in visual thalamus (LGN)

Stimuli:Drifting sinusoidalgratings

A B

0 150

red

magenta

cyan

green

yellow

blue

neutral

Time (ms)

5000trolands

• The psychophysical effect.A. Rotating disk patterns used by Fechner,Helmholtz, and Benham to induce subjective colors.B. Temporally-modulated illumination patternsproduce characteristic flicker colors (Festinger, Allyn& White, 1973).

• Electrical stimulation of the retina using similarpatterns produces phosphenes of the correspondingcolors (Young, 1977).

• Wavelength-dependent temporal dischargepatterns have been found in the optic nerve and inhigher visual centers by Granit(1943), Kozak &Reitboeck (1974), and others.

• Implications for neural coding. The existence ofthe psychophysical effect for both natural andelectrical stimulation and its correlate in temporaldischarge patterns of visual neurons suggests thepossibility of a central temporal code for color.

A temporal code for color?

Color vision

NTS

Chordatympani

NTS

Chordatympani

NaCl Quinine HCl

HClSucrose

Temporal coding of taste

Figure 12. Typical temporal patterns of response to four different stimuli recordedfrom the chorda typani nerve and the NTS. The stimuli were: 0.1M NaCl, 0.1Mquinine HCl, 0.5M sucrose and 0.1M HCl. Time markers indicate 5 sec intervals. Thechorda tympani recordings are from the whole nerve, NTS recordings are from singleneurons. The tracings of relative spike frequency shown in this figure were obtainedduring neural recording sessions by the output of amplified neural activity through aspike amplitude window discriminator to a counting rate meter, the output of whichwas displayed on a Brush pen writer. From: Ellen Covey, Temporal Neural Coding of Gustation (1980), Ph.D. thesis, Duke University.

NTS

Chordatympani

NTS

Chordatympani

NaCl Quinine HCl

HClSucrose

Temporal coding of taste

Figure 12. Typical temporal patterns of response to four different stimuli recordedfrom the chorda typani nerve and the NTS. The stimuli were: 0.1M NaCl, 0.1Mquinine HCl, 0.5M sucrose and 0.1M HCl. Time markers indicate 5 sec intervals. Thechorda tympani recordings are from the whole nerve, NTS recordings are from singleneurons. The tracings of relative spike frequency shown in this figure were obtainedduring neural recording sessions by the output of amplified neural activity through aspike amplitude window discriminator to a counting rate meter, the output of whichwas displayed on a Brush pen writer. From: Ellen Covey, Temporal Neural Coding of Gustation (1980), Ph.D. thesis, Duke University.

Temporal codingof taste

time

A1 B1 C1 D1 A2 B2 C2 D2 E2

cycle 2, slots A-Ecycle 1, slots A-E

E1

A. Time-division multiplexing in telephony (signals A-E)

B. Time-division multiplexing in neural systems

time

Scanning Synchrony

Sortingbysynchronousactivation

Featuredetector

units(neural

channels)

time

D. Code-division multiplexing in neural systems

AZ1:9808 DX1:8708 CV2:071389 DX2: 708773CV1:98458 BY2:8708 CV3:134389 AZ4:70767887

BY1:438074 AZ2:4329633 EW3:23342345EW1:43741541 BY3:432963323 EW2:2334245

C. Code-division multiplexing in telephony or computer networks

Sortingbyheader

ReceiversW-Z

Senders A-ESegments1-4

Sortingbytimepattern

Neural assemblyreceivers

Five primitivetime patterns

Neural timing nets

FEED-FORWARD TIMING NETS• Temporal sieves• Extract (embedded) similarities• Multiply autocorrelations

RECURRENT TIMING NETS• Build up pattern invariances• Detect periodic patterns• Separate auditory objects

Relative delay τ

Timet

Sj(t)

Si(t)

Si(t) Sj(t - τ)

individualmultiplicative

term

Si(tm) Sj(tm - t)Στ

convolutiontime-series

term m

two setsof input

spike trains

In p u t t im e s e q u e n c e

A l l t im e d e la y s p r e s e n t

τ0

τ1

τ2

τ3

C o in c id en c e

u n its

D irec t in p u ts

R ec u r ren t ,

in d irec t in p u ts

T im e p attern s rev erb erate

th ro u g h d elay lo o p s

Potential advantages of temporal pattern pulse codes & timing nets

• Multiplexed signal transmission

• Orthogonality of patterns; less interference

• Flexible multimodal integration

• Encoding of signal identity in itself (logical type)

• Liberate signals from wires

• Broadcast of signals + selective reception

• Nonlocal computational operations

• Mass action (statistical representations)

• Open-ended creation of new signal primitives

In the image of the digital computer, we conceptualize brains as distributed logic machines.

However, temporal correlation machines may prove to be a better metaphor.Temporal expectancies in perceptionTemporal patterning of body processesTemporal structure of movementTemporal expectations and reward structure

(dopamine system, conditioning) Temporal memory traces

Music may have the profound effects that it does because 1) it directly impresses its temporal structure on the activity of many neuronal populations, and 2) the neural codes & computations underlying experience are inherently temporal.

Music, brain, and time

Andy Partridge, xtc

Structural complexity alone is not sufficientPure computation alone is not sufficient

RequisitesSensors & effectorsMixed digital-analog designFeedback to structure, self-productionInheritable, replicable (digital) plansCombinatorics of digital stringsRich analog, ill-defined dynamicsGoal states and steering/selection mechanisms

Possibility of brain as temporally-coded self-organizing system

Conclusions Design principles for self-organizing systems

Temporal coding of sensory information.

Interspike interval (ms)

Pitch period


Pitch period

0 255 10 15 20

From cochlea to cortex

Primaryauditory cortex

(Auditory forebrain)

Auditory thalamus

Inferior colliculus(Auditory midbrain)

Auditory brainstem

Cochlea

Auditory nerve (VIII)

Lateral lemniscus

# spikes

Phase-locking to a 300 Hz pure tone

Evans, 1982

Period histogram (1100 Hz)

First-order interval histogram (1500 Hz)

.


10

1

0 5010 20 30 40

Auditory nerve

LowCF

HighCF

F0

F1

F2

F3


Time domain analysis of auditory-nerve fiber firing rates.Hugh Secker-Walker & Campbell Searle, J. Acoust. Soc. 88(3), 1990Neural responses to /da/ @ 69 dB SPL from Miller and Sachs (1983)

Vowel Formant Regions

Populationinterval

distribution

Interspikeinterval (ms)

150

Six stimuli that produce a low pitch at 160 Hz

Pure tone 160 Hz

AM tone Fm:160 Hz

Fc:640 Hz

Harms 6-12

AM tone

Click train

AM noise

F0 :160 Hz

Fm :160 Hz

Fc:6400 Hz

Fm :160 Hz

Waveform Power spectrum Autocorrelation

Pitch frequency Pitch period

F0 : 160 Hz

Lag (ms)Frequency (Hz)TIme (ms)150

WEAKPITCHES

STRONGPITCHES

Burst length, interburst interval

Higher-order interval pattern

Multiplexed intervals

Interspike interval code

PST or latency pattern

Spike latency

reference times

Interneural synchrony

Average discharge rateRate-

channelcodes

Neural pulse codes

Temporalpatterncodes

Time-of-arrival

codes

Codes are defined in termsof their functional roles

What constitutes a difference that makes a difference?

What spike train messages have the same meanings?(functional equivalence classes)

Temporal codes are neuralcodes in which timings ofspikes relative to each otherare essential to theirinterpretation.

global workspace

highersemantic

resonances

perceptualresonances

sensoryinputs

motoroutputs

ENVIRONMENT

controlledvariables

(consequencesof actions)

uncontrolledvariables(contingent events)

motorpreparations

pragmaticevaluations

deliberationplanning

long-termmemory

Primary sensorypathways

Unimodaland multimodalassociationareas

Motor systems

memory

memory

memory

memory

Limbic & paralimbic areas

Frontal cortical areas

Neural resonances

40

50

40

10

0 1000 2000 3000 4000 5000

20

1000

2000

1000

200

0 100 200 300 400 5000

50

All-order interval (ms)Peristimulus time (ms)

PST Histograms Interval Histograms

4 Hz

8 Hz

16 Hz

32 Hz

64 Hz

Phase-locking of an LGN unit to adrifting sinusoidal gratingTemporal

modulationfrequency


• Depending upon the self-modification process, adaptive systems change in different ways.

• They become tuned to their environments,on the percepton the action sideinternally: anticipating events, forecasting effects

• New sensors create new linkages with the external world new perceptual primitivesnew observablesnew modes of adjustment

• New effectors create new modes of action

Sensing ~ measurement

Adaptation ~ adjustmentAdaptive systems

A B

Resonantstate | A

Resonantstate | B

Contingentsensory input

Epistemiccut

S2

S3

S1

R2

R3

R1

CNS

Potentialstimuli

Potentialresponses

A BStimulus-driven switching betweenreverberant circuits (after Hebb, 1965)

Reverberant patternsas switchable eigenstates

Motorresponse

for A

Motorresponse

for B

Switching between reverberant states

receptor surfaces

Evaluation in termsof basic system-goals

(limbic system)

Evaluation in termsof manifold implications

(associations, planscognitive schemas)

Buildup ofsensoryimages

Sensorytransduction

Structure ofenvironmental

events

Attentionalfacilitationof imageformation

Attentionalfacilitationof imageformation

self-sustainingpatterns

Earlysensorycodng

Functional organization of the perceptual side

Frequency ranges of (tonal) musical instruments

27 Hz 4 kHz262Hz

440Hz

880Hz

110Hz

10k86543210.50.25

Measurement and tuning

Measurementmediates interactions with external world

permitting behavior contingent upon

perception

Tuninginvolves adjustment of internal relations to

external relations, i.e. adaptive resonance

Adaptive systems that create their own measurementsare possible (we may be such systems)

It is possible to envision brains and minds as resonant systems that operate on patternsrather than coupled via energetic relations

Areas of self-modifying media

Self-modifying computersCoevolution between humans and computersEmergent human-machine couplingsPask’s Conversation theoryComputers need means of independently

accessing the world and creating theirown concepts (epistemic autonomy)

Self-organizing materialsElectrochemicalFerromagneticBiological-silicon interfacesIntelligent materials

Mixed digital-analog feedback systems

40

50

40

10

0 1000 2000 3000 4000 5000

20

1000

2000

1000

200

0 100 200 300 400 5000

50

All-order interval (ms)Peristimulus time (ms)

PST Histograms Interval Histograms

4 Hz

8 Hz

16 Hz

32 Hz

64 Hz

Phase-locking of an LGN unit to adrifting sinusoidal gratingTemporal

modulationfrequency


Est. Error: 1.9%Est. Error: 0.4%

1000 50 10050

Interspike interval (ms) Interspike interval (ms)

16 Hz5743 intervals

32 Hz467 intervals

Temporal period estimation for an LGN unit(sinusoidal luminance modulation)

20

40

60Rate tuning curve

Frequency (Hz)10 10010.1

Rate(spikes/s)


Phase-locking in visual thalamus (LGN)

Stimulus:Drifting sinusoidalgratings

Phase-locking in visual neurons(Horseshoe crab ommatidium)

Miller, Ratliff, and Hartline (1961) How cells receive stimuli.Scientific American 215(3):222-238.

Neural timing nets

FEED-FORWARD TIMING NETS• Temporal sieves• Extract (embedded) similarities• Multiply autocorrelations

RECURRENT TIMING NETS• Build up pattern invariances• Detect periodic patterns• Separate auditory objects

Relative delay τ

Timet

Sj(t)

Si(t)

Si(t) Sj(t - τ)

individualmultiplicative

term

Si(tm) Sj(tm - t)Στ

convolutiontime-series

term m

two setsof input

spike trains

In p u t t im e s e q u e n c e

A l l t im e d e la y s p r e s e n t

τ0

τ1

τ2

τ3

C o in c id en c e

u n its

D irec t in p u ts

R ec u r ren t ,

in d irec t in p u ts

T im e p attern s rev erb erate

th ro u g h d elay lo o p s

Vowel [er]F0 = 125 HzPeriod = 8 ms

Period = 10 ms

Vowel [ae]F0 = 100 Hz

Build-up and separation of two auditory objects

Two vowels with different fundamental frequencies (F0s) are added together and passed through the simple recurrent timing net. The two patterns build upIn the delay loops that have recurrence times that correspond to their periods.

.

Time (ms)

0 10 20 30 40 50Time (msec)

Methodological issues:What distinguishes sensing from other kinds

of informational operations?

A sensing process must be contingent, it must have two or more possible outcomes to reduce uncertainty, whereas

A computation (formal operation) must be logically-determined, it must always produce the same outcome given the same initial state

Sensing vs computingContingent vs. logically-necessary “truths”

•Digital computers presently are capable of recombination-based creativity, but do not presently create new primitives for themselves.

• Brains, on the other hand, are self-modifying systems with rich analog dynamics that can serve as substrates for formation of new informational primitives.

• Contemplation of self-modifying systems is essential if we are to construct artificial systems that can create meaning for themselves.

• We need such systems when problems are ill-defined, or when we desire open-ended creative possibilities.

Computers and brains

global workspace

highersemantic

resonances

perceptualresonances

sensoryinputs

motoroutputs

ENVIRONMENT

controlledvariables

(consequencesof actions)

uncontrolledvariables(contingent events)

motorpreparations

pragmaticevaluations

deliberationplanning

long-termmemory

Primary sensorypathways

Unimodaland multimodalassociationareas

Motor systems

memory

memory

memory

memory

Limbic & paralimbic areas

Frontal cortical areas

Neural resonances

• We discuss the semiotics and functional organization of different adaptive systems.

• Adaptive systems reorganize their internal structure in order to improve their performance.

• We consider how systems with sensors, effectors, and coordinative faculties can adaptively modify their internal structures and functions.

• We consider how this adaptivity leads to emergent functions and behaviors.

Overview I: Measurement in adaptive systems

• Creativity has two levels: 1) Recombination of existing primitives

2) De novo creation of new kinds of primitives

• Inherent tradeoffs:

Specifiability vs. autonomy

Predictability/reliability vs. creativity

Overview IV: Creativity, autonomy, and specification

Homeostat

Grey Walter's device

Conceptions of “emergence”

• Structural emergence (appearance of new structures, org. levels)

• Computational emergence (unexpected results)

• Thermodynamic emergence (dissipative systems)

• Functional emergence (flight, color vision)

• Emergence-relative-to-a-model (perspectivist, operationalist)

• Appearance of new structures, functions, behaviors

• Novelty that was not predictable from what came before

Varieties

Methodological issues

• How can we identify the existence of information processing operations in artificial and natural systems?

•How can we distinguish measurement, computation, and effector operations from each other in an unknown material system?

•How can we detect changes in these functionalities, such that we know that our devices or organisms have modified them adaptively?

•We need operational distinctions.•We need to be able to parse a state-transition graph.

Recognizing determinate & contingent events

State-transitionsand

observer-operations

How do we distinguishmeasurementsand computations(such that wecan alsodetect changesin system behavior)?

referencestate

R1

A

B

two or morepossible

outcomes

Measurement Computation Prediction

R2

C

D

PA =

Test

Epistemic cuts(points of contingency)

PB

observed"pointerreading"

B

Emergence relative to an observer:What does the observer have to do to his/her own model to continue successfully predicting the material system’s behavior?

ENVMODELobservablessensorsPredictability Deviation

Recovery ofpredictability

new sensor evolves in devicestability stability

Observer

Organismor

Device

Evolution of observer's modelV2V1Original state spaceAdd new states to V2(no new state variables)V1V2V3Addition ofnew statevariable V3semantic

emergence

V2V1larger Nwithinsame D

larger Nwithinlarger D

syntacticemergence

same Nwithinsame D,differentstate-transitions

N = number of statesD = dimensionality of state space

Opening up the sensory interface:

Break-out strategies for creating new observables

1) construction of new sensors

2) modification of existing sensors

3) interposition of sensory prostheses

4) active measurements

5) creation of new internal sensors

coordinationfeature vector

performance

adaptiveconstruction of a

sensoryprosthesis

∆existingsensors

action vector

environment

test

structuraldevice

boundary

functionaldevice

boundary

prosthesis

Prosthesis:augmentation of functionalities

All technology is prosthesis.

Operational states and procedures in a scientific model

sequence of statesfollowing initial state A

sequence of states after B

referencestateR

1

A

Btwo or morepossible outcomes("pointer readings")

MeasurementComputation

Prediction

R2

C

D

PA =

Test

Epistemic cuts(points of contingency)

PB

Explicate realm of symbols (well-defined)

Implicate realm of material process

(ill-defined)

R

sequence of computationswith initial state A

sequence of computationswith initial state B

referencestate A

B

actions based on theoutcome of the computations

(contingent upon A)

actions based on theoutcome of the computations

(contingent upon B)two or more

possible outcomes("pointer readings")

R1

R2

R3

different preparedreference states

result in differentmeasurements

intentions to make measurementsR, R1, R2, or R3 (motor commands)

physical actions taken to bring thesystem into reference statesR1, R2 or R3 (motor actions)

(R is the passive reference state,without any active preparation)

R1

R2

R3

Computation

"preparingthe system"

Measurement Action

Active measurement

coordinationdiscretefeatures

test

performance

adaptiveconstruction ofnew sensors

∆internalsensors

action vector

environment

structural boundaryof device

internaliconic analog

representations

analog-digitalboundary

Neural assemblies as internal sensors

Design principles for adaptive self-organizing systems Finding Fluid Form Symposium University of Brighton December 9-10, 2005 Peter Cariani .

Documents

system slide

trajectory slide

creative emergence slide

permutations unbounded

material world sensing

worse philosophy slide

auditory system

material world need