Abstract: This overview of the free energy principle offers an
account of embodied exchange with the world that associates
conscious operations with actively inferring the causes of our
sensations. Its agenda is to link formal (mathematical)
descriptions of dynamical systems to a description of perception in
terms of beliefs and goals. The argument has two parts: the first
calls on the lawful dynamics of any (weakly mixing) ergodic system
from a single cell organism to a human brain. These lawful dynamics
suggest that (internal) states can be interpreted as modelling or
predicting the (external) causes of sensory fluctuations. In other
words, if a system exists, its internal states must encode
probabilistic beliefs about external states. Heuristically, this
means that if I exist (am) then I must have beliefs (think). The
second part of the argument is that the only tenable beliefs I can
entertain about myself are that I exist. This may seem rather
obvious; however, if we associate existing with ergodicity, then
(ergodic) systems that exist by predicting external states can only
possess prior beliefs that their environment is predictable. It
transpires that this is equivalent to believing that the world and
the way it is sampled will resolve uncertainty about the causes of
sensations. We will conclude by looking at the epistemic behaviour
that emerges under these beliefs, using simulations of active
inference.Key words: active inference autopoiesis cognitive
dynamics free energy epistemic value self-organization.
I am therefore I think Karl Friston, University College
London
Free Energy Workshop 2015
OverviewThe statistics of lifeMarkov blankets and ergodic
systemsSimulations of a primordial soupI am therefore I think
The anatomy of inferenceGraphical models and predictive
codingCanonical microcircuitsBirdsong and generalized synchrony
Active inferenceMinimizing expected uncertaintySaccadic searches
and salience
Anatomy of choiceMarkov decision processesMinimizing expected
free energyDopamine and precision
Knowing your place Multi-agent gamesMorphogenesis
How can the events in space and time which take place within the
spatial boundary of a living organism be accounted for by physics
and chemistry? (Erwin Schrdinger 1943)
The Markov blanket as a statistical boundary (parents, children
and parents of children)Internal states External statesSensory
statesActive statesThe Markov blanket in biotic systems
Active states
External statesInternal statesSensory states
The Fokker-Planck equation
And its solution in terms of curl-free and divergence-free
componentslemma: any (ergodic random) dynamical system (m) that
possesses a Markov blanket will appear to engage in active
inference
5But what about the Markov blanket?Reinforcement learning,
optimal control and expected utility theory
Information theory and minimum redundancy
Self-organisation, synergetics and homoeostasis
Bayesian brain, active inference and predictive coding
Value
Surprise
Entropy
Model evidence
PavlovHakenHelmholtz
Barlow
PerceptionAction
OverviewThe statistics of lifeMarkov blankets and ergodic
systemsSimulations of a primordial soupI am therefore I think
The anatomy of inferenceGraphical models and predictive
codingCanonical microcircuitsBirdsong and generalized synchrony
Active inferenceMinimizing expected uncertaintySaccadic searches
and salience
Anatomy of choiceMarkov decision processesMinimizing expected
free energyDopamine and precision
Knowing your place Multi-agent gamesMorphogenesis
Position
Simulations of a (prebiotic) primordial soupWeak electrochemical
attractionStrong repulsionShort-range forces
ElementAdjacency matrix2040608010012020406080100120Markov
BlanketHidden statesSensory statesActive statesInternal states
Markov Blanket = [B [eig(B) > ]]Markov blanket matrix:
encoding the children, parents and parents of childrenFinding the
(principal) Markov blanketADoes action maintain the structural and
functional integrity of the Markov blanket (autopoiesis) ?
Do internal states appear to infer the hidden causes of sensory
states (active inference) ?
Autopoiesis, oscillator death and simulated brain
lesionsDecoding through the Markov blanket and simulated brain
activation100200300400500-0.4-0.3-0.2-0.10TimeMotion of external
stateTrue and predicted motion
-505-8-6-4-20468PositionPositionPredictability2
TimeModesInternal states10020030040050051015202530
Christiaan Huygens
The existence of a Markov blanket necessarily implies a
partition of states into internal states, their Markov blanket
(sensory and active states) and external or hidden states.
Because active states change but are not changed by external
states they minimize the entropy of internal states and their
Markov blanket. This means action will appear to maintain the
structural and functional integrity of the Markov blanket
(autopoiesis).
Internal states appear to infer the hidden causes of sensory
states (by maximizing Bayesian evidence) and influence those causes
though action (active inference)
Interim summary
res extensa (extensive flow)res cogitans (beliefs)Belief
productionFree energy functionalI am [ergodic] therefore I
think
OverviewThe statistics of lifeMarkov blankets and ergodic
systemsSimulations of a primordial soupI am therefore I think
The anatomy of inferenceGraphical models and predictive
codingCanonical microcircuitsBirdsong and generalized synchrony
Active inferenceMinimizing expected uncertaintySaccadic searches
and salience
Anatomy of choiceMarkov decision processesMinimizing expected
free energyDopamine and precision
Knowing your place Multi-agent gamesMorphogenesis
Objects are always imagined as being present in the field of
vision as would have to be there in order to produce the same
impression on the nervous mechanism - von Helmholtz
Thomas BayesGeoffrey HintonRichard FeynmanThe Helmholtz machine
and the Bayesian brainRichard Gregory
Hermann von Helmholtz Objects are always imagined as being
present in the field of vision as would have to be there in order
to produce the same impression on the nervous mechanism - von
Helmholtz Richard Gregory
Hermann von Helmholtz Impressions on the Markov blanket
Bayesian filtering and predictive coding
prediction update
prediction error
Making our own sensations
Changing sensationssensations predictionsPrediction
errorChanging predictionsActionPerception
the
DescendingpredictionsAscending prediction errors
A simple hierarchy
whatwhereSensory fluctuations
Hierarchical generative models
frontal eye fieldsgeniculatevisual cortexretinal
inputponsoculomotor signalsTop-down or backward
predictionsBottom-up or forward prediction errorproprioceptive
inputreflex arcPerception
David MumfordPredictive coding with reflexesAction
Prediction error (superficial pyramidal cells)Expectations (deep
pyramidal cells)
Biological agents minimize their average surprise (entropy)
They minimize surprise by suppressing prediction error
Prediction error can be reduced by changing predictions
(perception)
Prediction error can be reduced by changing sensations
(action)
Perception entails recurrent message passing to optimize
predictions
Action makes predictions come true (and minimizes surprise)
Interim summary
OverviewThe statistics of lifeMarkov blankets and ergodic
systemsSimulations of a primordial soupI am therefore I think
The anatomy of inferenceGraphical models and predictive
codingCanonical microcircuitsBirdsong and generalized synchrony
Active inferenceMinimizing expected uncertaintySaccadic searches
and salience
Anatomy of choiceMarkov decision processesMinimizing expected
free energyDopamine and precision
Knowing your place Multi-agent gamesMorphogenesis
ThalamusArea X
Higher vocal centreHypoglossal Nucleus
creating your own sensations: Listening
time (sec)Frequency
(Hz)sensations0.20.40.60.811.21.41.61.8250030003500400045005000
ThalamusArea X
Higher vocal centreHypoglossal Nucleus
creating your own sensations: SingingMotor commands
(proprioceptive predictions)Corollary discharge(exteroceptive
predictions)
time (sec)Frequency
(Hz)sensations0.20.40.60.811.21.41.61.8250030003500400045005000
time (sec)Frequency
(Hz)percept1234567250030003500400045005000012345678-50050100time
(seconds)First level expectations (hidden
states)012345678-40-20020406080time (seconds)Second level
expectations (hidden states)Singing alone
time (sec)Frequency
(Hz)percept1234567250030003500400045005000012345678-50050100time
(seconds)First level expectations (hidden
states)012345678-40-20020406080time (seconds)Second level
expectations (hidden states)Singing together
-20-100102030405060-20-100102030405060Synchronizationsecond
level expectations (first bird)second level expectations (second
bird)-20-100102030405060-30-20-1001020304050No
synchronizationsecond level expectations (first bird)second level
expectations (second bird)Mutual prediction and synchronization of
chaossynchronization manifold
OverviewThe statistics of lifeMarkov blankets and ergodic
systemsSimulations of a primordial soupI am therefore I think
The anatomy of inferenceGraphical models and predictive
codingCanonical microcircuitsBirdsong and generalized synchrony
Active inferenceMinimizing expected uncertaintySaccadic searches
and salience
Anatomy of choiceMarkov decision processesMinimizing expected
free energyDopamine and precision
Knowing your place Multi-agent gamesMorphogenesis
saliencevisual inputstimulussamplingPerception as hypothesis
testing saccades as experiments
Sampling the world to minimise expected uncertainty
I am [ergodic] therefore I think I think therefore I am
[ergodic]
LikelihoodEmpirical priorsPrior beliefs
Frontal eye fields
Pulvinar salience mapFusiform (what)Superior colliculusVisual
cortexoculomotor reflex arc
Parietal (where)
30
Visual samplesConditional expectations about hidden (visual)
statesAnd corresponding perceptSaccadic eye movementsHidden
(oculomotor) states
Saccadic fixation and salience maps
200400600800100012001400-202Action (EOG)time
(ms)200400600800100012001400-505Posterior belieftime (ms)
vs.vs.
OverviewThe statistics of lifeMarkov blankets and ergodic
systemsSimulations of a primordial soupI am therefore I think
The anatomy of inferenceGraphical models and predictive
codingCanonical microcircuitsBirdsong and generalized synchrony
Active inferenceMinimizing expected uncertaintySaccadic searches
and salience
Anatomy of choiceMarkov decision processesMinimizing expected
free energyDopamine and precision
Knowing your place Multi-agent gamesMorphogenesis
Generative modelsHidden states
Action
Control states
Continuous statesDiscrete statesBayesian filtering (predictive
coding)Variational Bayes(belief updating)
Full priors control statesEmpirical priors hidden
statesLikelihoodThe (normal form) generative modelHidden states
Action
Control states
KL or risk-sensitive control
In the absence of ambiguity:Expected utility theory
In the absence of uncertainty or risk:Bayesian surprise and
Infomax
In the absence of prior beliefs about outcomes:
Prior beliefs about policiesQuality of a policy = (negative)
Expected free energy
Extrinsic valueEpistemic value, information gain or reduction of
expected uncertaintyPredicted divergenceExtrinsic valueBayesian
surprisePredicted mutual information
midbrainmotor Cortexoccipital Cortexstriatum
Variational updatesPerception
Action selection
Incentive salienceFunctional anatomy
prefrontal Cortex
hippocampus
00.20.40.60.81020406080PerformancePrior preferencesuccess rate
(%) FEKLRLDASimulated behaviour
Simulating conditioned responses and dopamine release (precision
updates)
OverviewThe statistics of lifeMarkov blankets and ergodic
systemsSimulations of a primordial soupI am therefore I think
The anatomy of inferenceGraphical models and predictive
codingCanonical microcircuitsBirdsong and generalized synchrony
Active inferenceMinimizing expected uncertaintySaccadic searches
and salience
Anatomy of choiceMarkov decision processesMinimizing expected
free energyDopamine and precision
Knowing your place Multi-agent gamesMorphogenesis
Generative modelIntercellular signalingIntrinsic
signalsExogenous signalsEndogenous signals
Knowing your place: multi-agent games and morphogenesis
-4-2024-4-3-2-101234Extracellular target signalposition
(Genetic) encoding of target morphologyPosition (place code)
Signal expression (genetic code)
-4-2024-4-3-2-101234Target formposition
05101520253035-4-3-2-101234Morphogenesistimelocation-4-2024-4-3-2-101234Solutionlocation51015202530-420-400-380-360-340Free
energytimeFree energy
Softmax
expectationscellcell123456781234567851015202530-1.5-1-0.500.511.522.53Expectationstime51015202530-2-1.5-1-0.500.511.522.5timeAction
Morphogenesis
DysmorphogenesisGradientGradientIntrinsicExtrinsicExtrinsicTARGET
Regeneration of
head05101520253035-4-3-2-101234morphogenesistimelocation-4-2024-4-3-2-101234location05101520253035-4-3-2-101234morphogenesistimelocationRegeneration
of tailRegeneration
Each movement we make by which we alter the appearance of
objects should be thought of as an experiment designed to test
whether we have understood correctly the invariant relations of the
phenomena before us, that is, their existence in definite spatial
relations.
The Facts of Perception (1878) in The Selected Writings of
Hermann von Helmholtz,Ed.R. Karl, Middletown: Wesleyan University
Press, 1971 p. 384
Hermann von Helmholtz And thanks to collaborators:
Rick AdamsRyszard AuksztulewiczAndre BastosSven BestmannHarriet
BrownJean DaunizeauMark EdwardsChris FrithThomas FitzGeraldXiaosi
GuStefan KiebelJames KilnerChristoph MathysJrmie MattoutRosalyn
MoranDimitri OgnibeneSasha Ondobaka Will PennyGiovanni PezzuloLisa
Quattrocki KnightFrancesco Rigoli Klaas StephanPhilipp
Schwartenbeck
And colleagues:
Micah AllenFelix BlankenburgAndy ClarkPeter DayanRay DolanAllan
HobsonPaul FletcherPascal FriesGeoffrey HintonJames HopkinsJakob
HohwyMateus JoffilyHenry KennedySimon McGregorRead MontagueTobias
NolteAnil SethMark SolmsPaul Verschure
And many othersThank you