Resurrecting Laplace's Demon: The Case for Deterministic ... · God was playing dice, in Einstein’s phrase. But science snatched victory from the jaws of defeat by moving the goal

Resurrecting Laplace's Demon: The Case for Deterministic Models

Edward A. Lee Robert S. Pepper Distinguished Professor UC Berkeley Invited Talk: Synchron December 8, 2016 Bamberg, Germany

Notjustinforma.ontechnology:•  Cyber+Physical•  Computa/on+Dynamics•  Security+SafetyProper.es:•  Highlydynamic•  Safetycri/cal•  Uncertainenvironment•  Physicallydistributed•  Sporadicconnec/vity•  Resourceconstrained

Doesitmakesensetotalkaboutdeterminis7cmodelsforsuchsystems?

Automotive

Context:Cyber-PhysicalSystemsApar/cularlychallengingcasefordeterminism

Biomedical

Military

Energy

Manufacturing

Avionics

Buildings

2 Lee, Berkeley

Modelsvs.Reality

Inthisexample,themodelingframeworkiscalculusandNewton’slaws.Fidelityishowwellthemodelanditstargetmatch

Lee, Berkeley 3

The model

The target (the thing being modeled).

Solomon Wolf Golomb

You will never strike oil by drilling through the map!

Lee, Berkeley 4

Engineers often confuse the model with its target

But this does not in any way diminish the value of a map!

Determinacy

Someofthemostvaluablemodelsaredeterminis7c.

Amodelisdeterminis7cif,giventheini/alstateandtheinputs,themodeldefinesexactlyonebehavior.

Determinis/cmodelshaveprovenextremelyvaluableinthepast.

Lee, Berkeley 5

Laplace’sDemon

“Wemayregardthepresentstateoftheuniverseastheeffectofitspastandthecauseofitsfuture.Anintellectwhichatacertainmomentwouldknowallforcesthatsetnatureinmo/on,andallposi/onsofallitemsofwhichnatureiscomposed,ifthisintellectwerealsovastenoughtosubmitthesedatatoanalysis,itwouldembraceinasingleformulathemovementsofthegreatestbodiesoftheuniverseandthoseofthe/niestatom;forsuchanintellectnothingwouldbeuncertainandthefuturejustlikethepastwouldbepresentbeforeitseyes.”— PierreSimonLaplace

Lee, Berkeley 6

Pierre-Simon Laplace (1749–1827). Portrait by Joan-Baptiste Paulin Guérin, 1838

Didquantummechanicsdashthishope?

“Atfirst,itseemedthatthesehopesforacompletedeterminismwouldbedashedbythediscoveryearlyinthe20thcenturythateventslikethedecayofradioac/veatomsseemedtotakeplaceatrandom.ItwasasifGodwasplayingdice,inEinstein’sphrase.Butsciencesnatchedvictoryfromthejawsofdefeatbymovingthegoalpostsandredefiningwhatismeantbyacompleteknowledgeoftheuniverse.”(StephenHawking,2002)

Lee, Berkeley 7

Nevertheless,Laplace’sDemoncannotexist.

In2008,DavidWolpert,thenatNASA,nowattheSantaFeResearchIns/tute,usedCantor’sdiagonaliza/ontechniquetoprovethatLaplace’sdemoncannotexist.Hisproofreliesontheobserva/onthatsuchademon,wereittoexist,wouldhavetoexistintheveryphysicalworldthatitpredicts.

Lee, Berkeley 8

David Wolpert

The Koptez Principle

Many properties that we assert about systems (determinism, timeliness, reliability) are in fact not properties of the system, but rather properties of a model of the system. If we accept this, then it makes no sense to talk about whether the physical world is deterministic. It only makes sense to talk about whether models of the physical world are deterministic.

Hermann Kopetz Professor (Emeritus) TU Vienna

Theques/onswitchesfromwhetheramodelisTruetowhetheritisUseful

“Essen/ally,allmodelsarewrong,butsomeareuseful.”

Box,G.E.P.andN.R.Draper,1987:EmpiricalModel-BuildingandResponseSurfaces.WileySeriesinProbabilityandSta/s/cs,Wiley.

Lee, Berkeley 10

Physicistscon/nuetodebatewhethertheworldisdeterminis/c

Determinismisapropertyofmodels,notapropertyofthesystemstheymodel.

Lee, Berkeley 11

Deterministic model

Deterministic system?

Schema/cofasimpleCyber-PhysicalSystem

Lee, Berkeley 12

What kinds of models should we use? Let’s look at the most successful kinds of models from the cyber and the physical worlds.

SocwareisaModel

PhysicalSystem Model

Single-threadedimpera7veprogramsaredeterminis7cmodels

Lee, Berkeley 13

Considersingle-threadedimpera/veprograms

Thetargetofthemodeliselectronssloshingaroundinsilicon.Ittakes/me,consumesenergy,andfailsifdroppedintheocean,noneofwhichareproper/esofthemodel.

This program defines exactly one behavior, given the input x. Note that the modeling framework (the C language, in this case) defines “behavior” and “input.”

Lee, Berkeley 14

Socwarereliesonanotherdeterminis/cmodelthatabstractsthehardware

PhysicalSystem Model

Instruction Set Architectures (ISAs) are deterministic models

Lee, Berkeley 15

Image: Wikimedia Commons Waterman, et al., The RISC-V Instruction Set Manual, UCB/EECS-2011-62, 2011

…whichreliesonyetanotherdeterminis/cmodel

PhysicalSystem Model

Synchronousdigitallogicisadeterminis7cmodel

Lee, Berkeley 16

Determinis/cModelsforthePhysicalSideofCPS

PhysicalSystem Model

Signal Signal

Differen7alEqua7onsaredeterminis7cmodels

Lee, Berkeley 17

Image: Wikimedia Commons

Signal Signal

18 Image: Wikimedia Commons Lee, Berkeley

AmajorproblemforCPS:combina/onsofdeterminis/cmodelsarenondeterminis/c

Correct execution of a program in all widely used programming languages, and correct delivery of a network message in all general-purpose networks has nothing to do with how long it takes to do anything.

Programmers have to step outside the programming abstractions to specify timing behavior. CPS designers have no map!

Lee, Berkeley 19

Timing is not part of software and network semantics

AStory

In“flybywire”aircrac,computerscontroltheplane,media/ngpilotcommands.

Abstrac/onLayersAllofwhicharemodelsexceptthebofom

Thepurposeofanabstrac/onistohidedetailsoftheimplementa/onbelowandprovideaplagormfordesignfromabove.

Abstrac/onLayersAllofwhicharemodelsexceptthebofom

Everyabstrac/onlayerhasfailedfortheaircracdesigner.Thedesignistheimplementa/on.

Determinism?Really?

CPSapplica/onsoperateinanintrinsicallynondeterminis/cworld.Doesitreallymakesensetoinsistondeterminis7cmodels?

23 Lee, Berkeley

•  Inscience,thevalueofamodelliesinhowwellitsbehaviormatchesthatofthephysicalsystem.

•  Inengineering,thevalueofthephysicalsystemliesinhowwellitsbehaviormatchesthatofthemodel.

Lee, Berkeley 24

In engineering, model fidelity is a two-way street!

For a model to be useful, it is necessary (but not sufficient) to be able to be able to

construct a faithful physical realization.

TheValueofModels

AModel

Lee, Berkeley 25

APhysicalRealiza/on

Lee, Berkeley 26

ModelFidelity

•  Toascien7st,themodelisflawed.

•  Toanengineer,therealiza/onisflawed.

I’manengineer…

Lee, Berkeley 27

ForCPS,weneedtochangetheques/on

Theques/onisnotwhetherdeterminis/cmodelscandescribethebehaviorofcyber-physicalsystems(withhighfidelity).Theques/oniswhetherwecanbuildcyber-physicalsystemswhosebehaviormatchesthatofadeterminis/cmodel(withhighprobability).

Lee, Berkeley 28

Determinism?Whataboutresilience?Adaptability?

Determinis/cmodelsdonoteliminatetheneedforrobust,fault-tolerantdesigns.

Infact,theyenablesuchdesigns,becausetheymakeitmuchclearerwhatitmeanstohaveafault!

Lee, Berkeley 29

Enter:SynchronousLanguages

•  Determinis/cconcurrency

But:•  Timebetween/cks?•  WCEToverallreac/ons?•  Distributedsystems?

Lee, Berkeley 30

Usefuldeterminis/cmodelsforCPS

Togetdeterminis/cmodelsforCPSwithfaithfulimplementa/ons,wecan:1.  Useprocessorswithcontrollable/ming

(PRETmachines).–  hfp://chess.eecs.berkeley.edu/pret

2.  Extendsynchronouslanguageswitha(superdense)modelof/me–  LeeandZheng,EMSOFT2007

3.  Synchronizeclocksandcreatedistributedreal-/meexecu/on(PTIDES)–  hfp://chess.eecs.berkeley.edu/p/des

Lee, Berkeley 31

Together,thesetechnologiesgiveaprogrammingmodelfordistributedandconcurrentreal-7mesystemsthatisdeterminis7cinthesenseofsingle-threadedimpera7veprograms,andalsodeterminis7cw.r.t.to7mingofexternalinterac7ons.

ExtendingSRtogetDE

•  Timetothenext/ckisdeterminedby/me-stampeddiscreteevents.

•  Ateach/ck,usealeastfixed-pointseman/cs,asusualwithsynchronouslanguages.

Lee, Berkeley 32

EMSOFT 2007

Abstract:Discrete-event(DE)modelsareformalsystemspecifica/onsthathaveanalyzabledeterminis/cbehaviors.Usingaglobal,consistentno/onof/me,DEcomponentscommunicatevia/me-stampedevents.DEmodelshaveprimarilybeenusedinperformancemodelingandsimula/on,where/mestampsareamodelingpropertybearingnorela/onshiptoreal/meduringexecu/onofthemodel.Inthispaper,weextendDEmodelswiththecapabilityofrela/ngcertaineventstophysical/me…

33 Lee, Berkeley

Ptides – A Robust Distributed Deterministic DE MoC

UsingSynchronizedClocksinDistributedSystems:RootsoftheIdea

ACMTransac/onsonProgrammingLanguagesandSystems,1984.Lee, Berkeley 34

GoogleSpanner–AReinven/on

Googleindependentlydevelopedaverysimilartechniqueandappliedittodistributeddatabases.

Lee, Berkeley 35

Proceedings of OSDI 2012

Bound C1 on computation

time Time stamp sensor data

Lee, Berkeley 36

Ptides: Time stamps bind to real time at sensors and actuators

Bound L on network latency

Bound E on clock synchronization

error

An event here with time stamp T can be processed when the local clock exceeds

T+C1+L+E

Bound C2 on computation

time

Logical delay D

Event is delivered to the actuator on time if D ≥ C1+C2+L+E

Determinis/cDistributedReal-Time

Assumeboundson:•  execu7on7me•  clocksynchroniza7onerror•  networklatencytheneventsareprocessedin.me-stamporderateverycomponentandeventsaredeliveredtoactuatorson.me.

Lee, Berkeley 37

See http://chess.eecs.berkeley.edu/ptides

All of the assumptions are achievable with today’s technology, and are requirements anyway for hard-real-time systems. The Ptides model makes the requirements explicit.

Lee, Berkeley 38

So Many Assumptions?

You will never strike oil by drilling through the map!

Violations of the requirements are detectable as out-of-order events and can be treated as faults.

Non-Synchronized Clocks

A fault manifests as out-of-order events.

… after an event here with a later time stamp has been processed, then fault!

If an event arrives here with an earlier time

stamp… Occurrence of a fault implies one or more of the assumptions was violated.

Lee, Berkeley 39

Handling Faults

But…

Determinismhasitslimits.

Lee, Berkeley 40

• Complexity • Uncertainty • Chaos •  Incompleteness

Complexity

•  Somesystemsaretoocomplexfordeterminis/cmodels.

•  Nondeterminis/cabstrac/onsbecomeuseful.

Lee, Berkeley 41

“Iron wing” model of an Airbus A350.

Complexity

•  Somesystemsaretoocomplexfordeterminis/cmodels.

•  Nondeterminis/cabstrac/onsbecomeuseful.

Lee, Berkeley 42

Deep Learning, draft book in preparation, by Yoshua Bengio, Ian Goodfellow, and Aaron Courville. http://www.deeplearningbook.org/

But…

Determinismhasitslimits.

Lee, Berkeley 43

• Complexity • Uncertainty • Chaos •  Incompleteness

Uncertainty

•  Wecan’tconstructdeterminis/cmodelsofwhatwedon’tknow.

•  Forthis,nondeterminismisuseful.

•  Bayesianprobability(whichismostlyduetoLaplace)quan/fiesuncertainty.

Lee, Berkeley 44

Portrait of Reverend Thomas Bayes (1701 - 1761) that is probably not actually him.

But…

Determinismhasitslimits.

Lee, Berkeley 45

• Complexity • Uncertainty • Chaos •  Incompleteness

Determinismdoesnotimplypredictability

Lee, Berkeley 46

Edward Lorenz

Determinismdoesnotimplypredictability

Lee, Berkeley 47

Edward Lorenz

The position of a point is not meaningfully predictable even though the system is deterministic.

Determinismdoesnotimplypredictability

[ThieleandKumar,EMSOFT2015]

Lee, Berkeley 48

But…

Determinismhasitslimits.

Lee, Berkeley 49

• Complexity • Uncertainty • Chaos •  Incompleteness

IncompletenessofDeterminism

Anysetofdeterminis/cmodelsrichenoughtoencompassNewton’slawsplusdiscretetransi/onsisincomplete.Lee,FundamentalLimitsofCyber-PhysicalSystemsModeling,ACMTr.onCPS,Vol.1,No.1,November2016

Lee, Berkeley 50

Illustra/onoftheIncompletenessofDeterminism

Lee, Berkeley 51

Illustra/onoftheIncompletenessofDeterminism

Lee, Berkeley 52

Illustra/onoftheIncompletenessofDeterminism

Lee, Berkeley 53

Illustra/onoftheIncompletenessofDeterminism

Lee, Berkeley 54

Illustra/onoftheIncompletenessofDeterminism

Lee, Berkeley 55

Illustra/onoftheIncompletenessofDeterminism

Lee, Berkeley 56

Illustra/onoftheIncompletenessofDeterminism

Lee, Berkeley 57

Illustra/onoftheIncompletenessofDeterminism

Lee, Berkeley 58

Illustra/onoftheIncompletenessofDeterminism

Lee, Berkeley 59

Illustra/onoftheIncompletenessofDeterminism

Lee, Berkeley 60

Illustra/onoftheIncompletenessofDeterminism

Lee, Berkeley 61

ArbitraryInterleavingYieldsNondeterminism

Lee, Berkeley 62

RecalltheHeisenbergUncertaintyPrinciple

Lee, Berkeley 63

IsDeterminismIncomplete?

•  InLee(2017),IshowthatthissequenceofmodelsisCauchy,sothespaceofdeterminis/cmodelsisincomplete(itdoesnotcontainitsownlimitpoints).

•  InLee(2014),Ishowthatadirectdescrip/onofthisscenarioresultsinanon-construc/vemodel.Thenondeterminismarisesinmakingthismodelconstruc/ve.

Lee, Berkeley 64

Rejec/ngdiscretenessleadstodeterminis/cchaos

Acon/nuousdeterminis/cmodelthatmodelstheballsasspringsischao/c.

Lee, Berkeley 65

Discretebehaviorscannotbeexcludedunlesswealsorejectcausality

Lee, Berkeley 66

Example from Lee, “Constructive Models of Discrete and Continuous Physical Phenomena,” IEEE Access, 2014

Summary

•  Determinis/cmodelsareextremelyuseful.

•  Combiningofourbestdeterminis/ccybermodelsandphysicalmodelstodayyieldsnondeterminis/cmodels.

•  Butdeterminis/cmodelswithfaithfulimplementa/onsexist(inresearch)forcyber-physicalsystems.

•  Determinis/cmodelsaren’talwayspossibleorprac/calduetocomplexity,unknowns,chaos,andincompleteness.

•  Determinismisapowerfulmodelingtool.Useitifyoucan.Backoffonlywhenyoucan’t.

Lee, Berkeley 67

Conclusion

Modelsplayacentralroleinreasoningaboutanddesigningengineeredsystems.Determinismisavaluableandsubtlepropertyofmodels.

Lee, Berkeley 68

Plato and the Nerd On Technology and Creativity

Edward Ashford Lee

MIT Press, 2017

Forthcoming book My first for a general audience