Top Banner
Breaking the Von Neumann Bottleneck And the future of Computing Bruce Damer for the Singularity University July 8, 2009
115

Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Nov 15, 2014

Download

Technology

Bruce Damer

Bruce Damer's talk on the Von Neumann Bottleneck, given to the Singularity University, NASA Ames Research Center, July 8, 2009
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Breaking the Von NeumannBottleneck

And the future of Computing

Bruce Damer for the Singularity UniversityJuly 8, 2009

Page 2: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Understanding what is going to be possible with computing in the future is inseparable with knowing its origins in the past. The birth of modern digital computing in the 1930s through the 1950s has placed an indelible mark on how digital spaces are structured today. The vast majority of computing is locked into the von Neumann architecture, originating from John von Neumann's work on Electronic Computer Project at the Institute for Advanced Study in Princeton, New Jersey six decades ago. Today, challenges in advanced computing where we wish to simulate and observe emergent phenomena that model nature at the quantum, chemical and higher levels run hard up against the insidiously hidden but still built-in von Neumann Bottleneck. Overcoming this bottleneck may simply be a matter of throwing more computers, cores or pipelines at the problem, or it might require a re-thinking of computing architectures and software entirely.

Page 3: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

In this presentation, we will take a whirlwind visual tour of the history of computing, beginning with the Colossus code-breaking machine of World War Two, through Von Neumann’s ECP project, the birth of interactive personal computing in the 1960s and 1970s, the coming of the network in the 1980s and 1990s, 3D virtual worlds and the GPU as a breakaway UI and computing paradigm of the 1990s, and multi-core, grid and cloud computing of the 2000s. We will then take a look at a recently launched visionary computing challenge called the EvoGrid and how it maps onto these architectures and suggest where we might be headed (or not headed).

Page 4: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

The Tour to Come

Where did the Von Neumann Machine Come from?

Why and how did von Neumann’s architecture come to dominate the Computing World?

How Von Neumann Machines May or May Not be able to Compute Nature (in which case concepts like a singularity or other

bio-inspired phenomena in digital technology are off the table)

The EvoGrid: An Origin of Artificial Life Project in the Von Neumann digital space

An Alternate View: Non-Von Neumann digital space: Systemic

Computing

Page 5: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

WWII Bletchley Park CodebreakersThe Colossus: a Non-Von Neumann Digital Computer

Page 6: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

1944 – The Colossus Rebuild

Page 7: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

1946 – Eniac, programming by plug board

Page 8: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 9: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 10: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 11: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Enter John von Neumann

Page 12: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Birth of the First True Digital Computer The Institute for Advanced Study, Princeton NJ - 1946

Page 13: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Von Neumann Architecture

Page 14: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

All Programming done in commoninstruction/data memory (cybernetic feedback)

Page 15: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Architecture of Contingency: Memory Clock

Page 16: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Public Unveiling of Electronic Computer, 1952

Page 17: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Robert OppenheimerJohn von Neumann

Alan Richards photographer.  Courtesy of The Shelby White and Leon Levy Archives Center, Institute for Advanced Study, Princeton, NJ, USA

Page 18: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

The ECP: Compact, fast, stored program,world changing

Page 19: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Forty Williams Tubes for Cache Memory

Page 20: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Programs and data loaded and stored on IBMPunched Cards

Page 21: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

ECP punched card

Page 22: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

ECP Program #3: Experiments in biometric evolution… 1953 report by Nils Baricelli

Page 23: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Baricelli Report & EvoGrid Design

Page 24: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Baricelli “Blueprints”

Page 25: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Baricelli “Blueprints”

Page 26: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Punched Card Output Photo Captures first directvisual output of a program state (memory)

Page 27: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

The world’s first Artifical Life program

Page 28: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Why and How did von Neumann’s architecture come to dominate the

computing world?

Answer: it was open sourced!

Page 29: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Entering the IAS Director’s Files

Page 30: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 31: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 32: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 33: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 34: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 35: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

The meteoric rise and dominance of von Neumann’s architecture

Page 36: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 37: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 38: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Interactive *personal* computing beginsthe LINC - 1962

Page 39: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 40: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 41: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 42: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 43: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

The ARPANET and the creation of the first networked multi-user graphical space – Maze War

1974

Page 44: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Maze War in Action

Page 45: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

The Xerox Alto and the coming of the Graphical User Interface and Networked Computing - 1973

Page 46: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Alto Screen (Draw) courtesy Al Kossow

Page 47: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

The birth of the modern networked computing environment Xerox Star - 1981

Page 48: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Xerox Star: The Coming of the Desktop Metaphor

Page 49: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Star Desktop Environment

Page 50: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Star Desktop EnvironmentThe Explosion of visual interfaces

Page 51: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Apple Lisa, 1983Apple Lisa

Page 52: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Apple Macintosh, 1984Apple Macintosh

Page 53: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

In Walk the Avatars, Lucasfilm Habitat - 1986

Page 54: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Explosion of Social Virtual Worlds platforms(Book Avatars by Damer – 1997)

Page 55: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

AlphaWorld “satellite map” 1995-96

Page 56: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 57: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Elements of Avatars98

Page 58: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Some Large Systems – Cray 1, Q2, LINC, TR-48, S-100 MultiCray Supercomputers: 1970s-80s (Digbarn)

Page 59: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Some Large Systems – Cray 1, Q2, LINC, TR-48, S-100 MultiGrid Supercomputers: 1990s-2000s (NASA ARC)

Page 60: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Some Large Systems – Cray 1, Q2, LINC, TR-48, S-100 MultiToday: Multi-Core Revolution, GPUs, Clouds, Grids

Page 61: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Computer history brought to you by the Digibarn

Page 62: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 63: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

2008: A Grand Scientific Challenge takes on the Von Neumann Bottleneck: The EvoGrid,

an “Origin of Artificial Life”

Page 64: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Origins of Life: Archaean to Cambrian1997: Digital Burgess - quest for life’s algorithmic

origins in the “Cambrian Explosion”, Biota.org

Page 65: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Early exemplar: Karl Sims’ Evolving Virtual Creatures (1991-4)

“Soft” Artificial Life Through the Ages: field named in the 1980s, progress through the

1990s, 2000s

Evolving Virtual Creatures by Karl SimsInspired a generation of Soft Alife developers in the 1990s-2000s

Page 66: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Karl Sims: Evolving Virtual Creatures

Page 67: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Early exemplar: Karl Sims’ Evolving Virtual Creatures (1991-4)

State of the art of “Soft” Artificial Life

Page 68: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

The Dawn of “Wet” ALife Protocells (Monnard, Rasmussen, Bedau et al)

Page 69: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Micelle SimulationExploring Life’s Origins Project (Harvard)

Page 70: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Micelle Division

Page 71: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Real Biology in Action: Ribosomes

Page 72: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Visit to FLiNT: Fundamental Living Technology Laboratory

University of Southern Denmark, Odense

Page 73: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 74: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 75: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 76: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 77: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 78: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 79: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Prebiotic Chemistry Fellermann (FLiNT, Univ Southern Denmark)

Page 80: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Models of Prebiotic Chemistry Monnard: Complexification & Formation of a Protocell

Page 81: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Mesoscale Molecular Simulation Fellermann: Formation of Simulated Membranes

Page 82: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Million Atom Satellite Tobacco Mosaic Virus simulation(NAMD and VMD, University of Illinois

Theoretical and Computational Biophysics Group

Page 83: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Enter the EvoGrid Could Artificial Life arise spontaneously from

Artificial Non-Lifeand could this shed light on

the Origins of Life from Non-Life?

New Book: Divine Action

and Natural Selection Gordon: Hoyle’s ChallengeDamer: The God Detector

Page 84: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

The Vision: EvoGrid The MovieA Thought Experiment - Storyboards

Page 85: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

EvoGrid The Movie

Page 86: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Farther Future Vision of the EvoGridProjecting life into the Solar System

Page 87: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Could we Artificially EvolveFreeman Dyson’s Trees?

Page 88: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Fanciful Concepts of Dyson Trees

Page 89: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Near Earth Objects: Threat or Future of Life in the Solar System?

Page 90: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Design for a human mission to a NEO

Page 91: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 92: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

EvoGrid the Movie: The Asteroid Eaters

Page 93: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

EvoGrid the Movie: The Asteroid Eaters

Page 94: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Building the EvoGrid

Variants

Page 95: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Building the EvoGrid

Ratcheting up the Complexity

Page 96: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Building the EvoGrid

Conceptual architecture

Page 97: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 98: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 99: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 100: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 101: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

EvoGrid Prototype 2009bond formation in GROMACS

Page 102: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

EvoGrid Prototype 2009movie

Page 103: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Final Question: Is digital technology based on the Von Neumann architecture up to the task of the EvoGrid or any substantial bio-inspired

computing as suggested by visions of the Singularity?

Answer: Probably not, but it is worth a try (get ready to work for decades)

Or: do we need to start thinking about Non-Von Neumann digital spaces as Von Neumann did 60 years ago?

Page 104: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Conventional vs Natural ComputationSystemic Computer model by Peter J. Bentley, UCL, Digital Biology Group

Conventional Natural Deterministic Stochastic Synchronous Asynchronous

Serial Parallel Heterostatic Homoestatic

Batch Continuous Brittle Robust

Fault intolerent Fault tolerant Human-reliant Autonomous

Limited Open-ended Centralised Distributed

Precise Approximate Isolated Embodied

Linear causality Circular causality

Table 1 Features of conventional vs Natural computation

Page 105: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Non-living natural world supports a massive number of parallel interactions but they are finite, bounded

Page 106: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Living natural world supports infinitely repeatable computations in a massively parallel fashion

Page 107: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Can this kind of machine do that?

Definitely not

Low level approximations (overhead)

How about a lot of these?

Perhaps… for the equivalent of a small volume of aqueous

chemicals

Page 108: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

You need this…. to originate and evolve complex life (and civilization)

Penny Boston, CONTACT Conference 2009, NASA Ames

Page 109: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

• Self-replication is perhaps one of the simplest forms of repeatable computations, and self-replicating systems in a systemic computer would therefore necessarily require simple metabolisms and the formation of a simple ecology in order for their behaviour to be maintained indefinitely.

• When the natural world is viewed from the perspective of systemic computation, the distinction between living systems and non-living systems becomes immediately clear: life is (or has the potential to be) an infinitely repeatable computation, non-living systems (for example, crystal growth) are bounded or finitely repeated computations.

Systemic Computing model by Peter J. Bentley, UCL, Digital Biology Group

Page 110: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Systemic computation

Everything is a system

Systems may comprise or share other nested systems.

Systems can be transformed but never destroyed

Interaction between systems may cause transformation of those systems, where the nature of that transformation is determined by a contextual system.

All systems can potentially act as context and affect the interactions of other systems, and all systems can potentially interact in some context.

The transformation of systems is constrained by the scope of systems.

Computation is transformation

Page 111: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 112: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009
Page 113: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Thanks EvoGrid External Advisors

•Prof. Richard Gordon, Professor-University of Manitoba•Tom Barbalet, director and community leader-Biota.org•Freeman Dyson-Institute for Advanced Study•Bob Taylor (retired), formerly of DARPA, Xerox PARC, DEC•Steen Rasmussen, Marin Hanczyc-FLiNT, Univ S. Denmark•Prof. Tom Ray, University of Oklahoma•Doron Lancet-Weizmann Institute, Israel•Dr. Nick Herbert, physicist and author•Larry Yaeger, Professor, University of Indiana•Brian Allen-MAGIX Lab, UCLA•Karl Sims, GenArts, Inc., Cambridge MA, USA•Dr. Ben Goertzel, CEO, Novamente, Silicon Valley, CA•Dan Miller, President/CTO, Singular Robotics, CA•Neil Datta, Imperial College London

Page 114: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Resources and Acknowledgements Project EvoGrid at: http://www.evogrid.orgProject Biota & Podcast at: http://www.biota.org DigitalSpace 3D simulations and all (open) source code at: http://www.digitalspace.com

We would also like to thank NASA and many others for funding support for this work. Other acknowledgements include: Dr. Richard Gordon at the University of Manitoba, Tom Barbalet, DM3D Studios, Peter Newman, Ryan Norkus, SMARTLab, Peter Bentley, University College London, FLiNT, Exploring Life’s Origins Project, Scientific American Frontiers, DigiBarn Computer Museum, The Shelby White and Leon Levy Archives Center, Institute for Advanced Study, Princeton, NJ, USA, and S. Gross.

Page 115: Bruce Damer's talk on the Von Neumann Bottleneck to the Singularity University, NASA, July 8, 2009

Closing Thought