A Bird's-Eye View of PetaVision, the World's First Petaflop/s Neural Simulation* Dan Coates Portland State University, Maseeh College of Engineering and Computer Science, Portland OR Parallel Implementations of Learning Algorithms: “What Have You Done For Me Lately?” NIPS08 Whistler, BC December 13, 2008 Dan Coates Garrett Kenyon, Craig Rasmussen Los Alamos National Laboratory, Los Alamos, NM * The authors acknowledge the support of the National Science Foundation, under a grant administered by the New Mexico Consortium
28
Embed
A Bird's-Eye View of PetaVision, the World's First Petaflop/s Neural ...dst/NIPS/nips08-workshop/Dan_Coates_slides.pdf · A Bird's-Eye View of PetaVision, the World's First Petaflop/s
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
A Bird's-Eye View of PetaVision, theWorld's First Petaflop/s Neural Simulation*
Dan CoatesPortland State University,
Maseeh College of Engineering and Computer Science,
Portland OR
Parallel Implementations of Learning Algorithms:“What Have You Done For Me Lately?”
NIPS08Whistler, BC
December 13, 2008
Dan Coates Garrett Kenyon,Craig Rasmussen
Los Alamos National Laboratory, Los Alamos, NM
* The authors acknowledge the support of the National Science Foundation, under a grant administered by the New Mexico Consortium
12/16/08 Coates NIPS08: PetaVision 2
PetaVision Project at LANL
Goal: Achieve human-level performancein a “synthetic visual cognition” system
On: IBM/DOE Roadrunner petascalesupercomputer (or a multicore PC)
Running: A spiking LIF neural networkinspired by visual cortex.
12/16/08 Coates NIPS08: PetaVision 3
What level of abstraction?
Emulate the cortical circuits formid/low-level visual processing.
We model the gross architecture ofvisual cortex, trying not to violateproven neural science.
Binzegger, et. al.
12/16/08 Coates NIPS08: PetaVision 4
What are the crucial features of V1?
Retinotopic mapping.
Edge detectorsof Hubel & Wiesel
Distinct laminar neuralpopulations.
12/16/08 Coates NIPS08: PetaVision 5
What are the crucial features of V1?
Retinotopic mapping.
Edge detectorsof Hubel & Wiesel
Distinct laminar neuralpopulations.
12/16/08 Coates NIPS08: PetaVision 6
What are the crucial features of V1?
Bannister. Laminar circuit.
Retinotopic mapping.
Edge detectorsof Hubel & Wiesel
Distinct laminar neuralpopulations.
12/16/08 Coates NIPS08: PetaVision 7
Spiking neurons and specific connectivity
- efficient, possibly asynchronous operation- sparse inter-node communication
What are the elements, and how does that help us?
12/16/08 Coates NIPS08: PetaVision 8
Spiking neurons and specific connectivity
- connections are primarily local
- function inherent in wiring
What are the elements, and how does that help us?
Bosking, et al. “Patchy” connectivity expresses orientation preference ofhorizontal connections.
12/16/08 Coates NIPS08: PetaVision 9
Example: edge detection
V1 simple cells have been shown to respondlike a Gabor functions. We use 8 orientations.
12/16/08 Coates NIPS08: PetaVision 10
Beyond edges: long-range association field
Ben-Shahar and Zucker have proposedadditional connectivity patterns formalizedusing differential geometry. [Neural Computation, 2004]
Besides curve integration, such a schemecould also be used for shape-from-shadingand natural texture identification.
12/16/08 Coates NIPS08: PetaVision 11
Ben-Shahar and Zucker have proposedadditional connectivity patterns formalizedusing differential geometry. [Neural Computation, 2004]
Besides curve integration, such a schemecould also be used for shape-from-shadingand natural texture identification.
Beyond edges: long-range association field
12/16/08 Coates NIPS08: PetaVision 12
Beyond edges: long-range association field
12/16/08 Coates NIPS08: PetaVision 13
Summary of Biological Inspiration
Network structure for a computer visionsystem can be modeled after architectureof mammalian visual cortex.
There are analytic correlates of thesetechniques, although closed-formderivations are difficult.
Note: these connections have beenshown to be learnable, although wehard-code as mathematical functions.
12/16/08 Coates NIPS08: PetaVision 14
Implementation: software abstractions
PVLayer: Population of neurons. Retina, LIF.
PVConnection: Connectivity pattern,represented by a mathematical weight function.Anything-to-anything routing possible.
12/16/08 Coates NIPS08: PetaVision 15
Implementation: LIF
12/16/08 Coates NIPS08: PetaVision 16
Implementation: PVConnection
Connection kernels are translation-invariant.
12/16/08 Coates NIPS08: PetaVision 17
Implementation: Parallel Algorithm
Process each presynaptic event
Process each PVConnection:
Update effected postsynaptic neurons
Update each layer
Perform I/O
12/16/08 Coates NIPS08: PetaVision 18
How to interpret results?
Readout: Spike trains are post-processed forfiring rate. Temporal correlations such assynchrony and oscillatory power are alsomeasured.