Computational Methods for Nonlinear Systems Cornell Physics 7682 Fall 2015 Chris Myers [email protected] Physical Sciences Building 424
Computational Methods for Nonlinear Systems
!
Cornell Physics 7682 Fall 2015
Chris Myers [email protected]
Physical Sciences Building 424
Computational Methods for Nonlinear Systems
• Graduate computational science laboratory course developed by Myers, Sethna & Mueller, starting in 2004 - developed originally to support interdisciplinary
IGERT program on Nonlinear Systems - class work focused on implementation of
computer programs from papers, hints and skeletal code
Computational Methods for Nonlinear Systems
• Hands-on introduction to computational science & scientific computing: modeling, algorithms, data structures, programming, simulation techniques, data analysis, etc. - a broad variety of scientific themes: complex networks,
biomechanics, pattern formation, chemical kinetics, statistical mechanics, molecular dynamics, chaos, random matrix theory, constraint satisfaction, infectious diseases, etc.
- a broad variety of numerical methods: ODEs, PDEs, graph theory, Monte Carlo, data analysis, fitting, root finding, etc.
- many exercises incorporated into Sethna’s textbook, “Statistical Mechanics: Entropy, Order Parameters, and Complexity”
- complementary to Physics 4480 / 7680 (“Computational Physics with Numerical Recipes”) which focuses more specifically on algorithms and numerical analysis
Course structure
• Primary focus is working on course modules in class
• Periodic lectures on science and/or computation to complement work on course modules
• Self-paced and autonomous: work on those modules you are most interested in, or propose other topics of interest you would like to explore in more detail
• Can work in groups if desired, although be sure to take turns driving
• Students can use PECF workstations in Rockefeller B3 (computing.physics.cornell.edu), or bring own laptop for use in class
Course structure
• No homework assignments — no graded assignments; reading background material outside of class is useful
• Attendance, engagement and participation are paramount
• Ask questions if you are confused, need help debugging, or want to discuss material
• Students will be expected to deliver at least one short lecture (~15 minutes) on a subject of interest, and to participate in other initiatives that might unfold (e.g., “Bring your problems to class day”)
• Opportunities to bring relevant projects of interest to class (e.g., related to research), or to work with me to implement some new modules for the course - random graphs, forest fires, flocking, parameter estimation,
sloppiness, zombies, mosh pits, etc.
Course structure
• Recommended to take S/U
• If a letter grade is needed for a particular program, we will conduct oral examination / review of work done at end of semester (can be based in part on extra project)
• Flexible within reason: can accommodate absences due to occasional conflicts (e.g., special lab meeting, travel to a conference); can make up missed time outside of class - talk to instructor if conflicts arise
Programming
• all programming done in Python
• 3rd party Python libraries for numerics, graphics, visualization, etc.
• graphical tools provided to give feedback, help debug, etc.
• why Python? - interpreted: rapid program development and interactive access - built-in data structures & high-level syntax - rich standard library and 3rd party libraries (for science,
visualization, databases, internet programming, etc.) - supports procedural programming, object-oriented programming,
functional programming, scripting, large systems - free software (monetarily and intellectually) - good job skill
Goals for semester
• Yours? - get exposed to a broad set of interesting scientific and
computational problems and methods - learn how to integrate a variety of tools to solve problems - learn how to conduct research that integrates computation with
theory, data, analysis, etc.
• Mine - teach you all of the above, and learn from all of you - convey how to think computationally: abstraction, interface &
information - add new course modules (suggestions are welcome!) - new features/modules to be rolled out over the course of the
semester (and you are welcome to help if you are interested in such things)
- continue to improve the course website and workflow
Scientific topics
Networks
Dynamics & Dynamical Systems
Biological dynamics & patterns
Statistical mechanics
etc.
Course modules
Small-world networks
Percolation
Pendulum
Walker
Invariant measure
Chaos & Lyapunov
Fractal dimensions
Period doubling
Cardiac dynamics
Random matrix theory
Stochastic cells
Repressilator
NP-completeness
Ising model
Molecular dynamics
Infectious diseases
Small-world networks
• Science - small world networks (Watts & Strogatz) - “six degrees of separation” - shortest path lengths in randomly wired
graphs
• Computing - data structure for undirected graphs
‣ good introduction to built-in Python containers (lists and dictionaries)
- object-oriented encapsulation of complex data structures
- graph traversal algorithms (breadth-first search) for shortest path and betweenness
- simple graph visualization software supports debugging and provides quick feedback
Percolation
• Science - statistical mechanics of percolation
‣ connected clusters in randomly wired graphs (e.g., bond percolation on a lattice)
‣ universality of phase transitions
• Computing - reuse of objects with generic interface
(reuse of UndirectedGraph class from small-world network module)
- graph traversal algorithms (breadth-first search) for cluster finding
- scaling collapses
Walker
• Science - Simple model of bipedal walker (Ruina and
coworkers)
‣ double pendulum with impulse for heelstrike
‣ single pendulum as warmup - period-doubling bifurcations in physical
system
• Computing - integration of ODEs
‣ finite differences, time-step dependence, integration schemes (stability, fidelity, accuracy)
‣ calling 3rd party numerical libraries - change of integration variables for event
detection (heelstrike) - tracking unstable periodic orbit - visualization tools for animation
Maps & dynamical systems
• Science - bifurcations and chaos in iterated maps - period doubling in logistic map - density (invariant measure) in chaotic
regime - Lyapunov exponents: divergence of
nearby trajectories - fractal dimensions of attractors - renormalization group of logistic map &
universality of period doubling route to chaos
• Computing - iterating maps - root-finding - fitting
Cardiac dynamics
• Science - pattern formation in excitable medium - FitzHugh-Nagumo model (type of
reaction-diffusion equation) - spiral defects forming in electrical
pulsing can lead to cardiac arrythmias - model extensions to simulate dead
tissue, cardiac chambers, etc.
• Computing - numerical solution of PDEs
‣ finite-differences, operator stencils - nullcline analysis for single cell (root-
finding) - simple animation tool allows interactive
steering of simulation (needed to provide targeted electrical pulses and “defribrillator” shock)
Cell dynamics: stochastic cells and Repressilator
• Science - Repressilator (Elowitz & Leibler): genetic
oscillator from realm of synthetic biology
‣ oscillatory mRNA/protein dynamics from mutually repressing proteins
- telegraph noise and shotgun noise in stochastic systems
• Computing - chemical kinetics and reaction networks
‣ Petri nets
‣ synthesis of aggregate kinetic equations from network
- Monte Carlo algorithms (Gillespie) - stochastic vs. deterministic descriptions
• Additional projects exploring other gene regulatory networks (switches, feed-forward loops, etc.)
NP-completeness & constraint satisfaction
• Science - NP-complete problems - phase transitions in 3SAT and
parametric complexity - integer partitioning problem
• Computing - algorithms for NP-complete
problems - backtracking, recursion
Random matrix theory
• Science - eigenvalue spacings in random
matrices - developed originally to describe
energy level spacings in quantum systems
• Computing - random matrix generation - eigenvalue computation (numpy)
Molecular dynamics
• Science - thermodynamics: emergence of
effective properties from molecular chaos
• Computing - design of large software systems - geometric data structures - integration of ODEs - thermodynamics
‣ pressure
‣ pair distribution functions
Ising model
• Science - phase transitions in simple
model of magnetic system
‣ nucleation
‣ self-similarity
‣ fluctuation-dissipation
• Computing - Monte Carlo algorithms
‣ heat bath
‣ Metropolis
‣ cluster flipping
Randomness: random walks & Gumbel distributions
• Science - ensembles and distributions
‣ averages
‣ extremal behavior
‣ emergent symmetries
• Computing - random number generation - statistical analysis of ensembles
The Python for Science Ecosystem
• SciPy and NumPy [www.scipy.org] - NumPy: provides (in a compiled library) high-level array syntax,
linear algebra, random number generation, Fourier transforms - Python interfaces to well-tested compiled numerical routines,
interfaced to work with NumPy arrays - provides routines for integration of functions and differential
equations, root-finding, minimization, etc.
• matplotlib (a.k.a. pylab) [matplotlib.org] - provides 2D (x-y) plotting, histograms, etc. (modeled after
MATLAB plotting)
• ipython [ipython.org] - enhancement of the standard python interpreter: command
completion & history, “magic” functions, etc. - notebook mode for integrated programs, documentation, results
The Python for Science Ecosystem (continued)
• Python standard library: pickle, time, os, collections, multiprocessing, string, url lib, xml, json, etc.
• Python Imaging Library (PIL, Pillow): image processing and analysis
• scikit-learn: machine learning
• NetworkX: graph theory and network analysis
• sympy: symbolic mathematics
• Pandas: data analysis
• mpmath: multiple precision mathematics
• h5py: HDF5 interface (btw, also interface to MATLAB .mat files)
• SWIG, Cython, weave, theano, …: tools for integrating compiled code with interpreted Python code
• etc.
Course mechanics & information
• Course web page - www.physics.cornell.edu/~myers/teaching/ComputationalMethods
• Module web pages - www.physics.cornell.edu/~myers/teaching/ComputationalMethods/
ComputerExercises - linked from “Computer Exercises” on course web page
• Navigation bar on course web page - News & Updates: information about new modules, bug fixes,
special projects, etc. - Lecture Notes: copies of in-class lecture slides - Python Resources: tutorials, documentation, guides, etc.
Course mechanics: working with modules
• For a given module: - possibly some background reading (e.g., journal paper) - text describing an exercise (pdf or on web page) — read this
first! - a “Hints” file (e.g., SmallWorldHints.py) ‣ Hints files have skeletal Python code with comments
(docstrings) ‣ Hints files have minimal Python code body consisting of
keyword pass
‣ Copy “Hints” file to new file (e.g., SmallWorld.py) and then fill in code body to implement desired functionality
‣ %run Python code file within IPython notebook to test code, run analyses, make plots, document work, etc.
Python installations
• Encourage use of Anaconda Scientific Python distribution (unless you are already a seasoned Python user with your own setup) !
• If using your own laptop: • Anaconda distribution (free): download & install Python 3.4
- https://store.continuum.io/cshop/anaconda/ !
• Physics Educational Computing Facility (PECF) - python, ipython, scipy, etc. all installed - add ~myers/anaconda3/bin to beginning of shell path - let me know if packages are missing or broken !
• Text editors have auxiliary python editing mode: syntax highlighting, indenting, etc. (e.g., python-mode in emacs)
Development Environments, Workflows & Interpreters
• python interpreter (“python”) - bare bones interpreter of Python code - useful to run programs in background/batch once fully developed
• ipython interpreter (“ipython” or “ipython qtconsole”) - enhanced Python interpreter with help, autocompletion, editing,
etc. - extra functionality (“magics”) outside of Python language - magics prepended by %
- %who : list of what variables, functions, etc. defined in namespace
- %timeit : provide timing information of specific command
• ipython notebooks (“ipython notebook”) - graphical interface integrating code, documentation, analyses,
plots, etc. (a la Mathematica notebooks)
Some new twists this year
• Python 3 - core numerical code for all the modules has been updated - some of the visualization tools will need to be updated
and/or replaced (anyone interested in helping?)
• Encouraging use of Anaconda python v. 3.4 by everyone
• IPython notebooks to organize and summarize analyses and results