Markov Chain Formalisation Long run behavior Cache modeling Synthesis Markov Chains and Computer Science A not so Short Introduction Jean-Marc Vincent Laboratoire LIG, projet Inria-Mescal UniversitéJoseph Fourier [email protected]Winter 2013 1 / 44 Markov Chains and Computer Science
31
Embed
Markov Chains and Computer Sciencepolaris.imag.fr/arnaud.legrand/teaching/2013/RICM4_EP_Markov.pdf · Markov Chain Formalisation Long run behavior Cache modeling Synthesis Nicholas
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Markov Chains and Computer ScienceA not so Short Introduction
Jean-Marc Vincent
Laboratoire LIG, projet Inria-MescalUniversitéJoseph Fourier
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Outline
1 Markov ChainHistoryApproaches
2 Formalisation
3 Long run behavior
4 Cache modeling
5 Synthesis
2 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
History (Andreï Markov)
An example of statistical investigation inthe text of "Eugene Onegin" illustratingcoupling of "tests" in chains.(1913) In Proceedings of AcademicScientific St. Petersburg, VI, pages153-162.
1856-1922
3 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
- Performance evaluation (quantification and dimensionning)- Stochastic control- Program verification-· · ·
12 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Nicholas Metropolis (1915-1999)
Metropolis contributed several original ideas tomathematics and physics. Perhaps the most widelyknown is the Monte Carlo method. Also, in 1953Metropolis co-authored the first paper on a technique thatwas central to the method known now as simulatedannealing. He also developed an algorithm (theMetropolis algorithm or Metropolis-Hastings algorithm) forgenerating samples from the Boltzmann distribution, latergeneralized by W.K. Hastings.
Simulated annealing
Convergence to a global minimum by a stochasticgradient scheme.
Xn+1 = Xn − ~gradΦ(Xn)∆n(Random).
∆n(random)n→∞−→ 0.
13 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Nicholas Metropolis (1915-1999)
Metropolis contributed several original ideas tomathematics and physics. Perhaps the most widelyknown is the Monte Carlo method. Also, in 1953Metropolis co-authored the first paper on a technique thatwas central to the method known now as simulatedannealing. He also developed an algorithm (theMetropolis algorithm or Metropolis-Hastings algorithm) forgenerating samples from the Boltzmann distribution, latergeneralized by W.K. Hastings.
Simulated annealing
Convergence to a global minimum by a stochasticgradient scheme.
Xn+1 = Xn − ~gradΦ(Xn)∆n(Random).
∆n(random)n→∞−→ 0.
13 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Nicholas Metropolis (1915-1999)
Metropolis contributed several original ideas tomathematics and physics. Perhaps the most widelyknown is the Monte Carlo method. Also, in 1953Metropolis co-authored the first paper on a technique thatwas central to the method known now as simulatedannealing. He also developed an algorithm (theMetropolis algorithm or Metropolis-Hastings algorithm) forgenerating samples from the Boltzmann distribution, latergeneralized by W.K. Hastings.
Simulated annealing
Convergence to a global minimum by a stochasticgradient scheme.
Xn+1 = Xn − ~gradΦ(Xn)∆n(Random).
∆n(random)n→∞−→ 0.
13 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Nicholas Metropolis (1915-1999)
Metropolis contributed several original ideas tomathematics and physics. Perhaps the most widelyknown is the Monte Carlo method. Also, in 1953Metropolis co-authored the first paper on a technique thatwas central to the method known now as simulatedannealing. He also developed an algorithm (theMetropolis algorithm or Metropolis-Hastings algorithm) forgenerating samples from the Boltzmann distribution, latergeneralized by W.K. Hastings.
Simulated annealing
Convergence to a global minimum by a stochasticgradient scheme.
Xn+1 = Xn − ~gradΦ(Xn)∆n(Random).
∆n(random)n→∞−→ 0.
13 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Nicholas Metropolis (1915-1999)
Metropolis contributed several original ideas tomathematics and physics. Perhaps the most widelyknown is the Monte Carlo method. Also, in 1953Metropolis co-authored the first paper on a technique thatwas central to the method known now as simulatedannealing. He also developed an algorithm (theMetropolis algorithm or Metropolis-Hastings algorithm) forgenerating samples from the Boltzmann distribution, latergeneralized by W.K. Hastings.
Simulated annealing
Convergence to a global minimum by a stochasticgradient scheme.
Xn+1 = Xn − ~gradΦ(Xn)∆n(Random).
∆n(random)n→∞−→ 0.
13 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Nicholas Metropolis (1915-1999)
Metropolis contributed several original ideas tomathematics and physics. Perhaps the most widelyknown is the Monte Carlo method. Also, in 1953Metropolis co-authored the first paper on a technique thatwas central to the method known now as simulatedannealing. He also developed an algorithm (theMetropolis algorithm or Metropolis-Hastings algorithm) forgenerating samples from the Boltzmann distribution, latergeneralized by W.K. Hastings.
Simulated annealing
Convergence to a global minimum by a stochasticgradient scheme.
Xn+1 = Xn − ~gradΦ(Xn)∆n(Random).
∆n(random)n→∞−→ 0.
13 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Nicholas Metropolis (1915-1999)
Metropolis contributed several original ideas tomathematics and physics. Perhaps the most widelyknown is the Monte Carlo method. Also, in 1953Metropolis co-authored the first paper on a technique thatwas central to the method known now as simulatedannealing. He also developed an algorithm (theMetropolis algorithm or Metropolis-Hastings algorithm) forgenerating samples from the Boltzmann distribution, latergeneralized by W.K. Hastings.
Simulated annealing
Convergence to a global minimum by a stochasticgradient scheme.
Xn+1 = Xn − ~gradΦ(Xn)∆n(Random).
∆n(random)n→∞−→ 0.
13 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Nicholas Metropolis (1915-1999)
Metropolis contributed several original ideas tomathematics and physics. Perhaps the most widelyknown is the Monte Carlo method. Also, in 1953Metropolis co-authored the first paper on a technique thatwas central to the method known now as simulatedannealing. He also developed an algorithm (theMetropolis algorithm or Metropolis-Hastings algorithm) forgenerating samples from the Boltzmann distribution, latergeneralized by W.K. Hastings.
Simulated annealing
Convergence to a global minimum by a stochasticgradient scheme.
Xn+1 = Xn − ~gradΦ(Xn)∆n(Random).
∆n(random)n→∞−→ 0.
13 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Nicholas Metropolis (1915-1999)
Metropolis contributed several original ideas tomathematics and physics. Perhaps the most widelyknown is the Monte Carlo method. Also, in 1953Metropolis co-authored the first paper on a technique thatwas central to the method known now as simulatedannealing. He also developed an algorithm (theMetropolis algorithm or Metropolis-Hastings algorithm) forgenerating samples from the Boltzmann distribution, latergeneralized by W.K. Hastings.
Simulated annealing
Convergence to a global minimum by a stochasticgradient scheme.
Xn+1 = Xn − ~gradΦ(Xn)∆n(Random).
∆n(random)n→∞−→ 0.
13 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Nicholas Metropolis (1915-1999)
Metropolis contributed several original ideas tomathematics and physics. Perhaps the most widelyknown is the Monte Carlo method. Also, in 1953Metropolis co-authored the first paper on a technique thatwas central to the method known now as simulatedannealing. He also developed an algorithm (theMetropolis algorithm or Metropolis-Hastings algorithm) forgenerating samples from the Boltzmann distribution, latergeneralized by W.K. Hastings.
Simulated annealing
Convergence to a global minimum by a stochasticgradient scheme.
Xn+1 = Xn − ~gradΦ(Xn)∆n(Random).
∆n(random)n→∞−→ 0.
13 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Nicholas Metropolis (1915-1999)
Metropolis contributed several original ideas tomathematics and physics. Perhaps the most widelyknown is the Monte Carlo method. Also, in 1953Metropolis co-authored the first paper on a technique thatwas central to the method known now as simulatedannealing. He also developed an algorithm (theMetropolis algorithm or Metropolis-Hastings algorithm) forgenerating samples from the Boltzmann distribution, latergeneralized by W.K. Hastings.
Simulated annealing
Convergence to a global minimum by a stochasticgradient scheme.
Xn+1 = Xn − ~gradΦ(Xn)∆n(Random).
∆n(random)n→∞−→ 0.
13 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Modeling and Analysis of Computer Systems
Complex system
System
Basic model assumptions
System :- automaton (discrete state space)- discrete or continuous timeEnvironment : non deterministic- time homogeneous- stochastically regular
Problem
Understand “typical” states- steady-state estimation- ergodic simulation- state space exploring techniques
14 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Modeling and Analysis of Computer Systems
Complex system
output
Environment
Input of the system
System
System
Basic model assumptions
System :- automaton (discrete state space)- discrete or continuous timeEnvironment : non deterministic- time homogeneous- stochastically regular
Problem
Understand “typical” states- steady-state estimation- ergodic simulation- state space exploring techniques
14 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Modeling and Analysis of Computer Systems
Complex system
output
Environment
Input of the system
System
System
Basic model assumptions
System :- automaton (discrete state space)- discrete or continuous timeEnvironment : non deterministic- time homogeneous- stochastically regular
Problem
Understand “typical” states- steady-state estimation- ergodic simulation- state space exploring techniques
14 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis
Modeling and Analysis of Computer Systems
Complex system
output
Environment
Input of the system
System
System
Basic model assumptions
System :- automaton (discrete state space)- discrete or continuous timeEnvironment : non deterministic- time homogeneous- stochastically regular
Problem
Understand “typical” states- steady-state estimation- ergodic simulation- state space exploring techniques
14 / 44Markov Chains and Computer Science
Markov Chain Formalisation Long run behavior Cache modeling Synthesis