UCSD - Thermodynamics and Statistical Mechanics

Lecture Notes on Thermodynamics and Statistical Mechanics

(A Work in Progress)

Daniel ArovasDepartment of Physics

University of California, San Diego

April 26, 2013

Contents

0.1 Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

0.2 General references . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv

1 Probability 1

1.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 A Statistical View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2.1 Distributions for a random walk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2.2 Thermodynamic limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.3 Entropy and energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.2.4 Entropy and information theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3 Probability Distributions fromMaximum Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.3.1 The principle of maximum entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.3.2 Continuous probability distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.4 General Aspects of Probability Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.4.1 Discrete and continuous distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.4.2 Central limit theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.4.3 Multidimensional Gaussian integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.5 Appendix : Bayesian Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2 Thermodynamics 17

2.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.2 What is Thermodynamics? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.2.1 Thermodynamic systems and state variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.2.2 Heat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

i

ii CONTENTS

2.2.3 Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.2.4 Pressure and Temperature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.2.5 Standard temperature and pressure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.3 The Zeroth Law of Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.4 Mathematical Interlude : Exact and Inexact Differentials . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.5 The First Law of Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.5.1 Conservation of energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.5.2 Single component systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.5.3 Ideal gases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.5.4 Adiabatic transformations of ideal gases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.5.5 Adiabatic free expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

2.6 Heat Engines and the Second Law of Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.6.1 Theres no free lunch so quit asking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.6.2 Engines and refrigerators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2.6.3 Nothing beats a Carnot engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

2.6.4 The Carnot cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.6.5 The Stirling cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.6.6 The Otto and Diesel cycles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

2.6.7 The Joule-Brayton cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

2.6.8 Carnot engine at maximum power output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

2.7 The Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

2.7.1 Entropy and heat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

2.7.2 The Third Law of Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.7.3 Entropy changes in cyclic processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

2.7.4 Gibbs-Duhem relation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

2.7.5 Entropy for an ideal gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

2.7.6 Example system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

2.7.7 Measuring the entropy of a substance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

2.8 Thermodynamic Potentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

2.8.1 Energy E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

2.8.2 Helmholtz free energy F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

CONTENTS iii

2.8.3 Enthalpy H . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

2.8.4 Gibbs free energy G . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

2.8.5 Grand potential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2.9 Maxwell Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2.9.1 Relations deriving from E(S, V,N) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2.9.2 Relations deriving from F (T, V,N) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

2.9.3 Relations deriving from H(S, p,N) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

2.9.4 Relations deriving from G(T, p,N) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

2.9.5 Relations deriving from (T, V, ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

2.9.6 Generalized thermodynamic potentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

2.10 Equilibrium and Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

2.11 Applications of Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

2.11.1 Adiabatic free expansion revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

2.11.2 Energy and volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

2.11.3 van der Waals equation of state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

2.11.4 Thermodynamic response functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

2.11.5 Joule effect: free expansion of a gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

2.11.6 Throttling: the Joule-Thompson effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

2.12 Phase Transitions and Phase Equilibria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

2.12.1 p-v-T surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

2.12.2 The Clausius-Clapeyron relation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

2.12.3 Liquid-solid line in H2O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

2.12.4 Slow melting of ice : a quasistatic but irreversible process . . . . . . . . . . . . . . . . . . . . 75

2.12.5 Gibbs phase rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

2.13 Entropy of Mixing and the Gibbs Paradox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

2.13.1 Computing the entropy of mixing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

2.13.2 Entropy and combinatorics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

2.13.3 Weak solutions and osmotic pressure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

2.13.4 Effect of impurities on boiling and freezing points . . . . . . . . . . . . . . . . . . . . . . . . 83

2.13.5 Binary solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

2.14 Some Concepts in Thermochemistry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

iv CONTENTS

2.14.1 Chemical reactions and the law of mass action . . . . . . . . . . . . . . . . . . . . . . . . . . 92

2.14.2 Enthalpy of formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

2.14.3 Bond enthalpies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

2.15 Appendix I : Integrating factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

2.16 Appendix II : Legendre Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

2.17 Appendix III : Useful Mathematical Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

3 Ergodicity and the Approach to Equilibrium 107

3.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

3.2 Modeling the Approach to Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

3.2.1 Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

3.2.2 The Master Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

3.2.3 Equilibrium distribution and detailed balance . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

3.2.4 Boltzmanns H-theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

3.3 Phase Flows in Classical Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

3.3.1 Hamiltonian evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

3.3.2 Dynamical systems and the evolution of phase space volumes . . . . . . . . . . . . . . . . . 111

3.3.3 Liouvilles equation and the microcanonical distribution . . . . . . . . . . . . . . . . . . . . . 114

3.4 Irreversibility and Poincare Recurrence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

3.4.1 Poincare recurrence theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

3.4.2 Kac ring model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

3.5 Remarks on Ergodic Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

3.5.1 Definition of ergodicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

3.5.2 The microcanonical ensemble . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

3.5.3 Ergodicity and mixing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

3.6 Thermalization of Quantum Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

3.6.1 Quantum dephasing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

3.6.2 Eigenstate thermalization hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

3.6.3 When is the ETH true? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

3.7 Appendix I : Formal Solution of the Master Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

3.8 Appendix II : Radioactive Decay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

CONTENTS v

3.9 Appendix III : Canonical Transformations in Hamiltonian Mechanics . . . . . . . . . . . . . . . . . . 131

4 Statistical Ensembles 133

4.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

4.2 Microcanonical Ensemble (CE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

4.2.1 The microcanonical distribution function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

4.2.2 Density of states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

4.2.3 Arbitrariness in the definition of S(E) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

4.2.4 Ultra-relativistic ideal gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

4.2.5 Discrete systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

4.3 The Quantum Mechanical Trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

4.3.1 The density matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

4.3.2 Averaging the DOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

4.3.3 Coherent states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

4.4 Thermal Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

4.5 Ordinary Canonical Ensemble (OCE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

4.5.1 Canonical distribution and partition function . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

4.5.2 The difference between P (En) and Pn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

4.5.3 Averages within the OCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

4.5.4 Entropy and free energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

4.5.5 Fluctuations in the OCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

4.5.6 Thermodynamics revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

4.5.7 Generalized susceptibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

4.6 Grand Canonical Ensemble (GCE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

4.6.1 Grand canonical distribution and partition function . . . . . . . . . . . . . . . . . . . . . . . 150

4.6.2 Entropy and Gibbs-Duhem relation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

4.6.3 Generalized susceptibilities in the GCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

4.6.4 Fluctuations in the GCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

4.6.5 Gibbs ensemble . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

4.7 Statistical Ensembles from Maximum Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

4.7.1 CE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

vi CONTENTS

4.7.2 OCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

4.7.3 GCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

4.8 Ideal Gas Statistical Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

4.8.1 Maxwell velocity distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

4.8.2 Equipartition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

4.8.3 Quantum statistics and the Maxwell-Boltzmann limit . . . . . . . . . . . . . . . . . . . . . . 159

4.9 Selected Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

4.9.1 Spins in an external magnetic field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

4.9.2 Negative temperature (!) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

4.9.3 Adsorption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

4.9.4 Elasticity of wool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

4.9.5 Noninteracting spin dimers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

4.10 Statistical Mechanics of Molecular Gases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

4.10.1 Separation of translational and internal degrees of freedom . . . . . . . . . . . . . . . . . . . 167

4.10.2 Ideal gas law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

4.10.3 The internal coordinate partition function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

4.10.4 Rotations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

4.10.5 Vibrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

4.10.6 Two-level systems : Schottky anomaly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

4.10.7 Electronic and nuclear excitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

4.11 Appendix I : Additional Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

4.11.1 Three state system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

4.11.2 Spins and vacancies on a surface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

4.11.3 Fluctuating interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

4.11.4 Dissociation of molecular hydrogen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

5 Noninteracting Quantum Systems 183

5.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

5.2 Statistical Mechanics of Noninteracting Quantum Systems . . . . . . . . . . . . . . . . . . . . . . . . 184

5.2.1 Bose and Fermi systems in the grand canonical ensemble . . . . . . . . . . . . . . . . . . . . 184

5.2.2 Maxwell-Boltzmann limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

CONTENTS vii

5.2.3 Single particle density of states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

5.3 Quantum Ideal Gases : Low Density Expansions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

5.3.1 Expansion in powers of the fugacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

5.3.2 Virial expansion of the equation of state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

5.3.3 Ballistic dispersion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

5.4 Entropy and Counting States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

5.5 Photon Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

5.5.1 Thermodynamics of the photon gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

5.5.2 Classical arguments for the photon gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

5.5.3 Surface temperature of the earth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

5.5.4 Distribution of blackbody radiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

5.5.5 What if the sun emitted ferromagnetic spin waves? . . . . . . . . . . . . . . . . . . . . . . . . 196

5.6 Lattice Vibrations : Einstein and Debye Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

5.6.1 One-dimensional chain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

5.6.2 General theory of lattice vibrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

5.6.3 Einstein and Debye models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

5.6.4 Melting and the Lindemann criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

5.6.5 Goldstone bosons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206

5.7 The Ideal Bose Gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

5.7.1 General formulation for noninteracting systems . . . . . . . . . . . . . . . . . . . . . . . . . . 207

5.7.2 Ballistic dispersion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208

5.7.3 Isotherms for the ideal Bose gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

5.7.4 The -transition in Liquid 4He . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

5.7.5 Fountain effect in superfluid 4He . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

5.7.6 Bose condensation in optical traps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

5.7.7 Example problem from Fall 2004 UCSD graduate written exam . . . . . . . . . . . . . . . . . 218

5.8 The Ideal Fermi Gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

5.8.1 Grand potential and particle number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

5.8.2 The Fermi distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

5.8.3 T = 0 and the Fermi surface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

5.8.4 Spin-split Fermi surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

viii CONTENTS

5.8.5 The Sommerfeld expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

5.8.6 Chemical potential shift . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

5.8.7 Specific heat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

5.8.8 Magnetic susceptibility and Pauli paramagnetism . . . . . . . . . . . . . . . . . . . . . . . . 226

5.8.9 Landau diamagnetism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

5.8.10 White dwarf stars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230

6 Classical Interacting Systems 233

6.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

6.2 Ising Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

6.2.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

6.2.2 Ising model in one dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

6.2.3 H = 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235

6.2.4 Chain with free ends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236

6.2.5 Ising model in two dimensions : Peierls argument . . . . . . . . . . . . . . . . . . . . . . . . 237

6.2.6 Two dimensions or one? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

6.2.7 High temperature expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

6.3 Nonideal Classical Gases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

6.3.1 The configuration integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244

6.3.2 One-dimensional Tonks gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244

6.3.3 Mayer cluster expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

6.3.4 Cookbook recipe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

6.3.5 Lowest order expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

6.3.6 Hard sphere gas in three dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

6.3.7 Weakly attractive tail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

6.3.8 Spherical potential well . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

6.3.9 Hard spheres with a hard wall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

6.4 Lee-Yang Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

6.4.1 Analytic properties of the partition function . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

6.4.2 Electrostatic analogy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

6.4.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

CONTENTS ix

6.5 Liquid State Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260

6.5.1 The many-particle distribution function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260

6.5.2 Averages over the distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261

6.5.3 Virial equation of state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265

6.5.4 Correlations and scattering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

6.5.5 Correlation and response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

6.5.6 BBGKY hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271

6.5.7 Ornstein-Zernike theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272

6.5.8 Percus-Yevick equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

6.5.9 Ornstein-Zernike approximation at long wavelengths . . . . . . . . . . . . . . . . . . . . . . 274

6.6 Coulomb Systems : Plasmas and the Electron Gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276

6.6.1 Electrostatic potential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276

6.6.2 Debye-Huckel theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277

6.6.3 The electron gas : Thomas-Fermi screening . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

6.7 Polymers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

6.7.1 Basic concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

6.7.2 Polymers as random walks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

6.7.3 Flory theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

6.7.4 Polymers and solvents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

6.8 Appendix : Potts Model in One Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288

6.8.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288

6.8.2 Transfer matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288

7 Mean Field Theory of Phase Transitions 291

7.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291

7.2 The van der Waals system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292

7.2.1 Equation of state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292

7.2.2 Analytic form of the coexistence curve near the critical point . . . . . . . . . . . . . . . . . . 295

7.2.3 History of the van der Waals equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298

7.3 Fluids, Magnets, and the Ising Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

7.3.1 Lattice gas description of a fluid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

x CONTENTS

7.3.2 Phase diagrams and critical exponents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

7.3.3 Gibbs-Duhem relation for magnetic systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

7.3.4 Order-disorder transitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

7.4 Mean Field Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304

7.4.1 h = 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305

7.4.2 Specific heat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

7.4.3 h 6= 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3077.4.4 Magnetization dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309

7.4.5 Beyond nearest neighbors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311

7.4.6 Ising model with long-ranged forces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312

7.5 Variational Density Matrix Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313

7.5.1 The variational principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313

7.5.2 Variational density matrix for the Ising model . . . . . . . . . . . . . . . . . . . . . . . . . . . 314

7.5.3 Mean Field Theory of the Potts Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317

7.5.4 Mean Field Theory of the XY Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319

7.6 Landau Theory of Phase Transitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321

7.6.1 Cubic terms in Landau theory : first order transitions . . . . . . . . . . . . . . . . . . . . . . 323

7.6.2 Magnetization dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324

7.6.3 Sixth order Landau theory : tricritical point . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326

7.6.4 Hysteresis for the sextic potential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326

7.7 Mean Field Theory of Fluctuations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329

7.7.1 Correlation and response in mean field theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 329

7.7.2 Calculation of the response functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330

7.8 Global Symmetries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333

7.8.1 Symmetries and symmetry groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333

7.8.2 Lower critical dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335

7.8.3 Continuous symmetries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336

7.8.4 Random systems : Imry-Ma argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337

7.9 Ginzburg-Landau Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339

7.9.1 Ginzburg-Landau free energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339

7.9.2 Domain wall profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340

CONTENTS xi

7.9.3 Derivation of Ginzburg-Landau free energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341

7.9.4 Ginzburg criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344

7.10 Appendix I : Equivalence of the Mean Field Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . 346

7.10.1 Variational Density Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347

7.10.2 Mean Field Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348

7.11 Appendix II : Additional Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348

7.11.1 Blume-Capel model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348

7.11.2 Ising antiferromagnet in an external field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

7.11.3 Canted quantum antiferromagnet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353

7.11.4 Coupled order parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355

8 Nonequilibrium Phenomena 361

8.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361

8.2 Equilibrium, Nonequilibrium and Local Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . 362

8.3 Boltzmann Transport Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363

8.3.1 Derivation of the Boltzmann equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363

8.3.2 Collisionless Boltzmann equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365

8.3.3 Collisional invariants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366

8.3.4 Scattering processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367

8.3.5 Detailed balance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368

8.3.6 Kinematics and cross section . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370

8.3.7 H-theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370

8.4 Weakly Inhomogeneous Gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372

8.5 Relaxation Time Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374

8.5.1 Approximation of collision integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374

8.5.2 Computation of the scattering time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374

8.5.3 Thermal conductivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375

8.5.4 Viscosity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376

8.5.5 Oscillating external force . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379

8.5.6 Quick and Dirty Treatment of Transport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380

8.5.7 Thermal diffusivity, kinematic viscosity, and Prandtl number . . . . . . . . . . . . . . . . . . 380

xii CONTENTS

8.6 Diffusion and the Lorentz model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381

8.6.1 Failure of the relaxation time approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381

8.6.2 Modified Boltzmann equation and its solution . . . . . . . . . . . . . . . . . . . . . . . . . . 382

8.7 Linearized Boltzmann Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384

8.7.1 Linearizing the collision integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384

8.7.2 Linear algebraic properties of L . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384

8.7.3 Steady state solution to the linearized Boltzmann equation . . . . . . . . . . . . . . . . . . . 386

8.7.4 Variational approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386

8.8 The Equations of Hydrodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390

8.9 Nonequilibrium Quantum Transport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390

8.9.1 Boltzmann equation for quantum systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390

8.9.2 The Heat Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394

8.9.3 Calculation of Transport Coefficients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395

8.9.4 Onsager Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396

8.10 Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397

8.10.1 Langevin equation and Brownian motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397

8.10.2 Langevin equation for a particle in a harmonic well . . . . . . . . . . . . . . . . . . . . . . . 400

8.10.3 Discrete random walk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401

8.10.4 Fokker-Planck equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402

8.10.5 Brownian motion redux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403

8.10.6 Master Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404

8.11 Appendix I : Boltzmann Equation and Collisional Invariants . . . . . . . . . . . . . . . . . . . . . . . 406

8.12 Appendix II : Distributions and Functionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408

8.13 Appendix III : General Linear Autonomous Inhomogeneous ODEs . . . . . . . . . . . . . . . . . . . 410

8.14 Appendix IV : Correlations in the Langevin formalism . . . . . . . . . . . . . . . . . . . . . . . . . . 416

8.15 Appendix V : Kramers-Kronig Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418

0.1. PREFACE xiii

0.1 Preface

This is a proto-preface. A more complete preface will be written after these notes are completed.

These lecture notes are intended to supplement a course in statistical physics at the upper division undergraduateor beginning graduate level.

I was fortunate to learn this subject from one of the great statistical physicists of our time, John Cardy.

I am grateful to my wife Joyce and to my children Ezra and Lily for putting up with all the outrageous lies Ivetold them about getting off the computer in just a few minutes while working on these notes.

These notes are dedicated to the only two creatures I know who are never angry with me: my father and my dog.

Figure 1: My father (Louis) and my dog (Henry).

xiv CONTENTS

0.2 General references

L. Peliti, Statistical Mechanics in a Nutshell (Princeton University Press, 2011)The best all-around book on the subject Ive come across thus far. Appropriate for the graduate or advancedundergraduate level.

J. P. Sethna, Entropy, Order Parameters, and Complexity (Oxford, 2006)An excellent introductory textwith a verymodern set of topics and exercises. Available online at http://www.physics.cornell.edu/sethna/StatMech

M. Kardar, Statistical Physics of Particles (Cambridge, 2007)A superb modern text, with many insightful presentations of key concepts.

M. Plischke and B. Bergersen, Equilibrium Statistical Physics (3rd edition, World Scientific, 2006)An excellent graduate level text. Less insightful than Kardar but still a good modern treatment of the subject.Good discussion of mean field theory.

E. M. Lifshitz and L. P. Pitaevskii, Statistical Physics (part I, 3rd edition, Pergamon, 1980)This is volume 5 in the famous Landau and Lifshitz Course of Theoretical Physics. Though dated, it stillcontains a wealth of information and physical insight.

F. Reif, Fundamentals of Statistical and Thermal Physics (McGraw-Hill, 1987)This has been perhaps the most popular undergraduate text since it first appeared in 1967, and with goodreason.

Chapter 1

Probability

1.1 References

F. Reif, Fundamentals of Statistical and Thermal Physics (McGraw-Hill, 1987)This has been perhaps the most popular undergraduate text since it first appeared in 1967, and with goodreason.

E. T. Jaynes, Probability Theory (Cambridge, 2007)The bible on probability theory for physicists. A strongly Bayesian approach.

C. Gardiner, Stochastic Methods (Springer-Verlag, 2010)Very clear and complete text on stochastic mathematics.

1

2 CHAPTER 1. PROBABILITY

1.2 A Statistical View

1.2.1 Distributions for a random walk

Consider the mechanical system depicted in Fig. 1.1, a version of which is often sold in novelty shops. A ballis released from the top, which cascades consecutively through N levels. The details of each balls motion aregoverned by Newtons laws of motion. However, to predict where any given ball will end up in the bottom row isdifficult, because the balls trajectory depends sensitively on its initial conditions, and may even be influenced byrandom vibrations of the entire apparatus. We therefore abandon all hope of integrating the equations of motionand treat the system statistically. That is, we assume, at each level, that the ball moves to the right with probabilityp and to the left with probability q = 1 p. If there is no bias in the system, then p = q = 12 . The position XN afterN steps may be written

X =Nj=1

j , (1.1)

where j = +1 if the ball moves to the right at level j, and j = 1 if the ball moves to the left at level j. At eachlevel, the probability for these two outcomes is given by

P = p ,+1 + q ,1 =

{p if = +1

q if = 1 . (1.2)

This is a normalized discrete probability distribution of the type discussed in section 1.4 below. The multivariatedistribution for all the steps is then

P(1 , . . . , N ) =Nj=1

P (j) . (1.3)

Our system is equivalent to a one-dimensional random walk. Imagine an inebriated pedestrian on a sidewalktaking steps to the right and left at random. AfterN steps, the pedestrians location is X .

Now lets compute the average of X :

X = Nj=1

j= N = N

=1

P () = N(p q) = N(2p 1) . (1.4)

This could be identified as an equation of state for our system, as it relates a measurable quantity X to the numberof steps N and the local bias p. Next, lets compute the average of X2:

X2 =Nj=1

Nj=1

jj = N2(p q)2 + 4Npq . (1.5)

Here we have used

jj = jj +(1 jj

)(p q)2 =

{1 if j = j

(p q)2 if j 6= j . (1.6)

Note that X2 X2, which must be so becauseVar(X) = (X)2 (X X)2 = X2 X2 . (1.7)

This is called the variance ofX . We haveVar(X) = 4Np q. The root mean square deviation,Xrms, is the square root

of the variance: Xrms =Var(X). Note that the mean value of X is linearly proportional to N 1, but the RMS

1The exception is the unbiased case p = q = 12, where X = 0.

1.2. A STATISTICAL VIEW 3

Figure 1.1: The falling ball system, which mimics a one-dimensional random walk.

fluctuations Xrms are proportional to N1/2. In the limit N then, the ratio Xrms/X vanishes as N1/2.

This is a consequence of the central limit theorem (see 1.4.2 below), and we shall meet up with it again on severaloccasions.

We can do even better. We can find the complete probability distribution forX . It is given by

PN,X =

(N

NR

)pNR qNL , (1.8)

where NR/L are the numbers of steps taken to the right/left, with N = NR + NL, and X = NR NL. There aremany independent ways to takeN

Rsteps to the right. For example, our firstN

Rsteps could all be to the right, and

the remaining NL= N N

Rsteps would then all be to the left. Or our final N

Rsteps could all be to the right. For

each of these independent possibilities, the probability is pNR qNL . How many possibilities are there? Elementarycombinatorics tells us this number is (

N

NR

)=

N !

NR!NL!. (1.9)

Note that N X = 2NR/L, so we can replace NR/L = 12 (N X). Thus,

PN,X =N !(

N+X2

)!(NX2

)!p(N+X)/2 q(NX)/2 . (1.10)

1.2.2 Thermodynamic limit

Consider the limit N but with x X/N finite. This is analogous to what is called the thermodynamic limitin statistical mechanics. Since N is large, x may be considered a continuous variable. We evaluate lnPN,X usingStirlings asymptotic expansion

lnN ! N lnN N +O(lnN) . (1.11)

4 CHAPTER 1. PROBABILITY

We then have

lnPN,X N lnN N 12N(1 + x) ln[12N(1 + x)

]+ 12N(1 + x)

12N(1 x) ln[12N(1 x)

]+ 12N(1 x) + 12N(1 + x) ln p+ 12N(1 x) ln q

= N[(

1+x2

)ln(1+x2

)+(1x2

)ln(1x2

)]+N

[(1+x2

)ln p+

(1x2

)ln q]. (1.12)

Notice that the terms proportional to N lnN have all cancelled, leaving us with a quantity which is linear in N .We may therefore write lnPN,X = Nf(x) +O(lnN), where

f(x) =[(

1+x2

)ln(1+x2

)+(1x2

)ln(1x2

)] [( 1+x2 ) ln p+ ( 1x2 ) ln q] . (1.13)We have just shown that in the largeN limit we may write

PN,X = C eNf(X/N) , (1.14)

where C is a normalization constant2. Since N is by assumption large, the function PN,X is dominated by theminimum (orminima) of f(x), where the probability is maximized. To find the minimum of f(x), we set f (x) = 0,where

f (x) = 12 ln(q

p 1 + x1 x

). (1.15)

Setting f (x) = 0, we obtain1 + x

1 x =p

q x = p q . (1.16)

We also have

f (x) =1

1 x2 , (1.17)

so invoking Taylors theorem,

f(x) = f(x) + 12f(x) (x x)2 + . . . . (1.18)

Putting it all together, we have

PN,X C exp[ N(x x)

2

8pq

]= C exp

[ (X X)

2

8Npq

], (1.19)

where X = X = N(p q) = Nx. The constant C is determined by the normalization condition,

X=

PN,X 12

dX C exp

[ (X X)

2

8Npq

]=2Npq C , (1.20)

and thus C = 1/2Npq. Why dont we go beyond second order in the Taylor expansion of f(x)? We will findout in 1.4.2 below.

2The origin of C lies in theO(lnN) andO(N0) terms in the asymptotic expansion of lnN !. We have ignored these terms here. Accountingfor them carefully reproduces the correct value of C in eqn. 1.20.

1.2. A STATISTICAL VIEW 5

Figure 1.2: Comparison of exact distribution of eqn. 1.10 (red squares) with the Gaussian distribution of eqn. 1.19(blue line).

1.2.3 Entropy and energy

The function f(x) can be written as a sum of two contributions, f(x) = e(x) s(x), where

s(x) = (1+x2 ) ln ( 1+x2 ) ( 1x2 ) ln ( 1x2 )e(x) = 12 ln(pq) 12x ln(p/q) .

(1.21)

The function S(N, x) Ns(x) is analogous to the statistical entropy of our system3. We have

S(N, x) = Ns(x) = ln

(N

NR

)= ln

(N

12N(1 + x)

). (1.22)

Thus, the statistical entropy is the logarithm of the number of ways the system can be configured so as to yield the same valueof X (at fixed N ). The second contribution to f(x) is the energy term. We write

E(N, x) = Ne(x) = 12N ln(pq) 12Nx ln(p/q) . (1.23)

The energy term biases the probability PN,X = exp(S E) so that low energy configurations are more probable thanhigh energy configurations. For our system, we see that when p < q (i.e. p < 12 ), the energy is minimized by taking xas small as possible (meaning as negative as possible). The smallest possible allowed value of x = X/N is x = 1.Conversely, when p > q (i.e. p > 12 ), the energy is minimized by taking x as large as possible, which means x = 1.The average value of x, as we have computed explicitly, is x = p q = 2p 1, which falls somewhere in betweenthese two extremes.

In actual thermodynamic systems, as we shall see, entropy and energy are not dimensionless. What we havecalled S here is really S/kB, which is the entropy in units of Boltzmanns constant. And what we have called Ehere is really E/k

BT , which is energy in units of Boltzmanns constant times temperature.

3The function s(x) is the specific entropy.

6 CHAPTER 1. PROBABILITY

1.2.4 Entropy and information theory

It was shown in the classic 1948work of Claude Shannon that entropy is in fact a measure of information4. Supposewe observe that a particular event occurs with probability p. We associate with this observation an amount ofinformation I(p). The information I(p) should satisfy certain desiderata:

1 Information is non-negative, i.e. I(p) 0.2 If two events occur independently so their joint probability is p1 p2, then their information is additive, i.e.I(p1p2) = I(p1) + I(p2).

3 I(p) is a continuous function of p.

4 There is no information content to an event which is always observed, i.e. I(1) = 0.

From these four properties, it is easy to show that the only possible function I(p) is

I(p) = A ln p , (1.24)where A is an arbitrary constant that can be absorbed into the base of the logarithm, since logb x = lnx/ ln b. Wewill take A = 1 and use e as the base, so I(p) = ln p. Another common choice is to take the base of the logarithmto be 2, so I(p) = log2 p. In this latter case, the units of information are known as bits. Note that I(0) =. Thismeans that the observation of an extremely rare event carries a great deal of information.

Now suppose we have a set of events labeled by an integer n which occur with probabilities {pn}. What isthe expected amount of information in N observations? Since event n occurs an average of Npn times, and theinformation content in pn is ln pn, we have that the average information per observation is

S =IN N

= n

pn ln pn , (1.25)

which is known as the entropy of the distribution. Thus, maximizing S is equivalent tomaximizing the informationcontent per observation.

Consider, for example, the information content of course grades. As we have seen, if the only constraint on theprobability distribution is that of overall normalization, then S is maximized when all the probabilities pn areequal. The binary entropy is then S = log2 , since pn = 1/ . Thus, for pass/fail grading, the maximum averageinformation per grade is log2(12 ) = log2 2 = 1 bit. If only A, B, C, D, and F grades are assigned, then themaximum average information per grade is log2 5 = 2.32 bits. If we expand the grade options to include {A+, A,A-, B+, B, B-, C+, C, C-, D, F}, then the maximum average information per grade is log2 11 = 3.46 bits.Equivalently, consider, following the discussion in vol. 1 of Kardar, a random sequence {n1, n2, . . . , nN} whereeach element nj takes one ofK possible values. There are thenK

N such possible sequences, and to specify one of

them requires log2(KN ) = N log2K bits of information. However, if the value n occurs with probability pn, then

on average it will occur Nn = Npn times in a sequence of length N , and the total number of such sequences willbe

g(N) =N !K

n=1Nn!. (1.26)

In general, this is far less that the total possible numberKN , and the number of bits necessary to specify one fromamong these g(N) possibilities is

log2 g(N) = log2(N !)Kn=1

log2(Nn!) NKn=1

pn log2 pn , (1.27)

4See An Introduction to Information Theory and Entropy by T. Carter, Santa Fe Complex Systems Summer School, June 2011. Availableonline at http://astarte.csustan.edu/ tom/SFI-CSSS/info-theory/info-lec.pdf.

1.3. PROBABILITY DISTRIBUTIONS FROMMAXIMUM ENTROPY 7

where we have invoked Stirlings approximation. If the distribution is uniform, then we have pn =1K for all

n {1, . . . ,K}, and log2 g(N) = N log2K .

1.3 Probability Distributions fromMaximum Entropy

We have shown how one can proceed from a probability distribution and compute various averages. We nowseek to go in the other direction, and determine the full probability distribution based on a knowledge of certainaverages.

At first, this seems impossible. Suppose we want to reproduce the full probability distribution for an N -steprandom walk from knowledge of the average X = (2p 1)N . The problem seems ridiculously underdeter-mined, since there are 2N possible configurations for an N -step random walk: j = 1 for j = 1, . . . , N . Overallnormalization requires

{j}P (1, . . . , N ) = 1 , (1.28)

but this just imposes one constraint on the 2N probabilities P (1, . . . , N ), leaving 2N1 overall parameters. What

principle allows us to reconstruct the full probability distribution

P (1, . . . , N ) =Nj=1

(p j ,1 + q j ,1

)=

Nj=1

p(1+j)/2 q(1j)/2 , (1.29)

corresponding to N independent steps?

1.3.1 The principle of maximum entropy

The entropy of a discrete probability distribution {pn} is defined as

S = n

pn ln pn , (1.30)

where here we take e as the base of the logarithm. The entropy may therefore be regarded as a function of theprobability distribution: S = S

({pn}). One special property of the entropy is the following. Suppose we have twoindependent normalized distributions

{pAa}and

{pBb}. The joint probability for events a and b is then Pa,b = p

Aa p

Bb .

The entropy of the joint distribution is then

S = a

b

Pa,b lnPa,b = a

b

pAa pBb ln

(pAa p

Bb

)=

a

b

pAa pBb

(ln pAa + ln p

Bb

)=

a

pAa ln pAa b

pBb b

pBb ln pBb a

pAa = a

pAa ln pAa

b

pBb ln pBb

= SA + SB .

Thus, the entropy of a joint distribution formed from two independent distributions is additive.

Suppose all we knew about {pn} was that it was normalized. Then

n pn = 1. This is a constraint on the values{pn}. Let us now extremize the entropy S with respect to the distribution {pn}, but subject to the normalizationconstraint. We do this using Lagranges method of undetermined multipliers. We define

S({pn}, ) =

n

pn ln pn (

n

pn 1)

(1.31)

8 CHAPTER 1. PROBABILITY

and we freely extremize S over all its arguments. Thus, for all n we have

S

pn= ( ln pn + 1 + ) = 0 (1.32)

as well asS

=n

pn 1 = 0 . (1.33)

From the first of these equations, we obtain pn = e(1+), and from the second we obtain

n

pn = e(1+)

n

1 = e(1+) , (1.34)

where n 1 is the total number of possible events. Thus,pn =

1

, (1.35)

which says that all events are equally probable.

Now suppose we know one other piece of information, which is the average value of some quantity X =nXn pn. We now extremize S subject to two constraints, and so we define

S({pn}, 0, 1) =

n

pn ln pn 0(

n

pn 1) 1

(n

Xn pn X). (1.36)

We then haveS

pn= ( ln pn + 1 + 0 + 1Xn) = 0 , (1.37)

which yields the two-parameter distribution

pn = e(1+0) e1Xn . (1.38)

To fully determine the distribution {pn} we need to invoke the two equations

n pn = 1 and

nXn pn = X ,which come from extremizing S with respect to 0 and 1, respectively:

e(1+0)n

e1Xn = 1 (1.39)

e(1+0)n

Xn e1Xn = X . (1.40)

General formulation

The generalization toK extra pieces of information (plus normalization) is immediately apparent. We have

Xa =n

Xan pn , (1.41)

and therefore we define

S({pn}, {a}) =

n

pn ln pn Ka=0

a

(n

Xan pn Xa), (1.42)

1.3. PROBABILITY DISTRIBUTIONS FROMMAXIMUM ENTROPY 9

with X(a=0)n X(a=0) = 1. Then the optimal distribution which extremizes S subject to the K + 1 constraints is

pn = exp

{ 1

Ka=0

aXan

}

=1

Zexp

{

Ka=1

aXan

},

(1.43)

where Z = e1+0 is determined by normalization:

n pn = 1. This is a (K + 1)-parameter distribution, with{0, 1, . . . , K} determined by the K + 1 constraints in eqn. 1.41.

Example

As an example, consider the random walk problem. We have two pieces of information:1

N

P (1, . . . , N ) = 1 (1.44)

1

N

P (1, . . . , N )Nj=1

j = X . (1.45)

Here the discrete label n from 1.3.1 ranges over 2N possible values, and may be written as an N digit binarynumber rN r1, where rj = 12 (1 + j) is 0 or 1. Extremizing S subject to these constraints, we obtain

P (1, . . . , N ) = C exp{

j

j

}= C

Nj=1

ej , (1.46)

where C e(1+0) and 2. Normalization then requires

Tr P {j}

= C (e + e)N , (1.47)hence C = (cosh)N . We then have

P (1, . . . , N ) =

Nj=1

ej

e + e=

Nj=1

(p j ,1 + q j ,1

), (1.48)

where

p =e

e + e, q = 1 p = e

e + e. (1.49)

We then haveX = (2p 1)N , which determines p = 12 (N +X), and we have recovered the correct distribution.

1.3.2 Continuous probability distributions

Suppose we have a continuous probability density P () defined over some set . We have observables

Xa =

d Xa()P () , (1.50)

10 CHAPTER 1. PROBABILITY

where d is the appropriate integration measure. We assume d =Dj=1 dj , where D is the dimension of .

Then we extremize the functional

S[P (), {a}

]=

d P () lnP ()Ka=0

a

(

d P ()Xa()Xa)

(1.51)

with respect to P () and with respect to {a}. Again,X0() X0 1. This yields the following result:

lnP () = 1Ka=0

aXa() . (1.52)

The K + 1 Lagrange multipliers {a} are then determined from theK + 1 constraint equations in eqn. 1.50.As an example, consider a distribution P (x) over the real numbers R. We constrain

dx P (x) = 1 ,

dx xP (x) = ,

dx x2 P (x) = 2 + 2 . (1.53)

Extremizing the entropy, we then obtain

P (x) = C e1x2x2 , (1.54)where C = e(1+0). We already know the answer:

P (x) =122

e(x)2/22 . (1.55)

In other words, 1 = /2 and 2 = 1/22, with C = e2/22/

22.

1.4 General Aspects of Probability Distributions

1.4.1 Discrete and continuous distributions

Consider a system whose possible configurations |n can be labeled by a discrete variable n C, where C is theset of possible configurations. The total number of possible configurations, which is to say the order of the set C,may be finite or infinite. Next, consider an ensemble of such systems, and let Pn denote the probability that agiven random element from that ensemble is in the state (configuration) |n . The collection {Pn} forms a discreteprobability distribution. We assume that the distribution is normalized, meaning

nCPn = 1 . (1.56)

Now let An be a quantity which takes values depending on n. The average of A is given by

A =nC

Pn An . (1.57)

Typically, C is the set of integers (Z) or some subset thereof, but it could be any countable set. As an example,consider the throw of a single six-sided die. Then Pn =

16 for each n {1, . . . , 6}. Let An = 0 if n is even and 1 if n

is odd. Then find A = 12 , i.e. on average half the throws of the die will result in an even number.

1.4. GENERAL ASPECTS OF PROBABILITY DISTRIBUTIONS 11

It may be that the systems configurations are described by several discrete variables {n1, n2, n3, . . .}. We cancombine these into a vector n and then we write Pn for the discrete distribution, with

n Pn = 1.

Another possibility is that the systems configurations are parameterized by a collection of continuous variables, = {1, . . . , n}. We write , where is the phase space (or configuration space) of the system. Let d be ameasure on this space. In general, we can write

d =W (1, . . . , n) d1 d2 dn . (1.58)

The phase space measure used in classical statistical mechanics gives equal weight W to equal phase space vol-umes:

d = Cr

=1

dq dp , (1.59)

where C is a constant we shall discuss later on below5.Any continuous probability distribution P () is normalized according to

dP () = 1 . (1.60)

The average of a function A() on configuration space is then

A =

dP ()A() . (1.61)

For example, consider the Gaussian distribution

P (x) =122

e(x)2/22 . (1.62)

From the result6

dx ex2

ex =

e

2/4 , (1.63)

we see that P (x) is normalized. One can then compute

x = x2 x2 = 2 . (1.64)

We call the mean and the standard deviation of the distribution, eqn. 1.62.

The quantity P () is called the distribution or probability density. One has

P () d = probability that configuration lies within volume d centered at

For example, consider the probability density P = 1 normalized on the interval x [0, 1]. The probability thatsome x chosen at random will be exactly 12 , say, is infinitesimal one would have to specify each of the infinitelymany digits of x. However, we can say that x [0.45 , 0.55]with probability 110 .

5Such a measure is invariant with respect to canonical transformations, which are the broad class of transformations among coordinatesand momenta which leave Hamiltons equations of motion invariant, and which preserve phase space volumes under Hamiltonian evolution.For this reason d is called an invariant phase space measure. See the discussion in appendix II of chapter 4.

6Memorize this!

12 CHAPTER 1. PROBABILITY

If x is distributed according to P1(x), then the probability distribution on the product space (x1 , x2) is simply theproduct of the distributions:

P2(x1, x2) = P1(x1)P1(x2) . (1.65)

Suppose we have a function (x1, . . . , xN ). How is it distributed? LetQ() be the distribution for . We then have

P() =

dx1

dxN PN (x1, . . . , xN ) ((x1, . . . , xN )

)

=

dx1

xN P1(x1) P1(xN )

((x1, . . . , xN )

),

(1.66)

where the second line is appropriate if the {xj} are themselves distributed independently. Note that

d P() = 1 , (1.67)

so P() is itself normalized.

1.4.2 Central limit theorem

In particular, consider the distribution function of the sum

X =

Ni=1

xi . (1.68)

We will be particularly interested in the case where N is large. For generalN , though, we have

PN (X) =

dx1

dxN P1(x1) P1(xN ) (x1 + x2 + . . .+ xN X

). (1.69)

1.4. GENERAL ASPECTS OF PROBABILITY DISTRIBUTIONS 13

It is convenient to compute the Fourier transform7 of P(X):

PN(k) =

dX PN (X) eikX (1.70)

=

dX

dx1

xN P1(x1) P1(xN )

(x1 + . . .+ xN X) eikX

=[P1(k)

]N,

where

P1(k) =

dxP1(x) eikx (1.71)

is the Fourier transform of the single variable distribution P1(x). The distribution PN(X) is a convolution of theindividual P1(xi) distributions. We have therefore proven that the Fourier transform of a convolution is the product ofthe Fourier transforms.

OK, now we can write for P1(k)

P1(k) =

dxP1(x)(1 ikx 12 k2x2 + 16 i k3 x3 + . . .

)= 1 ikx 12 k2x2+ 16 i k3x3+ . . . .

(1.72)

Thus,

ln P1(k) = ik 122k2 + 16 i 3 k3 + . . . , (1.73)where

= x2 = x2 x23 = x3 3 x2 x+ 2 x3

(1.74)

We can now write [P1(k)

]N= eiNk eN

2k2/2 eiN3k3/6 (1.75)

7Jean Baptiste Joseph Fourier (1768-1830) had an illustrious career. The son of a tailor, and orphaned at age eight, Fouriers ignoble status

rendered him ineligible to receive a commission in the scientific corps of the French army. A Benedictine minister at the Ecole Royale Militaireof Auxerre remarked, Fourier, not being noble, could not enter the artillery, although he were a second Newton. Fourier prepared for the priesthood,but his affinity for mathematics proved overwhelming, and so he left the abbey and soon thereafter accepted a military lectureship position.Despite his initial support for revolution in France, in 1794 Fourier ran afoul of a rival sect while on a trip to Orleans andwas arrested and verynearly guillotined. Fortunately the Reign of Terror ended soon after the death of Robespierre, and Fourier was released. He went on NapoleonBonapartes 1798 expedition to Egypt, where he was appointed governor of Lower Egypt. His organizational skills impressed Napoleon, andupon return to France he was appointed to a position of prefect in Grenoble. It was in Grenoble that Fourier performed his landmark studiesof heat, and his famous work on partial differential equations and Fourier series. It seems that Fouriers fascination with heat began in Egypt,where he developed an appreciation of desert climate. His fascination developed into an obsession, and he became convinced that heat couldpromote a healthy body. He would cover himself in blankets, like a mummy, in his heated apartment, even during the middle of summer.On May 4, 1830, Fourier, so arrayed, tripped and fell down a flight of stairs. This aggravated a developing heart condition, which he refusedto treat with anything other than more heat. Two weeks later, he died. Fouriers is one of the 72 names of scientists, engineers and otherluminaries which are engraved on the Eiffel Tower. Source: http://www.robertnowlan.com/pdfs/Fourier,%20Joseph.pdf

14 CHAPTER 1. PROBABILITY

Now for the inverse transform. In computing PN (X), we will expand the term eiN3k3/6 and all subsequent terms

in the above product as a power series in k. We then have

PN (X) =

dk

2eik(XN) eN

2k2/2{1 + 16 i N

3k3 + . . .}

=

(1 16N3

3

X3+ . . .

)1

2N2e(XN)

2/2N2

=1

2N2e(XN)

2/2N2 (N ) .

(1.76)

In going from the second line to the third, we have written X =N , in which case N

3

X3 = N1/2 3

3 , which

gives a subleading contribution which vanishes in the N limit. We have just proven the central limit theorem:in the limit N , the distribution of a sum of N independent random variables xi is a Gaussian with meanNand standard deviation

N . Our only assumptions are that the mean and standard deviation exist for the

distribution P1(x). Note that P1(x) itself need not be a Gaussian it could be a very peculiar distribution indeed,but so long as its first and second moment exist, where the kth moment is simply xk, the distribution of the sumX =

Ni=1 xi is a Gaussian.

1.4.3 Multidimensional Gaussian integral

Consider the multivariable Gaussian distribution,

P (x) (

detA

(2)n

)1/2exp

( 12 xiAij xj

), (1.77)

where A is a positive definite matrix of rank n. A mathematical result which is extremely important throughoutphysics is the following:

Z(b) =

(detA

(2)n

)1/2

dx1

dxn exp

( 12 xiAij xj + bi xi

)= exp

(12 biA

1ij bj

). (1.78)

Here, the vector b = (b1 , . . . , bn) is identified as a source. Since Z(0) = 1, we have that the distribution P (x) isnormalized. Now consider averages of the form

xj1 xj2k =dnx P (x) xj1 xj2k

=nZ(b)

bj1 bj

2k

b=0

=

contractions

A1j(1)

j(2) A1j

(2k1)j(2k)

.

(1.79)

The sum in the last term is over all contractions of the indices {j1 , . . . , j2k}. A contraction is an arrangement ofthe 2k indices into k pairs. There are C2k = (2k)!/2

kk! possible such contractions. To obtain this result for Ck,we start with the first index and then find a mate among the remaining 2k 1 indices. Then we choose the nextunpaired index and find a mate among the remaining 2k 3 indices. Proceeding in this manner, we have

C2k = (2k 1) (2k 3) 3 1 =(2k)!

2kk!. (1.80)

1.5. APPENDIX : BAYESIAN STATISTICS 15

Equivalently, we can take all possible permutations of the 2k indices, and then divide by 2kk! since permuta-tion within a given pair results in the same contraction and permutation among the k pairs results in the samecontraction. For example, for k = 2, we have C4 = 3, and

xj1xj2xj3xj4 = A1j1j2

A1j3j4 +A1j1j3

A1j2j4 +A1j1j4

A1j2j3 . (1.81)

1.5 Appendix : Bayesian Statistics

Let the probability of a discrete event A be P (A). We now introduce two additional probabilities. The jointprobability for events A and B together is written P (A B). The conditional probability of B given A is P (B|A). Wecan compute the joint probability P (A B) = P (B A) in two ways:

P (A B) = P (A|B) P (B) = P (B|A) P (A) . (1.82)

Thus,

P (A|B) = P (B|A) P (A)P (B)

, (1.83)

a result known as Bayes theorem. Now suppose the event space is partitioned as {Ai}. Then

P (B) =i

P (B|Ai) P (Ai) . (1.84)

We then have

P (Ai|B) =P (B|Ai) P (Ai)j P (B|Aj) P (Aj)

, (1.85)

a result sometimes known as the extended form of Bayes theorem. When the event space is a binary partition{A,A}, we have

P (A|B) = P (B|A) P (A)P (B|A) P (A) + P (B|A) P (A) . (1.86)

Note that P (A|B) + P (A|B) = 1 (which follows from A = A).As an example, consider the following problem in epidemiology. Suppose there is a rare but highly contagiousdisease A which occurs in 0.01% of the general population. Suppose further that there is a simple test for thedisease which is accurate 99.99% of the time. That is, out of every 10,000 tests, the correct answer is returned 9,999times, and the incorrect answer is returned only once8. Now let us administer the test to a large group of peoplefrom the general population. Those who test positive are quarantined. Question: what is the probability thatsomeone chosen at random from the quarantine group actually has the disease? We use Bayes theorem with thebinary partition {A,A}. Let B denote the event that an individual tests positive. Anyone from the quarantinegroup has tested positive. Given this datum, we want to know the probability that that person has the disease.That is, we want P (A|B). Applying eqn. 1.86 with

P (A) = 0.0001 , P (A) = 0.9999 , P (B|A) = 0.9999 , P (B|A) = 0.0001 ,

we find P (A|B) = 12 . That is, there is only a 50% chance that someone who tested positive actually has the disease,despite the test being 99.99% accurate! The reason is that, given the rarity of the disease in the general population,the number of false positives is statistically equal to the number of true positives.

8Epidemiologists define the sensitivity of a binary classification test as the fraction of actual positives which are correctly identified, andthe specificity as the fraction of actual negatives that are correctly identified. In our example in the text, the sensitivity and specificity are both0.9999.

16 CHAPTER 1. PROBABILITY

For continuous distributions, we speak of a probability density. We then have

P (y) =

dx P (y|x) P (x) (1.87)

and

P (x|y) = P (y|x) P (x)dx P (y|x) P (x) . (1.88)

The range of integration may depend on the specific application.

The quantities P (Ai) are called the prior distribution. Clearly in order to compute P (B) or P (Ai|B)we must knowthe priors, and this is usually the weakest link in the Bayesian chain of reasoning. If our prior distribution is notaccurate, Bayes theorem will generate incorrect results. One approach to obtaining the prior probabilities P (Ai)is to obtain them from a maximum entropy construction.

Chapter 2

Thermodynamics

2.1 References

E. Fermi, Thermodynamics (Dover, 1956)This outstanding and inexpensive little book is a model of clarity.

A. H. Carter, Classical and Statistical Thermodynamics(Benjamin Cummings, 2000)A very relaxed treatment appropriate for undergraduate physics majors.

H. B. Callen, Thermodynamics and an Introduction to Thermostatistics(2nd edition, Wiley, 1985)A comprehensive text appropriate for an extended course on thermodynamics.

D. V. Schroeder,An Introduction to Thermal Physics (Addison-Wesley, 2000)An excellent thermodynamics text appropriate for upper division undergraduates. Contains many illustra-tive practical applications.

D. Kondepudi and I. Prigogine, Modern Thermodynamics: From Heat Engines to Dissipative Structures(Wiley, 1998)Lively modern text with excellent choice of topics and good historical content. More focus on chemical andmaterials applications than in Callen.

L. E. Reichl, AModern Course in Statistical Physics (2nd edition, Wiley, 1998)A graduate level text with an excellent and crisp section on thermodynamics.

17

18 CHAPTER 2. THERMODYNAMICS

2.2 What is Thermodynamics?

Thermodynamics is the study of relations among the state variables describing a thermodynamic system, and oftransformations of heat into work and vice versa.

2.2.1 Thermodynamic systems and state variables

Thermodynamic systems contain large numbers of constituent particles, and are described by a set of state variableswhich describe the systems properties in an average sense. State variables are classified as being either extensiveor intensive.

Extensive variables, such as volume V , particle number N , total internal energy E, magnetization M , etc., scalelinearly with the system size, i.e. as the first power of the system volume. If we take two identical thermodynamicsystems, place them next to each other, and remove any barriers between them, then all the extensive variableswill double in size.

Intensive variables, such as the pressure p, the temperature T , the chemical potential , the electric fieldE, etc., areindependent of system size, scaling as the zeroth power of the volume. They are the same throughout the system,if that system is in an appropriate state of equilibrium. The ratio of any two extensive variables is an intensivevariable. For example, we write n = N/V for the number density, which scales as V 0. Intensive variables mayalso be inhomogeneous. For example, n(r) is the number density at position r, and is defined as the limit ofN/Vof the number of particlesN inside a volume V which contains the point r, in the limit V V V/N .Classically, the full motion of a system of N point particles requires 6N variables to fully describe it (3N positionsand 3N velocities or momenta, in three space dimensions)1. Since the constituents are very small, N is typicallyvery large. A typical solid or liquid, for example, has a mass density on the order of 1 g/cm3; for gases, 103 g/cm3. The constituent atoms have masses of 100 to 102 grams per mole, where one mole of X containsNA of X , and NA = 6.0221415 1023 is Avogadros number. Thus, for solids and liquids we roughly expectnumber densities n of 102 100mol/cm3 for solids and liquids, and 105 103mol/cm3 for gases. Clearlywe are dealing with fantastically large numbers of constituent particles in a typical thermodynamic system. Theunderlying theoretical basis for thermodynamics, where we use a small number of state variables to describe asystem, is provided by the microscopic theory of statistical mechanics, which we shall study in the weeks ahead.

Intensive quantities such as p, T , and n ultimately involve averages over both space and time. Consider forexample the case of a gas enclosed in a container. We can measure the pressure (relative to atmospheric pressure)by attaching a spring to a moveable wall, as shown in Fig. 2.2. From the displacement of the spring and the valueof its spring constant k we determine the force F . This force is due to the difference in pressures, so p = p0+F/A.Microscopically, the gas consists of constituent atoms or molecules, which are constantly undergoing collisionswith each other and with the walls of the container. When a particle bounces off a wall, it imparts an impulse2n(n p), where p is the particles momentum and n is the unit vector normal to the wall. (Only particles withp n > 0will hit the wall.) Multiply this by the number of particles colliding with the wall per unit time, and onefinds the net force on the wall; dividing by the area gives the pressure p. Within the gas, each particle travels fora distance , called the mean free path, before it undergoes a collision. We can write = v , where v is the averageparticle speed and is the mean free time. When we study the kinetic theory of gases, we will derive formulasfor and v (and hence ). For now it is helpful to quote some numbers to get an idea of the relevant distanceand time scales. For O2 gas at standard temperature and pressure (T = 0

C, p = 1 atm), the mean free path is 1.1 105 cm, the average speed is v 480m/s, and the mean free time is 2.5 1010 s. Thus, particles inthe gas undergo collisions at a rate 1 4.0109 s1. A measuring device, such as our spring, or a thermometer,

1For a system of N molecules which can freely rotate, we must then specify 3N additional orientational variables the Euler angles andtheir 3N conjugate momenta. The dimension of phase space is then 12N .

2.2. WHAT IS THERMODYNAMICS? 19

Figure 2.1: From microscale to macroscale : physical versus social sciences.

effectively performs time and space averages. If there areNc collisions with a particular patch of wall during sometime interval on which our measurement device responds, then the root mean square relative fluctuations in the

local pressure will be on the order of N1/2c times the average. Since Nc is a very large number, the fluctuations

are negligible.

If the system is in steady state, the state variables do not change with time. If furthermore there are no macroscopiccurrents of energy or particle number flowing through the system, the system is said to be in equilibrium. Acontinuous succession of equilibrium states is known as a thermodynamic path, which can be represented as asmooth curve in a multidimensional space whose axes are labeled by state variables. A thermodynamic processis any change or succession of changes which results in a change of the state variables. In a cyclic process, theinitial and final states are the same. In a quasistatic process, the system passes through a continuous succession ofequilibria. A reversible process is one where the external conditions and the thermodynamic path of the system canbe reversed (at first this seems to be a tautology). All reversible processes are quasistatic, but not all quasistaticprocesses are reversible. For example, the slow expansion of a gas against a piston head, whose counter-force isalways infinitesimally less than the force pA exerted by the gas, is reversible. To reverse this process, we simplyadd infinitesimally more force to pA and the gas compresses. A quasistatic process which is not reversible: slowlydragging a block across the floor, or the slow leak of air from a tire. Irreversible processes, as a rule, are dissipative.Other special processes include isothermal (dT = 0) isobaric (dp = 0), isochoric (dV = 0), and adiabatic (dQ = 0,i.e. no heat exchange):

reversible: dQ = T dS isothermal: dT = 0

spontaneous: dQ < T dS isochoric: dV = 0

adiabatic: dQ = 0 isobaric: dp = 0

quasistatic: infinitely slowly

We shall discuss later the entropy S and its connection with irreversibility.

How many state variables are necessary to fully specify the equilibrium state of a thermodynamic system? Fora single component system, such as water which is composed of one constituent molecule, the answer is three.These can be taken to be T , p, and V . One always must specify at least one extensive variable, else we cannotdetermine the overall size of the system. For a multicomponent system with g different species, we must specifyg + 2 state variables, which may be {T, p,N1, . . . , Ng}, where Na is the number of particles of species a. Anotherpossibility is the set (T, p, V, x1, . . . , xg1}, where the concentration of species a is xa = Na/N . Here,N =

ga=1Na

is the total number of particles. Note thatg

a=1 xa = 1.

If then follows that if we specify more than g + 2 state variables, there must exist a relation among them. Suchrelations are known as equations of state. The most famous example is the ideal gas law,

pV = NkBT , (2.1)

20 CHAPTER 2. THERMODYNAMICS

Figure 2.2: The pressure p of a gas is due to an average over space and time of the impulses due to the constituentparticles.

relating the four state variables T , p, V , and N . Here kB = 1.3806503 1016 erg/K is Boltzmanns constant.Another example is the van der Waals equation,(

p+aN2

V 2

)(V bN) = NkBT , (2.2)

where a and b are constants which depend on the molecule which forms the gas. For a third example, consider aparamagnet, where

M

V=CH

T, (2.3)

whereM is the magnetization,H the magnetic field, and C the Curie constant.

Any quantity which, in equilibrium, depends only on the state variables is called a state function. For example,the total internal energy E of a thermodynamics system is a state function, and we may write E = E(T, p, V ).State functions can also serve as state variables, although the most natural state variables are those which can bedirectly measured.

2.2.2 Heat

Once thought to be a type of fluid, heat is now understood in terms of the kinetic theory of gases, liquids, andsolids as a form of energy stored in the disordered motion of constituent particles. The units of heat are thereforeunits of energy, and it is appropriate to speak of heat energy, which we shall simply abbreviate as heat:2

1 J = 107 erg = 6.242 1018 eV = 2.390 104 kcal = 9.478 104BTU . (2.4)We will use the symbol Q to denote the amount of heat energy absorbed by a system during some given ther-modynamic process, and dQ to denote a differential amount of heat energy. The symbol d indicates an inexactdifferential, about which we shall have more to say presently. This means that heat is not a state function: thereis no heat function Q(T, p, V ).

2One calorie (cal) is the amount of heat needed to raise 1 g of H2O from T0 = 14.5 C to T1 = 15.5

C at a pressure of p0 = 1 atm. OneBritish Thermal Unit (BTU) is the amount of heat needed to raise 1 lb. of H2O from T0 = 63

F to T1 = 64 F at a pressure of p0 = 1 atm.

2.2. WHAT IS THERMODYNAMICS? 21

2.2.3 Work

In general we will write the differential element of work dW done by the system as

dW =i

Fi dXi , (2.5)

where Fi is a generalized force and dXi a generalized displacement3. The generalized forces and displacements are

themselves state variables, and by convention we will take the generalized forces to be intensive and the general-ized displacements to be extensive. As an example, in a simple one-component system, we have dW = p dV . Moregenerally, we write

dW =

Pj yj dXj (p dV H dM E dP dA+ . . . )

Pa a dNa (

1 dN1 + 2 dN2 + . . .)

(2.6)

Here we distinguish between two types of work. The first involves changes in quantities such as volume, mag-netization, electric polarization, area, etc. The conjugate forces yi applied to the system are then p, the magneticfieldH , the electric field E, the surface tension , respectively. The second type of work involves changes in thenumber of constituents of a given species. For example, energy is required in order to dissociate two hydrogenatoms in an H2 molecule. The effect of such a process is dNH2 = 1 and dNH = +2.As with heat, dW is an inexact differential, and workW is not a state variable, since it is path-dependent. There isno work functionW (T, p, V ).

2.2.4 Pressure and Temperature

The units of pressure (p) are force per unit area. The SI unit is the Pascal (Pa): 1Pa = 1N/m2 = 1kg/ms2. Otherunits of pressure we will encounter:

1 bar 105Pa1 atm 1.01325 105 Pa1 torr 133.3Pa .

Temperature (T ) has a very precise definition from the point of view of statistical mechanics, as we shall see. Manyphysical properties depend on the temperature such properties are called thermometric properties. For example,the resistivity of a metal (T, p) or the number density of a gas n(T, p) are both thermometric properties, and canbe used to define a temperature scale. Consider the device known as the constant volume gas thermometerdepicted in Fig. 2.3, in which the volume or pressure of a gas may be used to measure temperature. The gasis assumed to be in equilibrium at some pressure p, volume V , and temperature T . An incompressible fluid ofdensity is used to measure the pressure difference p = p p0, where p0 is the ambient pressure at the top ofthe reservoir:

p p0 = g(h2 h1) , (2.7)where g is the acceleration due to gravity. The height h1 of the left column of fluid in the U-tube provides ameasure of the change in the volume of the gas:

V (h1) = V (0)Ah1 , (2.8)whereA is the (assumed constant) cross-sectional area of the left arm of the U-tube. The device can operate in twomodes:

3We use the symbol d in the differential dW to indicate that this is not an exact differential. More on this in section 2.4 below.

22 CHAPTER 2. THERMODYNAMICS

Figure 2.3: The constant volume gas thermometer. The gas is placed in thermal contact with an object of temper-at