Higher-order Time Integration of Stochastic Differential ... · Higher-order Time Integration of Stochastic Differential Equations and Application to Coulomb Collisions! Presented

Higher-order Time Integration of Stochastic Differential Equations and Application to

Coulomb Collisions!

Presented at the 2011 Applied Mathematics Program Meeting Reston, VA, October 18, 2011

A.M. Dimits, B.I. Cohen, LLNL R. E. Caflisch, M. S. Rosin, UCLA

Work performed for U.S. DOE by LLNL under Contract DE-AC52-07NA27344, and by UCLA under grant DE-FG02-05ER25710.

LLNL-PRES-506061!

Main results

We have developed a higher (Milstein)-order Coulomb-Langevinscheme

� improved convergence demonstrated� correct mean behavior demonstrated

A new approach was needed� existing approach does not extend easily to higher order

New method developed for sampling area integral terms� simple, accurate, efficient

A.M. Dimits, B.I. Cohen, LLNL R. E. Caflisch, M. S. Rosin, UCLA (Presented at the 2011 Applied Mathematics Program Meeting Reston, VA, October 18, 2011 Work performed for U.S. DOE by LLNL under Contract DE-AC52-07NA27344, and by UCLA under grant DE-FG02-05ER25710)Higher-order Time Integration of Stochastic Differential Equations and Application to Coulomb CollisionsOctober 18, 2011 2 / 31

Coulomb collisions are important in many plasmaapplications

Any sufficiently dense plasma� Magnetic fusion (MFE), inertial fusion (ICF), plasma processing,

near-earth (or planetary) space plasma

Long history of study of Coulomb collisions in plasmas� Analytical results

� Landau ‘36-7; Rosenbluth et. al.; ‘57, Trubnikov‘65

� Monte-Carlo (SDE) methods� Langevin (+ field-term) methods - of interest in the present work:

Manheimer et. al., ‘97; Lemons et. al., ‘09; Cohen et. al., ‘10� Binary-collision methods - used in our hybrid work: Takizuke and Abe

‘77; Nanbu ‘97; Dimits et. al., ‘09

� Continuum (PDE) methods� e.g., Xiong, et. al., ‘08; Abel et. al., ‘08


Coulomb collisions are long-range, unlikeneutral-atomic/molecular collisions

Dominated by many small-angle scattering “events”� large-angle scattering events are subdominant

Appropriate description is a Fokker-Planck (forward Kolmogorov)equation (Landau, 1936/7 - not a Boltzmann equation):

∂fα

∂t

��coll

=∂

∂v·

πq2αL�

β

q2β

ˆdτ

�

�fα

∂f �β

∂v� − f�β∂fα

∂v

� �u2I− uu

�

u3


The Milstein method is the first in a hierarchy ofhigher-order methods for SDE’s

δYin,j = a

i(tn,j ,Yn,j)δt+ bi(tn,j ,Yn,j)δW

in,j

tn,j = tn + jδt, tn = t0 + n∆t, ∆t = Nδt

δW in,j are independent normal random numbers with variance δt .

∆W ≡ W (tn+1)−W (tn) = limN→∞�N

j=1 δWn,j - WienerincrementFirst-order (in �t) approximation to �Y N ≡ limN→∞

�Nk=1 δY k

∆Yin = a

i(tn,Yn)∆t+ bi(tn,Yn)∆W

in

+ bi,j(tn,Yn)b

j(tn,Yn)

ˆ ∆t

0dW

i(tn + s)

ˆ s

0dW

j(tn + η)

+

�O�∆t3/2

�− strong

O�∆t2

�− weak

→�O (T∆t) − strong

O (T∆t) − weak


The Milstein method is of interest because it is the firstin a hierarchy of higher-order methods for SDE’s

This hierarchy includes schemes with improved (higher-order)weak convergenceSignificantly improves efficiency of multi-(time-)level schemes(Giles ‘07), which have lower computational complexity for a givenoverall error than single-level Monte-Carlo schemes.


Higher-order methods for SDE’s have been applied ina variety of fields

FinanceChemical PhysicsSee, e.g., Kloeden and Platen. ‘92Almost all published Monte-Carlo treatments of Coulombcollisions have used the low-order Euler-Maruyama method.

� One exception: Lemons et. al., ‘09� Added higher order (Milstein) term for v, but� Did not do tests that might have shown a difference� Did not include higher order terms for angular scattering


Approach 1: Apply collisional drag and scattering in aframe aligned with particle velocity

Manheimer et al, ‘97; Lemons et. al., ‘09; Cohen et. al., ‘10Basic underlying equation:

dv (t) =�Fd (v) dt+Q� (v) dW

� (t)�v (t) +Q⊥ (v) dW (t) ,

dW⊥ (t) = dWx (t) x (t) + dW

y (t) y (t) .

Here� interpret in Ito sense� �v (t) ≡

´ t+∆tt dv (t)

� (x, y, v) - frame aligned with v

� e.g., x = y0 × v/ |y0 × v|, y = v × x


Milstein-order velocity step for Approach 1

First-order accurate (in �t) approximation to �v (t) ≡´ t+∆tt dv (t)

�v = Q�0∆t1/2�W

� v0 +Q⊥0∆t1/2 (�W

x x0 +�Wy y0)

+

��t Fd (v0) +

1

2Q�0Q

��0∆t

��W

��2 − 1�

−Q2

⊥0

2v0∆t

��[�W

x]2 − 1�+�[�W

y]2 − 1��

v0

+ Q�0Q�⊥0∆t

�A

x�x0 +Ay�y0

�+O

��t

3/2�

After applying �v, to get new v0 new ≡ v0 old +∆v, apply next �vusing a frame aligned with v0 new


Several possible choices for other unit vectors

1 Second vector along line of constant longitude in lab frame

x0 = θlab

y0 = v0 × x0

2 Second vector othogogonal to fixed plane

x0 = ylab × v0/ |ylab × v0|y0 = v0 × x0

3 Rotate unit vector system as a rigid body about the single axis thatgives the change in v0


Approach 1 achieves O (�t) strong convergence for v,but not for angular component of the evolution

400 realizations; time step range = 310; end time ν (vth) tend = 0.1

Green-Euler, red-Milstein fixed-plane, blue-Milstein rigid rot.

��v2end(dt)− v2end−fine

�� |vx(dt)− vxfine|


(better) Approach 2: formulate whole problem asSDE’s for spherical coordinates wrt a fixed (lab.) frame

Coordinates: v, µ = cos θ, φ; θ = polar angle, φ = azimuthal angleFrom Rosenbluth et. al., ‘57,

1

Γtf

�∂ft

∂t

�

c

= − 1

v2

∂

∂v

��v2∂h

∂v+

∂g

∂v

�ft

�+

1

2v2∂2

∂v2

�v2 ∂

2g

∂v2ft

�

+1

2v3∂g

∂v

�∂

∂µ

��1− µ

2� ∂ft∂µ

�+

1

(1− µ2)

∂2ft

∂φ2

�.

Γtf =4πq2t q

2fλ

m2t

.

For a Maxwellian field-particle plasma, have analyticalexpressions for g (v) and h (v) (Trubnikov, ‘65).


Coulomb test-particle problem as SDE’s for sphericalcoordinates wrt a fixed frame

Write as Ito form drag-diffusion (forward Kolmogorov) equation:�∂ft

∂ t

�

c

= − ∂

∂v

�Fd(v) ft

�+

∂2

∂v2

�Dv(v) ft

�+

∂

∂µ

�2Da(v)µ ft

�

+∂2

∂µ2

�Da(v)

�1− µ

2�ft

�+

∂2

∂φ2

�Da(v)

(1− µ2)ft

�,

where ft = 2πv2ftCorresponding Ito-Langevin equations:

dv (t) = Fd (v) dt+�

2Dv(v)dWv (t) ,

dµ (t) = −2Da(v)µdt+�2Da(v) (1− µ2)dWµ (t) ,

dφ (t) =

�2Da(v)

(1− µ2)dWφ (t) .


Milstein scheme for Coulomb test-particle problem

∆v = Fd0∆t+�

2Dv0∆Wv + κMD�v01

2

�∆W

2v −∆t

�,

∆µ = −2Da0µ0∆t+�2Da0

�1− µ2

0

�∆Wµ,

+κM

�−2Da0µ0

1

2

�∆W

2µ −∆t

�+

�Dv0

Da0

��1− µ2

0

�D

�a0Avµ

�,

∆φ =

�2Da(v)

1− µ20

∆Wφ + κM

��Dv0

Da0

D�a0�

1− µ20

Avφ +2Da0µ0

1− µ20

Aµφ

�,

∆ψ = ψ (ti+1)− ψ (ti) ,

ψ0 = ψ (ti) ,

Akl =

ˆ ti+1

ti

dWl (s)

ˆ s

ti

dWk (ξ) ,


New approach achieves O (�t) strong convergencefor v and for angular component

v evolution unaffected by angular evolution, and ∴ by area termsAngular evolution has poor convergence without area terms16 realizations; time step range = 38; end time ν (vth) tend = 0.1

Blue-Euler, Green-Milstein diagonal, Red-full Milstein

||vend(∆t)|− |vend−fine|| |µend(∆t)− µend−fine|


New approach gives correct dependences forvelocity-space density functions (“distributions”)

Blue - initial; other curves at tend; yellow - coarsest ∆t

10000 particles; end time ν (vth) tend = 10;∆/tend = 3−4, 3−5, 3−6, 3−7

f (v) f (µ)


Theory and numerical implementations exist for thesampling of the stochastic integral terms

ˆ ∆t

0dW

i(tn + s)

ˆ s

0dW

j(tn + η) =

12

��∆W i

n

�2 −∆t

�, i = j

12

�∆W i

n∆Wjn + L

i,jn

�, i �= j

Levy, ‘51

PcL�Li,jn |∆W

in,∆W

jn

�= PcL

�Li,jn |Ri,j

n

�

ri,jn =

�(∆W i

n)2 +

�∆W

jn

�2

φcL (k|R) ≡ �exp (−ikL)�|R

=k/2

sinh (k/2)exp

�R2

2

�1− (k/2) cosh (k/2)

sinh (k/2)

��.


We have developed a simple accurate method forsampling area integrals

Existing methods� Interpolation from 2D table based on Levy’s results (Gaines and

Lyons ‘94)� accurate and efficient� somewhat involved� challenging for conditional sampling - adaptive integration

� Discrete approximations (Clark and Cameron ‘80; Kloeden andPlaten ‘92; Gaines and Lyons ‘97)

� simple to implement� straightforward for adaptive integration� expensive for good accuracy (many random numbers per L sample)

Our method is a simplification of that of Gaines and Lyons ‘94� based on an accurate approximation to Levy’s PDF� can implement with 1D tables or analytical functions� can be used to significantly reduce memory and computation

requirements for conditional sampling


Our approximation for the Levy-area PDF is based onapproximate shape invariance of PcL (L|R)

Approximation to conditional PDF of L given R

PcL (L|R) ≈ Pc−anL (L|R) = s (R)P0L (s (R)L)

P0L (L) ≡ PcL (L|R = 0) =π

2

1

cosh2 (L/2)− exact

s (R) = PcL (L = 0|R)

Can calculate s (R) from 1D table or analytical fitResulting algorithm for sampling L

LR (R) =s (R)

2πlog

�u

1− u

�.


Our approximation for the Levy-area PDF is accurateto ∼ 1%

Exact and approximate conditional PDF’sof L given R vs. L for R = 0, 1, 2, 3


Our approximation for the Levy-area PDF is accurateto ∼ 1%

joint PDF of L and R absolute error in P (L,R)


For strong convergence studies, Wiener incrementsand area integrals must be compounded

Need to calculate trajectories representing a given underlyingrealization with different ∆t

Compounding is also needed for multilevel (Giles) schemesCompounding for Wiener increments: given δjW ≡

´ tjtj−1

dW (s),where tj = tj−1 + δt, and ∆t = nδt

∆W ≡ˆ ∆t

0dW (s) =

n�

j=1

δjW.

Compounding area integrals:

δA12j ≡ˆ tj

tj−1

dW1 (η)

ˆ η

tj−1

dW2 (ξ) ,

∆A12 ≡ˆ ∆t

0dW1 (η)

ˆ η

0dW2 (ξ) ,


Compounding of area integrals

∆A12 =n�

i=2

δiW1

i−1�

j=1

δjW2 + δA12,j

.

!

"

#$%&'(

#)*%+#&*&

,

!

" # $ % & '(


Our sampling and compounding algorithms andimplementations work

PDF for 9× 104 samples compounded by factor of 5

Strong scaling results for 2D Milstein (e.g., above collision results)provide a demonstration


Conditional sampling is needed for (time-) adaptiveSDE integration

Sample finer triplets (δjW1, δjW2, δjA12) given the coarser ones(∆W1,∆W2,∆A12)

Reverse of compoundingExisting methods are based on discrete representations

� expensive because many (pseudo)random numbers needed persample

Direct conditional sampling can be done� construct Pc (δL|δR,∆L,∆R) using Levy’s result for PcL (L|R)� store as 4D table� interpolate

Our approximation Pc−anL (L|R) = s (R)P0L (s (R)L) reducesdimensionality of conditional sampling PDF to 3

� much more manageable memory requirement


Summary

We have developed a higher (Milstein)-order Coulomb-Langevinscheme

� improved convergence demonstrated� correct mean behavior demonstrated

A new approach was needed� existing approach does not extend easily to higher order

New method developed for sampling area integral terms� simple, accurate, efficient� implemented (along with compounding)

Future work in this direction� higher-order weak schemes� implement Giles’ multilevel scheme� higher order adaptive SDE integrator


backup slides


Strong convergence resultsConvergence of trajectories (e.g., v at a given time) as �t → 0.

4 trajectories compted with different time steps


ReferencesBasic theory of Coulomb collisions in plasmas

� L. D. Landau, Phys. Z. Sowjet 10, 154 (1936); JETP 7, 203 (1937)� M. N. Rosenbluth, W. M. MacDonald and D. L. Judd, Phys. Rev.

107, 1 (1957)� review: B.A. Trubnikov, in Reviews of Plasma Physics (M. A.

Leontovich, ed., Consultants Bureau, New York) 1, 105 (1965)Monte-Carlo methods for Coulomb collisions

� Langevin (+ field-term) methods - of interest in the present work� W. M. Manheimer, M. Lampe, and G. Joyce J. Comp. Phys. 138,

563-584 (1997)� D. S. Lemons, D. Winske, W. Daughton, B. Albright, J. Comput. Phys.

228, 1391-1403 (2009)� B. I. Cohen, A. M. Dimits, A. Friedman, R. E. Caflisch, IEEE Trans.

Plasma Sci. 38, (2010)� Binary-collision methods - used in our hybrid work

� T. Takizuke and H. Abe, J. Comput. Phys. 25, 205-219 (1977)� K. Nanbu, Phys. Rev. E55, 4642 (1997)� A. M. Dimits, C. M. Wang, R. E. Caflisch, B. I. Cohen Y. Huang, J.

Comp. Phys. 228, 4881 (2009)A.M. Dimits, B.I. Cohen, LLNL R. E. Caflisch, M. S. Rosin, UCLA (Presented at the 2011 Applied Mathematics Program Meeting Reston, VA, October 18, 2011 Work performed for U.S. DOE by LLNL under Contract DE-AC52-07NA27344, and by UCLA under grant DE-FG02-05ER25710)Higher-order Time Integration of Stochastic Differential Equations and Application to Coulomb CollisionsOctober 18, 2011 29 / 31

References, contd.

Continuum methods for Coulomb collisions� Z. Xiong, R. H. Cohen, T. D. Rognlien, X. Q. Xu, J. Comp. Phys.

227 (2008)� I. G. Abel, M. Barnes, S. C. Cowley, W. Dorland, A. A.

Schekochihin, Phys. Plasmas 15, 122509 (2008)

Higher-order methods for SDE’s� G.N. Milstein, “Numerical Integration of Stochastic Differential

Equations,” (Kluwer Academic, Dordrecht, 1995)� M.B. Giles, “Improved multilevel Monte Carlo convergence using

the Milstein scheme,” in “Monte Carlo and Quasi-Monte CarloMethods 2006 (A. Keller, S. Heinrich, and H. Niederreiter, eds.,Springer-Verlag, 2007), 343

� P.E. Kloeden and E. Platen, “Numerical Solution of StochasticDifferential Equations” (Springer-Verlag, Berlin, 1992)


References, contd.

Levy areas, theory and numerical methods� P. Levy, Proc. 2nd Berkeley Symp. Math. Stat. and Prob. ,

University of California Press, Berkeley, Ca., 2 (1951)� J. M. C. Clark and R. J. Cameron, Stochastic Differential Systems,

B. Grigelionis, ed., Lecture Notes in Control and InformationSciences 25, Springer-Verlag, Berlin, (1980)

� J.G. Gaines and T.J. Lyons, SIAM J. Appl. Math. 54, 1132, (1994) &SIAM J. Appl. Math. 57, 1455 (1997)

� P.E. Kloeden and E. Platen, “Numerical Solution of StochasticDifferential Equations” (Springer-Verlag, Berlin, 1992)


Higher-order Time Integration of Stochastic Differential ... · Higher-order Time Integration of Stochastic Differential Equations and Application to Coulomb Collisions! Presented

Documents