Full-scale particle simulations of high- energy density science experiments W.B.Mori, W.Lu, M.Tzoufras, B.Winjum, J.Fahlen,F.S.Tsung, C.Huang,J.Tonge M.Zhou,

Full-scale particle simulations of high-energy density science experiments

W.B.Mori , W.Lu, M.Tzoufras, B.Winjum, J.Fahlen,F.S.Tsung, C.Huang,J.Tonge M.Zhou, V.K.Decyk, C. Joshi (UCLA)

L.O.Silva, R.A.Fonseca (IST Portugal)C.Ren (U. Rochester) T. Katsouleas (USC)

Directed high-energy density

• Pressure=Energy/Volume– Pressure=Power/Area/c

• PetaWatt with 10m spot– 3x1010 J/cm3

– 300 GBar

• Electric field in laser:– TeV/cm

• At SLAC:– N=2x1010 e- or e+

r=1m, z =60m

– E=50GeV

• Pressure:– 15x1010J/cm3

– 1.5TBar

• Electric field of beam:– 1.6TeV/cm

Lasers Particle beams

Radiation pressure and space forces of intense lasers and beams expel plasma electrons

Particle Accelerators Why Plasmas?

• Limited by peak power and breakdown

• 20-100 MeV/m

• No breakdown limit

• 10-100 GeV/m

Conventional Accelerators Plasma

€

a0 ≡eA

mc 2= 8.5x1010 I(W /cm2)λ μm

Why lasers?

Radiation pressure can excite longitudinal wakes

Laser Wake Field Accelerator(LWFA, SMLWFA, PBWA)

A single short-pulse of photons

Plasma Wake Field Accelerator(PWFA)

A high energy electron bunch

Concepts For Plasma Based Accelerators*

Drive beam Trailing beam

1. Wake excitation2. Evolution of driver and wake3. Loading the wake with particles

*Tajima and Dawson PRL 1979

Plasma Accelerator Progress and the “Accelerator Moore’s Law”

LOA,RAL

LBL ,RALOsaka

Slide 2

Courtesy of Tom Katsouleas

The blowout and bubble regimesRosenzwieg et al. 1990 Puhkov and Meyer-te-vehn 2002

Ion column provides ideal accelerating and focusing forces

Typical simulation parameters:

~109 particles~105 time steps

Full scale 3D particle-in-cell modeling is now possible:OSIRIS

Other codes:VLPL, Vorpal, TurboWAVE, Z3 etc., but no all the same!

Progress in computer hardware

The “Dawson” cluster at UCLA: <$1,000,000

$50,000,000

Progress in lasers

Courtesy of G.Mourou

Progress in hardware and software

• Era Memory particles speed max energy (full PIC)

•80’s 16MByte 105-106 5s/part-step 100 MeV (2D)

•Today ~6TByte/3 ~109 1x10-3s/part-step 1-10GeV (3D(e.g., NERSC) (~7.5 Tflops/3)

•Local ~500GByte ~109 2x10-3 s/part-step 1-10GeV (3D)Clusters (2.3Tflops) 1 TeV (3D)(e.g., DAWSON)

•Future 25-1000TByte >1011 5x10-5s/part-step 500 GeV (3D)

150Tflops - 10Pflops?

The simulations of Tajima and Dawsonwould take ~1 second on my laptop!

Beam-driven wake* Fully ExplicitΔz ≤ .05 /cωp

Δy, Δx ≤ .05 /cωp

Δt ≤ .02 /cωp

# grids in z ≥350

# grids in x , y ≥150# s teps ≥2 x 10 5

Npa rticles ~.25 x 1 08 (3D)~1 x 10 6 (2D)

Pa rticles x s teps ~.5 x 1013 (3D) - ≥ 10,000 h rs~1 x 10 11 (2D) - ≥ 75 h rs

*Lase r-driven GeV st age re quires on the o rder of (ωo/ωp)2=1000 x longer , however , the the res olution ca n usua lly be re laxed.

Computational challenges for modeling plasma-based acceleration

(1 GeV Stage)

Full-scale modeling: Challenges and expectations

• As a laser propagates through the plasma it encounters ~1013-1014

electrons

• There are ~106-109 self-trapped electrons

• Need to model accuracy of 1 part in O(106)

• Don’t know exact plasma profile.

• Don’t know laser intensity or spot size.

• Don’t know laser transverse, longitudinal, or frequency profile (not a diffraction limited Gaussian beam).

Challenges: What is excellent agreement?

Convergence of advances in

laser technology and

computer simulation

•Simulation Parameters–Laser:

• a0 = 1.1• W0=15.6 25 m• ωl/ωp = 10

–Particles• 2x1x1 particles/cell• 500 million total

–Plasma length• L=.2cm• 50,000 timesteps

Full scale 3D LWFA simulation using OSIRIS:6TW, 50fs

2340 cells56.18 m

512 cells100 m

512 cells100 m

State-of- the- art ultrashort laser pulse

0 = 800 nm, Δt = 50 fs

I = 2.5x1018 W/cm-2, W =12.5 m

Laser propagation

Plasma Backgroundne = 2x1019 cm-3

Simulation ran for 6400 hours on DAWSON

(~4 Rayleigh lengths)

Simulation ran for 6400 hours on DAWSON


Simulations: no fitting parameters!Nature papers, agreement with experiment

• In experiments, the # of electrons in the spike is 1.4 108.

• In our 3D simulations, we estimate of 2.4 108 electrons in the bunch.

3D Simulations for: Nature V431, 541 (S.P.D Mangles et al)

-2 1011

0

2 1011

4 1011

6 1011

8 1011

1 1012

1.2 1012

1.4 1012

-50 0 50 100 150 200

Imperial Data #( /((dE/E) Ω)3 D OSIRIS

( )Energy MeV

Movie of Imperial RunPlasma density and laser envelope

QuickTime™ and aMPEG-4 Video decompressor

are needed to see this picture.

3D PIC simulations:Tweak parameters

Parameters: E=1 J, 30 fs, 18 µm waist, 6×1018 cm-3

Scenario:• self-focusing (intensity increases by 10)• longitudinal compressionExcite highly nonlinear wakefield with cavitation: bubble formation

• trapping at the X point• electrons dephase and self-bunch• monoenergetic electrons are behind the laser field

Propagation: 2 mm

PIC

Experiment

•Simulation Parameters–Laser:

• a0 = 4• W0=24.4 5 m• ωl/ωp = 33

–Particles• 2x1x1 particles/cell• 500 million total

–Plasma length• L=.7cm• 300,000 timesteps

Full scale 3D LWFA simulation using OSIRISPredict the future: 200TW, 40fs

4000 cells101.9 m

256 cells80.9 m

256 cells80.9 m

State-of- the- art ultrashort laser pulse

0 = 800 nm, Δt = 30 fs

I = 3.4x1019 W/cm-2, W =19.5 m

Laser propagation

Plasma Backgroundne = 1.5x1018 cm-3

Simulation ran for 75,000 hours on DAWSON


Simulation ran for 75,000 hours on DAWSON


OSIRIS 200 TW simulation: Run on DAWSON Cluster

A 1.3 GeV beam!

The trapped particles form a beam. • Normalized emittance:The

divergence of the beam is about 10mrad.

• Energy spread:

Beam loading

Physical pictureEvolution of the nonlinear structure

• The blowout radius remains nearly constant as long as the laser intensity doesn’t vary much. Small oscillations due to the slow laser envelope evolution have been observed.

• Beam loading eventually shuts down the self injection.

• The laser energy is depleted as the accelerating bunch dephases. The laser can be chosen long enough so that the pump depletion length is longer than the dephasing length.



2-D plasma slab

Beam (3-D):

Laser or particles

Wake (3-D)

1. initialize beam

2. solve ∇⊥2ϕ =ρ, ∇⊥

2ψ =ρe ⇒ Fp,ψ

3. pushplasma, storeψ

4. stepslabandrepeat2.

5. useψ togiantstepbeam

QuickPIC loop:

Solved by 2D field solver

(1c2

∂2

∂t2 −∇2)A =4πc

j

(1c2

∂2

∂t2 −∇2)φ =4πρ

−∇⊥2A =

4πc

j

−∇⊥2φ =4πρ

€

plasma e− :

dPe

d(ξ /c)=

qe

1− vez /c(E +

1

cve × B)

dXe

d(ξ /c)=

ve

1− vez /c

⎧

⎨ ⎪ ⎪

⎩ ⎪ ⎪

beam e− :

dPb

d(s /c)= qb (E +

1

cvb × B)

dXb

d(s /c)= vb

⎧

⎨ ⎪ ⎪

⎩ ⎪ ⎪

€

Let :

s = z,ξ = ct − z

Assume :

(1) ∂s << ∂ξ

(quasi − static

approximation)

(2) vb ≈ c€

plasma e− :

dPe

dt=qe (E +

1

cve × B)

dXe

dt= ve

⎧

⎨ ⎪

⎩ ⎪

beam e− :

dPb

dt= qb (E +

1

cvb × B)

dXb

dt= vb

⎧

⎨ ⎪

⎩ ⎪

Maxell’s equations in Lorentz gauge Particle pusher(relativistic)

Full PIC(no approximation)

€

Let : Ψ = φ − A//

QuickPIC

QuickPIC: Basic concepts

€

−∇⊥2 φ = 4πρ

−∇⊥2 Ψ = 4π (ρ −

j//

c)

QuickTime™ and aTIFF (LZW) decompressor


QuickPIC: Code structure

-3

-2

-1

0

1

2

-5 0 5 10

OSIRISQuickPIC

ξ( /cωp)

-3

-2

-1

0

1

2

3

-8 -6 -4 -2 0 2 4 6 8

OsirisQuickPIC (l=2)QuickPIC (l=4)

ξ ( /cωp)

-0.1

-0.05

0

0.05

0.1

-10 -5 0 5 10

Osiris

QuickPIC (l=2)

ξ ( /cωp)

-1

-0.5

0

0.5

1

-6 -4 -2 0 2 4 6

Osiris QuickPIC (l=2)

ξ ( /cωp)

e- driver e+ driver

e- driver with ionization laser driver

QuickPIC Benchmark: Full PIC vs. Quasi-static PIC

Benchmark for different drivers

Excellent agreement with full PIC code. More than 100 times time-savings. Successfully modeled current experiments. Explore possible designs for future experiments. Guide development on theory.

100+ CPU savings with “no” loss in accuracy

A Plasma Afterburner (Energy Doubler) Could be Demonstrated at SLAC

Afterburners

3 km

30 m

S. Lee et al., Phys. Rev. STAB, 2001

0-50GeV in 3 km50-100GeV in 10 m!

Excellent agreement between simulation and experiment of a 28.5 GeV positron beam which has passed through a 1.4 m PWFA

OSIRIS Simulation Prediction:Experimental Measurement:

Peak Energy Loss64 MeV

65±10 MeV

Peak Energy Gain78 MeV

79±15 MeV

5x108 e+ in 1 ps bin at +4 ps

Head Tail Head Tail

OSIRIS E162 Experiment

Full-scale simulationof E-164xx is possible using a new code QuickPIC

• Identical parameters to experiment including self-ionization: Agreement is excellent!

QuickTime™ and aYUV420 codec decompressor


0

+2

+4

-4

-2

0 +5-5X (mm)

Rel

ativ

e E

nerg

y (G

eV)

Full-scale simulationof E-164xx is possible using a new code QuickPIC





5000 instead of 5,000,000 node hours

• We use parameters consistent with the International Linear Collider “design”

•We have modeled the beam propagating through ~25 meters of plasma.



Full-scale simulation of a 1TeV afterburner possible using QuickPIC

I see a day where particle simulations

will use 1 trillion particles

I see a day where theworld is fueled

by fusion energy.

I see a day when high energy

acceleratorswill fit on a

tabletop.

πρφ

π

4)1

(

4)

1(

22

2

2

22

2

2

=∇−∂

∂

=∇−∂

∂

tc

ctcjA

€

−∇⊥2 A =

4π

cj

−∇⊥2 φ = 4πρ

−∇⊥2ψ = 4π (ρ − jz /c)

∂(ρ − j /c)

∂ξ+∇⊥⋅ j⊥ = 0

Maxwell equations in Lorentz gaugeReduced Maxwell equations

€

φ,A = ϕ (ct − z = ξ ,x⊥),A(ξ ,x⊥)

Quasi-static approx.

€

ψ =φ−Az

We define

Wakefield equations:“2D-electro and magneto-statics

Antonsen and Mora 1997

Whittum 1997

Huang et al., 2005 (QuickPIC)

(1c2

∂2

∂t2 −∇2)A =4πc

j

(1c2

∂2

∂t2 −∇2)φ =4πρ

−∇⊥2A =

4πc

j

−∇⊥2φ =4πρ

€

quasi - static : ∂s ≈ 0

Maxwell equations in Lorentz gauge Reduced Maxwell equations

])([:

:

//⊥⊥

⊥ ×+−

=

⎥⎦

⎤⎢⎣

⎡ ×+−=

BV

EP

BV

EP

cVcq

dd

electronsplasmaFor

ccq

ds

delectronsbeamFor

e

e

ee

bbb

ξ

Initialize beam

Call 2D routine

Deposition

3D loop end

Push beam particles

3D loop begin

Initialize plasma

Field Solver

Deposition

2D loop begin

2D loop end

Push plasma particles Iteration

s =z, ξ =ct−z

2∂∂s

−ik0 +∂∂ξ

⎛

⎝⎜

⎞

⎠⎟a−∇⊥

2a=k02χ pa=−k0

2 ωp2

ω02γp

a

Quasi-static Model including a laser driver

Laser envelope equation:

Pipelining: scaling quasi-static PIC to 10,000+ processors

beam

solve plasma response

update beam

Initial plasma slab

Without pipelining: Beam is not advanced until entire plasma response is determined


update beam


update beam


update beam


update beam

Initial plasma slab

beam

1 2 3 4

1 2 3 4

With pipelining: Each section is updated when its input is ready, the plasma slab flows in the pipeline.

LWFA - Accelerating Field

512 cells40.95 m

•Isosurface values:

•Blue : -0.9

•Cyan: -0.6

•Green: -0.3

•Red: +0.3

•Yellow: +0.6

•Electric Field in normalized units me c ωp e-1

•Isosurface values:

•Blue : -0.9

•Cyan: -0.6

•Green: -0.3

•Red: +0.3

•Yellow: +0.6

•Electric Field in normalized units me c ωp e-1

SimulationsThe 200 TW run: Dephasing ~ Pump depletion

€

LT ≈1.0cm

Ldp ≈1.3cm

ΔE ≈1.5GeV

Laser plasma Given a we pick the density and we evaluate from our formulas:

After 5 Zr / 7.5 mm

Total charge = 1.1 nC

0

0.5

1

1.5

2

2.5

800 1200 1600 2000

f(E)

Energy (MeV)

f(E) (a.u.)

€

w0 = 20μm

€

τ =30 fs

€

a0 = 4

€

=0.8μm

€

P = 200TW

€

np =1.5 ×1018cm−3

€

P

Pc

≈10

Physical picture of an “optimal” regimeGeometry - fields

• The ponderomotive force of the laser pushes the electrons out of the laser’s way.

• The particles return on axis after the laser has passed.

• The region immediately behind the pulse is void of electrons but full of ions.

• The result is a sphere (bubble) moving with the speed of (laser) light, supporting huge accelerating fields.

• The ponderomotive force of the laser pushes the electrons out of the laser’s way.

• The particles return on axis after the laser has passed.

• The region immediately behind the pulse is void of electrons but full of ions.

• The result is a sphere (bubble) moving with the speed of (laser) light, supporting huge accelerating fields.

Physical pictureEvolution of the nonlinear structure

• The front of the laser pulse interacts with the plasma and loses energy. As a result the front etches back.

• The shape and size of the accelerating structure slightly change.

• Electrons are self-injected in the plasma bubble due to the accelerating and focusing fields.

• The trapped electrons make the bubble elongate.

• The front of the laser pulse interacts with the plasma and loses energy. As a result the front etches back.

• The shape and size of the accelerating structure slightly change.

• Electrons are self-injected in the plasma bubble due to the accelerating and focusing fields.

• The trapped electrons make the bubble elongate.

PIC Simulations of beam loading in blowout regime:Used the new code QuickPIC

(UCLA,USC,U.Maryland)

Wedge shape w/ beam load beam length = 6 c/ωp, nb/np= 8.4, Ndrive = 3x1010, Ntrailing = 0.5x1010

Bi-Gaussian shapez= 1.2 c/ωp, nb/np= 26

Full-scale particle simulations of high- energy density science experiments W.B.Mori, W.Lu, M.Tzoufras, B.Winjum, J.Fahlen,F.S.Tsung, C.Huang,J.Tonge M.Zhou,

Documents