The Plasmaturbulence Code GENEusers.cecs.anu.edu.au/.../events/petaCompWkshop12/Gene.pdf · 2012. 2. 22. · The Plasmaturbulence Code GENE Christoph Kowitz 13.02.2012 Christoph Kowitz:

Technische Universitat Munchen

The Plasmaturbulence Code GENE

Christoph Kowitz

13.02.2012

Christoph Kowitz: The Plasmaturbulence Code GENE

Computational Science at the Petascale and Beyond – Challenges and Opportunities, 13.02.2012 1


Fusion Energy Research

Confining a hot plasma well enoughto initiate a self-sustained fusion

Gyrokinetics

Numerical modelling of hot plasmas

GENE

• Gyrokinetic ElectromagneticNumerical Experiment

• highly parallel plasmaturbulencesimulation code






Gyrokinetics


GENE








Gyrokinetics


GENE






Fusion Reaction

Nuclear Fusion

fusion of tritium and deuterium

21H + 3

1H −→42He + 1

0n + 17.59MeV




Fusion for Energy Production:

• high temperatures required (200 million K)• efficient confinement

Plasma

At high temperature amatter gets in the stateof a plasma

Gas Plasma

Plasma Confinement

By magnetic fields




Devices

• different devices developed in the last 50 years• two main types for magnetically confined fusion

• Tokamak• Stellarator




Turbulent Transport

• confinement times are still too short for self-fed fusion• a lot of heat is transported out of the core zone due to

anomalous transport• that transport is larger than initially expected

Anomalous Transport

• result of microscopic turbulence• driven by macroscopic temperature and density gradients• so far only circumvented by larger and larger dimensions of

the experiments




ITER ≈ 2018• 10 times more energy

out than invested• more than 10 billion

Euros in cost• self-sustained fusion

(no external heating)• dimension are large

enough to reduce theturbulent transport

But:

Numerical modeling is still required to understand themacroscopic and microscopic behavior of the plasma. Thesuccess of ITER does heavily depend on simulations!




Numerical Models

Many Particle Description

• only for dilute plasmas ( e.g. astrophysics)• computation intensive• wide range of time and spatial scales

Magnetohydrodynamics

• fluid like description• for macroscopic description only• microturbulence and anomalous transport not

representable (collision dominated)




Kinetic Description

The time development of the 6D distribution function f (x,v, t) inphase space is simulated.

Vlasov Equation

∂f∂t

+ v∂f∂x

+ F∂f∂v

= ∆(f ) (1)

• F = qm (E + v× B)

• ∆(f ): collision operator −→ can be neglected• E and B depend on f −→ nonlinear• still a wide spectrum of scales




Wide Range of Time Scales

fast gyration ! slow drifts

• microturbulence is driven by drifts• resolution of the fast gyration is not required• exact angle does not have to be resolved




Coordinate Transformation

x,v x, v‖, µ =mv2

⊥2B

• the smallest timescale dropped out• the position of the particle is not resolved anymore• just the position of the gyro center is known




Further Approximations:

• perturbation theory• one-form formulation• . . .

Gyrokinetic Equations

∂f∂t

+ v∂f∂x

+ F∂f∂v‖

= ∆(f ) (2)

• 5D −→ f (x, v‖, µ)

• v and F are rather complex expressions, which contain theevaluation of the magnetic and electric fields




GENE - Gyrokinetic Electromagnetic NumericalExperiment

• implements the gyrokinetic equations• ab initio turbulence simulations• code developed at IPP in Garching• group of Prof. Frank Jenko (http://gene.rzg.mpg.de)• fully MPI parallelized (OpenMP is enabled)• successfully ported to petascale machines• can be ported to a variety of different machines• comes with a package of diagnostic tools• used in the fusion community



http://gene.rzg.mpg.de


Flux Tube




x , y – Fourier Space

• periodic boundaries• transformed to Fourier domain kx , ky

• accurate derivatives localised integration

z – real space

• complicated periodic boundary z due to a shear in themagnetic field

• simulation box is tilted, so coupling in z at different x

µ and v‖

• Dirichlet boundaries




∂f∂t

= L[f ] +N [f ] (3)

Linear Gyrokinetics

• fast

• microinstabilities getvisible, but no turbulence

• y direction decouples −→4D problem

• moderate grid sizes, butlarge parameter scans

• time integration

• eigenvalue decomposition

Nonlinear Gyrokinetics

• accurate

• huge grid sizes

• microscopic turbulenttransport is resolved

• structures due tononlinear effects arevisible

• time consuming (FFT . . . )




Global Simulations of whole Tokamaks

• local approximation does not hold anymore — no periodicboundaries in x direction — much more gridpoints in xdirection

• artificial sources and sinks have to be introduced• so far it only achievable for medium sized tokamaks• requires up to hundreds of thousands CPU hours

Computationally most expensive calculationsBut they can nearly simulate a whole tokamak!




Movie




Strong Scaling

Global Run

• grid: 512× 32×24× 64× 24× 2

• 1.2 billionunknowns, sumsup to 200 GBmemoryrequirement

• scales up to 10kcores

• on EPCC HectorCRAY XE6

Gorler et al. (2011). Journal of Computational Physics, 230(18), 7053-7071.




GENE in Petascale and Exascale?

• GENE is already highly efficient• it adopts itself to different architectures by competing

alternative implementations of certain parts in the code• checks automatically for the optimal parallelization strategy

But:

• ITER relevant setups require easily 10 times larger gridsand even longer computation times

• new techniques are required to be able to be handled oncoming large scale architectures



The Plasmaturbulence Code GENEusers.cecs.anu.edu.au/.../events/petaCompWkshop12/Gene.pdf · 2012. 2. 22. · The Plasmaturbulence Code GENE Christoph Kowitz 13.02.2012 Christoph Kowitz:

Documents