Accelerating incompressible fluid flow simulations using SIMD or GPU computing Yushan Wang 1 , Marc Baboulin 1,2 , Yann Fraigneau 1,3 , Olivier Le Maˆ ıtre 1,3 , Karl Rupp 4 1 Universit ´ e Paris-Sud, France 2 INRIA, France 3 CNRS, France 4 Argonne National Laboratory, USA Yushan Wang, LRI Navier-Stokes Solver 1/27
44
Embed
Accelerating incompressible fluid flow simulations using ...
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Accelerating incompressible fluid flowsimulations using SIMD or GPU computing
Yushan Wang1, Marc Baboulin1,2,Yann Fraigneau1,3, Olivier Le Maıtre1,3, Karl Rupp4
1 Universite Paris-Sud, France2 INRIA, France3 CNRS, France
4 Argonne National Laboratory, USA
Yushan Wang, LRI Navier-Stokes Solver 1/27
Outline
Solving Navier-Stokes equations via a prediction-projectionmethod
Helmholtz-like equationPoisson equation
Performance on a multicore architecture
Accelarating tridiagonal systems solutions using SIMD
GPU implementation
Conclusion and future work
Yushan Wang, LRI Navier-Stokes Solver 2/27
Navier-Stokes equationsThe Navier-Stokes equations describe mainly the motion of a viscousflow at all scales.
A Millennium Prize Problem of Navier-Stokes Euqations.http://www.claymath.org/millennium/Navier-Stokes_Equations/
Global Climate Models and the Navier-StokesEquations.http://climateaudit.org/2005/12/22/gcms-and-the-navier-stokes-equations/
Navier-Stokes simulation for the flow field around the Falcon business jet.http://mfquant.net/gallery_cfd.html
The Schur complement method is applied when there are multiplesubdomains along the solving direction.
I1
I2
y
z
x
Example for solving a tridiagonal systemalong z-direction.
This method results in information exchanges!
Yushan Wang, LRI Navier-Stokes Solver 13/27
Schur complement method
The Schur complement method is applied when there are multiplesubdomains along the solving direction.
I1
I2
y
z
x
Example for solving a tridiagonal systemalong z-direction.
This method results in information exchanges!
Yushan Wang, LRI Navier-Stokes Solver 13/27
Performance results: Time breakdown
0
50
100
150
200
250
300
350
1 2 3 4 5 6 8 10 12 15 16 20 24 30 40 48
Tim
e (s
)
Number of processes
HelmholtzPoisson
Other
Problem size : 2403
double precisionMagnyCours-48 system from University of Tennessee
4× 12 AMD Opteron Processor 6172
Yushan Wang, LRI Navier-Stokes Solver 14/27
Performance results: Strong scalability
0
5
10
15
20
25
30
35
40
45
50
0 5 10 15 20 25 30 35 40 45 50
Spee
dup
Number of processes
CPUHelmholtz
PoissonIdeal
Problem size : 2403
double precisionMagnyCours-48 system from University of Tennessee
4× 12 AMD Opteron Processor 6172
Yushan Wang, LRI Navier-Stokes Solver 15/27
Performance results: Weak scalability
0
50
100
150
200
250
300
0 5 10 15 20 25 30 35 40 45 50
Tim
e (m
s)
Number of processes
CPUHelmholtz
Poisson
Problem size per process : 240× 240× 10double precision
MagnyCours-48 system from University of Tennessee4× 12 AMD Opteron Processor 6172
Yushan Wang, LRI Navier-Stokes Solver 16/27
Tridiagonal Solver
At each time step, 10 tridiagonal systems to solve.
Helmholtz-like equation:
(I −2∆t
31
Re∆x)T′ = Si
(I −2∆t
31
Re∆y)T′′ = T′ i ∈ {x , y , z}
(I −2∆t
31
Re∆z)(V∗i − Vn
i ) = T′′
Poisson equation:
(Λx +Λy + Lz)φ′ = S′
Yushan Wang, LRI Navier-Stokes Solver 17/27
Tridiagonal Solver
The tridiagonal systems have the same block tridiagonal structure.
=⇒
Helmholtz-like equation:The tridiagonal blocks are identical =⇒ a smaller tridiagonal systemwith multiple RHS.Second order central difference scheme =⇒ diagonally dominantmatrix.
Yushan Wang, LRI Navier-Stokes Solver 18/27
Thomas Algorithm
b1 c1
a2 b2 c2. . . . . . . . .
ai bi ci. . . . . . . . .
an−1 bn−1 cn−1
an bn
x1
x2...xi...
xn−1
xn
=
s1
s2...si...
sn−1
sn
.
Forward eliminationfor i = 2 to n, do
bi = bi −ci−1 × ai
bi−1;
si = si −si−1 × ai
bi−1;
end
Backward subtitution:xn =
sn
bn;
for i = n − 1 to 1, do
xi =si − ci × xi+1
bi;
end
Yushan Wang, LRI Navier-Stokes Solver 19/27
Vectorization
Implemented using a generic SIMD abstraction library(BOOST.SIMD) for all SSEx variants and AVX.
Boost.SIMD, a C++ template library that simplifies the exploitationof SIMD hardware within a standard C++ programming model.
Scalable system that takes care of increasing wide of SIMDsystems (128 bits today, 512 bits in Intel Xeon Phi coprocessors).
See [ Esterie et al., Boost.SIMD: Generic Programming forportable SIMDization ] .
Yushan Wang, LRI Navier-Stokes Solver 20/27
Tridiagonal solver with vectorization
a6
R1=(a1, a2)
R2=(b1, b2)
b1 b2 b3
R1=(a1, b1)
R2=(a2, b2)b4 b5
Shuffle(R1, R2)
b6 b7 ...
a1 a2 a3 a4 a5 a7 ...
=c3a3
a4
c1
a2 c2
b3
b4
x11 x2
1
x12 x2
2
x13 x2
3
x14 x2
4
s13 s23
s14 s24
s11
s12
s21
s22
b1
b2
Yushan Wang, LRI Navier-Stokes Solver 21/27
Tridiagonal solver with vectorization
a6
R1=(a1, a2)
R2=(b1, b2)
b1 b2 b3
R1=(a1, b1)
R2=(a2, b2)b4 b5
Shuffle(R1, R2)
b6 b7 ...
a1 a2 a3 a4 a5 a7 ...
=c3a3
a4
c1
c2
x11 x2
1
x12 x2
2
x13 x2
3
x14 x2
4
s11
s12
s21
s22
b1
b2
b3
b4
s13
s14
s23
s24
Yushan Wang, LRI Navier-Stokes Solver 21/27
Tridiagonal solver with vectorization
a6
R1=(a1, a2)
R2=(b1, b2)
b1 b2 b3
R1=(a1, b1)
R2=(a2, b2)b4 b5
Shuffle(R1, R2)
b6 b7 ...
a1 a2 a3 a4 a5 a7 ...
=c3
c1
c2
x11 x2
1
x12 x2
2
x13 x2
3
x14 x2
4
s11
s12
s21
s22
b1
b2
b3
b4
s13
s14
s23
s24
Yushan Wang, LRI Navier-Stokes Solver 21/27
Tridiagonal solver with vectorization
a6
R1=(a1, a2)
R2=(b1, b2)
b1 b2 b3
R1=(a1, b1)
R2=(a2, b2)b4 b5
Shuffle(R1, R2)
b6 b7 ...
a1 a2 a3 a4 a5 a7 ...
=c3
c1
c2
s11
s12
s21
s22
b1
b2
b3
b4
s13
s14
s23
s24x14 x2
4
x23x1
3
x22x1
2
x21x1
1
Yushan Wang, LRI Navier-Stokes Solver 21/27
Performance: Cycle per value
10
20
30
40
50
60
70
1.0e+02 1.0e+03 1.0e+04 1.0e+05 1.0e+06
Cyc
le p
er v
alue
(c/v
)
Number of RHS
DGTSVvectorized Thomas
L2 cache sizeL3 cache size
Intel(R) Xeon(R) CPU E5645 @ 2.40GHzdouble precision
Yushan Wang, LRI Navier-Stokes Solver 22/27
Steps of NS solver
Domain initialization
Computation of eigen values and vectorsFor each time iteration:− Solve Helmhlotz equation− Solve Poisson equation− Variables increments− Record current numerical solution
Yushan Wang, LRI Navier-Stokes Solver 23/27
Steps of NS solver
Domain initialization
Computation of eigen values and vectorsFor each time iteration:− Solve Helmhlotz equation− Solve Poisson equation− Variables increments− Record current numerical solution
Yushan Wang, LRI Navier-Stokes Solver 23/27
Helmholtz-like equation
Tridiagonal block structure with identical blocks.
One GPU thread deals with one RHS value.
Data reordering after each solving step.
Poisson equation
Tridiagonal block structure with different blocks.
One GPU thread deals with one tridiagonal block.
Matrix-matrix multiplication.
Data reordering after each multiplication and solving step.
Transfers not included (needed only at the begining of the time itration)
Yushan Wang, LRI Navier-Stokes Solver 25/27
Conclusion
Scalable algorithm and CPU implementation of a 3DNavier-Stokes equations.
Tridiagonal solver acceleration using vectorization.
For discontinuous domains, we use an iterative method to solvethe Poisson equation. (SOR+multigrid)
GPU Helmholtz and Poisson solver.
Ongoing workMultiGPU solver for Navier-Stokes equations using partialdiagonalisation and ADI method. Collaboration with Argonne NationalLaboratory (Karl Rupp).