GTC On Demand | NVIDIA GTC Digital - Portability and ......GTC 2014 Session 4155 Gerhard Zumbusch Institut für Angewandte Mathematik Portability and Performance: A Functional Language

GTC 2014 Session 4155

Gerhard ZumbuschInstitut für Angewandte Mathematik

Portability and Performance:A Functional Language for Stencil Operations

SFB/TR 7 „gravitational wave astronomy“

Results: Standard Loop

0

5

10

15

20

25

30

35

40

45

50

single

double

GF

lop

/s

1D scalar,constant coefficientheat equation, periodic boundary conditions, explicit finite differences

time loop k { space loop i { difference stencil }}

Results: Optimized Space-Time Slicing

0

200

400

600

800

1000

1200

1400

1600

1800

2000

single

doubleGF

lop

/s

1D scalar,constant coefficientheat equation, periodic boundary conditions, explicit finite differences

● space-time loop● vectorization● OpenMP parallel● loop tiling● optimized parameters

Compiler

stencil code:functional language

stencil code:functional language

interface codeinterface code

grid sizegrid size target:parallel architecturevector unitdata type

target:parallel architecturevector unitdata type

optimization:thread block sizetile sizespace-time algorithm

optimization:thread block sizetile sizespace-time algorithm

compiler:parsedata dependence analysisloop generationloop transformationcode generation

compiler:parsedata dependence analysisloop generationloop transformationcode generation

target compilertarget compiler

code generation:

separate the specification of● kernel code● data distribution● data storage● parallelization strategy● vectorization strategy● loop and instruction order● code optimization● grid size● data type (float/ double)

Language: Scalar Example

heat [k, i] { //define a kernel, kernel name, iteration variables // k time dimension // i one space dimension r = .25; // CFL constant u [k, i] = // array u. scalar field in space/time if (k < bds0, // if 1, // then initial data if (i < bds0 || i >= bds1, // else if left/ right boundary 0, // then zero boundary values // else stencil operator: r * (u [-1, -1] + u [-1, 1]) + (1-2*r) * u [-1, 0] // left, right, center // old time step ) );}

differential equation:heat transfer

explicit finite difference stencil

C style standard loopsr = .25;for (k=bds0.k; k<bds1.k; k++) { u [k, bds0.i-1] = u [k, bds1.i] = 0; for (i=bds0.i; i<bds1.i; i++) u [k, i] = r * (u [k-1, i-1] + u [k-1, i+1]) + (1-2*r) * u [k-1, i];}

Language: Scalar Example (2)


C style syntaxfunctional language● single static assignement● assignement order not specified● ~ shader + data dependency

instruction order not specifiedresults can be computed● once● in parallel● cached, pre-fetched● or re-computed

storage scheme not specifiedvalues can be stored in● global, shared memory● Iocal memory, cache● register

Language: Scalar Example (3)


u [-1, -1..1]u [-1, -1..1]

u [0, 0]u [0, 0]

Language: Grammar

program := [repeat kernel]kernel := [kernelid] [ [list itervar+] ] { [repeat statement] }statement := [id] [opt [ [list itervar+] ]] = [expr] ;expr := [number] | [expr] [arithmeticop] [expr] | - [expr] | ( [expr] ) | [functionid] ( [list argexpr+] ) | [id] [opt [ [list integer+] ]] | [bds0 | bds1] [opt [ [itervar] ]]boolexpr := [expr] [compareop] [expr] | [boolexpr] [logicop] [boolexpr] | ! [boolexpr] | ( [boolexpr] ) | [functionid] ( [list argexpr+] )argexpr := [expr] | [boolexpr]arithmeticop := * | / | + | -logicop := || | && | ˆˆcompareop := == | != | < | > | <= | >=

C-style assignment syntax

basic data type:● const● const arrayscalar type: bult-inarray size: built-in bds

„if then else“ returns a valuefunctions return a value

● no control structures● no procedures● no side effects● no variables● no state

Language: 3D Stencil

heat3 [t, i, j, k] { //define a kernel // t time dimension // i,j,k three space dimensions r = 1 / 12; dx = 1 / (bds1 [i] - bds0 [i] + 1); u [t, i, j, k] = if (t < bds0, exp (- sqr(i * dx) - sqr(j * dx) - sqr(k * dx)), // initial data if (i < bds0 || i >= bds1 || j < bds0 || j >= bds1 || k < bds0 || k >= bds1, 0, // zero boundary values ((u [-1, -1, 0, 0] + u [-1, 1, 0, 0]) + // back/front (u [-1, 0, -1, 0] + u [-1, 0, 1, 0]) + // down/up (u [-1, 0, 0, -1] + u [-1, 0, 0, 1])) * // left/right r + (1 - 6 * r) * u [-1, 0, 0, 0] // center ) );}



Language: Space & Time Fields

varcoeff [k, i] { u [k, i] = // scalar field in space/time if (k < bds0, exp (i * dx), // initial data if (i < bds0 || i >= bds1, k * dt, // boundary values r * p[0] * c[0] * (u [-1, -1] + u [-1, 1]) + (1 - 2 * r * p[0] * c[0]) * u [-1, 0] ) ); c [i] = 1 + exp(- i * dx); // scalar field in space i p [k] = if (k < bds0, // scalar field in time .5, // initial data p [-1] * (1 + dt) // recursion ); r = .25; // CFL constant dx = 1 / (bds1 [i] - bds0 [i] + 1); // space step size dt = 1 / (bds1 [k] - bds0 [k] + 1); // time step size}



Language: Systems of Equations

wave [t, i] { dx = 3.1415927 / (bds1 [i] - bds0 [i] + 1); dt = .5 * dx; r = dt / sqr (dx); v [t, i] = // v field component in space/time if (t < bds0 || i < bds0 || i >= bds1, 0, dt * v [-1, 0] + r * (u [-1, -1] + u [-1, 1]) - 2 * r * u [-1, 0] ); // old u values u [t, i] = // u field component in space/time if (t < bds0 || i < bds0 || i >= bds1, sin (i * dx), u [-1, 0] + dt * v [0, 0] ); // current v, old u values}


wave equation

system of equations

Language: Systems of Equations (2)

wave [t, i] { dx = 3.1415927 / (bds1 [i] - bds0 [i] + 1); dt = .5 * dx; r = dt / sqr (dx); v [t, i] = // v field component in space/time if (t < bds0 || i < bds0 || i >= bds1, 0, dt * v [-1, 0] + r * (u [-1, -1] + u [-1, 1]) - 2 * r * u [-1, 0] ); // old u values u [t, i] = // u field component in space/time if (t < bds0 || i < bds0 || i >= bds1, sin (i * dx), u [-1, 0] + dt * v [0, 0] ); // current v, old u values}

u [-1, -1..1]u [-1, -1..1]v [-1, 0]v [-1, 0]

u [0, 0]u [0, 0]

v [0, 0]v [0, 0]

loop k

Language: Multi-Stages

RK4 [t, i] { r = .25; s0 [t, i] = if (i < bds0 || i >= bds1, // stage 0 0, (r / 2) * (u [-1, -1] + u [-1, 1]) + (1 - r) * u [-1, 0] ); // t old u values s1 [t, i] = if (i < bds0 || i >= bds1, // stage 1 0, u [-1, 0] + (r / 2) * (s0 [0, -1] + s0 [0, 1]) - r * s0 [0, 0] ); // t old u and new s0 values s2 [t, i] = if (i < bds0 || i >= bds1, // stage 2 0, u [-1, 0] + r * (s1 [0, -1] + s1 [0, 1]) - (2 * r) * s1 [0, 0] ); // t old u and new s1 values s3 [t, i] = if (i < bds0 || i >= bds1, // stage 3 0, (r / 2) * (s2 [0, -1] + s2 [0, 1]) + r * s2 [0, 0] ); // t old u and new s2 values u [t, i] = if (t < bds0, // new value 1, (s0 [0, 0] + 2 * s1 [0, 0] + s2 [0, 0] + s3 [0, 0] - u [-1, 0]) / 3 ); // t old u and new s0, s1, s2, s3 values}

4-stage Runge-Kutta scheme

heat transfer

Language: Multi-Stages (2)

RK4 [t, i] { r = .25; s0 [t, i] = if (i < bds0 || i >= bds1, // stage 0 0, (r / 2) * (u [-1, -1] + u [-1, 1]) + (1 - r) * u [-1, 0] ); // t old u values s1 [t, i] = if (i < bds0 || i >= bds1, // stage 1 0, u [-1, 0] + (r / 2) * (s0 [0, -1] + s0 [0, 1]) - r * s0 [0, 0] ); // t old u and new s0 values s2 [t, i] = if (i < bds0 || i >= bds1, // stage 2 0, u [-1, 0] + r * (s1 [0, -1] + s1 [0, 1]) - (2 * r) * s1 [0, 0] ); // t old u and new s1 values s3 [t, i] = if (i < bds0 || i >= bds1, // stage 3 0, (r / 2) * (s2 [0, -1] + s2 [0, 1]) + r * s2 [0, 0] ); // t old u and new s2 values u [t, i] = if (t < bds0, // new value 1, (s0 [0, 0] + 2 * s1 [0, 0] + s2 [0, 0] + s3 [0, 0] - u [-1, 0]) / 3 ); // t old u and new s0, s1, s2, s3 values}

u [-1, -1..1]u [-1, -1..1]

s0 [0, 0]s0 [0, 0]

s0 [0, -1..1]s0 [0, -1..1]

s1 [0, 0]s1 [0, 0]

u [-1, 0]u [-1, 0]

s1 [0, -1..1]s1 [0, -1..1]

s2 [0, 0]s2 [0, 0]

u [-1, 0]u [-1, 0]

s2 [0, -1..1]s2 [0, -1..1]

s3 [0, 0]s3 [0, 0]

u [-1, 0]u [-1, 0] s0 [-1, 0]s0 [-1, 0] s1 [-1, 0]s1 [-1, 0]

u [0, 0]u [0, 0]

Language: An Iterated Box Filter

box [t, i, j] { // t iteration dimension // i,j two image dimensions v [t, i, j] = if (t < bds0, 0, if (i < bds0 || i >= bds1 || j < bds0 || j >= bds1, 0, // zero boundary values (u [-1, -1, 0] + u [-1, 0, 0] + u [-1, 1, 0]) / 3 ) // old values u ); u [t, i, j] = if (t < bds0, sin (i) * sin (j), // initial image if (i < bds0 || i >= bds1 || j < bds0 || j >= bds1, 0, // zero boundary values (v [0, 0, -1] + v [0, 0, 0] + v [0, 0, 1]) / 3 ) // current values v );}

image processing: convolution filter

3-pt box filter

Interface: C++

#include ”fd32.hpf” // code generated from “fd32.pf”

kern k; // C++ interface class for kernel “kern”typedef kern::base pointer;int n = 2;pointer p = new pointer[n];for (int i=0; i<n; i++) p[i] = k.u.vecalloc (); // create a pool of memory blocks // size and shape of array uk.pool1.init (p, n); // hand blocks over to pool administrationk.launch0(); // initialize, compute initial datak.launch1(); // execute time loop

for (int i=k.u.low0(); i<k.u.low0()+k.u.size0(); i++) // one dimensional array u, array bounds in low and size cout<<k.u(i)<<” ”; // index arithmetic hidden by u(i)cout<<”\n”;

include generated code

kernel object with● dedicated memory management● i/o interface helper● kernel launch

transparent● memory layout● loop/ algorithm structure● vectorization● parallelization

Interface: Nvidia Cuda

#include ”fd32.hpf”kern k;typedef kern::base pointer;int n = 2;pointer p = new pointer[n];for (int i=0; i<n; i++) p[i] = k.u.devicealloc (); // device memory poolk.pool1.init (p, n);k.launch0();k.launch1();k.u.pt_host = k.u.vecalloc(); // additional host memoryk.u.readdevice (); // copy array u to host

GPU enabled code

host kernel object:● device kernel● device kernel launch● device memory management● array stored on device● user initiated host<>device transfer

Compiler Target: Vector Instruction Sets

scalar scalar arithmetic, 1 float or 1 double

sse x86 SSE1: 128 bit vector, 4 floats -msseSSE2: 4 floats or 2 doubles -msse2SSE4.1: more advanced ops -msse4

altivec PowerPC AltiVec: 128 bit vector, 4 floats or 2 doubles -maltivecspu Cell BE SPU: 128 bit vector, 4 floats (or 2 doubles) spu-g++neon ARM Neon: 128 bit vector, 4 floats -mfpu=neon

avx x86 AVX: 256 bit vector, 8 floats or 4 doubles -mavx

phi new Larabee instruction set: 512 bit vectorIntel Phi: 16 floats or 8 doubles -mmicAVX-512

cuda Nvidia GPU: synchronous warp, 32 floats/ doubles nvcccompute >=3.0: warp shuffle nvcc -arch=sm_30

opencl OpenCL: vector, on GPUs 32 or 64 floats/ doubles -lOpenCL

current CPU/ GPU architectures areparallel vector processors

● multi-core CPU● SMP CPUs

CPU vector units: SSE, AVX, AltiVec, Neon, …

Nvidia Cuda:● thread block % 32: vectors of length 32● thread block / 32: multi-threading on one

multi-processor● grid of blocks: multi-threading on all

multi-processors of a GPU

OpenCL:● work group of local work items: vectors

+ multi-threading on one processor● work groups on compute units:

multi-threading on processors

Optimization: Space-Time-Slicing

array

iterations

start-up slices

cache

// p stencil width// s slice widthfor (i=(stepi-1)*s; i>=0; i=i-s) { // load memory cache[0..s-1] = u[i..i+s-1]; for (k=0; k<stepk*p; k=k+p) { // load cache cache[s..s+p-1] = u[i+k+s..i+k+s+p-1]; cache[0..s-1] = stencil (cache[0..s+p-1]); // store cache u[i+k+p..i+k+2*p-1] = cache[0..p-1]; } // store memory u[i+(stepk+1)*p..i+stepk*p+s-1] = cache[p..s-1];} 2NT/s+T² cache load

2NT/s cache storeN+2T global loadN global storeNT+T² stencil ops

Optimization: Data Interleaving for Vectors & parallel Processors

vec u0 = rotate_right (u[n-1]);vec u1 = u[0];u[n] = rotate_left (u1);for (int i=0; i<n; i++) { // unroll vec u2 = u[i+1]; v[i] = stencil_vec (u0, u1, u2); u0 = u1; u1 = u2;}

grid4 sub-domains

memoryvectors of 4

Portability and Performance: A Functional Language for Stencil Operations

functional description of finite difference stencils in● differential equations● image processing● stencils in space-time

generate● high performace kernel codes● find good storage and execution

order in space-time● for many current computer

architectures

● S4953 - Hangout: GTC SpeakersWednesday, 03/26, 13:00 – 14:00Concourse Pod C

● Workshop PHSP1407/14-15, Jena, Germanyhttp://cse.mathe.uni-jena.de/phsp14

http://cse.mathe.uni-jena.de/phsp14

GTC On Demand | NVIDIA GTC Digital - Portability and ......GTC 2014 Session 4155 Gerhard Zumbusch Institut für Angewandte Mathematik Portability and Performance: A Functional Language

Documents