Tobias Neckel Lehrstuhl Informatik V Summer Semester 2017 · nested Clenshaw-Curtis points: Rl points in each dimension d Rl # points sparse grid full tensor grid 2 5 13 25 9 29 81

Algorithms for Uncertainty QuantificationTobias Neckel

Lehrstuhl Informatik V

Summer Semester 2017

Lecture 8:

Sparse grids in UncertaintyQuantification

Repetition from previous lectureThe stochastic Galerkin approach

• idea− insert polynomial expansions into model− modify model to compute coefficients

• Galerkin projection as in FEM• comparison with non-intrusive methods− needs model modifications− good convergence properties

• example: damped linear oscillator

Dr. rer. nat. Tobias Neckel | Algorithms for Uncertainty Quantification | Summer Semester 2017 3

Today’s lectureTopic

Sparse grids in Uncertainty Quantification

Content

• how to realise quadrature efficiently in higher dimensions

• generic approach to sparse grids

• example: damped oscillator


Multi-dimensional forward propagation of uncertainty

stochastic inputs Ω

stochasticmodel f (t ,ω)

stochastic output(s) Y

Problem

• assumption: f computationally expensive

• what happens when Ω is multi-dimensional?

What we want

• use polynomial chaos-based methods at a reasonable computational cost


Remember: multivariate polynomial chaos expansion• random vector Ω consisting of independent random variables Ωi , i = 1, . . . ,d

• multiindices n = (n1, . . . ,nd),k = (k1, . . . ,kd) ∈ Nd0

• multivariate polynomials: product of univariate polynomials

φn(ω) = φn1(ω1) · · ·φnd (ωd),

< φn(ω),φm(ω) >w = δnm, δnm = δn1m1 · · ·δndmd

• multivariate polynomial chaos expansion

f (t ,ω)≈N−1

∑|n|1=0

fn(t)φn(ω),

where |n|1 = n1 + . . .+ nd

• use the multivariate pseudo-spectral approach to obtain fn

fn(t) =K−1

∑|k |∞=0

f (t ,xk )φn(xk )wk ,

where |k |∞ = maxi|ki |


Multivariate polynomial chaos expansion

• multivariate polynomial chaos expansion

f (t ,ω)≈N−1

∑|n|1=0

fn(t)φn(ω)

• n typically chosen such as n1 + . . .nd ≤ N for a given N

• with this setup: P =(d+N

d

)is the number of serialised summation terms

• multivariate pseudo-spectral approach

fn(t) =K−1

∑|k |∞=0

f (t ,xk )φn(xk )wk

• in the standard approach, M = K d , where K is the number of quadrature points in one direction

• M grows exponentially fast with d

• M drives the overall computational cost


Multivariate pseudo-spectral approachStandard pseudo-spectral approach

• e.g. 10 (equidistant) points in 1D

• 100 points in 2D

• 1000 points in 3D

• . . .


Question:

Can we reduce the overall computationalcost without affecting too much the

accuracy?

The sparse grid ideaIntuition

• the full tensor-grid approach assumes that all directions are equally well coupled

• idea: weaken the assumed coupling

• discard the components that do not contributed much to the overall solution


The sparse grid ideaAlgorithm

• let U (i), i = 1, . . . ,d be 1D continuous linear operators (e.g. integration)

• in d-dimensions: take all possible combinations (i.e. tensor product)

U (d) = U (1)⊗ . . .⊗U (d)

• generally: U (i) available only theoretically

• assume numerical approximation U(i)

k ≈U (i) s.t.

||U (i)−U(i)

k || → 0, k → ∞

• use following intuition: for k = 2, e.g., write U(i)

2 as a telescoping sum

U(i)

2 = U(i)

0 + (U(i)

1 −U(i)

0 ) + (U(i)

2 −U(i)

1 )


The sparse grid ideaAlgorithm (2)

• remember: assume U(i)

k ≈U (i) s.t.

||U (i)−U(i)

k || → 0, k → ∞

⇒ U (i) may be written as series:

U (i) = U(i)

0 + (U(i)

1 −U(i)

0 ) + (U(i)

2 −U(i)

1 ) + . . . =∞

∑k=0

(U(i)

k −U(i)

k−1), U(i)−1 := 0

• for simplicity: define ∆(i)0 := U

(i)0 , ∆

(i)k := U

(i)k −U

(i)k−1 ⇒ U (i) = ∑

∞k=0 ∆

(i)k

• in d-dimensions, let k = (k1, . . . ,kd) ∈ Nd

U (d) = U (1)⊗ . . .⊗U (d) =∞

∑|k |1=0

∆(1)k1⊗ . . .⊗∆

(d)kd

• Note: The above sum is exact, but has an infinite number of terms!

• Q: How to truncate the above sum?


The sparse grid ideaIntuition example

• consider the exact integration of 2D monomials

• for simplicity: let X = 1,x ,x2,Y = 1,y ,y2• tensor product basis (i.e. all possible combinations):

X⊗Yfull tensor = 1,x ,y ,xy ,x2,y2,x2y ,xy2,x2y21

• note: mixed terms x2y ,xy2,x2y2 have order > 2⇒ higher-order quadrature degree necessary to integrate exactly BUT error contribution typicallylow (products of (powers of) epsilons)

• idea: truncate the above basis→ “diagonal cut”

X ⊗Ysparse = 1,x ,y ,xy ,x2,y2

2

1source: https://github.com/jonathf/chaospy/blob/development/tutorial/tutorial_2.pdf2source: https://github.com/jonathf/chaospy/blob/development/tutorial/tutorial_2.pdf


The sparse grid ideaAlgorithm (3)

• remember: in d-dimensions, let k = (k1, . . . ,kd) ∈ Nd

U (d) = U (1)⊗ . . .⊗U (d) =∞

∑|k |1=0

∆(1)k1⊗ . . .⊗∆

(d)kd

• idea: truncate above sum by weakening the assumed coupling between input dimensions

• take k ∈K s. t. K contains all multiindices that contributed significantly to the overall solution

• intuition: “truncate on the diagonal”

• for a user-defined level L:

K = k ∈ Nd : |k |1 ≤ L + d−1

where|k |1 =d

∑i=1|ki |

• Remark: if K = k ∈ Nd : |k |∞ ≤ L→ full tensor grid


Sparse grid example 1: nested Leja points

3

1source: https://github.com/jonathf/chaospy/blob/development/tutorial/tutorial_2.pdfDr. rer. nat. Tobias Neckel | Algorithms for Uncertainty Quantification | Summer Semester 2017 15

Full vs. sparse grid example 1: nested Leja points


Sparse grid example 2: Newton-Cotes nodes

4

4source: D. Pflüger, Spatially Adaptive Sparse Grids for High-Dimensional Problems, 2010Dr. rer. nat. Tobias Neckel | Algorithms for Uncertainty Quantification | Summer Semester 2017 17

Full vs. sparse grid example 2: Newton-Cotes nodes


Non-nested vs. nested sparse gridsImportant ingredients• important ingredients of the sparse grid construction:− 1D discrete operator (interpolation, quadrature etc.)− underlying 1D grid

• remember: sparse grids constructed on ∆(i)n = U

(i)n −U

(i)n−1

• let Gn denote the grid at level n− if Gn−1 ⊂Gn⇒ grid is nested⇒ evaluating ∆

(i)n = U

(i)n requires only Gn \Gn−1

− if Gn−1 6⊂Gn⇒ grid is not nested⇒ evaluating ∆(i)n = U

(i)n requires Gn∪Gn−1

5

5source: Gerstner T., Griebel M., Numerical Integration using Sparse Grids, Numerical Algorithms, 1998Dr. rer. nat. Tobias Neckel | Algorithms for Uncertainty Quantification | Summer Semester 2017 19

Sparse grids – possible savings• SG designed to save grid points

• Q: How much can be saved?• A: It depends− on the dimension d− on the SG parameter choices (w/o nesting, w/o boundary points, linear vs. exponential growth

over grid levels)• example:− quadrature operator− nested Clenshaw-Curtis points: Rl points in each dimension

d Rl# points

sparse grid full tensor grid2 5 13 25

9 29 815 5 61 3,125

9 241 59,04910 5 221 9,765,625

9 1,581 > 3×109

source: Smith, Chapter 11, p. 248


Sparse grids - fields of usage• quadrature• approximation/interpolation− classification problems− financial mathematics− visualisation/evaluation of simulation results− reduced-basis approaches− . . .

• numerical discretization of (elliptic) PDEs (original work of C. Zenger, 1991)

always: low- to mid-size dimensionality (rule of thumb: 4≤ d ≤ 20)

Adaptivity

• adaptivity in subspaces (dimension-adaptive SG): whole (larger) chunks of grid points

• adaptivity in (single) grid points (spatially-adaptive SG)


Sparse grids in Uncertainty QuantificationSparse grid quadrature

• focus of this lecture

• apply above algorithm when U(i)

k = quadrature scheme

• use obtained (sparse) scheme to compute the polynomial chaos expansion coefficients

Sparse grid interpolation

• sparse grid idea not restricted to numerical quadrature

• when U(i)

k = interpolation scheme→ sparse interpolation

• use obtained scheme to approximate underlying model

• alternative to polynomial chaos expansion


Model problem – damped linear oscillator

d2ydt2 (t) + c dy

dt (t) + ky(t) = f cos(ωOt)y(0) = y0dydt (0) = y1

• c – damping coefficient

• k – spring constant

• f – forcing amplitude

• ωO – frequency

• y0 – initial position

• y1 – initial velocity


Damped linear oscillator: deterministic setup

Setup• t ∈ [0,30]

• assume− c = 0.100− k = 0.035− f = 0.100− ωO = 1.000− y0 = 0.500− y1 = 0.000

Results

• at T = 15, ydet(T ) =−1.5137e−01


Damped linear oscillator: UQ scenario 1Setup

• t ∈ [0,30]

• T = 15, ydet(T ) =−1.5137e−01• assume− c ∼U (0.08,0.12)− k ∼U (0.03,0.04)− f ∼U (0.08,0.12)− ωO = 1.000− y0 ∼U (0.45,0.55)− y1 ∼U (−0.05,0.05)

Full grid results• 7776 quadrature nodes• E[y(T )] =−1.499e−01• Var[y(T )] = 2.507e−03

Sparse grid (Gauss-Legendre nodes) results• 2203 quadrature nodes• E[y(T )] =−1.499e−01, rel. error ∈ O(10−10)

• Var[y(T )] = 2.503e−03, rel. error ∈ O(10−9)


Damped linear oscillator: UQ scenario 2Setup

• t ∈ [0,30]

• T = 15, ydet(T ) =−1.5137e−01• assume− c = 0.10− k ∼U (0.03,0.04)− f ∼U (0.08,0.12)− ωO ∼U (0.8,1.2)− y0 ∼U (0.45,0.55)− y1 ∼U (−0.05,0.05)

Full grid results• 7776 quadrature nodes• E[y(T )] =−2.422e−01• Var[y(T )] = 9.833e−03

Sparse grid (Gauss-Legendre nodes) results• 2203 quadrature nodes• E[y(T )] =−2.422e−01, rel. error ∈ O(10−10)

• Var[y(T )] = 9.833e−03, rel. error ∈ O(10−8)


Literature• Chapter 11 in R. C. Smith, Uncertainty Quantification – Theory, Implementation, and Applications,

SIAM, 2014

• D. Xiu, Numerical Methods for Stochastic Computations – A Spectral Method Approach, PrincetonUniv. Press, 2010


SummarySparse grids in Uncertainty Quantification

• concept of sparse grids (SG)− basic idea: “truncate on diagonal”− SG save many grid points but often provide similar accuracy compared to full tensor grids− rule of thumb: SG useful for 4≤ d ≤ 20• specific SG versions: depend on− 1D grid point sequence (w/o nesting, point positions/stretching, boundary points)− 1D discrete operator

• focus in this lecture: SG for quadrature in UQ

• adaptivity possible

• example: damped oscillator


Tobias Neckel Lehrstuhl Informatik V Summer Semester 2017 · nested Clenshaw-Curtis points: Rl points in each dimension d Rl # points sparse grid full tensor grid 2 5 13 25 9 29 81

Documents