Steps in Uncertainty Quantification Challenge: • How do we do uncertainty quantification for computationally expensive models? • Example: – We have a computational budget of 5000 model evaluations. – Bayesian inference and uncertainty propagation require 120,000 evaluations.
32
Embed
Steps in Uncertainty Quantification Polynomial Representations Nonintrusive PCE: to obtain Take weighted inner product of y(q)= P 1 k=0 k k(q) k = 1 k Z y(q) k(q) (q)dq Note: Sample
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Steps in Uncertainty Quantification
Challenge:• How do we do uncertainty quantification for computationally expensive models?
• Example:
– We have a computational budget of 5000 model evaluations.
– Bayesian inference and uncertainty propagation require 120,000 evaluations.
Uncertainty Quantification ChallengesExample: MFC model – Fourth-order PDE
Capacitor probe
MFC Patch
Bayesian Inference: Took 6 days!Macro-Fiber Composite
Solution: Highly efficient surrogate models
Problem:
M = -c
E
I
@2w
@x
2 - c
D
I
@3w
@x
2@t
- [k1e(E ,�0)E + k2"irr
(E ,�0)]�MFC
(x)
⇢@2
w
@t
2 -@2
M
@x
2 = f
cE I1.2 ⇥ 10
5
PDE solutions
138
Surrogate Models: MotivationExample: Consider the heat equation
Notes:• Requires approximation of PDE in 3-D
• What would be a simple surrogate?
t
1 x , y , z
with the response
Boundary Conditions
Initial Conditions
y(q) =
Z1
0
Z 1
0
Z 1
0
Z 1
0u(t , x , y , z)dxdydzdt
@u
@t
=@2
u
@x
2 +@2
u
@y
2 +@2
u
@z
2 + f (q)
139
Surrogate Models: MotivationExample: Consider the heat equation
with the response
Boundary Conditions
Initial Conditions
y(q) =
Z1
0
Z 1
0
Z 1
0
Z 1
0u(t , x , y , z)dxdydzdt
@u
@t
=@2
u
@x
2 +@2
u
@y
2 +@2
u
@z
2 + f (q)
Question: How do you construct a polynomial surrogate?• Regression
• Interpolation
t
1 x , y , z
Surrogate: Quadraticys(q) = (q - 0.25)2 + 0.5
Surrogate ModelsRecall: Consider the model
Question: How do you construct a polynomial surrogate?• Interpolation
• RegressionM=7k=2
with the response
Boundary Conditions
Initial Conditions
y(q) =
Z1
0
Z 1
0
Z 1
0
Z 1
0u(t , x , y , z)dxdydzdt
@u
@t
=@2
u
@x
2 +@2
u
@y
2 +@2
u
@z
2 + f (q)
t
1 x , y , z
Surrogate: Quadraticys(q) = (q - 0.25)2 + 0.5
Surrogate Models
Question: How do we keep from fitting noise?• Akaike Information Criterion (AIC)
• Bayesian Information Criterion (AIC)
Likelihood:
M=7k=2
Minimize
Maximize
SSq =MX
m=1
[ym - ys(qm)]2
AIC = 2k - 2 log[⇡(y |q)]
BIC = k log(M)- 2 log[⇡(y |q)]
⇡(y |q) =1
(2⇡�2)M/2 e-SSq/2�2
142
Surrogate Models
Question: How do we keep from fitting noise?• Akaike Information Criterion (AIC)
• Bayesian Information Criterion (AIC)
Likelihood:
M=7k=2
Maximize
SSq =MX
m=1
[ym - ys(qm)]2
AIC = 2k - 2 log[⇡(y |q)]
BIC = k log(M)- 2 log[⇡(y |q)]
⇡(y |q) =1
(2⇡�2)M/2 e-SSq/2�2
Example: Exercise:• Construct a polynomial surrogate using the code
response_surface.m.
• What order seems appropriate?
y = exp(0.7q1
+ 0.3q2
)
!
q 2
q1
Minimize
Data-Fit ModelsNotes:• Often termed response surface models, emulators, meta-models.
• Constructed via interpolation or regression.
• Data can consist of high-fidelity simulations or experiments.
y = f(q)
Example: Steady-state Euler-Bernouilli beam model with PZT patch
Data: Displacement observations
Parameter:
YI
d
4w
dx
4 (x) = k
p
V�pzt
(x)
YI
144
Data-Fit ModelsExample: Steady-state Euler-Bernouilli beam model with PZT patch
Data: Displacement observationsParameter: YI Training points: 5000
Bayesian Inference
YI x
disp
lace
men
t
Polynomial surrogate: 6th order
YI
d
4w
dx
4 (x) = k
p
V�pzt
(x)
Data-Fit ModelsNotes:• Often termed response surface models, surrogates, emulators, meta-models.
• Rely on interpolation or regression.
• Data can consist of high-fidelity simulations or experiments.
• Common techniques: polynomial models, kriging (Gaussian process regression), orthogonal polynomials.
Sparse Grid TechniquesTensored Grids: Exponential growth Sparse Grids: Same accuracy
p R` Sparse Grid R Tensored Grid R = (R`)p
2 9 29 81
5 9 241 59,049
10 9 1581 > 3⇥ 109
50 9 171,901 > 5⇥ 1047
100 9 1,353,801 > 2⇥ 1095
155
Surrogate Construction: Nuclear Power Plant Design
Subchannel Code (COBRA-TF): 33 VUQ parameters reduced to 5 using SA
Surrogate: Total pressure drop
• Kriging (GP) emulator constructed using 50 COBRA-TF runs perturbing 5 active inputs.
• Use remaining computational budget to evaluate quality of surrogate using post-processed Dakota outputs.
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
1.10 1.15 1.20 1.25
1.10
1.15
1.20
1.25
Calculated Total Pressure Drop
Pred
icte
d To
tal P
ress
ure
Dro
p
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
1.10 1.15 1.20 1.25
−10
12
Calculated Total Pressure Drop
Stan
dard
ized
Res
idua
l Tot
al P
ress
ure
Dro
p
Pred
icte
d To
tal P
ress
ure
Dro
p
Calculated Total Pressure Drop Theoretical Quantities
Sam
ple
Qua
ntiti
es
Out-of-Sample Validation
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
−2 −1 0 1 2
−2−1
01
Leave−one−out Cross Validation
Theoretical Quantiles
Sam
ple
Qua
ntile
s
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
−2 −1 0 1 2
−10
12
Out−of−sample Validation
Theoretical Quantiles
Sam
ple
Qua
ntile
s
156
Example: SIR Cholera ModelModel:
dSdt
= bN - �LSBL
L + BL- �HS
BH
H + BH- bS
dIdt
= �LSBL
L + BL+ �HS
BH
H + BH- (�+ b)I
dRdt
= �I - bR
dBH
dt= ⇠I - �BH
dBL
dt= �BH - �BL
Model Parameter Symbol Units ValuesRate of drinking BL cholera �L
1week 1.5
Rate of drinking BH cholera �H1
week 7.5 (⇤)
BL cholera carrying capacity L# bacteria
m` 106
BH cholera carrying capacity H# bacteria
m`L700
Human birth and death rate b 1week
11560
Rate of decay from BH to BL � 1week
1685
Rate at which infectious individuals ⇠ # bacteria# individuals·m`·week 70
spread BH bacteria to waterDeath rate of BL cholera � 1
week7
30Rate of recovery from cholera � 1
week75
δ
BH
BL
βH
βL
I RSγ
ξ
χ
157
�L �HL b � ⇠ �L �
0
0.1
0.2
0.3
0.4
0.5
parameter
generalizedSobolindex
MC (Nmc = 10
4)
Alg 1 (Nquad = 609)
Alg 1 (Nquad = 2177)
Alg 2 (Nmc = 150/Nkl = 2)
Alg 2 (Nmc = 150/Nkl = 5)
Alg 2 (Nmc = 150/Nkl = 15)
Example: SIR Cholera ModelStrategy: Employed collocation and discrete projection with sparse grids to compute time-dependent global sensitivity indices.
Conclusion: Sensitive indices�: Recovery rate
�H : Rate of drinking BH cholera
L: BL carrying capacity; Note H = L/700
⇠: Rate at which BH bacteria spread
10
�110
010
110
20
0.2
0.4
0.6
0.8
1
time [weeks]
totalindices
�L
�HL
b�⇠�L�
10
�110
010
110
20
0.2
0.4
0.6
0.8
1
time [weeks]
generalizedtotalindices
�L
�HL
b�⇠�L�
δ
BH
BL
βH
βL
I RSγ
ξ
χ
Example: SIR Cholera ModelModel:
dSdt
= bN - �LSBL
L + BL- �HS
BH
H + BH- bS
dIdt
= �LSBL
L + BL+ �HS
BH
H + BH- (�+ b)I
dRdt
= �I - bR
dBH
dt= ⇠I - �BH
dBL
dt= �BH - �BL
Model Parameter Symbol Units ValuesRate of drinking BL cholera �L
1week 1.5
Rate of drinking BH cholera �H1
week 7.5 (⇤)
BL cholera carrying capacity L# bacteria
m` 106
BH cholera carrying capacity H# bacteria
m`L700
Human birth and death rate b 1week
11560
Rate of decay from BH to BL � 1week
1685
Rate at which infectious individuals ⇠ # bacteria# individuals·m`·week 70
spread BH bacteria to waterDeath rate of BL cholera � 1
week7
30Rate of recovery from cholera � 1
week75
159
Steps in Uncertainty Quantification
160
6. Quantification of Model Discrepancy – Thin Beam“Essentially all models are wrong, but some are useful” George E.P. Box
with
Example: Thin beam driven by PZT patches
Euler-Bernoulli Model:
Statistical Model:
Note: 7 parameters, 32 states
For all � 2 V
ZL
0
⇢(x)
@2w
@t
2 + �@w
dt
��dx +
ZL
0
YI(x)
@2w
@x
2 + cI(x)@3
w
@x
2@t
�� 00
dx
= k
p
V (t)
Zx2
x1
� 00dx
⇢(x) = ⇢hb + ⇢p
h
p
b
p
�p
(x) , YI(x) = YI + Y
p
I
p
�p
(x)
cI(x) = cI + c
p
I
p
�p
(x)y(t
i
, q) = w(ti
, x , q)
ti
YiYi = y(ti , q) + "i 161
Quantification of Model Discrepancy – Thin BeamExample: Good model fit
Note: Observation errors not iid
Model Fit to Data
0 20 40 60 80 100 1200
0.2
0.4
0.6
0.8
1
x 10−7
Frequency
Fre
quency
Conte
nt
Mathematical Model y(ti;q)
Data
Reference: Additive observation errors
Yi = y(ti , q) + "i
Yi = y(ti , q) + �(ti , eq) + "i
• M.C. Kennedy and A. O’Hagan, Journal of the Royal Statistical Society, Series B, 2001.
Quantification of Model Discrepancy – Thin BeamExample: Good model fit
Problem: Observation errors not iid
Result: Prediction intervals wrong
Prediction Intervals and Data
Model Fit to Data
Approaches: • GP Model: Inaccurate for extrapolation• Control-based approaches: difficult to extrapolate.• Problem: correct physics or biology required for extrapolation!
Quantification of Model Discrepancy – Thin BeamProblem: Measurement errors not iid
Result: Prediction intervals wrong
Prediction Intervals and Data
Model Fit to Data
One Approach: • Determine components of model you trust (e.g., conservation laws) and don’t trust (e.g., closure relations). Embed uncertainty into latter.• T. Oliver, G. Terejanu, C.S. Simmons, R.D. Moser, Comput Meth Appl Mech Eng, 2015.
2018-19 SAMSI Program: Model Uncertainty: Mathematical and Statistical (MUMS)
Quantification of Model Discrepancy – Thin Beam
Our Solution: “Optimize” calibration interval
• Use damping/frequency domain results to guide.
0.75 0.8 0.85 0.9 0.95 1−60
−40
−20
0
20
40
60
Time (s)
Dis
pla
cem
en
t (µ
m)
2 2.1 2.2 2.3 2.4 2.5−50
0
50
Time (s)
Dis
pla
cem
en
t (µ
m)
3 3.1 3.2 3.3 3.4 3.5−40
−30
−20
−10
0
10
20
30
40
Time (s)
Dis
pla
cem
en
t (µ
m)
0 1 2 3−30
−20
−10
0
10
20
30
Time (s)
Dis
pla
cem
ent (µ
m)
0 1 2 3−40
−30
−20
−10
0
10
20
30
Time (s)
Dis
pla
cem
en
t (µ
m)
Note: We have substantially extended calibration regime.Calibrate on [0,1] Calibrate on [0.25,1.25]
Concluding RemarksNotes:• UQ requires a synergy between engineering, statistics, and applied mathematics.
• Model calibration, model selection, uncertainty propagation and experimental design are natural in a Bayesian framework.
• Goal is to predict model responses with quantified and reduced uncertainties.
• Parameter selection is critical to isolate identifiable and influential parameters.
• Surrogate models critical for computationally intensive simulation codes.
• Codes and packages: Sandia Dakota, R, MATLAB, Python, nanoHUB.
• Prediction is very difficult, especially if it’s about the future, Niels Bohr.