Computational Methods for Design Lecture 4 – Introduction to Sensitivities John A. Burns C enter for O ptimal D esign A nd C ontrol I nterdisciplinary.

Computational Methods for Design Lecture 4 – Introduction to Sensitivities

John A. Burns

Center for Optimal Design And Control

Interdisciplinary Center for Applied MathematicsVirginia Polytechnic Institute and State University

Blacksburg, Virginia 24061-0531

A Short Course in Applied Mathematics

2 February 2004 – 7 February 2004

N∞M∞T Series Two Course

Canisius College, Buffalo, NY

A Falling Object

( ) ( )F t ma t“Newton’s Second Law”

. y(t)

( ) ( ) ( ) ( ) ( )g dampmy t F t F t mg y t y t

)()()( tytym

gty

)()()( tvtvm

gtv

)()( tvty

0)0(

000,10)0(

v

y

{

{

AIR RESISTANCE

System of Differential Equations

)()(

)(

)(

)(tvtv

mg

tv

tv

ty

dt

d

)()()( tvtvm

gtv

)()( tvty

0)0(

000,10)0(

v

y

)()(

)(),,),(()(

22

2

txtxm

g

txmgtxftx

dt

d

)(

)()(

2

1

tx

txtx

Parameters

),,,,(),,,( 2122

2

mgxxfxx

mg

xmgxf

IN REAL PROBLEMS THERE ARE PARAMETERS

SOLUTIONS DEPEND ON THESE PARAMETERS

),,,( mgtx

WE WILL BE INTERESTED IN COMPUTINGSENSITIVITIES WITH RESPECT TO THESE PARAMETERS

),,,(

mgtx

Examples: n=m=1

5)0( ),()( xtqxtxdt

d

qxqxf ),(

qqxfx

),(

CONTINUOUS EVERYWHEREqteqtxtx 5),()(

UNIQUE SOLUTION

1)0( ,)(

)(

xqt

txtx

dt

d

qt

xqxtf

),,(

qtqxf

x

1

),(

CONTINUOUS WHEN 0 qt

22 /)(),( qqqtqtxq

qqtqtxtx /)(),()(

UNIQUE SOLUTION

qtteqtxq

5),(

Logistics Equation

)()(1

1)(2

1 txtxq

qtxdt

d

xx

qqqqpf

2121

11),,(

),),(()( 21 qqtxftxdt

d

tqexqx

xqqqtx

1020

0221 ),,(

),,( 211

qqtxq

),,( 212

qqtxq

Computing Sensitivities

),,,,(),,,( 2122

2

mgxxfxx

mg

xmgxf

HOW DO WE COMPUTE THE SENSITIVITIES …

),,,(

),,,(),,,(

2

1

mgtx

mgtxmgtx

),,,(

),,,(),,,(

2

1

mgtx

mgtxmgtx

SEM Example 1

5)0( ),()( xtqxtxdt

d

DIFFERENTIATE

qteqtxtx 5),()(

qtteqtxq

5),(

qtqtqt eteqett

qtxqt

qtsdt

d

555),(),(

qtdefine

teqtxq

qts 5),(),(

),(),(55),( qtxqtsqeteqqtsdt

d qtqt

SEM Example 1

),(),( qtqxqtxdt

d

),(),(55),( qtxqtsqeteqqtsdt

d qtqt

),(),(),( qtxqtsqqtsdt

d

5)0( x

0)0( s

)()( tqxtxdt

d 5)0( x

)()()( txtsqtsdt

d 0)0( s

qttets 5)(

SEM Method

SOLVE THE SYSTEM (DE) – (SE)

)(),( tsqtxq

(DE)

(SE)

)()( tqxtxdt

d 5)0( x

)()()( txtsqtsdt

d 0)0( s

1. WHY DO IT THIS WAY ?2. WE DERIVED (SE) BY USING THE KNOWN

SOLUTION …HOW DO WE FIND (SE) IN GENERAL?

3. HOW GENERAL IS THIS PROCESS?

Derivation of SEN Eq

5)0( x)()( tqxtxdt

d

),()( qtxtx

),(),( qtqxqtxdt

d 5),0( qx

DIFFERENTIATE THE EQUATION WITH RESPECT TO q

),(),( qtqxq

qtxdt

d

q

),(),( qtxqtxq

q

),(),( qtxqdt

dqtx

dt

d

q),(),( qtxqtx

qq

),( qts ),( qts

INTERCHANGE THE ORDER OF DIFFERENTIATION

Derivation of SEN Eq

),(),(),( qtxqtsqqtsdt

d

0),0( qs

q

(DE)

(SE)

)()( tqxtxdt

d 5)0( x

)()()( txtsqtsdt

d 0)0( s

5),0( qx 05),0(

qqx

q

SEM Method

(DE)

(SE)

)()( tqxtxdt

d 5)0( x

)()()( txtsqtsdt

d 0)0( s

)()(

)(

)(

)()(

12

1

2

1

txtqx

tqx

tx

tx

dt

dtx

dt

d

)()( ),()( 21 tstxtxtx

)(

)()(

2

1

tx

txtx

0

5)0(x

Explicit Euler for SEQs

t0

(x0 ,s0)t

kt1t 2t 1kt

ihttth i 0 ,

),(1 qxfhxx kkk

R2


)),(()()(

)(

)(

)()(

12

1

2

1 qtxftxtqx

tqx

tx

tx

dt

dtx

dt

d

),(1 qxfhxx kkk

kkkxxq

xqh

x

x

x

x

12

1

2

1

12

1


)][][(][][

][][][

12112

1111

kkkk

kkk

xxqhxx

xqhxx

kkkxxq

xqh

x

x

x

x

12

1

2

1

12

1

SOLVE BOTH DE AND SE TOGETHER

HOW DOES IT WORK?

MATLAB Code for SEM

Set q

Set x0 and s0

Set h

Time interval

Set ICs

Explicit Euler

DE Solution x(t)

qteqtxtx 5),()(

05.h

01.h

1.h

SE Solution s(t)

qtteqtsts 5),()(

05.h

01.h

1.h

Special Structure of SE’s

(DE)

(SE)

)()( tqxtxdt

d 5)0( x

)()()( txtsqtsdt

d 0)0( s

(DE) )()( tqxtxdt

d 5)0( x

(SE) )()()( txtsqtsdt

d 0)0( s

FIRST: SOLVE (DE)qtetx 5)(

qte5

SECOND: SOLVE (SE)

Logistics Equation

)()(1

1)(2

1 txtxq

qtxdt

d

xx

qqqqpf

2121

11),,(

),),(()( 21 qqtxftxdt

d

tqexqx

xqqqtx

1020

0221 ),,(

),,( 211

qqtxq

),,( 212

qqtxq

SEQ for the Logistics Equation

DIFFERENTIATE THE EQUATION WITH RESPECT TO q1

),,(),,(1

1),,( 21212

121 qqtxqqtxq

qqqtxdt

d

1q

),,(),,(1

1),,( 21212

11

211

qqtxqqtxq

qq

qqtxdt

d

q

),,(),,(1

1),,( 21212

121 qqtxqqtxq

qqqtxdt

d


),,(),,(1

1),,( 21212

11

211

qqtxqqtxq

qq

qqtxdt

d

q

221

2

1211

1

)],,([),,( qqtxq

qqqtxq

q

221

2

1

1211

1

)],,([),,( qqtxq

q

qqqtxq

q

),,(),,( 21211

1 qqtxqqtxq

q

2211

12

)],,([1

qqtxqqq


),,(),,( 21211

1 qqtxqqtxq

q

),,(),,( 21211

1 qqtxqqtxq

q

22121

1211

2

)],,([),,()],,([21

qqtxqqtxq

qqtxqq

2211

12

)],,([1

qqtxqqq


),,( 211

qqtxdt

d

q

),,(),,( 21211

1 qqtxqqtxq

q

),,( 211

qqtxqdt

d

),,(),,( 21211

1 qqtxqqtxq

q


22121

1211

2

)],,([),,()],,([21

qqtxqqtxq

qqtxqq

22121

1211

2

)],,([),,()],,([21

qqtxqqtxq

qqtxqq


),,( 211

qqtxqdt

d

),,(),,( 21211

1 qqtxqqtxq

q

),,()],,([2)],,([1

211

2112

212

qqtxq

qqtxqqqtxq

),,(),,( 211

211 qqtxq

qqtsdefine

),,( 211 qqts ),,( 211 qqts

),,( 211 qqts

),,( 211 qqtsdt

d ),,(),,( 212111 qqtxqqtsq

)),,()],,([2)],,(([1

2112112

212

qqtsqqtxqqqtxq


),,( 211 qqtsdt

d ),,(),,( 212111 qqtxqqtsq

)),,()],,([2)],,(([1

2112112

212

qqtsqqtxqqqtxq

)(1 tsdt

d )()(11 txtsq

))()]([2)](([1

112

2

tstxqtxq

)(1 tsdt

d

2

21

21 )]([

1)()()]([

21 tx

qtxtstx

qq


NEED SENSITIVITY WITH RESPECT TO q2

)(1 tsdt

d

2

21

21 )]([

1)()()]([

21 tx

qtxtstx

qq

SEQ for the Logistics Equation 2

DIFFERENTIATE THE EQUATION WITH RESPECT TO q2

),,(),,(1

1),,( 21212

121 qqtxqqtxq

qqqtxdt

d

2q

),,(),,(1

1),,( 21212

12

212

qqtxqqtxq

qq

qqtxdt

d

q

),,(),,(1

1),,( 21212

121 qqtxqqtxq

qqqtxdt

d


),,(),,(1

1),,( 21212

12

212

qqtxqqtxq

qq

qqtxdt

d

q

221

2

1211

2

)],,([),,( qqtxq

qqqtxq

q

221

2

1

2211

2

)],,([),,( qqtxq

q

qqqtxq

q

),,( 212

1 qqtxq

q

221

22

2212

21 )],,([

1)],,([

)(

1qqtx

qqqqtx

qq


),,( 212

1 qqtxq

q

),,()],,([21

)],,([)(

121

221

2

2212

21 qqtx

qqqtx

qqqtx

qq

),,( 212

1 qqtxq

q

221

22

2212

21 )],,([

1)],,([

)(

1qqtx

qqqqtx

qq


),,( 212

1 qqtxq

q

2

2122

1 )],,([)(

1qqtx

qq

),,()],,([2 212

212

1 qqtxq

qqtxq

q

),,( 212

qqtxdt

d

q


),,(),,( 212

212 qqtxq

qqtsdefine

),,( 212

qqtxqdt

d ),,( 212 qqts ),,( 212 qqts

),,( 212 qqts


)(2 tsdt

d)(21 tsq

2

22

1 )]([)(

1tx

qq )()]([2 2

2

1 tstxq

q

)(2 tsdt

d)()](

21[ 2

21 tstx

qq

2

22

1 )]([)(

1tx

qq

SEQ’s for the Logistics Equation

FROM THE FIRST PARTIAL

)()(1

1)(2

1 txtxq

qtxdt

d

THE LOGISTICS EQUATION

0)0( xx

0)0(1 s

0)0(2 s

)(2 tsdt

d)()](

21[ 2

21 tstx

qq

2

22

1 )]([)(

1tx

qq

)(2 tsdt

d)()](

21[ 2

21 tstx

qq

2

22

1 )]([)(

1tx

qq

)(1 tsdt

d

2

21

21 )]([

1)()()]([

21 tx

qtxtstx

qq

SEQ’s for the Logistics Equation

)()(1

1)(2

1 txtxq

qtxdt

d

0)0( xx

FIRST: SOLVE (DE) )(tx

SECOND: SOLVE (SEs)

)(2 tsdt

d)()](

21[ 2

21 tstx

qq

2

22

1 )]([)(

1tx

qq

)(1 tsdt

d

2

21

21 )]([

1)()()]([

21 tx

qtxtstx

qq

Model Problem #1

dx)(w(x,q

(x),w)-w(x,)(dqd

1

0) ˆ

J qqq

SENSITIVITY

The sensitivity equation for s(x, q ) = q w(x , q) in the“physical” domain (q) = (0,q) is given by

Can be made “rigorous” by the method of mappings.MORE ABOUT THIS NEAR THE END

x) ,w(xq

),s(xs(x) 0

qq q

q ,

q x

)x(w)(s,)(s dxd | 00

q

x xsdxd,xw

dxdxs

dxd 0 ,0)(

2)(

83+)(

2

2q

Typical Cost Function

WHERE w( x , q ) USUALLY SATISFIES A DIFFERENTIAL EQUATION AND q IS A PARAMETER (OR VECTOR OF PARAMETERS)

1

0

22

1 |)(ˆ) ,(|) ), ,((=) ( dxxwxwwFJ qqqq

1

0

]) ,([])(ˆ) ,([) ), ,((=) ( dxxwxwxww qdqd

dqd FJ q q q q q

1

0

)] ,( [),(ˆ) ,( ) ), ,( (=)] ( [ dxxwxwxww qdqd

dqd FJ q q q q q

THE CHAIN RULE PRODUCES

OR (Reality) USING NUMERICAL SOLUTIONS

hh h h

CONTINUOUSSENSITIVITY

DISCRETESENSITIVITY

Computing Gradients

(I) BY FINITE DIFFERENCES

( )- ) (

)( qd

d JJJ

qq0 q0

q0

q

hh

h

TYPICAL APPROACHES TO COMPUTE

)(dqd J q

q =q0

h

(II) BY DISCRETE SENSITIVITIES

1

0

)] ,( [),(ˆ) ,( ) ), ,( (=)] ( [ dxxwxwxww qdqd

dqd FJ q0

q0q0q0 q0hh h h

Computing Gradients

FINITE DIFFERENCES

• REQUIRES 2 NON-LINEAR SOLVES

• IF SHAPE IS A DESIGN VARIABLE, FD REQUIRES 2 MESH GENERATIONS

DISCRETE SENSITIVITIES

• REQUIRES THE EXISTENCE OF THE DISCRETE SENSITIVITY

• IF SHAPE IS A DESIGN VARIABLE, THE DISCRETE SENSITIVITY LEADS TO MESH DERIVATIVES COMPUTATIONS

WHAT IS THE “CONTINUOUS / HYBRID”SENSITIVITY EQUATION METHOD? --- SEM

1

0

)] ,( [),(ˆ) ,( )] ( [ dxxwxwxw qdqd J q0

q0q0hh h, k

APPROXIMATE

A Sensitivity Equation Method

FOR q > 1 AND h=q/(N+1) CONSIDER (FORMAL)

h h hh h

NUMERICAL APPROXIMATION

x=0 x=1 x=q

x

w(x)

w h(x) = Finite Element Approximation

4 00 )(w,)(w qq ,

x,xwdxdxw

dxd 0 0

3)(

81+)(

22

DISCRETE STATE EQUATION


h h

h h hh h 4 00 )(w,)(w qq ,

x,xwdxdxw

dxd 0 0

3)(

81+)(

22

q

x)x(w)(s,)(s dx

d | 00q

q ,

x xsdxd,xw

dxdxs

dxd 0 ,0)(

2)(

83+)(

2

2q

h

IMPORTANT OBSERVATIONS The sensitivity equations are linear The sensitivity equation “solver” can be constructed

independently of the forward solver -- SENSE™ When done correctly “mesh gradients” are not required


FOR q > 1 AND k = q/(M+1) CONSIDER (FORMAL)

2nd NUMERICAL APPROXIMATION

x=0 x=1 x=q

x

s(x)= qw(x,q)

s h,k(x) = Finite Element Approximation ofh

h,k

) ,(),( xwq

xsh,k

q q

q ,

x xsdxd,xw

dxdxs

dxd 0 ,0)(

2)(

83+)(

2

2q

q

x)x(w)(s,)(s dx

d | 00q

h

h

h

Convergence Issues

),,,( )] ,([),(ˆ) ,( )] ( [1

0

xdxxwxwxwdef

qdqd GJ

q qqhhhq k

h,k

THEOREM. The finite element scheme is asymptotically consistent.

0-)] ( [00

),,,x(lim dqd GJ qq

h

hh k

k

a trust region method should (might?) converge.

),,,(-)] ( [ xdqd GJ qqh

h kWhen the error is small, thenIDEA:

J. T. Borggaard and J. A. Burns, “A PDE Sensitivity Equation Method for Optimal Aerodynamic Design”, Journal of Computational Physics, Vol.136 (1997), 366-384.

R. G. Carter, “On the Global Convergence of Trust-Region Algorithms Using Inexact Gradient Information”, SIAM J. Num. Anal., Vol 28 (1991), 251-265.J. T. Borggaard, “The Sensitivity Equation Method for Optimal Design”, Ph.D. Thesis, Virginia Tech, Blacksburg, VA, 1995.

Convergence Issues

N=16, M=32

Convergence Issues

),,,(-)] ( [ xdqd GJ qqh

h h

THE CASE k = h is often used, but may not be “good enough”

NOT CONVERGENT

N=M=16 Tol = 0.00001 Tol = 0.0001 Total: 378.82Iter q Grad. Norm Step Time (secs) Cost Time Grad. Time

0 1.2000 4.3998E+00 -3.6427E-02 0.1231 1 1.1636 3.1583E+03 1.4051E-03 31.3210 31.2697 0.04782 1.1650 3.0910E+03 -1.4339E-03 36.2310 36.1798 0.04803 1.1635 5.8909E+02 7.8372E-03 46.1160 46.0075 0.10434 1.1714 5.0139E+03 -8.8462E-04 45.3550 45.3006 0.05115 1.1705 2.9396E+03 -1.5052E-03 43.6720 43.6208 0.04706 1.1690 1.7238E+04 2.5880E-04 42.5810 42.5301 0.04687 1.1693 2.5888E+03 1.7342E-03 46.3470 46.2965 0.04728 1.1710 4.6995E+04 -9.4732E-05 44.1900 44.1396 0.04689 1.1709 1.5743E+02 0.0000E+00 42.8790 42.8265 0.0485

Timing Issues

THE CASE k = 2h offers flexibility and ),,,(-)] ( [ xdqd GJ qqh

h 2h

convergence. But, what about timings?

Approximately 96 .6% of cpu time spent in function evaluationsApproximately 02 .4% of cpu time spent in gradient evaluations

N=16, M=32 Tol = 0.00001 Tol = 0.0001 Total: 39.81Iter q Grad. Norm Step Time (secs) Cost Time Grad. Time

0 1.2000 4.8489E+00 -3.2414E-02 0.1968 1 1.1676 2.0720E+01 4.0347E-01 34.9270 34.8053 0.09112 1.5711 4.4544E+00 3.7808E-01 1.2613 1.1234 0.10753 1.9491 6.9846E-02 -1.2442E-02 0.9941 0.8714 0.09254 1.9367 1.5779E-02 2.7472E-03 0.4190 0.2907 0.09385 1.9394 3.3558E-03 -5.8723E-04 0.4095 0.2845 0.09436 1.9389 7.3235E-04 1.2801E-04 0.4525 0.3135 0.10837 1.9390 1.5892E-04 -2.7785E-05 0.8602 0.6327 0.19688 1.9390 3.0703E-05 0.0000E+00 0.2914 0.1451 0.1083

Mathematics Impacts “Practically”

UNDERSTANDING THE PROPER MATHEMATICAL FRAMEWORK CAN BE EXPLOITED TO PRODUCE BETTER SCIENTIFIC COMPUTING TOOLS

A REAL JET ENGINE WITH 20 DESIGN VARIABLES PREVIOUS ENGINEERING DESIGN METHODOLOGY

REQUIRED 8400 CPU HRS ~ 1 YEAR USING A HYBRID SEM DEVELOPED AT VA TECH AS

IMPLEMENTED BY AEROSOFT IN SENSE™ REDUCED THE DESIGN CYCLE TIME FROM ...

8400 CPU HRS ~ 1 YEAR TO 480 CPU HRS ~ 3 WEEKS

NEW MATHEMATICS WAS THE ENABLING TECHNOLOGY

Special Structure of SE’s

(DE)

(SE)

)()( tqxtxdt

d 5)0( x

)()()( txtsqtsdt

d 0)0( s

(DE) )()( tqxtxdt

d 5)0( x

(SE) )()()( txtsqtsdt

d 0)0( s

FIRST: SOLVE (DE)qtetx 5)(

qte5

SECOND: SOLVE (SE)

END

Computational Methods for Design Lecture 4 – Introduction to Sensitivities John A. Burns C enter for O ptimal D esign A nd C ontrol I nterdisciplinary.

Documents

logistics equationfirst

logistics equationfrom

logistics equationinterchange

thephysical domain q

selogistics equationseq

order of differentiationseq

seqsr2explicit euler

seqsexplicit euler