Optimization in Engineering Design

1

Optimization in Engineering Design

Mark Abramson, USAF Institute of Technology

Charles Audet, Gilles Couture,

Ecole Polytecnique. de Montreal

John Dennis, Rice University

Andrew Booker, Evin Cramer,Paul Frank, Joerg Gablonski, Boeing Phantom Works

Thanks to: AFOSR, Boeing, LANL, SANDIA, ExxonMobil, NSF

IMA2003

2

Find the deepest point in a lake

Is it possible to promise the deepest point?

IMA2003

3

Not with a deterministic code

Global optimization is generally impossible

IMA2003

4

If we are lucky...

Let f(x) represent the depth at position x. Sometimes f

and at least its gradient are available so we can use local

models - eg assume that f(x) is a quadratic lake bottom

around the current best point

A quasi-Newton method:

Using information gathered so far, and assuming that the

lake bottom is quadratic, predict the deepest point

Find the depth and the gradient of the depth at the

predicted deepest point and update the depth model

IMA2003

5

Finite difference derivatives?

If the gradient is not available, can we approximate each

partial NorthSouth and EastWest by finite differences?

∂f(x)∂x NorthSouth

uf(x + ∆x NorthSouth)− f(x)

∆x NorthSouth

Maybe, but how to choose ∆x is a serious problem

IMA2003

6

Most cases

What if the only way to measure the depth is to row out

there in 30 dimensions and drop a line?

Or, to wait days for a simulation of the depth to run

based on geophysics,topography, water temperature, etc

And for the resulting estimate to be pretty rough

And if you aren’t sure if depth is what you want to

measure

IMA2003

7

Conflicting objectives in design

Performance Decisions

Payloads Decisions

Propulsion Decisions

Structures Decisions

Systems Decisions

Airplane Architecture

Airplane Level Architecture

IMA2003

8

Design Explorer Applications

Multidisciplinary wingplanform design

777 Engine Duct Seals

Helicopter Rotor Design

Machining,riveting,and drilling database

3-D Fighter Aerodynamics

Engine NozzlePerformance

Shot peen forming of wing skinsSpace Station Power System

Aerospike Nozzle

IMA2003

9

The target optimization problem

The design problem is:

minimize fp(x)

subject to x ∈ X ∩ {x ∈ <n : Cp(x) ≤ 0},

There are parameters and variables:

� Contextual parameters p, supposedly fixed, and

� Optimization or design or control variables x

IMA2003

10

Properties of the target problem

� p is fixed in the simulation, but it is subject to

uncertainty because of incomplete knowledge (e.g.

material properties) or because the product is used

differently (e.g., the altimeter is off by 5%)

� x ∈ X must be satisfied when the underlying simulations

for fp, Cp are called. X is simple, say a polygon

� Cp(x) ≤ 0 only has to hold at the solution

IMA2003

11

Properties of variables and constraints

� Function evaluations typically involve numerically linking

legacy PDE solvers. Evaluations may fail unexpectedly

� There are few correct digits. Accurate approximation of

derivatives is problematic

� Some variables are categorical, i.e., the simulations

only run with certain discrete choices, e.g., choice and

sequencing of manufacturing processes - Lecture 3.

IMA2003

12

Uncertainty in design variables

Instead of specified design x∗, the delivered product

might be the design specified by a nearby x

Very common in engineering design. Good data may be

tough to get

Robustify against such “variability” by giving up some

optimality at a given point in return for optimality over

nearby designs

IMA2003

13

Uncertainty in context variables

Products are designed for optimal performance under

specified operating conditions, loads, etc

We would be willing to give up some performance at the

exact conditions p in return for optimal performance over

all likely nearby conditions q - or we can constrain

probability of failure for the nominal p

Seems to be “chance constrained” formulation - nothing

new

IMA2003

14

Hubris or Sanguinity?

Both types of uncertainty can be modelled for

optimization by incorporation into f, C

minimize f(x)

subject to x ∈ X ∩ {x ∈ <n : C(x) ≤ 0},

where f(x), cj(x) incorporate uncertainty modelled by

the user’s scheme of choice

Still belongs to target class, Bring ‘em on for now, but

seek tighter coupling

IMA2003

15

A confidence builder

Meckesheimer, Booker, and Torng report very good

performance of our approach on some structural

engineering problems in dealing with probability of failure

by constraints or as the objective

“Reliability based design optimization using Design

Explorer” available from [email protected]

From this point on, assume formulation incorporates

uncertainty

IMA2003

16

More problem structure

Typically, fp(x) is actually fp(x, u(x)), where u(x) are

ancillary variables gotten from some simulation with

input x

I.e., given x, solve Fp(x, u) = 0 for u(x). Usually a

discretized PDE or several such legacy solvers. This is a

major reason why fp(x) fails to evaluate - the scheme to

couple the solvers fails to compute u(x)

These are called MDO problems, and they are hard...

IMA2003

17

Target to end 1st Lecture here

IMA2003

18

The target optimization problem

The design problem can be formulated:

minimize f(x, u(x))

subject to x ∈ X ∩ {x ∈ <n : C(x) ≤ 0},

� Optimization or design or control variables x

� x ∈ X must be satisfied when the underlying simulations

for f, C are called. X is simple, say a polygon

� C(x) ≤ 0 may include constraints on u(x)

IMA2003

19

Properties of variables and constraints

� State or ancillary variables u(x) found by solving

F (x, u) = 0

� Function evaluations typically involve numerically linking

legacy PDE solvers, so F (x, u(x)) = 0 can be

complicated

� Solving F (x, u(x)) = 0 when there are multiple

disciplines is called a ”multidisciplinary analysis”

IMA2003

20

An oversimple multidisciplinary analysis

Let

u1(x) = A1(x, u2(x)) or u1(x)−A1(x, u2(x)) = 0

u2(x) = A2(x, u1(x)) or u2(x)−A2(x, u1(x)) = 0

Newton’s method would require differentiating the PDE

solver codes Ai w.r.t. their input - pray for automatic

differentiation to succeed

IMA2003

21

MDA = Multidisciplinary analysisJanuary 3, 2003 6

D1

Aerodynamics

�

?E12

?M1 ≡ M12

A1

?U1

F21

?µ21

-������������������

D2

Structures

-

?E21

?M2 ≡ M21

A2

?U2

F12

?µ12

�DDDDDDDDDDDDDDDDDD

Figure 1: Aeroelastic System

the so-called “design point”, like the viscosity of the air, angle of attack of the wing,and speed. We have completed the MDA for a given decision vectorx when we havefoundu(x) such that the following hold foru = (M1, M2).

F1(x, M1, M2) = E21 ◦ F21 ◦ A1(M1)−M2, (3)

F2(x, M1, M2) = E12 ◦ F12 ◦ A2(M2)−M1. (4)

andA1 is the the flow solver (maybe finite difference) andA2 denotes the structurescode (maybe finite element). TheE, F routines are interface routines that do suchbookkeeping as converting pressures to loads and deflections to shape. Figure 2 pro-vides a more detailed idea of what theE andF boxes look like in that there is probablyone required for each of the other disciplines connected toAi. These interface boxesare the source of many of the difficulties with MDO.

– Why is successive replacements is used in MDA and not Newton? When faced witha system of nonlinear equations to solve, we automatically think of Newton’s method.The problem is the availability of derivatives ofF with respect tou. Each finite dif-ference step would require a single discipline solve, and the number of correct digitsin the difference might be small anyway. Still, one suspects that sparsity is there to beexploited, and the number of variables involved may be too large to use Broyden-typeupdating methods. The most common approach is for the user to apply intuition to asequencing of the single discipline solvers and use successive replacements, i.e., fixu2, . . . uns , solve the first equation block for a newu1. Now fix all but u2 with thenewu1, and useF2 to get a newu2, etc, and iterate. If convergence is a problem, thenreorder the equation and unknowns and try again.

• Formulations of MDO using aeroelastic design example

– MDF or black box or closed equations or control theory method (different fields call itdifferent names. Different engineering disciplines are often different worlds.): This is

� Disciplines D1, D2 with analysis codes A1, A2

� µij is compressed info from Dj needed by Di

IMA2003

22

Mathematical formulation of MDA

Write F (x, u) = 0, but solve F (x,M1,M2) = 0 for M ,

where

F1(x,M1,M2) = E21 ◦ F21 ◦A1(M1)−M2,

F2(x,M1,M2) = E12 ◦ F12 ◦A2(M2)−M1

� A1 finite difference and A2 finite element maybe

� E,F are interface routines that do such bookkeeping as

converting pressures to loads and deflections to shape

IMA2003

23

A generic discipline in an MDO problemJanuary 3, 2003 7

Di

XD

?

µi1

?Ei1

PPPPPPPqMi1

µi2

?Ei2

ZZ

Z~Mi2

• • •

µin

?Ein�������)

Min

µi1

?Ei1

� ��?Mi

Ai

?Ui� ��

µ1i?

F1i

�������)

µ2i?

F2i

��

�=

• • •

µni?

Fni

PPPPPPPq

Figure 2: Generic Discipline

the traditional approach in which a complete MDA foru(x) is done for each choice ofthe design variablesx. Thus, the optimization problem to be solved is a function onlyof the design variablesx, and it looks like (1). An important point is that the objectivefunctionf(x) is almost surely of the formf(x) = f(x, u(x)). The point being that thestate or system variablesu are needed to evaluate the design objective function. Thus,givenx, we solve an MDA problem foru, and then we have the arguments to evaluatef , C. Notice that if there are bounds onu, then they become bounds onu(x), and hencethey are really nonlinear constraints. If bounds onu(x) must be closed, ( for examplebounds onu may represent safety considerations - like operating temperatures of somechemical process), then we may have a nasty difficulty for the optimization algorithm.

∗ Advantage is optimization variables are the design variables (generallynd << ns

and every iterate can be made feasible. Thus, when we stop, we only need torestore feasibility inC(x), often called the side constraints.

∗ Disadvantage is computational effort to complete an MDA for every functionvalue. Even finite difference derivatives with respect tox require an MDA foreach forward difference quotient.

– All-at-once (AAO) or Simultaneous Analysis and Design (SAND) or open equationsor nonlinear programming method: This is the ideal in many ways. It doesn’t alwayswork. There is some analytic and practical evidence that nonsmooth behavior in the so-lution u(x) gets smoothed as it passes throughf(x, u(x)) and so the blackbox methodmay work when the AAO method doesn’t. However, when the AAO approach does

E,F routines are a major headache to write

IMA2003

24

The MDF or blackbox formulationJanuary 3, 2003 8

Optimizer (Controls calculation off , CD)

?XD 6U1, U2

Aeroelastic Analysis Solver

D1Aerodynamics

�

?E12

?M1≡M12

A1

?U1

F21

?µ21 -�������������

D2Structures

-

?E21

?M2≡M21

A2

?U2

F12

?µ12�DDDDDDDDDDDDD

Figure 3: Multidisciplinary Feasible (MDF) Method

Optimizer (Controls calculation off , CD)

?

XD, XU1 , XU2

?

XD, XU1 , XU2

AerodynamicsResidualComputation

? -W1

6

StructuresResidualComputation

?�W2

6

Figure 4: All-At-Once or AAO Formulation

work, it seems most efficient. One solves the AAO problem as:

minx,u

f(x, u) s.t. x ∈ X, F (x, u) = 0, andC(x) ≤ 0 . (5)

Notice that this requires that we “open” the single discipline solvers in order to beable to evaluate their internal residuals ( remember, U is produced by solving an inter-nal system of equations). This is expensive, and generally it is resisted by the singlediscipline specialists.

∗ Advantage is less expensive function values and derivatives as well as a generallyless nonlinear optimization problem

∗ Disadvantage is many more variables (the design variables + the ancillary vari-ablesu, as well as the inability to use legacy disciplinary solvers. Feasible pointalgorithms are not compatible

� Requires a full MDA for every evaluation

� Derivatives must be through solvers - requires dudx

IMA2003

25

The All-At-Once or SAND formulation

January 4, 2003 8

Optimizer (Controls calculation off , CD)

?XD 6U1, U2

Aeroelastic Analysis Solver

D1Aerodynamics

�

?E12

?M1≡M12

A1

?U1

F21

?µ21 -�������������

D2Structures

-

?E21

?M2≡M21

A2

?U2

F12

?µ12�DDDDDDDDDDDDD

Figure 3: Multidisciplinary Feasible (MDF) Method

Optimizer (Controls calculation off , CD)

?

XD, XU1 , XU2

?

XD, XU1 , XU2

AerodynamicsResidualComputation

? -U1

6

StructuresResidualComputation

?�U2

6

Figure 4: All-At-Once or AAO Formulation

work, it seems most efficient. One solves the AAO problem as:

minx,u

f(x, u) s.t. x ∈ X, F (x, u) = 0, andC(x) ≤ 0 . (5)

Notice that this requires that we “open” the single discipline solvers in order to beable to evaluate their internal residuals ( remember, U is produced by solving an inter-nal system of equations). This is expensive, and generally it is resisted by the singlediscipline specialists.

∗ Advantage is less expensive function values and derivatives as well as a generallyless nonlinear optimization problem

∗ Disadvantage is many more variables (the design variables + the ancillary vari-ablesu, as well as the inability to use legacy disciplinary solvers. Feasible pointalgorithms are not compatible

� (x, u) = (xd, xu) all are optimization variables, and

Fp(xd, xu) = 0 is enforced only in the limit

� Can’t use legacy solvers, but derivatives are simpler - dudx

avoided

IMA2003

26

The IDF or DAO formulationJanuary 3, 2003 9

Optimizer (Controls calculation off , CD, C12, C21)

?

XD, Xµ12

?

XD, Xµ21

AerodynamicsAnalysis

? -F21(XD, U1(X))

(including, e.g.,U1)

6

StructuresAnalysis

?�F12(XD, U2(X))

(including, e.g.,U2)

6

Figure 5: Individual Discipline Feasible or IDF Formulation

– Individual Discipline Feasible (IDF) or In-Between methods: Generally, a small per-centage of the state variables from one discipline’s solver are needed as input to othersolvers. Suppose we reflect this by writingu = (z, u), andz is the smaller subvectorof the solver variablesu actually needed as input to various solvers. We can now posethe MDO problem as:

minx,z

f(x, z, u(z, x)) s.t. x ∈ X, C(x) ≤ 0, (z, u(z, x))− u(x) = 0. (6)

∗ Advantage is ability to use legacy disciplinary solvers as they are and the opti-mization problem is nearer the size of the MDF than the AAO problem. Seems tobe more efficient than the blackbox.

∗ Disadvantage is that the derivatives needed are like those needed for MDA, andfeasible point algorithms are not compatible

Alexandrov and Lewis seem to have rediscovered this formulation, although there maybe some difference we missed, and they do not compare it to IDF. They call it DAOor “distributed analysis optimization”, a nicer name than IDF [3]. Alexandrov andKodiyalam give computational evidence that this is an effective formulation [2].

� MDA only in the limit - many fewer variables than AAO

� Uses legacy solvers, but requires dudx

IMA2003

27

Summary

� Blackbox/MDF is least efficient, but most stable, and

most often used in practice. Derivatives can sometimes

be gotten by adjoint approaches, but not if there are

many constraints

� AAO is most efficient when it works, but incompatibility

with legacy codes is a huge barrier to practical use in

most fields Very common in chemical process control

� IDF/DAO seems a good compromise - generalizes

domain decomposition

IMA2003

28

Summary

� Many designers mistrust a method like AAO or IDF that

doesn’t have all feasible iterates

� I got the idea for IDF from Georg Bock’s direct multiple

shooting

� Blackbox or MDF will be used in my Math Coll.

numerical results for the SMF, which is derivative-free

� A heuristic called “collaborative optimization” mimics

the “by hand” traditional approach

IMA2003

29

� DAKOTA project at SANDIA attacks same problem

class by more heuristic approaches

� Automatic differentiation is the critical technology for

MDO

IMA2003