EVOLUTIONARY ALGORITHMS: What are EAs? Mathematical Formulation & Computer Implementation Multi-objective Optimization, Constraints Computing cost reduction Kyriakos C. Giannakoglou Associate Professor, Lab. of Thermal Turbomachines (LTT), National Technical University of Athens (NTUA), GREECE
71
Embed
What are EAs? Mathematical Formulation & Computer …velos0.ltt.mech.ntua.gr/kgianna/optdpms/distr/theory_EA.pdf · 2007. 12. 4. · From traditional problem solving techniques to
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
K.C. Giannakoglou, Associate Professor NTUA, Greece
Outline
From traditional problem solving techniques to EAs.Generalized EA: Basic and advanced operators.Mathematical foundations of EAs.EAs for multi-objective optimization.Distributed Evolutionary Algorithms (DGAs). Hierarchical Evolutionary Algorithms (HGAs).Constraints’ handling.Efficient ways for reducing the computing cost of EAs.Applications in the field of aeronautics, turbomachinery, energy production, logistics.
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Effective/Efficient Problem Solving Techniques
The number of possible solutions in the search space is so large as to forbid exhaustive searchSeeking the best combination of approaches that addresses the purpose to be achievedFinding the solution using the available computing resourcesFinding the solution within the available timeOne or more (contradictory) targetsSolving the problem under a number of (hard/soft) constraints
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Basic Concepts of Problem Solving Techniques
Representation: how to encode alternative candidate solutions for manipulationObjective: describes the purpose to be fulfilledEvaluation function: returns a value that indicates the (numeric of ordinal) quality of any particular solution, given the representation
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Hill-Climbing: A Traditional (Deterministic) PST
Useful Definitions:Neighborhood of a solutionLocal Optimum
x1
x2
Ideas for creating Hill-Climbing Method variantsHow to select the new solution for comparison with the current solution (how to compute the gradient) …To use more than one starting solutions, if necessary …To be “less deterministic” …
Requirements:A starting pointComputation of gradientTermination criteria
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Algorithms Relying on Analogies to Natural Processes
K.C. Giannakoglou, Associate Professor NTUA, Greece
Schema Theorem (3/4) :
The effect of mutation:
Possibility of maintaining the schema S after mutation:
( )(1 ) 1 ( )o Ssurv mut mutp p p o S= − ≈ −
****11010*01*1*11*
1,
( ) ( )( ) ( ) (1 ( ) )
( ) 1g g Xover mutmean g
F S d Sm S m S p o S p
F S m+ ≥ − −−
Final Schema Growth Equation:
Short, low-order, above-average
schemata should receive an
(exponentially) increasing number of strings in the next generations
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Schema Theorem (4/4) :
Lessons Learned:Short, low-order, above-average schemata sould receive an (exponentially) increasing number of strings in the next generations (Schema Theorem).GA explore the search space by short, low-order schemata.GAs seek near-optimal performance through the juxtaposition of short, low-order, high-performance schemata (the so-called building blocks, Building Block Hypothesis).
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Exploration vs. Exploitation :
Exploration: seeking the global optimum in new and unknown areas in the search space.Exploitation: making use the knowledge gained from the previously examined points to guide the search towards new better points in the search space.
Holland 1975: GAs
but:Infinite population sizeFitness function value accurately reflects the utility of a solutionGenes in a chromosome do not interact significantly
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
EAs: Binary or Real Coding?
Binary Coding:Creates lengthy binary strings if high accuracy is required.Offers the maximum number of schemata per bit of information, compared to any other coding.Facilitates theoretical analysis and the development of new genetic operators.Smallest alphabet that allows a natural expression of the problem (Goldberg, 1989).
Real Coding:Problem-tailored genetic operators can readily be devised.High-cardinality alphabets contain more schemata (Antonisse, 1989).
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Generalized Evolutionary Algorithm (EA)
λ,gS λ
, 1,mx m M=
μ,gS μ
e,g eS
, 1,kF k K=
( )y F x=
1: Kφ →
eΤ
Τμκ
ρΤrΤm
Offspring Parent Elite
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
ΕΑ
–
Schematic Presentation:
( )
( )( )
if( ( )) end
2
,
1, , ,
, , 1,
, , 1,
1, , ,
1, 1, 1,
( ),
( )
( ),
( )
( )
1
converge
g
g e g g ee
g g g e
g g g ee
g g g
g g g em r
y F x s S
S T S S
y s S S S
S T S S
S T S S
S T S S
g g
g
λ
λ
μ λ
λ λ
μ μ λμ
λ μ
φ
+
+
+
+
+ + +
⎡ = ∀ ∈⎢⎢⎢ = ∪⎢⎢ ∀ ∈ ∪ ∪⎢⎢⎢ = ∪⎢⎢⎢ = ∪⎢⎢
= Τ ∪⎢⎢⎢ = +⎢⎢⎢⎢⎣
, , ,0, , , g e g g randomg S S S Sμ λ= = ∅ = ∅ =
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Parent Selection Operator
Τμ
(1/5)
(Neglecting Elitism)
Phase 1:
Phase 2:
( ), , ,,1
prov g gS S Sμ μ λμ= Τ ∪
( )1, ,,2
g provS Sμ μμ
+ = Τ
Life-span κFitness φ
Selective Pressure
(Fitness φ)
(ES)
(GA)
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Parent Selection Operator
Τμ
(2/5)
Proportional selection
Linear ranking
Roulette wheel
Probabilistic Tournament selection
Indirect Selection (μ<λ)
( )( )
1, , ,,1
1, ,,2
g g g
g prov
S S S
S S
μ μ λμ
μ μμ
+
+
= Τ ∪
= Τ
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Parent Selection Operator
Τμ
(3/5)
Proportional selection:
Select (λρ)
individuals
out of (μ) preselected
ones
( )
( )
1
( )( )
( )
1
1
_ _ int
s
i
ss
i
number of copies
μι
μι
μφ
φ
λρμφ
φ
=
=
≥
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
∑
∑
Premature convergence due to
the presence of a “super-fit”
individual.
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Parent Selection Operator
Τμ
(4/5)
Roulette Wheel:
Select (λρ)
individuals
out of (μ) preselected
ones
( )
( )
1
_ 360s
o
i
angle slot μι
φ
φ=
=
∑
Premature convergence due to
the presence of a “super-fit”
individual.
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Parent Selection Operator
Τμ
(5/5)
Fitness Ranking:Individuals are sorted in the order of fitness valuesReproductive trials are assigned according to rankLinear RankingExponential Ranking, etc
Overcomes the problem of the presence of an extreme individualFitness ranking performs better than fitness scaling.
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Recombination Operation
Tr
–
Binary Coding
}One-point
Two-pointOne- or two-point per variableDiscrete (variables’ interchanging)Uniform (with parent-depending probability)
01101001110100001100
0110001100
011010011101000011001010001010
0110000010
ρ=2
ρ=3
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Recombination Operation
Tr
– Real Coding
One-point, ρ=2 x1
x2
x3
x4
x5
x6x1
x2
x3
x4
x5
x6 } x1
x2
x3
x4
x5
x6
x4
=x4
+r
(x4
-x4
),
r∈ [0,1]Two-pointM-point
Discrete, ρ=2Discrete Panmictic ρ=Μ (50% selection probability
from each parent {1,2}, {1,3}, ..., {1,Μ} )Generalized Intermediate Panmictic
xm
=xm
+rm
(xm,ρ
-xm
),
m=1,M,
rm
∈ [0,1]
xm
=xm
+rm
(xm
-xm
),
m=1,M,
rm
∈ [0,1]
Blend Xover (BLX-a), c=(1+2a)r-a, a=0.5
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Mutation Operator
Tm
-
Binary Coding
011000110000101011001
1...5m M
mm
Pb
=∑∼
Dynamic Adjustment
of
Pm , depending on
• the number of generations without any improvement
• the number of evaluations without any improvement
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Mutation Operator
Tm –
Real Coding
Dynamic mutation probability per variable
Pm
max
1
1
2max
(1 )
2
( , ), 0.5
( , ), 0.5
( , ) 1
( , ) 1p
m m m
mm m m
p
gg
x D g U x rx
x D g x L r
gD g a a r
g
D g a a r−
⎧ + − <⎪⎪⎪= ⎨⎪ − − ≥⎪⎪⎩⎛ ⎞⎟⎜= ⋅ ⋅ − ⎟⎜ ⎟⎜⎝ ⎠
⎛ ⎞⎟⎜ ⎟= ⋅ −⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠
ή
…
or, using number of evaluations, instead of number of generations
p~0.2
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Mutation Operator
Tm –
Real Coding
( )exp (0,1) (0,1)
(0,1)
m m m
m m m m
´
x x N
σ σ τ τ
σ
= ⋅ ⋅Ν + ⋅ Ν
= + ⋅
( )( )
1
1
2
' 2
τ
τ
−
−
= Μ
= Μ
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Genetic Algorithms or Evolution Strategies or …
Binary Codingμ=λ, parents=offspringρ=2, two-parent recombinationκ=0, zero life-spanΡr<1, recombination probabilityK=1, one target
[Holland,
1970]
[Goldberg, 1989][Michalewicz, 1994][Fogel]
Real Coding, including the evolution parametersWithout parent Selection Operator (μ<λ)μ<<λκ=0 (μ,0,λ) = (μ,λ) or κ=∞ (μ,∞,λ) = (μ+λ)ρ=2Pr=1Κ=1
[Schwefel, Rechenberg, 1965]
[Bäck, 1996]
GA
ES
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Multi-objective Optimization
Non-Pareto Front
Techniques
Pareto Front Techniques
(K = number of objectives)
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Pareto Front
Front
0 (Pareto) F1
F2
Front
1
Front
2
,g eS
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Pareto Front –
Definition (Minimization Problems):
Dominant Solution:( ) ( ) ( ) ( )
( ) ( )
( ) ( )
{1,..., } :
{1,..., } :
p q p q
p qk k
p qk k
x x s s
k K F F
k K F F
< ⇔ < ⇔
∀ ∈ ≤ ∧
∃ ∈ <
' ' :Mx x x x/⇔ ∃ ∈ >
Pareto Optimal Solution:
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Multi-Objective Optimization
–
Computation of
φ
1
K
k kk
w Fφ=
= ⋅∑
, [1, ]kF k Kφ = ∈
Pareto_Rank( ), k 1,kF Kφ = =
For parent selection, VEGA [Schaffer, 1984]
[Goldberg, 1989]
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Vector Evaluated Genetic Algorithm (VEGA)
K=3
Selection based on φ=F1
Selection based on φ=F2
Selection based on φ=F3
Recombination, Mutation
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
VEGA: Guess the final solutions…
Front
0 (Pareto) F1
F2
Front
1
Front
2
,g eS
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
φ
computation using the Pareto front:
Front Ranking [Goldberg, 1989]
Niched Pareto GA (NPGA) [Horn, Nafpliotis, 1993]
Nondominated Sorting GA (NSGA) [Srinivas, Deb, 1994]
Multilayer PerceptronRadial Basis Function Networks
Kriging
Ways of using the surrogate evaluation model:Decoupled from the exact evaluation tool (+Design of Experiments, DoE)In combination with the exact evaluation tool
Regular Training (depending on the number of new entries in the DB)Dynamical Training (separately, for each new individual)
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Use of Surrogate Models (with Off-Line Training)
Design of Experiments
Exact Evaluations
Surrogate model
Evaluations using the Surrogate
Model
Genetic Operators
“Optimal”
Solution
Inner Loop
Outer Loop
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Use of Surrogate Models (with On-Line Training)
best
Evaluations using the Surrogate Model(s)
Evaluations using the
Exact Model
Genetic Operators
Fitness Value Homogenization
New Population
Build the Surrogate Model(s)
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Inexact Pre-Evaluation (IPE) –
The Concept:
Exact Evaluation of the most promising solutions
λevaluations
Evolution (to the next generation)
Inexact Evaluations
L<λ evaluations
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Local Surrogate Models -
Training Set:
x1
x2min max[ , ]T T T∈
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Surrogate Models –
What else do they tell us???
err
errF
Fitness Function ApproximationConfidence IntervalsHessian Matrix ApproximationSensitivity Derivatives (Importance Factors)
Optimal
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Optimization and Multi-processing
CPUs ≠ λ (ή L )
Loading Distribution:Heterogeneous PlatformsLoading per processorVariable evaluation cost
Synchronization (in each generation)
Master: ΕΑ Module - waiting list for evaluationsSlave: discrete (remote) evaluation process
Parallelization Level
Reduction of the Optimization Wall-Clock Time
CFDCFDEAEA
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Distributed
ΕΑ
Why ?L (<< λ) < CPUs
Persistant diversity in populationsStraightforward parallelization
Additional Parameters:Number of islandsCommunication topologyCommunication frequencyMigration algorithmEA parameters per island
?
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Distributed
ΕΑ
on a Multi-Processor System
Evaluations Server
EA EA EA
Migrations
Islands
Asynchronous Requests for Evaluations
MASTER
SLAVE1 SLAVE2 SLAVE3 SLAVE4
Thread 1
Thread 2
Thread 3 Thread
4 Thread
5
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
v1.3
The Evolutionary Algorithms SYstem
Applications
Developed by the National Technical University of Athens, (NTUA), Greece
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Problem: Rastrigin’s
Function
2
1
1
1( ) 20 20 exp( 0.2 )
1exp( cos(2 ))
M
mm
M
mm
F x e xM
xM
π
=
=
= + − −
−
∑
∑
M=30 30 30mx− ≤ ≤
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Results: Rastrigin’s
Function
1
10
100
0 500 1000 1500 2000 2500 3000 3500 4000
Cos
t Fun
ctio
n V
alue
(Ras
trigi
n)
Cost Measured in Exact Evaluations
GADGA
GA-IPE-IFD(GA-IPE-IF)
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Problem: Three-Element Airfoil, Lift Maximization
Initial
Rotation Angle 28.1ο
Δx -0.020
Δy 0.0269
Rotation Angle -37ο
Δx 0.020
Δy 0.0249
0.12, 17.18oM a∞ ∞= =
LF C= −
Initial
Optimal
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Results: Three-Element Airfoil, Lift Maximization
slat
Δy
flap
Initial OptimalRotation Angle 28.1ο 28.02ο
Δx -0.020 -0.03078
Δy 0.0269 0.01982
Rotation Angle -37ο -36.96ο
Δx 0.020 0.02016
Δy 0.0249 0.02469
IFs
Design Variable
Imp
ort
an
ce
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Results: Three-Element Airfoil, Lift Maximization
-5.24
-5.22
-5.2
-5.18
-5.16
-5.14
-5.12
0 500 1000 1500 2000
Cos
t Fun
ctio
n V
alue
(-Li
ft)
Cost Measured in Exact Evaluations
GADGA
GA-IPE-IFD(GA-IPE-IF)
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Problem: 3D Compressor Blade DesignNURBS
ΑκροπτερύγιοΠόδι
ωref =0.09
Evaluations
Hub Section Tip Section
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Problem: Design of a Compressor Cascade
c=0.07ms=0.68c γ=30ο
Minimum Total Pressure LossesConstraint on the minimum (maximum thickness)Desirable Flow turning
1 1 2F P Pω= ⋅ ⋅
max1 max, maxexp( ), 0.9 , thres
thres ref thresthres
t tP t t t t
t−
= = <
2 1 2exp( max(1, )), 0ref
refP
α αα α α
αΔ −Δ
= − Δ = − >Δ
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Results: Design of a Compressor Cascade
0
0.02
0.04
0.06
0.08
0.1
0.12
0 500 1000 1500 2000
Cos
t Fun
ctio
n V
alue
Cost Measured in Exact Evaluations
GADGA
GA-IPE-IFD(GA-IPE-IF)
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Problem: Compressor Multi-Point Design
MISES 2.53
c=0.07ms=0.68c γ=30ο
2, 2
1 1
t is t
t
p pp p
ω−
=−
1 2 3 4 5
α1
43o
45o
47o
49o 52o
Μ1 0.6184
0.6182
0.6180
0.6195
0.6214
Re
8.61E5
8.50E5
8.41E5
8.20E5
7.63E5
1/AVDR
1.0909
1.0965
1.1021
1.1027
1.1032
ω
(exp) 0.0232
-
0.0186
-
0.0417
ω
MISES
0.0234
0.0208
0.0189
0.0237
0.0374
α2
(πexp)
20.79
20.80
20.92
21.69
22.74
α2
MISES
20.80
20.80
20.90
21.70
22.701
3 5
ASME 90-GT-140
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Compressor Multi-Point Design (1 OP –
1 target)(20,2,100)
Evaluations
1 2 3 4 5
ω
L=5
0.0244
0.0182
0.0155
0.0275
-
ω
REF
0.0234
0.0208
0.0189
0.0237
0.0374
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Compressor Multi-Point Design (3 OP –
2 targets)
1 3 1 2F P Pω= ⋅ ⋅
2 1 5 3 1 2( 2 ) 0F P Pω ω ω= + − ⋅ ⋅ >
Evaluations
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Compressor Multi-Point Design (5 OP –
3 targets)
Evaluations
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Compressor Multi-Point Design (5 OP –
5 targets)
1 2, 1,..., 5i iF P P iω= ⋅ ⋅ =
Evaluations
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Problem: High-Lift, Low-Drag Optimization
61 : 0.20, 10.8 , Re 5 10oC M a∞ ∞ ∞= = = ×
NS, k-ε
WF 6000
3000
200
62 : 0.77, 1.0 , Re 10 10oC M a∞ ∞ ∞= = = ×
4min
F1
F2
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Results: High-Lift, Low-Drag Optimization
500 evaluations
C1 C2
1500
evaluations 2500 evaluations
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece
Optimization of Combined Cycle GT Power Plants
ST ST
C T1
2
3
4
5
6
7
1
3
2
7
45
6
HP
LP
G1
G28
9
8
WATER TANK
Natural gas fired, dual-pressure CCGTPP configuration
GT: 260 MWe, 38% efficiency, exhaust gas mass flow 615 kg/sec at 600C.
Design variablesHP steam pressureLP steam pressuresuperheated HP steam temperaturefeedwater temperature at the inlet to the HP evaporator feedwater temperature at the outlet from the first HP economizerfeedwater temperature at the inlet to the LP evaporator superheated LP steam temperaturesteam pressure fed to the water tank exhaust gas mass flow ratio (percentage of mass flowratetraversing the LP economizer) exhaust gas temperature at the HRSG outlet steam extraction pressure from the LP steam turbine exhaust gas temperature at the inlet to the condensate preheater
______________________________________________
Introductory Course to Design Optimization
K.C. Giannakoglou, Associate Professor NTUA, Greece