Miroslav KrsticUniversity of California, San Diego
Extremum Seeking Feedback Tools forReal-Time Optimization
Large Scale Robust Optimization, Sandia, August 2005
1
Example of Single-Parameter Maximum Seeking
f (θ) = f * + f ''2
θ −θ*( )2
asinωt sinωt
ks
ss + h+ ×
y
ξ
θ
θ̂
f * θ* Plant
2
Example of Single-Parameter Maximum Seeking
3
Theory
•History
•Single parameter ES, how it works, and stability analysis byaveraging
•ES with plant dynamics and compensators for performanceimprovement
• Internal model principle for tracking parameter changes
•Multi-parameter ES
•Slope seeking
•ES in discrete time
•Limit cycle minimization via ES
4
Applications
•Anti-skid braking
•Compressor instabilities in jet engines
•Combustion instabilities
•Flow separation control in diffusers
•Thermoacoustic coolers
•Automotive engine mapping
•Beam matching in particle accelerators
•Formation flight
•Bioreactors
•PID tuning
•Autonomous vehicles without position sensing
5
History• Leblanc (1922) - electric railways
• Early Russian literature (1940’s) - many papers
• Drapper and Li (1951) - application to IC engine spark timing tuning
• Tsien (1954) - a chapter in his book on Engineering Cybernetics
• Feldbaum (1959) - book Computers in Automatic Control Systems
• Blackman (1962 chap. in book by Westcott) - nice intuitive presentation of ES
• Wilde (1964) - a book
• Chinaev (1969) - a handbook on self-tuning systems
• Papers by[Morosanov], [Ostrovskii], [Pervozvanskii], [Kazakevich], [Frey, Deem,and Altpeter], [Jacobs and Shering], [Korovin and Utkin] - late 50s - early 70’s
• Meerkov (1967, 1968) - papers with averaging analysis
• Sternby (1980) - survey
• Astrom and Wittenmark (1995 book) - rates ES as one of the most promisingareas for adaptive control
6
Recent Developments
• Krstic and Wang (2000, Automatica) - stability proof for single-parametergeneral dynamic nonlinear plants
• Choi, Ariyur, Wang, Krstic - discrete-time, limit cycle minimization, IMC forparameter tracking, etc.
• Rotea; Walsh; Ariyur - multi-parameter ES
• Ariyur - slope seeking
• Tan, Nesic, Mareels (2005) - semi-global stability of ES
• Other approaches: Guay, Dochain, Titica, and coworkers; Zak, Ozguner, andcoworkers; Banavar, Chichka, Speyer; Popovic, Teel; etc.
• Applications outside of my group:o Electromechanical valve actuator (Peterson and Stephanopoulou)o Artificial heart (Antaki and Paden)o Exercise machine (Zhang and Dawson)o Chemical reactors and petroleum processing (several research groups)o Shape optimization for magnetic head in hard disk drives (UCSD)
7
ES Book
8
Basic Extremum Seeking - Static Map
f (θ) = f * + f ''2
θ −θ*( )2
asinωt sinωt
−ks
ss + h+ ×
y
ξ
θ
θ̂
f * θ* Plant
y = output to be minimizedf * = minimum of the mapf " = second derivative (positive - f (θ) has a min.)θ* = unknown parameter
θ̂ = estimate of θ*
k = adaptation gain (positive) of the integrator 1s
a = amplitude of the probing signalω = frequency of the probing signal
h = cut-off frequency of the "washout filter" ss + h
+/× = modulation/demodulation
9
How Does It Work?
f (θ) = f * + f ''2
θ −θ*( )2
asinωt sinωt
−ks
ss + h+ ×
y
ξ
θ
θ̂
f * θ* Plant
Estimation error: θ = θ* − θ̂
y = f * + a
2 f "4
+f "2θ 2 − af " θ sinωt + a
2 f "4cos2ωt
y ≈ f * + a
2 f "4
+f "2θ 2 − af " θ sinωt + a
2 f "4cos2ωt
Loc. Analysis - neglect quadratic terms:
10
How Does It Work?
f (θ) = f * + f ''2
θ −θ*( )2
asinωt sinωt
−ks
ss + h+ ×
y
ξ
θ
θ̂
f * θ* Plant
y ≈ f * + a
2 f "4
− af " θ sinωt + a2 f "4cos2ωt
ss + h
[y] ≈ f * + a2 f "4
− af " θ sinωt + a2 f "4cos2ωt
11
How Does It Work?
f (θ) = f * + f ''2
θ −θ*( )2
asinωt sinωt
−ks
ss + h+ ×
y
ξ
θ
θ̂
f * θ* Plant
ξ = sinωt s
s + h[y] ≈ −af " θ sin2ωt + a
2 f "4cos2ωt sinωt
Demodulation:
ξ ≈ −
a2 f "4θ +
a2 f "4θ cos2ωt + a
2 f "8
sinωt − sin 3ωt( )
12
How Does It Work?
f (θ) = f * + f ''2
θ −θ*( )2
asinωt sinωt
−ks
ss + h+ ×
y
ξ
θ
θ̂
f * θ* Plant
θ = − ̂θthen
Since θ = θ* − θ̂
13
How Does It Work?
f (θ) = f * + f ''2
θ −θ*( )2
asinωt sinωt
−ks
ss + h+ ×
y
ξ
θ
θ̂
f * θ* Plant
θ ≈ks
−a2 f "4θ +
a2 f "4θ cos2ωt + a
2 f "8
sinωt − sin 3ωt( )⎡
⎣⎢
⎤
⎦⎥
high frequency terms - attenuated by integrator
14
How Does It Work?
f (θ) = f * + f ''2
θ −θ*( )2
asinωt sinωt
−ks
ss + h+ ×
y
ξ
θ
θ̂
f * θ* Plant
θ ≈ −
ka2 f "4θ
Stable because k,a, f " > 0
15
Stability Proof by Averaging
f (θ) = f * + f ''2
θ −θ*( )2
asinωt sinωt
−ks
ss + h+ ×
y
ξ
θ
θ̂
f * θ* Plant
θ = θ* − θ̂
e = f * − hs + h
y[ ]τ =ωt
ddτθ =
kω
f "2θ − asinτ( )2 − e⎛
⎝⎜⎞⎠⎟sinτ
ddτ
e = hω
−e − f "2θ − asinτ( )2⎛
⎝⎜⎞⎠⎟
Full nonlinear time-varying model:
16
Stability Proof by Averaging
f (θ) = f * + f ''2
θ −θ*( )2
asinωt sinωt
−ks
ss + h+ ×
y
ξ
θ
θ̂
f * θ* Plant
θ = θ* − θ̂
e = f * − hs + h
y[ ]τ =ωt
ddτθav = −
kaf "2ωθav
ddτ
eav =hω
−eav −f "2θ 2av +
a2
2⎛⎝⎜
⎞⎠⎟
⎛
⎝⎜⎞
⎠⎟
Average system:
θav = 0
eav = −a2 f "4
Average equilibrium:
17
Stability Proof by Averaging
f (θ) = f * + f ''2
θ −θ*( )2
asinωt sinωt
−ks
ss + h+ ×
y
ξ
θ
θ̂
f * θ* Plant
θ = θ* − θ̂
e = f * − hs + h
y[ ]τ =ωt
Jav =−kaf "2ω
0
0 −hω
⎡
⎣
⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥
Jacobian of the average system:
18
Stability Proof by Averaging
f (θ) = f * + f ''2
θ −θ*( )2
asinωt sinωt
−ks
ss + h+ ×
y
ξ
θ
θ̂
f * θ* Plant
θ = θ* − θ̂
e = f * − hs + h
y[ ]τ =ωt
θ2π /ω (t) + e2π /ω (t) −a2 f "4
≤O 1ω
⎛⎝⎜
⎞⎠⎟,→→∀t ≥ 0
Theorem. For sufficiently large ω there exists a unique exponentially stableperiodic solution of period 2π/ω and it satisfies
Speed of convergence proportional to 1/ω, a2, k, f "
19
Stability Proof by Averaging
f (θ) = f * + f ''2
θ −θ*( )2
asinωt sinωt
−ks
ss + h+ ×
y
ξ
θ
θ̂
f * θ* Plant
θ = θ* − θ̂
e = f * − hs + h
y[ ]τ =ωt
y − f * → f "O 1ω 2 + a
2⎛⎝⎜
⎞⎠⎟
Output performance:
Extremum-Seeking Control of Combustion Instabilities
Andrzej Banaszuk
United Technologies Research Center, E. Hartford, CT, U.S.A.
Acknowledgements:
Youping Zhang, Numerical Technologies
Kartik Ariyur, Honeywell
Miroslav Krstic, UCSD
Jeff Cohen, Pratt & Whitney
Clas Jacobson, UTRC
Workshop on Real-Time Optimization by Extremum-Seeking Control, ACC 05
Partially sponsored by AFOSR
Thermo-acoustic instability
Fluctuating acoustic pressure and velocity driven by unsteady heat release
Acoustics
Combustion
Coupling of acoustics with heat release results in pressure oscillations
Fuel
CombustorFuel/air mixing nozzle
Air
Fluctuating heat release driven by fluctuating velocity
•Power generation: pressure oscillations prevent low emission operation
•Military aeroengines: afterburner screech and rumble limit performance, passive fixes increase weight, maintenance costs
• Rockets: passive fixes increase weight, limit payload
Thermo-acoustic instabilities affect cost and performance of gas turbines and rocket engines
Fuel/Air Ratio
01020304050N
Ox 60
708090
100
0.400.420.440.460.480.500.520.540.560.580.60
NOx
CombustionInstability
25MW Pratt & Whitney FT8 gas turbine engine
Active fuel control can suppress oscillations
Fuel
CombustorFuel/air mixing nozzle
Air Acoustics
Combustion
Control
Fuel valve
Controller
Pressure measurement
0 200 400 600 800 10000
2
4
6
8
10
12
14
Am
plit
ude
(psi
)
Uncontrolled
Controlled
6X reduction in amplitudeDecrease in NOX & CO emissions
Experiment in UTRC 4MW Single Nozzle Rig
• Numerous demonstrations at universities and industrial rigs
• Rolls-Royce demonstrated control in afterburner
• Siemens implemented control in 260MW gas turbine engine
• Phase-shifting control effective, but no models to guide selection of parameters
Summary of results: two extremum seeking schemes were successfully demonstrated on UTRC 4MW single nozzle rig in August 1998.
Excellent performance at high power Improved performance at low power
0 5 10 15 20 25 30−150
−100
−50
0
50
file r54p18 ,Date & Time of Capture: Thu Aug 13 11:57:35 1998
contr
ol ph
ase
0 5 10 15 20 25 3001234567
Performance measure 1 = 8.0355, Performance measure 2 = 20.6
magn
itude
of bu
lk mo
de
0 5 10 15 20 25 30
−50
0
50
100
file r56p37 ,Date & Time of Capture: Wed Aug 19 12:00:34 1998
contr
ol ph
ase
0 5 10 15 20 25 300
2
4
6
8
10
Performance measure 1 = 27.9418, Performance measure 2 = 79.3275
magn
itude
of bu
lk mo
de
−200 −150 −100 −50 0 50 100 150 2001
2
3
4
5
6
7
control phase
pres
sure
and i
ts ma
gnitu
de
file r54p18 ,Date & Time of Capture: Thu Aug 13 11:57:35 1998
−200 −150 −100 −50 0 50 100 150 2004
5
6
7
8
9
10
11
control phase
pres
sure
and i
ts ma
gnitu
de
file r56p37 ,Date & Time of Capture: Wed Aug 19 12:00:34 1998
Control phase in degrees
Optimal level
Uncontrolled level
Optimal level
Phase-magnitude map
Uncontrolled level
Optimal phase
Time in seconds Time in seconds
Control phase in degrees Control phase in degrees
Pressure magnitude in psi
Pressure magnitude in psi
1
Formation Flight Optimization
(Acknowledgement: Paolo Binetti)
2
Benefits of Formation Flight
• Reduction in power demand
• Alleviation of airspace congestion
Obstacles to Attainment
• Sensitivity to positioning
• Difficulty in precise measurements
• Absence of precise modeling
6
Physics of Aircraft Wakes and Wake Modeling
• Wake structure—modeled as two NASA counter-rotating vortices
• Wake transport—a modeling uncertainty
• Wake decay—neglected
( ) ( ) bbbW
rrr
rrV red
redc 4! ,
"V ,
!2 22
2
# ==Γ+
Γ=∞
5
Selection of the Problem
Why the C-5?
• Large savings in fuel consumption
• Representative of large transports
• They will be in service for the next 40 years
• Availability of wake data
13
Extremum Seeking for Formation Flightz!
Aircraft with formation autopilot
in the wake
sk 1−
1hss+
sk 2−
2hss+
zref
yref+
Estimator
Objective:-
( )ta 11 "sin ( )11"sin ϕ−t
( )22"sin ϕ−t( )ta 22 "sin
w!
18
Simulation for brief CAT Encounter …
Extremum Seeking Control of Thermoacoustic Coolers
Mario RoteaJune 7, 2005
Purdue [email protected]
REF. Extremum Seeking Control of Tunable Thermo-acoustic Coolers, Y. Li, M. Rotea, G. Chiu, L. Mongeau, I. Paek, IEEE Transactions on Control Systems Technology, Vol. 13, No. 4, July 2005
Thermoacoustic CoolingElectric energy Acoustic energy Heat pumping
Standing sound wave creates the refrigeration cycle
Resonance tubeStackHot-end heat
exchangers
Electro-dynamic driver
Cold-end heat exchangers
Pressurized He-Ar mixture
32
Heat Pumping
1-2: adiabatic compression and displacement2-3: isobaric heat transfer (gas to solid)3-4: adiabatic expansion and displacement4-1: isobaric heat transfer (solid to gas)
Solid surface (stack plate)
Gas particle in a standing wave
1
4
QLQH
Thermoacoustic Cooling• Benefits
– Environmentally benign (inert gases)– Simple, easy to maintain configuration
• Limitations– Very hard to model ! hard to tune key parameters (driver, stack
location, duct geometry) for best performance• Existing prototypes
– Acoustic Stirling Heat Engine (Los Alamos National Lab)– Triton 10 kW refrigerator (Penn State)– Space Thermo-acoustic Refrigerator (NASA)– Purdue 300 W standing wave unit– Miniature thermo-acoustic cooler (Rockwell Science Center)
Tunable Thermoacoustic Cooler
Moving piston (varying resonator’s stiffness)
Heat exchangers
Helmholtz resonator
Neck (mass) Volume (stiffness)
Electro-dynamicDriver
Performance and efficiency sensitive to variations in – Piston position (acoustic impedance)– Driver frequency
Tuning Variables
Extremum Seeking Controller
PDIntegratorLPF + +sin( )x x xa t! "#sin( )xt!
PDIntegratorLPF + +
sin( )f f fa t! "#sin( )f t!
HPF
11fT s #Tunable
Cooler
CoolingPower
Calculation 11xT s #
POS Command
FREQ Command
cQ!
ESC Parameters– High pass filter (HPF)– Low pass filters (LPF)– Dither frequencies, amplitudes
and phases (!x,ax,"x; !x,ax,"x)– PD gains (4 gains)
(Tuning mechanism)
Experiment – Varying Operating Condition
0 50 100 150 200 250 300 350 4000
50
100
Coo
ling
Pow
er (W
att)
0 50 100 150 200 250 300 350 4004
6
8
10
12
Pis
ton
Pos
ition
(in)
0 50 100 150 200 250 300 350 400140
145
150
155
Driv
ing
Freq
uenc
y (H
z)
0 50 100 150 200 250 300 350 40020
40
60
80
100
Time (sec)
Flow
Rat
e (m
l/s)
Cold SideHot Side
ESC ON
Flow Rate Change
ESC tracks optimum after cold-side flow rate is increased
Extremum-Seeking Control of Flow Separation in a Planar Diffuser
Andrzej Banaszuk
United Technologies Research Center, E. Hartford, CT, U.S.A.
Acknowledgements:
Satish Narayanan, UTRC: experiment
Youping Zhang, Numerical Technologies: algorithm
Workshop on Real-Time Optimization by Extremum-Seeking Control, ACC 05
Partially sponsored by AFOSR
Objective of pressure recovery control
Control effortPerformance
221
)(inlet
U
pptC inletexit
p !"
#HU
hutC
inlet
neck
2
2
)( #$
H
h
inletp exitp
neckuinlet
U
$C
Cp bang
buck=
Speaker command
Experimental Setup
% & 14o% & 18o
Pressure recovery as function of diffuser angle (no control)
%
no stall
•Optimum uncontrolled performance•Insignificant improvement with control
fully-developed2D stall jet flow
unsteady stall
•Poor uncontrolled performance•Significant improvement with control
)(tCpUTRC diffuser rig•fully turbulent BL•40,000 < ReH < 140,000•Re%e > 300; M < 0.1•Actuation: C$ ~ 0.001
Need: control algorithm to optimize performance
-150 -100 -50 0 50 100 15010
15
20
25
30
35
40
45
50
0.05
0.05
0.1
0.1
0.1
0.15
0.15
0.2
Frequency f
Phase !
+
-150 -100 -50 0 50 100 15010
15
20
25
30
35
40
45
50
-0.05
0
0
0
0.05
0.05
0.05
0.1
0.1
0.15
Frequency f
Phase !
+
Optimal operating point: f=31Hz, ! =60, Cp =0.21 Optimal operating point: f=36Hz, ! =60, Cp =0.16
U = 30m/sU = 20m/s
Objective:•Optimize performance without exhaustive search
Challenges:•Noisy measurement•Flow transients•Keeping up with operating condition change
Two frequency control law: U(t)=A1*(sin(2*"*f*t) + sin(2*"*2f*t-!))
Adaptive control used to optimize performance
Speaker command
Pressure recoverycalculation
Pressure recoverycalculation
Multi-frequency forcing
Multi-frequency forcing
Extremum-seeking algorithm
Extremum-seeking algorithm
Adjustable parameters )2sin()(1
ii
N
ii tfAtU !" #$%
$NifA iii ..1,,, $!
)(tCp
Filter noise, wait for transient to settle, adapt parameters to improve performance
Excite multiple vortices, explore their interactions
Measure performance
consttC $)(&
Automatic Control Parameter Tuning to Optimum Values
On-line optimization of pressure recovery using extremum-seeking algorithm demonstrated.
0 10 20 30 40 50 600
0.2
0.4
0 10 20 30 40 50 600
20
40
60
Frequency
0 10 20 30 40 50 60-200
0
200
Control phase
Time in seconds
•Mean pressure recovery, control frequency, and phase in four independent adaptive control experiments. •The control frequency and phase initialized away from the optimal values.
Transient behaviorOptimal pressure recovery reached & held
-150 -100 -50 0 50 100 15010
15
20
25
30
35
40
45
50
0.05
0.05
0.1
0.1
0.1
0.15
0.15
0.2
Phase
Fre
quen
cy
Optimal pressure recovery:Cp =0.21
optimal control parametersf=31Hz! =60
Dynamics determines transient time!
Pressure recovery
Autonomous Vehicle Controlvia Extremum Seeking
Antranik A. Siranosian and Miroslav Krstic
2
Model
ω ,v
Unicycle Dynamics
x = vcos(θ)y = vsin(θ)θ =ω
Unicycle Inputs
peak) itsat output (from leUnobservab
and ) inputs (from blencontrollalinearly U is System
f(x,y)
v,w
Unicycle
3
Control InputsUnicycle
4
Block DiagramUnicycle
5
Simulation Results – TrajectoryUnicycle
6
Simulation Results – ES ValueUnicycle
BEAM MATCHING IN PARTICLE ACCELERATORS
Eugenio Schuster
Complex Control Systems LaboratoryMechanical Engineering and Mechanics
Workshop: Real Time Optimization by Extremum Seeking ControlAmerican Control Conference 2005
June 7, 2005
LEHIGHU N I V E R S I T Y
PARTICLE ACCELERATORS
The Spallation Neutron Source (SNS), being built in Oak Ridge, Tennessee, by the U.S. Department of Energy with a cost of $1.4 billion, is the most intense accelerator-based neutron source in the world.
SPALLATION NEUTRON SOURCE
BEAM MATCHING CHANNEL
! "z#
z
1$#3$#
2$#
4$#
qL dL
L
! "
! " 02
02
3
2
3
2
%&'
&'((
%&'
&&((
YYXKYzY
XYXKXzX
Y
X
)#
)#Drift Lens Drift Lens Drift DriftLens Lens Drift
z
1# 2# 3# 4#
0 L
inix finx
BEAM MATCHING OPTIMIZATION
332211 JkJkJkJ ''%
****
+
,
----
.
/
(
(%
****
+
,
----
.
/
(
(%
****
+
,
----
.
/
(
(%
tar
tar
tar
tar
tar
fin
fin
fin
fin
fin
ini
ini
ini
ini
ini
YYXX
x
YYXX
x
YYXX
x ,,Drift Lens Drift Lens Drift DriftLens Lens Drift
z
1# 2# 3# 4#
0 L
inix finx
! " ! "! " ! "! " ! " ! "0 12 &'&%
(&('(&(%
&'&%
L
o desiYdesiX
tarfindYtarfindX
tarfinYtarfinX
dzzYzYKzXzXKzwJ
YYKXXKJ
YYKXXKJ
223
222
221
)()()()(
BEAM MATCHING OPTIMIZATION
! "#J
*# *J
hqq'&1
1&&q3
! "ka 4cos ! "54 &kb cos
67! "k8
Plant
Low-PassFilter
High-PassFilter
! "k# ! "! "kJ #
! "k#̂ ! "kJ f
! " ! " ! " ! "! " ! " ! "
! " ! " ! "! " ! " ! ")1(cos1ˆ1
ˆ1ˆcos
11
'''%'
&%'
&%
&&'&&%
kakk
kkk
kbkJkkJkJkhJkJ
f
ff
4##
38##
548
BEAM MATCHING OPTIMIZATION – 4D
50 100 150 200 250 300 350 400 450 500−55
−50
−45
−40
−35
−30
−25
−20
−15
−10
J [d
B]
Iteration
Cost Function
0 5000 10000 15000−70
−60
−50
−40
−30
−20
−10
0
10
Case Number
J [d
B]
50 100 150 200 250 300 350 400 450 500−50
−40
−30
−20
−10
0
10
20
30
40
50
Iteration
Filte
red θ
Parameter Evolution
θ1θ2θ3θ4
0 0.2 0.4 0.6 0.80
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5 x 10−3 Beam profile for estimated θ
z
X Y XtarYtar
., k, kk, KK,, K, K, KK iYiXdYdXYX
0110 1110002000
321 !!!!!!!!!
""""
#
$
%%%%
&
'(
!
011034.0003289.0006730.0
001070.0
finx
""""
#
$
%%%%
&
'(
!
011726.0003290.0007864.0
001094.0
tarx
TERMINAL CONSTRAINTS ONLY
) *) *) *) *) * dB - if J ....a
dB -J dB if -....adB -J dB if -....a
dB -J dB if -....aJdB if -...a
,,,
,,,
,,,
,,,
,,,
60025002500100105560050050050050
505510102501040505050750250
40517512252
4321
4321
4321
4321
4321
+!
+,!
+,!
+,!
,!
PID Tuning using ExtremumSeeking
Extremum Seeking Tuning Scheme
Stepfunction
!-
"kExtremum
Seeking Algorithm
# $krC "
# $kyC "
Gy(t)!r(t) u(t)
!"#$%#&"&'()%*+
J("k)
,%'-.+$+()%*+
)201)(101(
51)(4 ss
ssG
!!"
#
c) Step Response of output
b) Evolution of PID Parameters
d) Step Response of controller
a) Evolution of Cost Function