AFOSR Workshop, Dayton, OH, August 4-6, 1999 1 M. G. Safonov Robust Control, Feedback and Learning Michael G. Safonov University of Southern California
AFOSR Workshop, Dayton, OH, August 4-6, 1999 1 M. G. Safonov
Robust Control, Feedback and
LearningMichael G. Safonov
University of Southern California
AFOSR Workshop, Dayton, OH, August 4-6, 1999 2 M. G. Safonov
The Problem• Control theory relies too much on models:
– Step 1: System ID to obtain model– Step 2: Design controller for model
• System ID relies too much on assumptions:– Assume noise probabilities & model structure– Optimize model fit to data & assumptions
• Models & assumptions can be wrong• Need theory that responds appropriately
when observed data falsifies prior belief
AFOSR Workshop, Dayton, OH, August 4-6, 1999 3 M. G. Safonov
The Approach
• Let the data speak...
• Unfalsify (validate) models and controllers against hard criteria:– Choose criteria expressible directly in terms of
observed data (sensor outputs, actuator inputs)– Avoid criteria that that rely on “noise model”
and other prior beliefs
AFOSR Workshop, Dayton, OH, August 4-6, 1999 4 M. G. Safonov
UnfalsificationAlgorithm
Brugarolas & Safonov, CCA/CACSD ‘99
UNFALSIFIED CONTROL:
The ability of each candidate controller to meet the performance goal is treated as a hypothesis to be tested directly against evolving real-time measurement data. The controller need not be in the loop to test the hypothesis.
UnfalsifiedHypotheses
D-H Test
FalsificationTest
pass
Goal
Data
Hypotheses
pass
Tests: D-H Test: Tests data-
hypothesis consistency(Def . 3).
Falsification Test(Thm. 1).
fail
fail
Choosebest
opt
FalsifiedHypotheses
AFOSR Workshop, Dayton, OH, August 4-6, 1999 5 M. G. Safonov
Plato introspective learning
Data approximates Unobserved TRUTH
Reality is an ideal, observable only through noisy sensors.
‘Probabilistic Estimation’
Two views on how we learn: Introspection vs. Observation
vs.
Galileo open-eyed learning
MODELS approximate Observed Data
Reality is what we observe.
‘Curve-Fitting’
AFOSR Workshop, Dayton, OH, August 4-6, 1999 6 M. G. Safonov
• Traditional control theory (‘Platonic’):– contains many assumptions about the plant.– some assumptions are unrealistic.
• Unfalsified control theory (‘Galilean’):– eliminates hypotheses that are not consistent
with evolving experiment data.
AFOSR Workshop, Dayton, OH, August 4-6, 1999 7 M. G. Safonov
Impact on SYSID
• Statisticians treat prior probabilities as part of model to be validated/unfalsified – validation via one- ‘confidence intervals’– ‘Platonic’ probability becomes ‘Galilean’
• Ljung (e.g., CDC ’97) proposed ‘confidence interval’ reinterpretation of SYSID– model validation/unfalsification
AFOSR Workshop, Dayton, OH, August 4-6, 1999 8 M. G. Safonov
Two Views of SYSID: Curve-Fitting vs. Estimation
CURVE FIT: (Galilean)Given data (yi,ui), i=1,2,...
iii bauy
21||||min
(y1,u1)
(y2,u2)
(y3,u3)
(y4,u4)
(y5,u5)
(y6,u6)
(y7,u7)
height 2 {
vbauy
BAYESIAN ESTIMATE (Platonic): Given data (yi,ui), i=1,2,...
‘noise’ v=N(0, )
), ba(y |x,max probsubject to prior beliefs
AFOSR Workshop, Dayton, OH, August 4-6, 1999 9 M. G. Safonov
CURVE FIT (Galilean):The Galilean may reject model if 2/3 of future data fails to fall in his prior 2 confidence bound
If the fit is bad, do you reject the data or the model?
(y1,u1)
(y2,u2)
(y3,u3)
(y4,u4)
(y5,u5)
(y6,u6)
(y7,u7)
height 2 {
BAYESIAN ESTIMATE (Platonic): The Platonist may reject data if 2/3 of future data fails to fall in his prior 2 confidence bound
AFOSR Workshop, Dayton, OH, August 4-6, 1999 10 M. G. Safonov
candidate controllers
FALSIFIED
COMPUTER SIEVE
LEARNING FEEDBACK LOOPS
K
UnfalsifiedControllers
K
evolving I/O datagiven
M. G. Safonov. In Control Using Logic-Based Switching, Spring-Verlag, 1996.
How about Validation-Based Direct Controller ID?
AFOSR Workshop, Dayton, OH, August 4-6, 1999 11 M. G. Safonov
Canonical Representation
Signals:
• manifest
control signal
measurement signal
• latent
reference signal
performance signal
controller signals
controller parameters
EXTENDED PLANT
PLANTACTUATOR SENSOR
dA dP n
UnfalsifiedController
u y
yK uK
ControlArchitecture
r
p
LearningProcessor
ControllerK
KK yu
p
r
y
u
,
Brugarolas & Safonov, CCA/CACSD ‘99
AFOSR Workshop, Dayton, OH, August 4-6, 1999 12 M. G. Safonov
Formulation in Truncated Space
• Observations operator maps input-output signals to measurement signals.
• Truncated Space results from applying the observations operator to a signal space.
Basic sets:goalshypothesishypothesesdata
ZPzPJzPZ goals 1)(0
ZPzPKzPZ hypothesis 0)()(
ZPhypotheses 2: Z
ZPyuzPzPZ tsmeasuremenmanifestdata ),(
P
ZPZ
AFOSR Workshop, Dayton, OH, August 4-6, 1999 13 M. G. Safonov
Formulation in Truncated Space, cont.
Problem 1 (Truncated Space Unfalsified Control Problem): Given a performance specification (goal set), a set of candidate controllers (hypotheses), and experimental data then determine the subset of candidate controllers (hypothesis) which is not falsified.
goalhypothesisdata ZZZ )(
AFOSR Workshop, Dayton, OH, August 4-6, 1999 14 M. G. Safonov
Main Results
• Definition (Data-hypothesis consistency): Given a truncated space unfalsified control problem, we say that a hypothesis is consistent with the data if
).(hypothesisZdataZ ZPZP
manifestmanifest
• Theorem 1 (Truncated space unfalsified control): Given truncated space unfalsified control problem, then a candidate control law (hypothesis) consistent with the experimental data is unfalsified by data if and only if
.)( goalhypothesisdata ZZZ
AFOSR Workshop, Dayton, OH, August 4-6, 1999 15 M. G. Safonov
—————
Jun & Safonov, CCA/CACSD ‘99
Example: Adaptive PID
dttff
ruwyrw2
0
2
222
2
2
1
)( where
*)(*
Performance Spec:
AFOSR Workshop, Dayton, OH, August 4-6, 1999 16 M. G. Safonov
Adaptation Algorithm• Procedure (at each time t = kt) :
1. Measure u(kt) and y(kt).
2. For each • calculate and
• calculate
• if then delete the controller index element i from I; else continue.
3. If the set I is empty, terminate; else set the current to
and increment time.
))1((~ tkxi )(~ tkri
),(~
tkiJ ),(
~tkiJ
} ),,(~
{min arg )(ˆ ,)(ˆ I itkiJki
kiK
Ii
AFOSR Workshop, Dayton, OH, August 4-6, 1999 17 M. G. Safonov
Controller Parameter Adaptation
• The i-th candidate PID controller Ki is unfalsified if and only if
where],,0[ ,0),(~ ttiJ
0 ))(),(),(~(),(~
0
t
ispecdxxuxyxrtiJ T
i for the signal reference fictitious :)(~dataplant past measured :)(),(
Ktr
tytu
i
AFOSR Workshop, Dayton, OH, August 4-6, 1999 18 M. G. Safonov
Simulation
Jun & Safonov, CCA/CACSD ‘99
AFOSR Workshop, Dayton, OH, August 4-6, 1999 19 M. G. Safonov
Simulation
Jun & Safonov, CCA/CACSD ‘99
control input u(t)
proportional gain kP (t)
integral gain kI (t)
derivative gain kD (t),
plant output y(t)
Evolution of unfalsified set Time responses
Tsao & Safonov, IEEE Trans, AC-42, 1997.
AFOSR Workshop, Dayton, OH, August 4-6, 1999 20 M. G. Safonov
Example: Missile Autopilot• Learns control gains
• Adapts quickly to compensate for damage & failures
• Superior performance
Brugarolas, Fromion & Safonov, ACC ’98
Specified target response bound
Actual response
Commanded response
Brugarolas, Fromion & Safonov, ACC ’98
Unfalsified adaptive missile autopilot:• discovers stabilizing control gains as it flies, nearly instantaneously• maintains precise sure-footed control
AFOSR Workshop, Dayton, OH, August 4-6, 1999 21 M. G. Safonov
Robot Example:
Tsao & Safonov, CCA/CACSC’99Tsao & Safonov, CCA/CACSD ’99
AFOSR Workshop, Dayton, OH, August 4-6, 1999 22 M. G. Safonov
Other People’s Successes with Unfalsified Control
• Emmanuel Collins et al. (Weigh Belt Feeder adaptive PID tuning, CDC99)
• Kosut (Semiconductor Mfg. Process run-to-run tuning, CDC98)
• Woodley, How & Kosut (ECP Torsional disk control, adaptive tuning, ACC99
• … maybe others soon?
AFOSR Workshop, Dayton, OH, August 4-6, 1999 23 M. G. Safonov
• Unfalsified Control Learning/Adaptation• Adaptation = Direct Controller ID• Unfalsified control focuses on the knowable
– ‘Galilean’ emphasis on observation – precise information in each datum– real-time learning (the essence of feedback)
• Practical, reliable adaptive feedback control• Greater tolerance of evolving uncertainties, failures &
battle damage
goalhypothesisdata ZZZ )(
Conclusions
AFOSR Workshop, Dayton, OH, August 4-6, 1999 24 M. G. Safonov
Acknowledgment
Bob Kosut’s mid-1980’s work on time-domain model validation and identification for control played a key role in laying the foundations of this work, as did later contributions of Jim Krause, Pramod Khargonekar, Carl Nett, Kamashwar Poolla, Roy Smith and many others who have advanced the use of validation methods in control-oriented identification. Tom Mitchell’s early 1980’s “candidate elimination algorithm’’ for machine learning is closely related to the unfalsified control methods presented here. And of course, none of this would have been possible without the superb graduate education that I received at MIT so many years ago under the guidance of first Jan Willems and later Michael Athans.
AFOSR Workshop, Dayton, OH, August 4-6, 1999 25 M. G. Safonov
References[1]M. G. SafonovandT. C. Tsao. Theunfalsi¯edcontrol concept and learning.
IEEE Trans. Autom. Control, 42(6):843{847, J une1997.
[2]M. G. Safonov. Focusing on the knowable: Controller invalidation andlearning. In A. S. Morse, editor, Control Using Logic-Based Switching,pages224{233. Springer-Verlag, Berlin, 1996.
[3]M. J un and M. G. Safonov. Automatic PID tuning: An application ofunfalsi¯ed control. In Proc. IEEE CCA/ CACSD, KohalaCoast{Island ofHawaii, HI, August 22-27, 1999(to appear).
[4]P. B. Brugarolas and M. G. Safonov. A canonical representation for unfal-si¯ed control in truncated spaces. In Proc. IEEE CCA/ CACSD, KohalaCoast{Island of Hawaii, HI, August 22-27, 1999(toappear).
[5]T.-C. Tsaoand M. G. Safonov. Unfalsi¯ed direct adpativecontrol of a two-link robot arm. In Proc. IEEE CCA/ CACSD, Kohala Coast{Island ofHawaii, HI, August 22-27, 1999(to appear).
Also, see webpage http://routh.usc.edu