PROSPECTS OF GRADIENT METHODS FOR NONLINEAR CONTROL Ivo BUKOVSKÝ 1 Jiří BÍLA 1 Noriasu HOMMA 2 Ricardo RODRIGUEZ 1 1 Czech Technical University in Prague 2 Tohoku University, Japan
Dec 19, 2015
PROSPECTS OF GRADIENT METHODS FOR NONLINEAR CONTROL
Ivo BUKOVSKÝ1
Jiří BÍLA1
Noriasu HOMMA2
Ricardo RODRIGUEZ1
1Czech Technical University in Prague
2Tohoku University, Japan
PROSPECTS OF GRADIENT METHODS FOR NONLINEAR CONTROL
• We consider sample-by-sample adaptation of discrete-time models and controllers by gradient descent
2( )( 1) ( ) ;
... adaptable parameter of a model or controler
kk ki i
i
thi
Qw w i
w
w i
weight update system
• Stability monitoring and maintenance of weight update system of adaptively tuned models and controllers significantly contributes to a stable and convergent control loop
PROSPECTS OF GRADIENT METHODS FOR NONLINEAR CONTROL
PROSPECTS OF GRADIENT METHODS FOR NONLINEAR CONTROL
• In the paper, we introduce derivation of stability condition for gradient-descent tuned models and controllers
• The approach is valid for models and controllers that are nonlinear (incl. linear), but they are linear in parameters– Not suitable for conventional neural networks (MLP,
RBF)– Suitable for Higher-Order Neural Units (HONU, also
known as polynomial neural networks) (not limited to)
PROSPECTS OF GRADIENT METHODS FOR NONLINEAR CONTROL
Further in this presentation
• Fundamental gradient descent schemes for adaptive identification and control
• Static or dynamic Higher Order Neural Units (HONU)
• Stability conditions for static and dynamic HONU and its maintenance at every adaptation step
• Demonstration of achievements with ONU( NOx prediction – EME I, lung motion prediction, nonlinear control loop of a laboratory system)
Plant
Adaptive model-linear
- neural network,
+-
2( )( 1) ( ) ;
kk ki i
i
ew w i
w
( )ku( )krealy
Fundamental gradient descent schemes for adaptive identification and control
Plant Identification by Gradient Descent
( )ke
( )ky
... neural weights
(adaptable parameter)
... control variableu
w
weight update system
(Základní schemata adaptivní identifikace a řízení gradientovými metodami)
Automatické ladění adaptivního stavového regulátoru
Regulovanásoustava
Adaptivní regulátor-lineární
- polynomiální-- klasická neuronová síť
2( )( 1) ( ) ;
kk ki i
i
ew w i
w
( )kv ( )krealy
Referenční model (požadované chování regulované soustavy)
+-
Žádaná hodnota
+-
( )kdesiredy
... adaptovatelný parametr
(váhy u neuronových sítí)
... žádaná hodnota
w
v
Systém adaptovaných
vah
( )ke
Žádaný průběh chování
Fundamental gradient descent schemes for adaptive identification and control (continue)
Tuning of Adaptive Controller in a Feedback Control Loop with Gradient Descent
Plantadaptive controller
- linear PID - neural network,
2( )( 1) ( ) ;
kk ki i
i
ew w i
w
( )kv
( )krealy
Model of desired behavior
+-
( )kdesiredy
... neural weight ,
(adaptable parameter)
... desired value
w
v
+-
2( )( 1) ( ) ;
kk ki i
i
ew w i
w
Plant
Adaptive model-linear
- neural network,
+-
( )ku( )krealy
Fundamental gradient descent schemes for adaptive identification and control (continue)
Updating Control Inputs Directly by Gradient Descent
( )ke
( )ky
2( )( 1) ( ) c
kck ki
eu u
w
+-
( )kdesiredy
eC(k)
The question is:
• How do we assure stability of nonlinear adaptive control loop?• The ways is to assure stability and convergence of adaptive
components in a control loop (plant model + controller)• What nonlinear model to use?
• MLP or RBF networks as models and controllers– Not linear in parameters– Guaranteeing stability is complicated (not
suitable for undergraduate level, difficult for PhD students from non-heavy-math schools)
– Guaranteeing stability is complicated and theoretically heavy for practicioners (thus not attractive for practice)
Static & Dynamic Higher-Order Neural Units
How do we assure stability of the nonlinear adaptive control loop? What model to choose?
Static & Dynamic Higher-Order Neural Units
How do we assure stability of the nonlinear adaptive control loop? What model to choose?
2( )( 1) ( ) ;
kk ki i
i
ew w i
w
Weight-update system:
Example of 2nd-order HONU: 1
( ...)
( ...)
( )
k
k
k
y
y
u
( )kx
( )sk ny
0
r rn n
i j iji j i
x x w
20 0 0 1 0 2 i j ny x x x x x x x x x 0,0 0,1 0,2 i,j n,nw w w w w
“axis of adapted neural weights”
LNU
HONU
convetional NN
2( )k
k
e
0
Approximation strength of neural networks can be improved by adding more neurons or even layers, GA, PSO,…
Static & Dynamic Higher-Order Neural Units (continue)
Sketch of optimization error surfacesLinear x MLP Networks x HONU
Static & Dynamic Higher-Order Neural Units (continue)
Static MLP vs. QNU as MISO models of hot steam turbine averaged data (“steady states”, batch training by Levenberg-Marquardt)
• double hidden layer FFNN
• single hidden layer FFNN
• static QNU• measured data
Static & Dynamic Higher-Order Neural Units (continue)Respiration time series: Training Accuracy for Predicting Exhalation Time -Instances of trained neural architectures trained from different initial conditions by L-M algorithm
2-hidden-layer static MLPs (static feedforward networks)
1-hidden-layer static MLPs (static feedforward networks)
static
QNUs
0 50 100 150
0
20
40
60
80
100
trénovacích epoch
JRNN
JDLNU
JDQNU
trénovacích epoch
Trénování predikce Mackey-
Glass
0 50 100 150
0
20
40
60
80
100
trénovacích epoch
JDQNU
JDLNU
JRNN
Trénování predikce polohy plic0 20 40 60
0
20
40
60
80
trénovacích epoch
JRNN
JRNN
JDQNU
Trénování predikce nelineárního periodického
signálu
0 50 100 150
0
20
40
60
80
100
trénovacích epoch
JRNN
JDLNU
JDQNU
trénovacích epoch
Static & Dynamic Higher-Order Neural Units (continue)
0,0 0 0 0,1 0 1 0,2 0 2
2, ,... i j i j n n n
y w x x w x x w x x
w x x w x
0,0
0,1
0,20 0 0 1 0 2
,
n n
n n
w
w
wy x x x x x x x x
w
rowx colW
1( ...)
( ...)
( )
k
k
k
y
y
u
( )kx
( )sk ny
0
r rn n
i j iji j i
x x w
Static & Dynamic Higher-Order Neural Units (continue)
Stability of weight-update system
• Condition for STATIC HONU
• Condition for DYNAMICAL HONU
( ) ( ) 1k k 1 M colx rowx
( )( ) ( ) ( ) 1
kk n k kse
rowx1 M colx rowx
colW
,
HONU
1( ...)
( ...)
( )
k
k
k
y
y
u
( )kx
0 50 100 150 200 250 300 350 400-2
-1
0
1One Epoch of GD Adaptation of Recurrent QNU to Predict MacKey-Glass Equation (training data vs. neural output)
0 50 100 150 200 250 300 350 400-0.5
0
0.5Prediciton Error during the Epoch of Adaptation
0 50 100 150 200 250 300 350 4000.98
1
1.02
1.04
k
Spectral Radius during the Epoch (stability of weight update system at each adaptation step)
0 100 200 300 400 500 600 700 800
-2
0
2
k
GD Adaptation of Recurrent QNU to Predict MacKey-Glass Equation (training data vs. neural output)
0 100 200 300 400 500 600 700 800-10
0
10
k
Prediciton Error during Adaptation
0 100 200 300 400 500 600 700 800
1
1.5
2
k
Spectral Radius during Adaptation (stability of weight update system at each adaptation step)
600 620 640 660 680 700 720 740 760 780 800
-2
0
2
k
GD Adaptation of Recurrent QNU to Predict MacKey-Glass Equation (training data vs. neural output)
600 620 640 660 680 700 720 740 760 780 800-10
0
10
k
Prediciton Error during Adaptation
600 620 640 660 680 700 720 740 760 780 800
1
1.05
1.1
k
Spectral Radius during Adaptation (stability of weight update system at each adaptation step)
Achievements with QNU
250 300 350 400
-1
-0.5
0
0.5
1
1.5
t [min]
NOx,CO prediction – EME I
trénování testování
Obr. 1: Dobře natrénovaná síť TptRNN pro 3-minutovou predikci klouzavých 3-minutových průměrů NOx, externí měřené vstupy jsou klapky a výkon , (klouzavé průměry se počítají jako průměry předchozích, současných a následujících hodnot, při intervalu predikce 3 minuty to znamená, že externí vstupy jsou již dostupné ale model v principu predikuje 3-minutový průměr který má být za 2 minuty), včase cca 415 ignoruje výpadek měření NOx a výstup modelu dobře nahrazuje měření.
Lung Tumor Motion Prediction
0 500 1000 1500 2000 2500 3000 3500-2
-1
0
1
2
Late
ral a
xis
[mm
]
0 500 1000 1500 2000 2500 3000 3500-10
-5
0
5
Ceph
aloc
auda
l axi
s [m
m]
0 500 1000 1500 2000 2500 3000 3500-2
-1
0
1
2
k
Ante
ropo
ster
ior A
xis
[mm
]
y1
y2
y3
20 40 60 80 100 120
t [sec]
-8
-6
-4
-2
0
2
4
6testing MAE= 0.853120295578 [mm], RMSE= 1.14143756682, treatment time = 86[sec], computing time= 83.385[sec]
20 40 60 80 100 120t [sec]
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0absolute value of prediction error
Lung Tumor Motion Prediction by static QNU
sampling 15 Hz, epochs=100, Ntrain=360, 492 neural weights
Lung Tumor Motion Prediction by static QNU
10^0 10^1 10^2
epochs
0.000
0.005
0.010
0.015
0.020
0.025
0.030
0.035
0.040 Averaged normalized SSE of Retrainings
Nonlinear Control Loop of a Laboratory System
[ ] Ladislav Smetana: Nonlinear Neuro-Controller for Automatic Control,Laboratory System, Master’s Thesis, Czech Tech. Univ. in Prague, 2008.
Nonlinear Control Loop of a Laboratory System
PID Control and Nonlinearity of the Plant
0 100 200 300 400 500 600-30
-25
-20
-15
-10
-5
0
t [s]
y [c
m]
Prubeh PID regulace v zavislosti na hloubce ponoru batyskafu, serizeno na hloubku 20 cm
5 cm
10 cm
15 cm20 cm
25 cm
0 100 200 300 400 500 600-40
-35
-30
-25
-20
-15
-10
-5
0
t [s]
y [c
m]
Prubeh PID regulace v zavislosti na hloubce ponoru batyskafu, serizeno na hloubku 10 cm
5 cm
10 cm
15 cm20 cm
25 cm
Tunned PID controller for 10 cm
30
Tunned PID controller for 20 cm
Nonlinear Control Loop of a Laboratory System
0 100 200 300 400 500 600-30
-25
-20
-15
-10
-5
0
t [s]
y [c
m]
Prubeh PID regulace v zavislosti na hloubce ponoru batyskafu, serizeno na hloubku 20 cm
5 cm
10 cm
15 cm20 cm
25 cm
0 100 200 300 400 500 600-40
-35
-30
-25
-20
-15
-10
-5
0
t [s]
y [c
m]
Prubeh PID regulace v zavislosti na hloubce ponoru batyskafu, serizeno na hloubku 10 cm
5 cm
10 cm
15 cm20 cm
25 cm
310 20 40 60 80 100 120 140
-25
-20
-15
-10
-5
0
t [s]
y [c
m]
Prubeh regulace neuro-regulatoru zavislosti na hloubce ponoru batyskafu
5 cm
10 cm
15 cm20 cm
25 cm
QNU as Adaptive Controller (simplest gradient descent)
Linear PID
Nonlinear Control Loop of a Laboratory System
False Neighbor Analysis is a single-scale analysis
x yyf )(x
( )
( ) ( )
i
j i
x
x x
( )
( ) ( )
i
j i
y
y y
To train neural networks , input (state) vector must be estimated to minimize uncertainty in training data
Děkuji za pozornost
y=f(x)x input data y output data
False Neighbors
1 2 IF AND
THEN and are False Neighbors
=> How much is correct Rx and Ry? - we do not know
=> Let's characterize false neighbors over whole intervals
of Rx and Ry, an
x yR y y R 1 2
1 2
x x
x x
d not just for their single setup
False Neighbor Analysis is a single-scale analysis
Slope of FN in Log-Log plot
FN = 4.2239*log2(id) - 4.5879
-2
0
2
4
6
8
1 1.5 2 2.5 3
log2(id)
log2(FN) Linear (log2(FN))
q(k ) c r (k )H
( )
( )
log
log log
k
k
q
c H r
MULTI-SCALE ANALYSIS approach (MSA)
number of false neighbours on a main diagonal
0
50
100
150
1 2 3 4 5 6
id...index of a diagonal cell
FN
• To characterize a system over the range of setups
• Power law
•What is the fundamental idea?
MULTI-SCALE ANALYSIS approach (MSA)• What is the fundamental idea?
q(k ) c r (k )H
q … quantityH … characterizing exponentr(k) … discretely growing radius
r(k)=2,4,8
•To characterize a system over the range of intervals•The power-law concept
MULTI-SCALE ANALYSIS approach (MSA)• What is the fundamental idea?
q(k ) c r (k )H
q … quantityH … characterizing exponentr(k) … discretely growing radius
r(k)=2,4,8
•To characterize a system over the range of intervals
•The power-law concept
MULTI-SCALE ANALYSIS approach (MSA)• What is the fundamental idea?
k r(k) q A q B
1 2 4 22 4 13 113 8 44 44
r(k)=2,4,8
log2(qB) = 2.2297*log2(r) - 1.1531
log2(qA) = 1.7297*log2(r) + 0.2605
0.9
1.9
2.9
3.9
4.9
1 1.5 2 2.5 3log2(r(k))
q(k ) c r (k )H
MULTI-SCALE ANALYSIS approach (MSA) (cont.)
• How can MSA help to create better neural network models?
j =1 j =2 j =3 j =4 j =5
i=1
max FN (highest chance that y1≠y2
when x1=x2 )
i=2 FN (2,2)
i=3 FN (3,3)
i=4 FN (4,4,)
i=5
min FN (lowest chance that y1≠y2
when x1=x2 )
FN (i ,j ) … count of False Neighbors for Rx (i ) and Ry ( j )
Rx(i
)
Ry ( j )
Smallest Rx - maximum of different states of a system
Largest Rx - minimum of different states of a system
Smallest Ry - maximum of recognized different outputs
Largest Ry - minimum of recognized different outputs
FN decrease
FN decreases
ffecf F
N d
ecre
ase
ffecf F
N d
ecre
ase
( )f yxj
i
False Neighbors Matrix:
Multiscale False Neighbor Approach
MULTI-SCALE ANALYSIS approach (MSA) (cont.)
• What are other potentrials for MSA for signal processing?
• MSA based signal processing
• Variance Fractal Dimension Trajectory (VFDT)
• Mutual Information
– Multiscale approach to calculate mutual information itself
– Mutual information of VFDT processed signals
• Everywhere, where a common analysis is subject to a
single-parameter setup and changing the setup disqualifies
the analysis results.