Linear multivariable systems : preliminary problems inmathematical description, modelling and identificationCitation for published version (APA):Hajdasinkski, A. K. (1980). Linear multivariable systems : preliminary problems in mathematical description,modelling and identification. (EUT report. E, Fac. of Electrical Engineering; Vol. 80-E-106). TechnischeHogeschool Eindhoven.
Document status and date:Published: 01/01/1980
Document Version:Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)
Please check the document version of this publication:
• A submitted manuscript is the version of the article upon submission and before peer-review. There can beimportant differences between the submitted version and the official published version of record. Peopleinterested in the research are advised to contact the author for the final version of the publication, or visit theDOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publication
General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.
• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.
If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:www.tue.nl/taverne
Take down policyIf you believe that this document breaches copyright please contact us at:[email protected] details and we will investigate your claim.
Download date: 06. Aug. 2021
LINEAR MLLTIVARIABLE SYSTEMS
Preliminary Problems in Mathematical Description, Modelling and Identification.
by
A. K. H ajdasinski
E I N D H 0 V E NUN I V E R SIT Y 0 F TEe H N 0 LOG Y
Department of Electrical Engineering
Eindhoven The Netherlands
LINEAR MULTIVARIABLE SYSTEMS.
Preliminary Problems in Mathematical Description, Modelling and Identification.
By
A.K. Hajdasinski
TH-Report 80-E-106 ISBN 90-6144-106-4
Eindhoven
April 1980
Contents
1.
1.1.1.
1. 2.
1.2.1.
1.2.2.
1.2.3.
1.2.4.
1. 3.
2.
2.1.
2.1.1.
2.1.2.
2.2.
2.2.1.
2.2.2.
2.3.
2.3.1.
2.3.2.
2.3.3.
2.4.
2.4.1.
2.4.2.
Acknowledgements
Abstract
ii
Introductory informations
Preliminaries - definitions of some important notions
Mathematical models commonly used for the multivariable dynamical system description
Transfer function matrix model
Decomposition of the transfer matrix and classification of the multivariable dynamical systems
State space representations
Nonuniqueness of the state space equations
controllability and observability in multivariable dynamical systems
Basic structures of the multivariable dynamical systems and canonical forms
Definition of the order of the multivariable dynamical system
Advantages and disadvantages of the transfer function matrix models
iv
v
1
1
12
13
19
21
27
28
31
31
34
Advantages and disadvantages of the state space models 35
Observable and controllable canonical forms for the state 36 space models
Canonically observable form
Canonically controllable form
Innovation state space models
Optimum estimation and conditional expectation
Optimum estimation and orthogonal projection
The discrete-time innovation problem
Generation of canonical forms from Hankel matrices
The Ho-Kalman minimal realization algorithm
The minima realization algorithm with the use of singular value decomposition of the Hankel matrix
38
42
45
46
51
54
59
62
64
3.
3.1.
3.2.
3.3
3.4.
3.5.
3.5.1.
3.5.2.
3.5.3.
3.6.
3.6.1.
3.6.2.
3.6.3.
3.7.
4.
4.1.
4.2.
4.3.
iii
Identification of the multi variable dynamical system structure
Estimation of structural invariants - Guidorzi's method
Order test based on the innovation-approach to the state space modelling-Weinert - Tse's method
Structural identification proposed for the transfer function matrix model of the MIMO system - Furuta's approach
Miscellaneous order test
Akaike's FPE (final prediction error) and AIC (Akai~s
maximum Information criterion) as the order test for MIMO Systems
Statistical predictor identification - Final Prediction Error Approach
Akaike's maximum Information Approach
Concluding remarks
Structural identification based on the Hankel model
Behaviour of the error function
Behaviour of the determinant of the Hankel Matrix
Singular value decomposition of the Hankel Matrix
Conclusions and remarks
Multivariable system identification
The Tehter's minimal partial realization algorithm
Gerths's algorithm
The approximate Gauss - Markov scheme with the singular value decomposition minimal realization algorithm
References
72
72
79
86
90
91
92
104
114
115
115
116
117
123
124
126
130
134
141
iv
ACKNOWLEDGEMENTS
This report, being a part of the project "Identification in MIMO
Systems", has been written with the kind help and financial support
of the Samenwerkingsorgaan between Katholieke HQ2eSchool' Tilbur2 and
Technische Hogeschool Eindhoven.
The author feels honoured to express his acknowledgements to
Ir. A.J.W. van den Boom, the project leader, who took the responsibility
of co-ordination, discussion and correction of this report.
The author also feels indebted to Mrs. Barbara Cornelissen, whose
devotion in typing this report within a short time, including lunches,
wins appreciation and gratitude.
The writing of this report in a relatively short time was also possible
due to the generosity of the author's wife and son, who were left --alone
for four months.
Present address of the author:
Dr. Ing. Andrzej K. Hajdasifiski, G16wne Biuro Studi6w i Projekt6w G6rniczych, Plac Grunwaldzki 8/10, 40-950 KATOWICE (Poland)
v
Abstract
This report contains a partial knowledge about linear multivariable
systems. It starts with very simple concepts from multivariable
system theory, and closes with some proposals of further research
in the field of MIMO systems identification.
Selected subjects were mainly discussed, however, forming a comprehensive
set. The choice was certainly subjective, but presented methods were
either applied with good experience or convincing records about their
application were found.
There is, however, an exception which still needs further research,
namely the Akaike FPE method, which intuitively is quite obvious, but
practically never well explained.
This work deals with subjects to be found in generally available
literature, but also (this is a subjective feeling) with subjects which
are presented in an artificially complicated way (e.g. innovation
approach) or which are mainly authors' studies (e.g. Markov parameters,
. order tests for MIMO systems).
- 1 -
1. INTRODUCTORY INFORMATION
The notion of the multi variable dynamical system has appeared in literature
and in practice as a natural evolution of the scalar dynamical system being
the very first approximation of real processes. R.W. Brocket and
H.H. Rosenbrock in their foreword to the series" Studies in Dynamical
Systems" have written: "During the last twenty years there has been
a progressive increase in the complexity and degree of interconnection of
systems of all kinds. The reasons are clear: recent progress in communication
data processing, and control have made possible a much greater degree of
coordination between the parts of a system than ever before."(+) Such a
development demanded new techniques, new mathematical models and methods
suitable for handling more complex and intercorrelated tasks of the agregated
systems. A quick development of the multi input - multi output systems theory
had to go in line with a very advanced mathematical apparatus application
and unavoidable incorporation of digital computers and numerical methods. In
this study we will try to give a comprehensive description of selected problems
being of particular interest for a system designer. For the rest of already
tremendously imposing material we will refer .to an extended bibliography.
1.1. Definition of a multivariable dynamical system
The definition of the multivariable (multi input - multi output system)
dynamical system as proposed by Wolovich (1974), Niederlinski (1974) and
Rosenbrock (1970) is as follows:
Definition The multivariable system is the system having more than one
input and one output and inputs to this system may influence more than one
output at a time.
(+) Published by Thomas Nelson, London.
- 2 -
The block diagram of such a system is shown in Fig. 1.
u, --- .. --...... ,-- .,.,.--:::---........ -~ ........ ~,.",
-::...~_ ...c._--- <...---~~ . , .-:::- _ >0 -. ~ ......... , , -::::- -- - - ---"':!...
t".!. i..
Further we will consider only a relatively simple class of multivariable
dynamical systems, namely those which are linear, time invariant and finite
dimensional.
1.1.1. Preliminaries - definitions of some important notions
Definition 2 For the multivariable dynamical, linear, time invariant and
finite dimensional system having p inputs u,(k) .••.•• u (k) (forming the p
input vector ~(k)) and q outputs Yl(k) ...•• Y (k) (forming the output vector q
y(k». Here is defined the q x p matrix !(z). called the transfer matrix
(being considered the rational matrix of the argument z) fulfilling the
following condition:
l.(z)
where
l.(z)
!(z)~(z)
y (z) ,
y (z) q
~(z) u (z) ,
u (z) p
(1 )
- 3 -
and l.(z) , .o!.(z) are the "z" transforms of l.(k) and u(k) respectively, under
zero initial conditions. (see also Zadeh L.A., C.A. Desoer (1963),
Eykhoff P. (1974), Rosenbrock H.H.(1970), Niederlinski (1974), Schwarz H.
(1971), Wolovich (1974~.
The coordinates of the .o!.(k) vector can be both control variables and
disturbances, while the coordinates of the l.(k) vector are the output variables.
Further there will be considered only such linear systems, for which outputs
are linearly independent i.e. the outputs cannot be described as a linear
combination of remaining ones. This simply means that the inputs and the
outputs must fulfil the following condition:
q ~ p
This condition is always fulfilled while the rank of the ~(z) matrix is
equal to q.
{rank {~(z)} = q}<:> {q ~ p}
(2)
(3)
Definition 3 The aharacteristia polynaminal w(z) of the strictly proper+)
or proper transfer matrix !(z) is defined as the least Common Denominator of
all minors\nE(z), having by the greatest power of "z" the coefficient equal to
one. (see also Wo10vich (1974), Schwarz, (1971), Rosenbrock (1970».
Proceeding further with definitions we have to define the degree of a
parametric (po1ynomina1) matrix and the state representation of the multivariab1e
dynamical system.
Defini tion 4 The degpee 6{K(z)} of the strictly proper or proper transfer
matrix ~(z) is defined as the degree of its characteristic polynominal.
- 4 -
(Practically it is the smallest number of shifting elements necessary to
model the dynamics of this system).
+),. a multivariable system is called "a proper system" or its transfer
matrix is called "a proper transfer matri;x:"
if
lim ~(z) '" 0
z + 00
2. a multivariable system is called "a striatZy propel' system" or its
transfer matrix is called "a striatZy proper tronsfer matrix"
if
lim ~(z) 0
z + 00
3. a multivariable system is called "an impl'oper system" if at least for
one component of a transfer matrix it holds that the degree of a
nominator is greater than that of a denominator.
Definition 5 For the multivariable, linear, time invariant, dynamical system,
the state of the system of an arbitrary time instant k = k is defined as a o
minimal set of such numbers Xl(k), X2(k), ••••• X (k) the knowledge o 0 " 0
of which, together with the knowledge of the system model and inputs for
k ~ k is sufficient for determination of the system behaviour for k ~ k o 0
X(k) = - 0
X2 (k ) • 0
Xl(k) o
X (k ) n 0
is called the state vector, and members Xl(k ) .•••• X (k ) are
o n 0
called state variables.
- 5 -
(see also DeRusso P.II., Roy R.J., Close Ch. M. (1965), Kalman R.E.,
Falb P.L., Arbib M.A. (1969), Rosenbrock H.H. (1970), Schwarz (1971),
Wolovich (1974), Niederlinski (1974) and many other~.
Defini tion 6 The set of difference equations
~(k + I) = ~~(k) + !~(k)
where ~(k + I) - is a (n xl) state vector
~(k) - is a (p x I) input vector
is called the state equation, while the set of difference equation
where ~(k) - is a (q x I) output vector
is called the output equation.
Definition 7 The triplet of matrices {~t !t £} is defined as the
reaZization of the dynamical, linear, time invariant, multivariable system.
Defini don 8 The number of state variables "nit in the state equation is
defined as the dimension of the state vector or the state space and also
denoted as the dimension of the complete system.
Definition 9 Any polynominal fez)
fez)
for which holds
f(~) = Ak + Cl~k-t C A2 CAe AO ~ + ••••••• + k-2- + k-l- + k- = 0/
(4 )
(5)
(6)
(7)
- 6 -
is called the annihilating polynominal of the ~ matrix.
Investigating various properties of multivariable system the following
Lemma drawn from the Cayley-Hamilton Theorem can be of great help.
Lemma The characteristic polynominal of the ~ matrix - WA(z) is one
of the annihilating polynominals of the A matrix.
Definition 10 The polynominal f(z) of the smallest. nonequal zero.
degree k. fulfilling definition 9. is called the minimal polynominal of the
A matrix.
Definition II The matrix coefficient ~ = C AkB for k = 0.1.2 •..••. is
referred to as the k-~ ,Markov Parameter of the system defined by the
realization {~.~. ~}. (see also Ho B.L •• Kalman R.E. (1966). Schwarz (1971).
Gerth. W. (1971). Tether A.J. (1970). Hajdasiiiski A.K. (1976. 1978).
Hajdasinski A.K .• Darnen A.A.H. (1979».
Definition 12 The following description of the multivariable dynamical
system is referred to as the Hankel model (H - model) of this system.
y MUll + !!J<~ ~(i) = 0 for i < 0 (8)
where
1. '''l ~(o) S -properly
y U = dimensioned block vector containing
1.(1) ~(lL initial conditions
1.(5) ~(2 )
- 7 -
MO ....... 0 -0
= M -0
Generalized Toeplitz Matrix
H --k
M -2
M -0
M -1
M -:1
M -0
o
M ••••• ~ •••• -1 -1
M :i. •••• ~ ••••••
M ; ••• '~+1 ••••
Generalized Hankel Matrix
and ~ - for k = 0,1,2, ...• are the Markov parameters of the considered
system.
For a rigorous -derivation and more facts about the H-model, the reader is
referred to: Schwarz H. (1971), Gerth, H. (1971), Hajdasiiiski A.K. (1976),
Hajdasiiiski A.K. (1978).
In aiming for equivalency conditions for different types of models of multi-
variable systems, we must pass through two fundamental theorems and the
definition of the order of the multivariable dynamical system.
Theorem 1 The sequence of Markov parameters {~} for k = 0,1,2,3, ••.•..
has a finite dimensional realization {~, !, C} if and only if there are an
integer r and constants ai such that:
M --r + j =t: o'(i) M
r + j - i for all j ~ 0
i = 1
where r is the degree of the minimal polynominal of the state matrix A
(assuming we consider only minimal realizations).
(9)
- 8 -
Remark: Theorem 1 is called the reatizabitity criterion and the r is
called the reatizabitity index.
The proof of this theorem is to be found in the Ho L.B •• Kalman R.E •• (1966).
Schwarz H •• (1971). Kalman R.E •• Falb P.L •• Arbib M.A •• (1969).
Theorem 2 If the Markov parameters sequence {~} for k = 0.1.2 ••••. has
a finite dimensional realization {A. ~. ~}. with realizability index r. then
the minimal dimension n in the state space (also of the realization) for o
this realization fulfils
rank [.!!..-] = n o
where n - minimal state space dimension o
and
[M.] - (q x p) matrix -:l
n ~ r x min (P.q) o
H -r
M M -<> -I
M M -I -.
M M -r-l -r
......
......
M -r-I
M -r
- the Hankel Matrix (finite)
( 10)
(II)
The most correct proof of this theorem is to be found in Schwarz H •• (1971).
Remark: From linear dependence of Markov parameters it follows that
n o
(12)
- 9 -
Discussion of the Chapter 1.1.
With the aid of these 12 definitions, I Lemma and 2 Theorems it is possible
now to find a link between different types of multivariable system
descriptions. There will be no ~igorous mathematical derivation presented
here. To visualize this link we will draw a block scheme showing inter
dependence of different type models. The arrows in this scheme show only
possible direct links (fig. 2).
From this scheme we learn that while from the state space description there
is a straighforward way to get the transfer matrix !(z), the reverse
procedure must be completed employing the realization theory, which is a lot
more complicated.
OTHER REALIZATIONS
~ Ho-Kalman STATE SPACE
REALIZATION - ~, ~, ~ !(z)
DESCRIPTION
t MARKOV
PARAMETERS
l , H
MODEL
fig. 2. Interdependence of different
type models
- 10 -
On the contrary, knowing Markov parameters, it is equally easy to get any
required form of description. For the sake of modelling, Markov parameters
can be derived as easily from the state space description as from the
transfer matrix. Obviously Markov parameters are also used in the H - model.
Example I Let us consider a simple two input - two output system described
in the following transfer matrix ~(z):
u (z) 1
y (z)
:1 ~(z) :1 u~(~z)~--1----------1----~y (z)
2 2
- 1.0 z - 0.5
= (z - 0,8) 2(z - O,2)(z - 0,8)
I .0 1.0
(z - 0,4) (z - 0,4)
the characteristic polynominal w(z) of the strictly proper matrix !(z) is:
W(z) (z - 0.2)(z - 0.4)(z - 0.8)
the degree of the ~(z) is:
Markov parameters for this system are:
M = --Q
[
-1.0 0.0]
1.0 1.0
!:!I = [-0.8 0.6]
0.4 0.4
M """"2
-0.64
0.4
0.6
0.4
M = [-0.512
-3
0.064
M. = -1.
- II -
0.504 1 0.064
. . . . and so on •
One of the possible realizations of this system is:
0.2 0.0
A = 0.0 0.4
0.0 0.0
0.0 -1.0
B 1.0 1.0
1.0 1.0
c = [
1.0
0.0
0.0
1.0
-1.0 0.0
1.0 1.0
-1.0 0.0
1.0 1.0
H = 0.6 -. -O.B
0.4 0.4
0.0
0.0
O.B
-1.0 I 0.0
= M -0
-O.B
0.4
-0.64
0.4
rank H = 2 -I
0.6 rank H
0.4 -.
0.6
0.4
c 3
- 12 -
rank H = rank H = rank H -3 -" -" + N
N > 2
Thus r - realizability index = 2
n o
- dimension of the realization = 3
Coefficients of the minimal polynominal are: a1
f ",.in (~) = A2 - A + 0.16 I =
0.16
Remark: It is made evident now that not every annihilating polynominal of
A (~z.W(~) = (~- 0.2)(~ - 0.4)(~ - 0.8» must be of minimal order. In
this case the characteristic polynominal of !, being one of the annihilating
polynominals for ~, is of the .3-rd order while the minimal order is 2.
M . -,,+J
for j ~ 0
1.2. Mathematical models commonly used for the multivariable dynamical system description
Solving problems in multivariable dynamical systems requires implementation
of quite a huge and advanced mathematics: theory of sets, matrix algebra
and analysis with special attention payed to polynominal matrices and
functions of matrices, theory of linear spaces, theory of limiting processes,
advanced mathematical analysis, some topics in functional analysis, theory
of differential e.quations, complex analysis, Laplace and "z" transform
techniques and many supplementary topics from related disciplines.
It is not possible to give a review of even selected problems and the only
possibility is to direct a reader to references.
Unfortunately there does not exist any comprehensive publication treating a
- 13 -
subject of "Mathematical methods ll for multivariable systems entirely. Moreover,
the best references are with literature concerning the control problems,
because the control engineering science was stimulating the development of
certain methematical disciplines.
In this report we will attempt to give the most intuitive and simple
description of rnultivariable systems.
The practical applications show that sometimes sitnpler models may better
serve the control tasks than very sophisticated ones. This always is a
compromise between achievable accuracy, "common sense" and a scientifically
formal approach.
Extensive references for further readings will also be given here.
1.2.1. Transfer function matrix models
The main interest will be focused on the discrete-time systems. However,
it seems to be useful to start with the continuous-time, linear systems and
generalize derived results using the "z" transform concept.
Assuming that there are given: !(s) - the transfer function matrix and ~(s) -
the Laplace's transform of the input vector, it is always possible to find the
output vector Z(t) with the zero initial conditions.
and using the convolution integral: t J !(t - T)u(T)dT
o
I.( t)
where
( 13)
( 14)
is the (q x p) "weighing matrix" (see Zadeh L.A., Polak E. (1969), Schwarz H.
- 14 -
(1971), Niederlinski (1974~. The weighing matrix has an interesting
physical interpretation for columns of this matrix can be interpreted as
impulse responses to the separately applied input Dirac pulses.
if
t
1.( t)
o
o
u (T) I
0. (T) •
where Uk(T) = 0
A k f i, i = 1.2 - - P
k (t - T)k (t - T) ... k .(t - T) ... k (t - T) 0 II 12 I' IP
k (t - T)k (t - T) .•• k .(t - T) ... k (t - T) 0 21 22 21 2P
k. (t - T)k. (t - T) .•• k .. (t - T) .•• k. (t - T) O.(T) • 11 12 11 1p
o
k (t - T)k (t - T) ••• k .(t - T) ••• k (t - T) 0 ql q2 q1 qp
k .(t - T) I'
k .(t - T) 2'
k .(t - T) q1
o.(T)dT = k.(t) • -:L
i - th. column of
the ~(t)
dT =
( 15)
- 15 -
Example 2:
As an example to start with, we can consider a simple two input and two
output system of the level control.
,,~ Q. [,;:'J Qz.r-w-\] VI.
I M-
-------- -- ---- --
h, hI.
! A~[m~l Q ...
. A,,[m'] QIA
-
h
J
The task is to maintain levels h and h at a certain 2
range, manipulating
valves v and v such that the volumetric flows Q and Q can be properly I 2 I 2
adjusted. In order to propose a control algorithm, we have to find a model
- 16 -
of the given system, having the following block diagram:
Q. ~ ..
Writing linearized mass balance equations
llQ 1
llQ 2
llQ 1 1
llQ 21
A 1
= A 2
dt
dllh 2
dt
dllh llQ + llQ - llQ a A ---
II 21 dt
VI
111
where llQ, llQ, llQ ,llQ ,llQ, llh , llh ,llh are small deviations of 1 2 II 12 1 2
variables around the working points.
assuming
Q = f(h)
AQ = ~ a~~h) ) h
• llh = h
0
( af(h)
) h =
a a ah 0
h 0
llQ = a llh 0
also llQ = B llh 1 1 1 0 1
llQ = B llh 21 20 2
. .60,(5)
AQz.(S)
finally,
- 17 -
after applying the Laplace transform:
llQ (s) = A sllh (s) + S llh (s) I I I I 0 I
llQ (s) = A sllh (s) + fl llh (s) 2 2 2 20 2
llQ (s) = fl llh(s) + fl llh (s) - Asllh(s) I 0 I 20 2
a llh(s) + Asllh(s) o fl llh (s) + fl llh (s)
1 1 20 2
The block diagram of this dynamical process is following:
-I ,.,
A.s + (3.0 (3.0 ~ As T <:i..o
~h2(!»
~"o r r--
-1 I A'l.S + (310
Ant!»
An2.(~1
- 18 -
and a
i\.b(s) 0
= (8~: 'S+I)(f-'S+I) 0
i\.h (s) 2
0.0
introducing
820
k =--. k =-- k • a I I a 12 0 0
AI A2 T =-- ·T = T
I 8 2 B I 0 H
we get
!(s) !(S) !!.(S)
where k
I I = ---------
(I + 5 T )( I + s T) I
0.0
is the transfer matrix, and
k I I
22
-1.u..... a
i\.Q (s) 0 I
(~S+I~' S+I) 820 a
i\.Q (S)
'V~~ 2
At '" .... 1 ?S;o i\.b(s)
= !(s) =
820 i\.b (s) 2
A i\.Q (s) .. . I = !!.(s) • CXo i\.Q (5)
2
k 12
(I + sT )(1 + sT) 2
k 22
(I + sT) 2
k 12 -t -t
!(t) = (Te - - T e -) T I T
-t -t (Te - - T e -)
T 2 T T - T I T I
0.0
- T 2
k -t eT" 22 2
2
- 19 -
is the weighting matrix of the dynamical system
(I + sT1)(1 + sT2 )(1 + sT)
3
1.2.2. Decomposition of the transfer matrix and classification of the multivariable dynamical systems
Following Niederli~ski (1974) we will decompose the input vector ~(s) into
q - control inputs and p - q - disturbing inputs:
u (s) 1
~(s) = u (s) 2
(q x I)
. u (s)
q
u q + 1(9) (p - q) x I
u (9) p
which leads to the following relation:
1.( s) Q(s)] [~(s) 1 !.(s)
= t(s) ~(s)
u (9) p
+ Q(s)z(s)
t(s) + q x q transfer matrix of control inputs
Q(s) + q x (p - q) transfer matrix of disturbances
Following Niederlinski(1974) and Iserrnann (1977) we will classify the multi-
variable dynamical systems in the following way:
1. stable multivariable dynamical systems - Le. all those systems for which
(16)
- 20 -
all poles of the transfer function matrices lie in the left half plane of
the complex variable "s" and there are no poles on the imaginary axis of the
plane.
2. nonstab1e mu1tivariab1e dynamical systems - i.e. all those not fulfilling
the stability definition.
3. minimum-phase m.d.s. - i.e. all those for which all zeros of the
determinant det{!(s)} are in the left half plane of the complex variable "s".
4. non-minimum phase m.d.s. - if at least one zero of the det{!(s)} appears
in the right half plane of the complex variable "s".
It will be noticed that the mu1tivariab1e system may be non-minima1 phase.
However, each of its components p .. (s) (i,j, = 1,2, .... q) is the minimal 1J
phase object. The non-minima1 phase objects, are much more difficult for
handling than minimal phase ones.
Another classification due to Nieder1inski (1974) and Isermann (1977) is made
according the internal couplings of the multivariable dynamical systems:
I. m.d.s. with a negative coupling - i.e. such a system for which
det !(o) 1 - < 0
rlpu(O) (17)
i=1
II. m.d.s. with a positive coupling - i.e. such a system for which
det !(o)
1 - > o (18) q
TI P .. (0) 11
i~l
- 21 -
III. m.d.s. with a zero coupling - i.e. such a system for which
det !(o)
= 0 (19) q
[1 P .. (o) 11
i=1
!f.d.s. with a positive coupling resemble very much simple input-output systems
with a positive feedback, inclining to monotonic instability.
Another classification, credited to MacFarlane (1970)"Ostrowski(1952) and
Rosenbrock (1970), is suggested according to a very convenient stability criterion
by MacFarlane (1970):
a) m.d.s. with a dominant main diagonal - i.e. systems for which
q
\ P ii (jld) \ > L: P ik (jW)\
k=1 k,&i
i=I,2, .... q
i=1,2, .... q
(20)
(21)
b) m.d.s. without a dominant main diagonal - i.e. all those for which (20)
or (21) does not hold.
For further readings, the reader is referred to: Schwarz H. (1971),
Rosenbrock H. (197Q), MacFarlane A. (1970), Zadeh, Desoer (1963), Wolovich'
(1974), Kalman, Falb, Arbib (1969), Niederlinski (1970), Rij nsdorp (1961,
1971), Isermann (1977).
1.2.3. State space representations
Let us start with a definition.
Definition 5 in the chapter 1.1.1. is valid for both the continuous and discrete
time systems. The physical meaning of this definition is that the dynamical
- 22 -
system has a certain "memory" which contains information about influence
of past events on the present and the future. For the continuous time
systems definition 6 changes into the set of equations
d ~ (t) A x (t) + !~(t)
dt
assuming x(t ) = x(o) - initial conditions. - 0 -
A solution to this problem is
~(t)
t
A(t - t ) r e- 0 ~(o) + J
t o
l.( t) Ce _A (t - to ).!( 0) + erA (t - T) () ( ) J e- ! ~ T dT + !!. ~ t
t o
For the discrete time systems following definition 6 we have:
~(k + I) A x (k) + !~(k)
l.(k) = ~ ~ (k) + ~ ~(k)
where matrices {A, B, C, n} from eq. (22) are not the same as from
eq. (25) (26)
Solutions to (25) and (26) are:
x (N) N !!. ~(o) +
N - I ~ AN - k - IB u (k)
k = 0
N - I :leN) N = ~!!. ~(o) + c L:: AN-k-IBu() () k + ~ ~ N
k = 0
(22)
(23)
(24)
(25)
(26)
(27)
(28)
- 23 -
Using equations (22) and (25),(26) it is easy to find relations between state
equations and transfer functions for both types of systems:
a) continuous-time m.d.s.
applying the "Laplace" transform and combining two operator equations:
assuming
, y(s) = £ (s.!. - ~t ~ ~(s) + £(s.!. - ~) ~(o) + Q. ~(s) (29)
for the "zero" initial conditions are required for the transfer function def-
inition, ~(o) = 0
-' !(s) = £(s.!. -~) ~ + Q., where.!. = [Unitary matrix] (30)
of dimension (n x n)
b) discrete-time m.d.s.
applying the "z" transform and combining eq. (25)(26)
, -' y(z) = £(z.!. -~) ~ ~(z) + z £(z.!. - A) ~(o) + Q. ~(z) (31 )
and again for the "zero" initial condition
(32)
UlS\
L.1.l'L)
- 24 -
The block scheme of the multivariable dynamical system described in terms of
state equations is as follows:
--") B
Example 3:
equations:
dlW 1
dt
dfIV 2
dt
dV
dt
choosing t.V , 1
t.Q =
t.Q = 11
D -
;(): ~ I~ (4) -
~ 6
~(Sl :!lzl
- ----
lr ....
c -X -
':J.(,S ) ) 'itz
Reconsidering example 2, we can write the following set of
t.Q - t.Q t.V = A t.h 1 1 1 1 1 1
= t.Q - t.Q t.V = A t.h 2 21 2 2 2
= t.Q + t.Q - t.Q t.V Mh 1 1 2
lW and t.v as state variables and writing 2 3
Ct tN
0-
A
t.V B 1 10---
A 1
- 25 -
!W l>Q ~ 8 2
21 HA 2
we get:
deW I B,O = - --!W, + l>Q, dt A,
dl>V2 ~ - 820
dt -- l>V 2 A2 + l>Q2
t. I I' 'I
dl>V 810 820 () I,
l>V , + V - _0_ l>V f:
= --, A;""-dt A, A 'I; '1
[ "'j. [ I'
l>V l> l>V I l> l> '"] ).
l>h c __
x· u • za A tN2 l>Q2 lIh2 , = ..§!..L r
lIh2 A2 lIV
b. . /:;.V. - A. 0 0 I:Ni ~ 0 Aq.
~ AV1 = 0 _is. 0 tN,. T 0 1 ~Qa. i A~
h .J2v 0<.0 ,
AV -- AV 0 0 Ai AI. A
[!~J [ 0 0 1 ] 6V, 1 = 0 1 0 AVa.
AV
-NA. 0 0
[~ ~l A= 0 -A./A~ 0 B=
-AjV'A. J3m/A2. -oly'A
~ =( 0 0 ~ ] 0 1
~ ,. Q
- 26 -
Assuming that first q inputs to the system are the control quantities and
remaining p - q are the disturbing quantities, we get the following state
equation.
d .! (t) A x (t) + l1!p '!!Q ] [~(t)l !.( t)
dt
1.( t) = ~.! (t) +[!!.p !!.q] [~(t)l
z(t)
=
= C x (t) +.!!p ~(t) + !!.q ~(t)
.!!p(n x q), ~(n x (p - q)), ~(q x q), Qq(q x (p - q)) which corresponds to
the transfer matrix of control inputs
1
~(s) = ~(!s -!) .!!p + !!.p
and transfer matrix of disturbances
(33)
(34)
(35)
(36)
The matrix (!s -!) is called the characteristic matrix of A and the polynominal
(37)
is the charateristic polynominal of A.
Example 4
For the dynamical system from examples 2 and 3 we have
W(s) (s + ~ ) (s + ~) (s + Clo )
A
- 27 -
WA(s) - calculated as the Characteristic Polynominal of A
W(s)
Discussion
=
)(s + ~)(s +
A2
Ct o
A
)
The degrees of the W(s) and WA(s) are the invariants of the multivariable
dynamical system. The degree of the WA(s) is equal to the dimension of the
state space (see def. 8). The degree of W(s) is the degree of the transfer
matrix ~(s) (see def. 4). In general, W(s) I WA(s) and 0 WA(s) I 0 W(s) .
This will be discussed in the sequel.
1.2.4. Nonuniqueness of the state space equations
It is sometimes difficult to find a state space description which has a physical
interpretation. This happens while estimating the state space equations basing
an experimental input-output data without a prior knowledge of the physical
structure of the considered object. This is due to nonuniqueness of the state
space equations.
It can be proven that for a m.d.s. it is possible to find and infinite number
of state equations.
Let
!.(t) = T x (t) (38)
where! (n x n) is any nonsingular matrix. Thus equations (22) or (25)(26) will
result in
d x (t)
dt
_1 TAT x - --- (39)
- 28 -
~(t) = C T x (t) + ~~ (t) (40)
This new set of state equations results in the same transfer function matrix
as equations (22):
1 I 1
~ !(!s -! ~!) T B + D = =
= =
= I
~(~s -~) B + D = !(s) (41 )
The relation A -I
.!. !:..!. is called the "similarity relation" and there is
as special type equivalence between matrices ~ and A. For more details see:
DeRusso,P.M.,Roy R.J., Close Ch. M. (1965), Rosenbrock H.H. (1970), Wolovich
W.A. (1974), Birkhoff G., MacLane S. (1965), Gantmacher (1959) and many others.
1.3. Controllability and observability in multivariable dynamical systems
The multivariable dynamical system described by means of the state equation
!(t) = ~ ~(t) + ~P 'y'(t)
is sain to be COMPLETELY CONTROLLABLE i~ Kalman's sense if given any initial
state x(t ) there exist such the control vector _v(t), which will drive the - 0
m.d.s. to the final state ~(tf) = ~ for finite (tf - to)'
(42)
The necessary and sufficient conditions for complete controllability in Kalman's
sense are usually formulated in the following way: Kalman R.E., (1960),
Chen C.T., Desoer C.A., Niederli~ski A. (1966), Kuo B.C. (1970), Wolovich W.A.
(1974), Niederlinski A. (1974):
- 29 -
Theorem 3
The necessary and sufficient condition for the multivariable dynamical system
to be controllable in Kalman's sense is that the block matrix (n x qk)
(43)
where K ~ n is the degree of the minimal polynominal of ~, has the rank equal n.
(Alternative conditions are also discussed by Kuo B.C. (1970), Paul C.R. and
Kuo Y.L. (1971), Zadeh A.L., Desoer C.A. (1963) and many others.
The proof of this theorem can be found in KalmanR.E. (196~, Kuo B.C. (1970),
DeRusso P.M., Roy R.J., Close Ch. M. (1965).
From condition (43) it follows that the complete controllability in Kalman's
senSe is the property of the pair of matrices (~, ~) and does not depend on
the way outputs are produced by the system.
The multivariable dynamical system described by means of the state equation (42)
and the output equation
1.( t) £. x (t)
is said to be completely observable in Kalman's sense if given any input ~(t)
and output 1.(t) for to ~ t ~ t f is sufficient to determine the initial state
~(to) for a finite interval [to' tfJ.
The necessary and sufficient conditions for complete observability in Kalman's
sense are usually formulated in the following way.
Theorem 4
The necessary and sufficient condition for the multivariable dynamical system
(44)
to be completely observable in Kalman's sense is that the block matrix (n x qk)
(45)
- 30 -
where k ~ n is the degree of the minimal polynominal of !, has the rank equal n.
(Alternative conditions are also discussed by Kuo B.C. (J970), Paul. C.R. and
Kuo Y.L. (1971».
In the classical analysis of control systems, transfer functions are often
used for the modelling of linear time-invariant systems. Although
controllability and observability are concepts of modern control theory, they
are closely related to the properties of the transfer function. The following
theorem gives the relationship between controllability and observability and
the pole zero cancellation of a transfer function.
Theorem 5
If the input-output transfer function of a linear system has pole-zero
cancellation, the system will be either not state controllable or unobservable,
depending on how the state variables are defined. If the transfer function of
a linear system does not have pole-zeros cancellation, the system can always
be represented by dynamic equations as a completely controllable and observable
system. Kuo B.C. (1975). (This is an excellent reference for further readings
for everyone who wants to gain more information about dynamical systems being
considered in a very physical way).
Concepts of controllability and observability in the Kalman's sense are rather
useless while dealing with noisy systems. Also for some theoretical systems
those concepts can cause misunderstandings. An example will be given while
discussing a closely related concept of the multivariable system order.
- 31 -
2. Basic structures of the multivariable dynamical systems and canonical forms
Three main types of models incorporated in multi variable dynamical systems
design, analysis and identification have been described in brief. According
to very specific properties of each of these models, slightly deeper insight
must be done into their structural properties and utility possibilities for
different types of tasks. The concept of the multivariable dynamical system
order will be of an essential importance for the following part of the text.
2.1. Definition of the order of the multivariable dynamical system
In trying to model the reality, one has to answer first a question: what type
of applications for this model is considered?, thus facing the problem
of the "structure" choice for this model. This structure presents a desired
type of relations between inputs and outputs. However, two additional steps
must be performed - those are: demarcation of the "degree of complexity" for
this model (corresponds to the order determination) and parametric estimation
of the chosen model being of the pre-estimated degree of complexity
Hajdasifiski - Damen (1979) •
This procedure in most practicaZ cases is an iterative seeking for the modeZ
order and inaorporated set of parameters, matching it to reaZ data according
to a given optimaZity criterion and comparing with resuZts of previous runs.
- 32 -
input data output data
- - OBJECT )
choice of structures I'
choice of models
• > complexity
d a t a
test (O'yeSII 'llllll/mlq
• decisions
parameter
~ estimation ~
'F'T('1\'I'T""
mOdel output computation
~ .J ~ ~ ~ no
MODEL . ~ f", ")( ATCHIN y-
~ yes
FINAL MODEL
Definition 13 The order of the multivariable dynamical system will be defined
as the minimal number of Markov-parameters necessary and sufficient to re-
construct the entire realizable sequence of Markov -parameters. Hajdasinski,
Damen (1979) •
- 33 -
Remark It means that the system order is equal to the realizabilty index
fir" - see relation (9) Theorem J ..
This definition favours the Hankel model description and it really is the
intention for this type of model to give the most general possibilities of
descriptiveness without causing any ambiguity. Having a properly described
model complexity it is very easy to generate proper structures in the state
space and a properly structured transfer function matrix.
For example for the state space description, the multivariable system order
can be defined as the degree of the minimal polynominal. For the transfer
function matrix description, however, the order definition in the general case
is not possible.
Defini tion 14 The dimension of the multivariable dynamical system is defined
as the number "n" being equal to the rank of the H - Hankel matrix for this r
system, where fir" is the order of the system.
Remark Compare def. 14 with the theorem 2, relations (10) and (II).
Alternatively for the state space description it is the dimension of the state
matrix A. And again for the transfer matrix description there does not exist
a unique definition of the system dimension. Only in the cases when all poles
1n elements of the transfer matrix are either different or "equal and common"
the dimension can be determined as the degree of this matrix (see definition 4.)
To illustrate this, the following example shows the case of equal but distinct
or non-common poles. Hajdasinski - Damen (1979).
0.0 K(z)
(z - 0.8)(z - O,25)(z - 0.5)
0.0 (z - 0.8)(z - 0.6)(z - 0.01)
- 34 -
Dimension of this system is n = 6, but degree 0 k(z) 5 because the equal
poles z = 0.8 are noncommon and they refer to different state variables.
However, in practical cases it will be seldom that distinct poles have exactly
the same value i.e. are equal. Nevertheless, when poles are given up to a
certain accuracy (for example numerically evaluated poles) it may be difficult
to decide whether they are really distinct or not. This problem is also one
of the drawbacks of the transfer matrix description and one more argument for
the state space and Hankel description, where this ambiguity never arises.
2.1.1. Advantages and disadvantages of the transfer function matrix models
The following features of transfer function matrix models are worth noticing.
I. the transfer matrix description is a unique description of the multi-
variable dynamical system given a unique ordering of inputs and outputs.
It means that there exists one and only one transfer function matrix k(z)
for a given order of inputs and outputs.
II. the transfer function matrix has a very easy physical interpretation for
elements K .. (z) of K(z) are transmitances between y.(k) outputs and u.(k) 'J - 'J
inputs of the considered system.
III. the transfer function matrix description is not very economical for the
analog computer modelling.
IV. the transfer function matrix is very inconvenient for digital modelling.
V. the transfer function matrix may not encounter all dynamical properties of
the system (see 2.1.).
VI. from knowledge of the transfer function matrix it is difficult to derive
state space equations, but it is fairly easy to derive the Hankel model.
- 35 -
For further comments see: DeRusso P.M.,Roy R.J., Close Ch. M. (1965),
Wolovich (1974), Kuo B.C. (1975), Isermann (1977), Niederlinski (1972, 1974).
2.1.2. Advantages and disadvantages of the state space description
The following features of the state space models are worth noticing:
I. the state space model is a non-unique description of the multivariable
dynamical system given an unique ordering of inputs and outputs (see 1.2.4.)
II. in a general case there is a Zaak of physiaaZ interpretation for the state
space model.
III. the state space model is more eaonomiaaZ for analog modelling than the
transfer function matrix model.
IV. the state space model is very aonvenient for digital modelling
V. the state space model enaounters aZZ dynamiaaZ properties of the system
being modelled.
VI. the state space model provides equaZZy easy transformation into the transfer
function matrix as into the'Hankel model.
- 36 -
2.2. Observable and controllable canonical forms for the state space models
This chapter will start with the phase canonical forms for single input -
single output controllable systems. The generalization for multi input-
multi output systems will be then easier. The major results in the field
of canonical forms are due to Kalman R.E. (1963), Lue.berger G.D. (1966),
Mayne D.Q. (1972a, 1972b) Popov V.M. (1972).
Assuming that there is given the transfer function K(z):
y(k) b + bIZ + + biz m-I
K(z) 0 m-= =
u(k) a2 z2 n-I n a + alZ + + ••••• + a 1 z + z
0 n-
for m ,n, and there is no pole-zero cancellation, the corresponding state
space equations in the minimaZ canonicaZ form may take the following form:
o o
X2 (k + 2) o o
Xn
_1 (k + 1 ) 0 0 0
x (k + u
1 ) -a 0
-al -a2
y(k) = lbo b l b2 •••••• b m-I
o
o
.... 1
.. .. -a n-I
•••• 0 •• I • 0
XI (k)
x2(k)
xn
_1 (k)
x (k) n
]
+
xI(k)
x (k) n
o
o u(k)
o
Equations (47) are referred to as the "phase-vanabZe canonicaZ form" and
(46)
(47)
- 37 -
state variables x (u) are called the "phase variables". i
The state A o o •... 0
o o o
o o o ..•• I
is called the "F:t'obenius matm" or the "companion matm of the poly-
nominaL ao + a z + ...... . u-l u" + a 1 z + z • u-
Another phase canonical form will be
~
)1.,(\(+1) 0 0 0 0 - a. A
1 0 0 0 -0... x~( k +1)
=
><,,)\(+4) 0 0 1 0 - a...-1 A
X" (\1.+1) 0 0 0 1 - a ".\
\jl k) :: [ 0 0 . ·0 0 o 1 ]
~
)(, (It.) b.. Xl tic.) b,
+ u'l'-'\
b"'_1 0
A
X "t\C.) 0
A
)(. (k) • )(l l \()
The development of phase canonical form for single output - single input
system was an attractive area of research for two main basic reasons:
(I) simplicity of derivation
(2) a convenient starting point for r.ertain control design problems.
The canonical forms for MIMO systems are even more important than
for 5I50 systems. The canonical form will be defined as t~s-
(48)
- 38 -
formation of the state vector to a new coordinate system in which
the system equations take a particular simple form". (see
Niederlinski - Hajdasinski (1979)).
Unlike the SISO case, the corresponding canonical forms for
multivarible systems are not unique. Among the most used canonical
forms, the canonically observable and controllable forms are of the
greatest importance.
2.2.1. Canonically observable form
Consider the discrete completely observable multivariable system represented
by state equations:
~(k +
x.(k)
Let H =
I)
h T -1
h T -:z
h T --q
H x
I x(k) + ~ u(k)
(K)
Constructing the vec.to..,. sequences
and selecting them in the following order
h -2
, .....
.....
retaining a vector (FT)sh. in (52) if and only if it is independent from - -1
(49)
(50)
(51 )
(52)
- 39 -
T . all previously selected ones and all the vectors (F )J h . (0 < j < s) have
- -1
already been selected. Let v , v ,v .... v be the numbers of vectors I 2 3 q
selected from the first, second and ----- q-th sequence in (51);
h , FTh , -2 -~
~~-----------v.--------
FROM THIS SEQUENCE ::;'V I
FROM THIS SEQUENCE ~ v 2
Tv' Thus the vectors (F ) Jh. are therefore linearly dependent on previously - -J
selected ones (because of the choosing procedure and because v. is counted J
from zero).
The complete onservabi1ity of the system implies that
v + v + •••• + v = n I 2 q
where n - is the dimension of the considered system.
. 1 . TT Let us now construct the nons1ngu ar matrIx _
h -q
and use the T as a transformation matrix for the similarity transformation
(see 1.2.4.) providing a new state vector x:
x Tx
Remark: The T is a nonsingular (n x n) matrix which consists of linearly
nondependent columns, each product
v + v ..... v = n. Lue~berger I 2 q
The state equations are:
~(k + I) =!! (k) + B u (k)
(FT)ih . - -J
(1967).
is the vector and
(53)
(54)
(55)
x.(k) C x (k) (56)
- 40 -
where matrices ~,~, and £ are of the following form
A = T F T- I - -- {A .. }
-1J i, j = I, 2, ..... , q
and (v. x v.) matrices A .. show the structure ~ ~ -],,1.
A .. = -11
o
o
a .. I 11,
.••••• a. . . 1.1,Vl.
which is exactly as already discussed for 8180 systems Frobenius matrix.
Because of the order followed in the selection of vectors in (51) and
of the consequent structure of !, in every matrix A .. -1]
at most v. + 1 1
elements are non-identically zero if j < i and v. if j 1
number v .. is given by 1J
L: min (v. , 1 v .. =
1J
(v., v.) 1 J
v. - 1 ) for j < i J
for j > i
and the (v. x v.) matrices A .. are of the type 1 J -].J
A .. = -1J
Further more
o .•..•.•...•......•.•.. 0
o ..............•....•.. 0
a .. 1J,
a.. 0 ...... lJ ,v ....... 0 1J
> i. Thus the
o ...... 0 0 ..................... 0
= o o o
0 ........... : .... : ..... 0 o .... 0
t • (v + I) 1
t (v + '" +v 1 + I)
1 q-
(57)
(58)
(59)
(60)
(61)
- 41 -
b b •••• b
B = T G does not possess 11 12 IP (62)
any special structure
b b •••• b nl n2 np
It should be noticed that the structure of the canonically observable couple
(!, £) is completely determined by v. - indices which are called Kponecke~ ~
invariants. These are structuraZ invariants of the dynamical system. (see
Luenberger D.G. (1967), Guidorzi R.P. (1973), , Popov V.M. (1972),
Niederli~ski, Hajdasifiski (1979). A single-output, single-input system has
only one structural invariant, namely the system order r or the system
dimension n (which are related to each other by the relation n = rank Hr -
see Theorem 2, relation (10».
The transformation! decomposes the original system (!, ~) into q inter-
connected subsystems having a structure guaranteeing the complete
observability of the jth subsystem from the jth output component. This
properly justifies the name "canonically observable form". (Niederlinski-
Hajdasinski (1979».
Remark: The dependent vectors (FT)vi h. are a linear combination, with - -~
coefficients given by the ith significant row of ! matrix, of
previously selected ones i.e. as demonstrates Popov (1972).
i-I
=L: j=1
min (v. ,v .-~) + ~ 1 J
~ aij'k k=1
min(v. ,v.) ~ 1 J
L- aij,k k=1
(See also Niederlinski - Hajdasifiski (1979».
Remark: Given Kroneckey invariants vi, the number of aij.k invariants
is well determined with the following properties (Niederlinski
Hajdasifiski (1979».
( 1 ) of v. and a .. k invariants are independent, i.e. for any ~ 1J,
the set
private integers v. satisfying v + v + •••• + van ~ 1 2 q
- 42 -
and any numbers aij,k eR' with properly ranging indices (i,j,k),
there exists a pair of matrices (F,H) whose invariants are
precisely the above integers vi and -numbers aij ,k.
(2) the set of invariants vi and aij,k are aompZete, i.e. if for two
pairs of matrices (!,~) and (!~) of the same dimension the
invariants are respectively equal, there exists a nonsingular matrix
such that F = T F T- 1 and H =
(3) the set of vi and aij,k invariants in the emaZZeet set of parameters
determining a canonical form. It is impossible to construct a
canonical form having the universality properly with less parameters
than the number of aij,k parameters, with the vi parameters det-
ermining their position in the canonical form.
2.2.2. Canonically controllable form
For the controllable couple (G,F) from equations (49), (50) also exists
a canonically controllable form derived by writing the G matrix in the form:
G [ . " ] = ,g,: g ~ ... :g-1 t """"2 •• -p
(64)
and constructing the sequence
P , g , .... g., Fll ., F" ..... Fa.., F2", ,F2,. ... F2Q. , ... (65)
.... ,..., -p -:!:' -~ -~I' - ~1 -!L~ - !.p
Similarly, as before, the choice of linearly independent vectors !s.s.j can be
made and again n ,n ..... n are the numbers of vectors selected from 1 2 P
sequences.
(66)
- 43 -
Then the vectors Fnl~. are linear combinations of previously chosen - L
s F a. and following relations hold: - .£J
n + n + ••••• + n = n '2 P
(n· - are Kronecker invariants of the controllable form), l
n; F 0.' - 11'
(67)
(68)
where ai.j.k - are invariant parameters corresponding to the· aq.k.' The sets
of the {n t} and {a tj.k.} invariants share all properties of the sets of
{Vi) and {a:i.j.k J invariants,
Introducing the new state vector z = !~, with
-I R = ,IF IF I"'IF
[
I I n,-I I I np-I ] ~, I -~, I - ~.. ,- ~,p
the system equations (49) (50) become
Z(k) ~ !.(k)
where A R F R- 1 fA, .} -LJ
i,j = J,2, •.•. p
with o •••.... 0
n. x n. matrices A .. L L -n
I. 1 -nL -
... 0 •••••• 0 and n. x n. matrices A •• =
L J -LJ 0 •••••• 0
o ...... 0
a .. n,
aii ,
a .. 1J ,1
a .. 1J ,2
n. L
ai . 0 .. J, LJ
(69)
(70)
(71 )
(72)
(73)
(74)
and
n .. 1J
Further
B
and c
c
{" min
R G =
- 44 -
min(n. , 1 ) for j < i n. -1 J
(n .• n. ) for j > i 1 J
0 • •••••• 0 .. 1
0 0 • •••••• a
o O ••••••• 0
o · ..•... 0 ~ n + I 1
•••• 0 · ...... . • • • • • • • • • • • • • 0
o o ...•... 1 ~ n + •••.• +n + 1 1 p-l
o O ••••••• 0
H R- I does not reflect any specific structure
c C 1 1 1 2.
(75)
(76)
(77)
It can be noticed that the transformation (69) has decomposed the original
couple (!. ~) into p interconnected subsystems having a structure
guaranteeing the complete controllability of the ith subsystem by the ith
input component.
This property justifies the name "aanoniaaZly controllable form". (see
Niederlinski. Hajdasifiski (1979».
Remark: The observable and controllable canonical forms are only two
among many possible canonical forms, but also very important ones.
- 45 -
As it will be discussed later, these two canonical forms play
an important role in determination of the multivariable system
structure.
For further reading in the subject of canonical forms see
Denham (1974), Mayne (1972a,b), Niederli~ski, Hajdasifiski (1979).
2.3. Innovation state space models
The essence of the innovation approach to the state space modelling can be
expressed in words of Kailath T. (1968 ):
" ....•.• the innovation approach is first to convert the observed process
to a white-noise process, to be called the innovation process, by means of
+ a causal and causally invertible linear transformations. The point is that
the estimation problem is very easy to solve with white-noise observations.
The solution to this simplified problem can then be re-expressed in terms of
the original observations by means of the inverse of the original '~hitening"
filter .•... "
The given observations are recorded in the form
l.(k)
E{!. (k)}
~(k) + !.(k) k 0,1,2 •..•
Q - expected value of the !.(k)
E{!. (k)!. T (l)} !(k)Okl - covariance matrix for !.(k)
(!.(k) - white noise) k=l Kronecker del ta
kfl
(79)
(80)
(81 )
+ The external properties of a physical system can frequently be characterized by an operator relation of the form
g T f (78)
- 46 -
{z(k)} a zero-mean finite-variance process, i.e.
T ~ E{~(k)~ (k)} < 00 (82)
Let us introduce further the linear least-squares estimate of
what is the meaning of the last, it is necessary to solve the problem of the
optimum estimation and conditional expectation. Also, for the sake of the
innovation derivation we will need a theorem of the orthogonal projection.
To solve the final problem, we will refer to sub-chapters 2.3.1. and 2.3.2.
2.3.1. Optimum estimation and conditional expectation
The innovation approach to multivariable dynamical systems plays a still more
and more important role in modelling identification and the state reconstruction.
+ in which g and f are real n-vector valued functions of t, such that gTf
represents the total instantaneous power entering the system. (For example
an electromechanical system, the jth component oft might be voltage or force
at jth accessible point of the system, in which case g would be the
corresponding n-vector of currents or velocities. The notions of energy and
causality playa central role in the study of physical systems. Let us agree
to say that! is causaZ if for every 0 >- 00, If, and !f2 agree on the
interval (-00,0) whenever the n-vector-valued functions f, and f2 are
permissible inputs to the system fepresented by 1 and fl and £2 agree on
(-00,0). Let us say also that T is passive if for all
real x and permissible f. It has been proven that under some weak assumptions,
if an operator T is linear and passive, it is also causal.
- 47 -
For many other reasons, such as simplification of derivations, equivalence
to the Kalman filter without necessity to solve the Riccatti equation,
easier implementation of the identifiability notion, to mention but a feW,
we will show a very formal derivation of the innovation state space model.
This requires more complicated mathematics and a few notions which may seem
to be complicated at first sight. However, it will profit in deeper
understanding of the more general properties of the state space models and
related to it realizations.
Let {Z(o), X(I) .••••. ~(k - I)} be a set of measured random variables of
y (i). We can determine in principle, the probability of z.(j) (where z.(j) 1 1
is the j ith component of the ~(j)) assuming value Vi or smaller for
i = 1,2, ..... n.
This follows from the definition of the conditional probability function.
Considering a random variable z of a random process, given the actual value
n with which the random variable y has occurred, the conditionaZ p~obabiZity
density function of z, given y is denoted by p(~\n), and is defined by the
following relationship:
In general, the conditional p~obabiZity density of z, given
~(O) = no, ~(I) = n 1
is denoted by p(~\nOl n1···nk _ I)'
~(k - I) = n\l._1
The conditionaZ p~obabiZity dist~bution function of z given
yeo) no, .•..• , ~(k - I) = nl( _ I
(83)
- 48 -
is written
a Tl ) a k -I
(84)
Defining the joint probability distribution fUnction of the random variables
z (j), z (j) ..... Z (j) as the probability of the simultaneous occurrence I 2 n
of:
z.(j) ~ ~ , z (j) ~ ~ •••• z (j) ~ ~n 1. 1 2 2 n
(85)
and denoting it P(~ ,~ , •••• ~ ), and defining the ma~inal probability I 2 n
distribution function of random variables z, y(O), y(l) •••• 1(k- I)
Pr(z ~ ~, Z(O) ~ Tl o ' ••.•• Z(k - I) ~ Tlk _ I) = P(~.Tlo'Tll····Tlk _ I) (86)
the conditional probability distribution function is written
(87)
which is nothing but a Bayesian relation.
Let us use the expression
Z(k - I)} (88)
to describe the probability that the random variable z.(j) will be less than 1
or equal to some value ~. simultaneously for i = 1,2, •.• n, at the time j, 1
given measurements y(O), y(l) •••• y(k - I). This expression of the conditional
expectation distribution function is clearly also a statistical estimate of the
- 49 -
random variable ~(j), given information on ~(j). This statistical estimate
is represented by i(j k - I), which is assumed to be a fixed vector whose
elements are known whenever ~(O), ~(I) ••.•• ~(k - I) are known.
The general estimate i(jlk - I) will be different from the actual random
variable ~(j), which is unknown. For the purpose of comparison, it is
desirable to define an error function vector e,
~(j) = -'!.(j I k - I) + ~(j) (89)
Let us define, according to the statement made,that we are looking for the
linear least-squares estimate of ~(j), the loss function as L(~) = ~T(j)~(j).
We want to show that minimization of the average or expeated vaZue of L(~) is
equivalent to minimizing aonditionaZ expeatation. Given the random variable
vector ~(j) = ~(j) + ~(j), where ~(j) is the actual signal vector and ~(j) is the
noise vector and 1.(0),1.(1) .•••• ~(k - I) are k given measured random vectors.
It is desired to find the best estimate of ~(j) at j ~ k or j < k from
knowledge of the measured random variables. (j > k - smoothing (interpolation),
j = k - filtering, j > k - predicitng).
The problem is to find the optimum estimate z (jlk - I), given the measurement -opt
on ~(O), ...•• Z(k - I), so that the expected value of the loss function L(~)
is minirunized.
E!{~T(j) ~(j)1 Z(O),~(I) '" ~(k-J) }.
!{[~(j) - iOlk - 1)]T[~(j) - i(jlk - I)~Z(O), ~(1) ""Z(k - I)} =
= min (90)
Performing operations under the expectation operator we get
!{~T(j)~(j)1 ~(O)'Z(I) .. ". ~(k - I)} = E{~T(j)~(j)1 1.(0), 1.(0 ... Z(k - I)}-
- E{zT(jl k - 1)~(j)1 1.(0),1.(1) ..... Z(k - I)} -E{~T(j)i(jlk-l) ~(O)'Z(I) .. ~(k-I)}+
(91 )
- 50 -
Since ~(jlk - I) is a known constant vector when Z(O), Z(I) .••. Z(k - I)
are given, we have:
E{l(j)e(j)lz(O) .... Z(k - I)} = E{~.T(j)~(j)lz(O) .. 'Z(k - I)}-
- i(jlk - I)E{~(j)lz(O)""Z(k - I)}- E{~T(j)lz(o),Y(I)""Z(k - I)} +
+ zT(jlk - I)z(jlk - I) - -. (92)
Taking the partial derivative of the last equation with respect to
~(jlk - I) and set it equal to O.
A
d Z.<jlk-1) =
.-a Zn<jlk-1)
d I "'I' " l o ~ljl"-1) l Z lj'k-1)~q'k-1)j =
... :: 2. ~(j' k-1)
(%)
- 51 -
and
(98)
This shows that the optimum estimate of ~(j) in the sense of the minimum
least squares error is the conditional expectation of ~(j).
2.3.2. Optimum estimation and orthogonal projection
Consider the set of random vectors {~(j)} j = 0.1.2 •...• where ~(j) is an
n-vector. The set of all linear combinations of these random vectors of the
form:
i = a A. z(i) -,,-
(99)
(where A. is an n x n matrix with constant coefficients) forms a vector space -1
which is denoted Z(j).
Similarly. we will call the vector space generated by the set of vectors
k - I
L: Y(k - I) ( 100)
i = 0
The dimension of the matrix B. with constant elements is n x q. If k - J ~ j, -1 -
the vector space Y(k - I) is a subspace of the vector space Z(j).
Now let us consider that the random vector ~(j) is composed of two components.
that is:
~(j) = :!(jlk - I) + l(jlk - I) (10 I)
where ~(jlk - 1) is the orthogonaZ projection of ~(j) on the subspace Y(k - I).
- 52 -
The vector ~(j\k - I) is the component which is orthogonal to the subspace
Y(k - I)(i(jlk - I) 1 Y(k - I). that is. orthogonal to every vector in
Y(j). Now we shall prove the following theorem:
Theorem 6 The orthogonal projection ~(j\k - I) of ~(j) is that
Proof:
vector in Y(k - I) which minimizes the loss function
Let ~ be any vector n x I in the vector space Y(k - I).
Forming the conditional expectation
lo(k - I)}
and substituting (101) into (102) we get
Eq~U) - £f [~(j) -~] I lo(O). lo(I) .... • lo(k - I)} =
E{iTUlk - l)iU Ik - I) + iTU Ik - 1)[iU Ik - I) - f] +
+ [~(jlk - I) - ~JTiUlk - I) +
+ [fOlk - I) -~n~.(jlk - I) - §] I lo(O).loW .... x.(k - I)}
Since ~(jlk - I) - ~ is the vector which is in Y(k - I). and
I) is orthogonal to Y(k - I) and therefore also to
i(j/k - I) - ~. equation (103) can be written:
Eq~(j) -~IT[~(j) - ~11x.(0).x.(1) .... lo(k - I)} =
= E(~.T(jlk - I)i(jlk - 1)\ lo(O).lo(I) ..... x.(k - I)} +
(102)
(103)
+ E{["i(jlk - I) - i1T!E(jlk - I) - i]Ix.(O).X.(I) ..... X.(k - I)} (104)
Because the term E{lUlk - l)iUlk - I) I x.(O).x.(I) '" x.(k - I)}
will remain always positive definite. it is apparent that to
=
Remark I:
Remark 2:
- 53 -
minimize (104), we must have ~ equal z(J'\k - I) = Z (J'\k - I) - - -opt '
so that the minimum loss function is:
E{[~(j) - ~pt(j\k - I)1TI~(j) - ~Pt(j\k - 1)]\ ~(O),~(I) ••• y(k-I)}
E{iT(jlk - I)i(j\k - I) I ~(O),~(I) ..... ~(k - I)}
Q.E.D;
Relation (101) is of a great importance for consideration.
It shows that ~(jlk - I), which is an optimal estimate of
~(j), can be expressed as a difference between a fixed vector
(for a measured ~(O), ~(I), ...•. ~(k - I) and the correcting
term i (j I k - I)
i(j\k - I) = ~(j) - i(jlk - I)
To find the correcting term l(j\k - I) is due to the Kalman
filtering theory and the innovational approach. (106) - is
called a "correction equation".
Based on the discussion carried out in these last two sections,
we can now summarize:
(105)
(106)
Z (j\k - I) = OPTIMAL ESTIMATE OF =.(j) GIVEN ~(O) .... ~(k - I) = -opt
Kalman filter approach ~ =
Innovation approach -= E{~(j) I ~(O), l.(I) .... ~(k - I)}
= ~(jlk - I) of =.(j) on Y(k - I)
- 54 -
2.3.3. The discrete-time innovation problem
Let us return to equations (79)(80)(81)(82).
Z(k) = ~(k) + ~(k) K=O,l,2 •••..
E{~(k)} = Q ; E{~(k)} = 0
E{~(k)VT(k)} = ~(k)6kl T tr E{~(k)~ (k)} < 00
Let us define the innovation proce88 by
where as z(k\k - I) = z (k\k - I) - the linear least squares estimate - -opt
(79)
(80)
(81)
(82)
(107)
~(k) given {Z(l) O't~k - I} (108)
Thus we can calculate
E~(k)} = E{y(k)} - E{z(klk - I)} = ~
~(k) = Z(k) - ~(k\k - I) K X(k) - ~(k/k - I) =
~(k) + ~(k) - ~(klk - I) = I(klk - I) + z(klk - I) -
- ~(klk - I) + ~(k) = ~(k) + l(k/k - I)
thus for k > 1
E~(k~1t)} = E{~(k)lU,)} + E{~(k) 1(.9. 19. - I } +
+ E{l(klk - l)lT(9.19. - I)} + E{l(klk - 1)~T(9.)}
= E{~(k)~T(9.)} + E{l(klk - 1)[~(9.) + ~(9.)1T} =
= E{~(k)l (9.)}
k > 9,
the same can be shown for k < ~.
(109)
(110)
( I I I )
- 55 -
For k = £ we have
E~(k)~T(£)} = E{~(k)~T(k)} + 2E{~(k)i(klk-l)} + E{i(klk-l)iT(klk-l)}
E{~(k)~T(k)} + E{i(klk-l)iT(klk-I)}
covariance of the covariance matrix of the error noise in the estimate i(klk-I)
~(k)oik
E{i(klk-l)iT(k\k-I)} = E{[~(k) - ~(k\k-I)1(z(k) - ~(k\k-l)]T)
= P (k) -z
So that~(k) like ~(k) is white but with a different varaiance.
Remark: The ~02:t!n~o~s-tim~case can be approached by a limiting procedure
(I 12)
(I 13)
in which R(k) becomes indefinitely large, while P (k) remains finite, -z
so that the variance of~(k) and ~(k) are the same. (see Kailath T.
(1968).
Let us refer to our initial equation:
and let
~(k + I) = ~~(k) + !~(k)
r(k) = £ ~(k) + y(k)
~(k) = £~(k)
E{~(k) ~T(£)} = g(k)oki; E{~(k)~T(i)} = ~(k)oki
Assuming that the matrix [~(.) + ~(.)1-1 exists, for £ ~ k we get
~(k + I) = ~(k + II k) + ~(k + II k), where ~1!!.
Let an optimal estimate of ~(k + I) be
k ~(k + II k) = L. gi (k)!: (£)
£ = 0
(114)
( 115)
( 116)
( 117)
so that
- 56 -
k ~(k + I) - ~(k+I\k) = L g~(k)!:.(~)
~ = 0
according to the projection theorem, Multiplying both sides of (118) by
/(k) we get:
k T - 4. g~ (k)!:.(~)1!. (k) ~ ·.0
( 118)
( 119)
E{~(k + 1)1!.T(k) - R(k + l{k)~T(k)} = ~{~(k + 1)~T(k)} (120)
according to the projection theorem:
thus
and
k
L ~ D °
k = L: g~ (k) [Kz (~) + !(~)~~I< D
~ D 0
= g~ (k) (Kz (~) + ~(R.)]
(121 )
( 122)
(I23)
Putting (123) into (117) we get
k !.(k + Ilk) = L. E{~(k + 1)1!.T(~)} [Kz(~) + !(~) rl}!.(R.) (124)
~ = 0
Rearranging (124) in the way
k-I R(k + 11k) = L E{~(k + l)iO,) [~(~) + !(R.)rl!:.(R.) +
~=O
+ (125)
- 57 -
and substituting
and k-I E{~(k + l)lU,)} [~(R,) + ~(R.)]-Il:!.(R,) A x(k\k - I) L: =
R.=O
because k-I
l)i(R.)} r~W + ~(J/,)rll:!.(J/,) L: E{~(k + =
J/,=O
=
k-I ~ E{~ x(k)l:!.T(R.) + !u(k)l:!.T(J/,)} [~(R.) + ~(J/,)]-Il:!.(R.) = R.=O
k-I E{~ x(k)l (J/,) [ ~ (R.) + ~(R.)rl T
= L l:!.(J/,) = (for E{~ l:!. }
£=0 k-I
A L: E{~(k)l(J/,)} [~(R.) + ~(J/,)rl!:.(R.) = ~!(k\k - I)
R.=O
we finally get
form (107)
~(k + 1\ k) = ~ ~(k\k - I) + .!S.(k)l:!.(k) }
Z(k) = £ x(klk - I) + l:!.(k)
l:!.(k) = Z(k) - !(k\k - I) - innovation equation
!(klk - I) = ~~(klk - 1)
= Q
Equations (128), (129) constitute the innovation modeZ of the state space •
.!S.(k) is called the Kalman filter gain matrix, and to evaluate it we have to
consider
E{~(k)l (k)} = E~ ~(k) + ! ~(k) J [~T (kl k - I) £T + l (k) J} T T
= ~ E{~(k) ~ (klk - I)} £ +! ~(k) =
= A!(k)~T + !.~(k)
(J 26)
( 127)
)
(128)
(J 29)
(130)
- 58 -
where
~(k) ~ E{~(klk - I) ~T(klk - I)}
using relation y(k) = ~(k) + ~(k) and the orthogonal projection of
~(t) it is straightforward to show
so
!(k) = ~[~(k)£T + l! ~(k)][£ ~(k)£T + !(k)rl
Finally it can be shown (see Kalman R.E. (1960). Falb P.L. (1967).
Kalman R.E .• Bucy R.S. (1961) for example).
~(k + I) = ~ ~(k)~T + Q(k)
~(k)
(131 )
(132)
( 133)
(134 )
(135 )
Equations (128) and (134)(135) define the so called discrete-time Kalman filter.
An innovation approach to least squares estimation of the state has many
advantages and is broadly discussed and used in the modern system theory. Many
identification methods refer to this model, which incorporates less parameters
than ordinary state space model to identify from the same set of input-output
data. (it will be demonstrated in the next chapter). Also some order test
algorithms refer to this method. The complete derivation of the innovation
model was presented in order to introduce the reader to more advanced, however
intuitional1y quite simple, mathematical formalisms, which are not to be
avoided when studying literature on the multivariable system identification.
The following references are also warmly suggested to enlarge experience in
state space models handling: Jazwinski (1970). Kailath T. (1968). Kailath T.
Frost P. (1968). Kailath T. (1970). Kailath T •• Geesey R.A. (1971). Kailath T .•
Geesey R.A. (1973). Eykhoff (1974). Aesnaes H.B •• Kailath T. (1973). Gevers M ••
Kailath T. (1973). Meh,.a R.K. (1971). Akaike H. (1973). Akaike H. (1974a).
Akaike (1974b). Niederli~ski. A. Hajdasinski A.K. (1979) and others.
- 59 -
2.4. Generation of canonical forms from Hankel matrices
State equations for the purely conceptual noiseless system deliver the
already discussed transfer function matrix:
-I !.(z) = ~(z ~ -~) !
where !(z) is assumed to be proper or strictly proper.
According to Schwarz (1971), Ho-Kalman (1966), Hajdasinski (1976)
!.(z) can be described in the form of the exponentional expansion:
!.(z) f= i=O
M. -, i+1
z
where M. are already known Markov parameter matrices -, i M. = CAB - (q x p)
-1 ---
see Ho-Kalman (1966), Gantmache~ (1959).
Alternatively, applying the z-transform to the state and output equations,
( 136)
(137)
(138)
we may describe this system by means of the so-called "weighting sequence".
(See Hajdasinski (1978».
with B 13 = x - "-Q -0
= C Ak x (0) +
k-I
=~i!.o + 2: i=O
M. u(k - i-I) -, -
M. u(k - i-I) -J. -
=
(139)
It is very easy and intuitievely simple to define the equivalence of dynamical
systems in terms of Markov-parameters.
Definition 15 Two dynamical systems
are said to be equivalent if and only if their Markov-parameters
fulfill the following condition (Ho-Kalman (1966»,
for i = 0,1,2,3, ..... (140)
- 60 -
With the knowledge of Markov-parameters it is possible to construct the finite
Hankel matrix Hr - (qr x pr), which is a submatrix of the double infinite
Hankel matrix form relation (8).
M M ...... H --0 -1 = -r
M M ...... -1 -2
M M -r-1 -r
M r-1
M -r
M -2 r -2
It is very easy to notice that Hr can be expressed as a product of the
observability and controllability matrices (see Ho-Kalman (1966».
W = L~; ~ ~! ~2~ ••.• ~r-1~ 1 v
H -r
[~T i ~T~Ti (~T)2~T .•••• (~T)r-1l]
= C A B
C A
C :. ....
observability Matrix
(141 )
( 142)
( 143)
(144)
From (144) it is obvious that a row (column) of ~r is dependent if and only if
it corresponds to a dependent row (column) of the observability (controllability)
matrix (see Niederlinski, Hajdasiii.ki (1979». This means that there exists
equivalence between canonically observable and canonically controllable forms
(described in 2.2.1. and 2.2.2.) and forms which can be generated from the
Hankel matrix. This is stated in the following theorem by Candy, Warren and
Bullock (1978).
Theorem 7
- 61 -
1. If the rows of the Hr matrix are examined for predecessor
independence, then ~th (dependent) row, where ~ = i + qv., 1
i = 1,2, .... q is given by
l-.f rnin(Vi ,vj-d
L L a.'i,\( yj .. ",k
i"~ k~O
where a .. k and v. are invariant parameters for canonically 1. J , l.
observable couple (~,f).
2. If the columns of the Hr matrix are examined for predecessor
independence, then the ~th (dependent) column, where
~ = i + pn., i = 1,2 •.• p is given by 1
-l-1 W\i~(V\~,nj-d
= I. L Q'j,d j ... pI( 1-1;1 '.0 -
where a .. k and n. are invariants for the canonically controll-1J , 1.
able couple (~,~),
Candy, Warren and Bullock (1978) present an algorithm for the transformation of
the Hr matrix to a form enabling the simultaneous determination of the v. and 1
n. invariants by inspection. This algorithm may be treated as the determination 1
of the canonical structure of the cenceptual, nondist~bed system, having the
advantage that both sets of Kroneck~indices are formed in parallel.
An infinite number of canonical forms can be derived from the Hankel matrix ~r,
depending on the decomposition of the latter. No completely satisfactory
research has been made in order to reflect the hidden possibilities of the
Hankel canonical forms. As an illustrative example the minimaZ reaZization
algorithm by Ho-Kalman will be presented as a method using a "general"
decomposition of the Hankel matrix and the minimal realization algorithm
- 62 -
with use of the singular value decomposition as an illustration of a
particularly interesting decomposition of the Hankel matrix, giving
possibilities for interpretation of the noisy experiements in view of
least squares fit to a given data set.
3.4.1. The Ho-Kalman minimal realization algorithm
The minimal realization problem is referred to as finding the triplet of
matrices {~,~,~} using the external data like input-output signals or the
transfer function matrix. This problem is very well known in the system
theory, and quite extensively worked out at present. There is a huge number
of minimal realization algorithms, but still the originelwork by Ho-Kalman
(1966) shows the majority over another method. Let us present the Ho-Kalman
algorithm in the form of the following theorem:
Theorem 8 For an arbitrary, finite dimensional, linear, dynamical system,
given the input-output map, the canonical realization exists in
the following form:
Let q - be the number of outputs
p - the number of inputs
n - dimension of the realization
I. There is defined the following matrix:
k x J/, matrix [~~-k ] if k < J/, EJ/,
k J/, matrix
[~J (145 ) =k x
if k > J/,
k x k matrix [~ ] if k J/,
- 63 -
2. Choose r such that the relation
M • = -r+J
for all j ~ 0 (146)
holds.
3. Find a nonsingular matrix P(q x q ) and a nonsingular matrix - r r
where
P H Q --r-
H = -r
Q(p x p ) such - r r
=
Ik -"k
Opr-n
n
On Opr-n
qr-n qr-n
M -0
•.•.••..• •• M 1 -r-
MI' ••••••••• M2 -r- - r-
=
oH = -r
En qr
Epr
n
~l ••••••••.•.
M ••••••••••• -r
4. A canonical realization of the considered system is given by:
qr A = E P(OH).9, En
--n - -r ."r
B = Eqrp H EP --n --r."r
C =
M -r
(147)
shifted (148) Hankel Matrix
M -2r;'1
(149)
- 64 -
The proof of this theorem can be found in Ho B.L., Kalman R.E. (1966),
Kalman R.E., Arbib, Falb (1967), Schwarz H. (1971), Hajdasinski (1976).
In the Ho-Kalman algorithm the Hankel matrix decomposition is non-unique
and depends on the choice of the transformation matrices P and ~. Kalman
(1967) suggested the upper and lower triangular structure for the (!, ~)
or (~,!) respectively. For this sake the Andree (1951) algorithm was
perfectly suited. But it can be shown (Hajdasinski (1976) that by a different
choice P and ~ the phase-canonical form of the state equations can be achieved.
However, in general, when state variables are not well defined, it is not possible.
3.4.2. The mini~al realization algorithm with the use of Singular Value Decomposition of the Hankel matrix
The following decomposition of the Hankel matrix will be called the Singular
Value Decomposition (s.v.d.) (see Hajdasinski - Damen (1979)):
where
u - (r~ x n)
V - (r p x n)
H = -r
D is the n x n diagonal matrix
D = diag. (01. 02 ••••• • ,8 ) n
cr., for i = 1,2, ..... ,n are called the singular values 1
matrix consisting of n orthogonal columns u .• J
UTU I --n
matrix consisting of n orthogonal columns v j ,
VTV I --n
(ISO)
(151 )
It can be easily demonstrated that the Moore-Penrose inverse of Hr given the
- 65 -
s.v.d. (150) is:
as the other hand from (147) we have.
P H Q = En Epr - --r-" -qr ~
by definition P H HtH Q J - -r -r--r-"
P H Q i; P H Q --------r-~ '---r-"
J J
thus
Comparing (152) and (155)
=
v
J
J
A canonical realization of the considered system is given by
A D-IUT(OH )V - - ---T-
B = D-IUTU D VTEP = VTEP -----pr pk
C =
( 152)
(153)
(154)
( 155)
(156)
( 157)
(158)
Once the (s.v.d.) of the ~r is given, the realization (158) is unique. This
canonical form plays a dominant role in indentification of both the order and
parameters of the noise corrupted dyncamical multivariable systems, and will
be more broadly explained in chapters 3 and 4.
- 66 -
More facts about the realization and minimal realization theory employed for
the multivariable system identification are to be found in: Ackermann J.E.
Bucy R.S. (1971), Ackermann J.E. (1972), Anderson B.D.O. (1977), Anderson B.H.,
Brasch F.M. Jr., Lopresti P.V. (1975), Audley D.R., Rugh W.J. (1973a), Audley
D.R., Rugh W.J. (1973b), Barraud A., de Larminat P. (1973), Bingulac S.P.
Djorovic M. (1973), Bingulac S.P. (1976), Budin M.A. (1971), Budin M.A. (1972a,
1972b), Dickinson B.W. Morf M. Kailath T. (1974), Dickf.nson B.W. Kailath T.
Morf M. (1974) Faurre P., Marmorat J.P. (1969), Faurre (1976), Furuta K. (1973),
Gerth W. (1971), Gopinath B. (1969), Gupta R.D., Fairman F.W. (1973),
Hajdasifiski A.K. (1976), Hajdasifiski A.K. (1978), Hajdasifiski A.K. (1979),
Hajdasifiski A.K. Darnen A.A.H. (1979), Ho-KaZman R.E. (1966), Kalman R.E.(1963)
Kalman R.E., Falb P.L., Arbib M.A. (1969), Lal M. Singh H., Parthasarthy R. (1975),
Mayne D.Q. (1968), Rissanen J., Kailath T. (1972), Rissanen J. (1974), Roman J.R.,
Bullock T.E. (1975), Rossen R.H. Lapidus L. (1972), Rozsa P. Sinha N.K. (1974),
Shmask Y. (1975), Shiek L.S. (1975), Silverman L.M., Meadows H.E. (1969),
Silverman L.M. (1971), Sinha N.K. (1975), Sinha N.K. Sen A. (1976), Rozsa P.
Sinha N.K.(1975), Tether A. (1970), ThamQ.C. (1976), Zeiger H.P. McEwen J. (1974).
Example 5 Let us consider a two input, two output dynamical system, whose
realization {!,~,!! } is:
0 -I -I 0 -I
0 0 0 0
[: 0 0
:] -5 -I -2 G = H 0 F = -2 -3 I -I 0
0 0 0 0 0
-I 0 -2
constructing
H =
[~ 1
- 67 -
where h = -,
o o o
h -2
o o o
o
and performing the test for linear independence according to relation (52)
we get
v = 3, v = 2, V -1 2 12
v = 3 2 ,
so the transformation matrix TT will be (see (54»
FT 0 0 -2 0 -1 (!.T) 2
0 -5 0
-1 -3 0
-1 -1 0 0
0 0 -2 -2
thus
TT = 0 0 0 0 T =
0 0 0
0 0 0 0
0 0 0
0 0 0 0
2 -2 8 -1 0
5 -5 11 -8
4 -3 4 0 -3
2 -1 0 0
-1 9 -2 2
0 0 0
0 0 0 0
0 0 0
0 0 0 0
0 0 0 0
- 68 -
-I -I 0 0 0 T =
0 0 0 0
0 0 -I 0
0 0 0 0
0 0 0 0
Now the canonical form may be completed:
0 0 I 0 0 A I A
A = T F T- I I -11 I -12 I = ,
--- 0 0 I 0 0 I I I (see (58), (60»
-2 -3 -3 I 2 -I ---+----I ----------+------ I I
0 0 0 I 0 I I I I A I A I
-I 2 \-1 -2 -21 I """"'22 I
B = T G • o o
-I 0
o
o o o =
(see (61» o o
The realization {~,!,£,} is a canonically observable realization of the considered
system, and is equivalent (in the sense of Definition 15) to the realization
- 69 -
Example 6 Let us consider a two input, two output dynamical system,
characterized by the following series of Markov-parameters:
{M.L -1 1 0, ) ,2 ....
={[I.O 0.0
M -0
o.oJ; [0.8
1.0 0.0
0.21 ;[0.64
0.6 0.0
0.28];[0.512
0.36 0.0
0.296];
0.216
[0.4096
lO.O 0.28 1 0.129~' [
0.32608
0.0
0.24992 1;
0.07776 [
0.260864
0.0
0.215488) ••••• }
0.46626
M -. M -5
M -6
I. Applying the Ho-KaUnan.algorithm we will derive the realization
It is sufficient to consider H for -1
det H = I -1
det H = ° -,
rank H 2 -1
rank H = 2 -,
thus the system is of the first order and has the dimension n = 2. In such
a case it is trivial to find ~ and g matrices, which are:
P a I = [:' :,1 P-2
.9- a I = [:, : 1 q-2
q
and a = p
aq
- 70 -
The Ho-Kalman algorithm (149) provides us with
A = a (oH) a = aH p -I q -I
B a H = a I P-I p--,
c = H a -I q
a I q-2
~I = [0.8 0.2]
0.0 0.6
Eigenvalues: A 1
0.8
A - 0.6 2
Transfer function matrix of this system is:
!5.(z) -1
= ~(!z -~) ! = 0.2
(z - 0.8) (z - 0.8)(z - 0.6)
0.0 (z - 0.6)
II. Applying the s.v.d. realization algorithm for example for H • we get -. a following set of singular values:
0 = 2.5701194 1
0 = I. 4734138 2
0 = 8.6947604 . 10- 12 ;; 0 3
04
= 0
thus if it is clear that the system dimension ~ = 2, r - order = I.
Set of equations (158) provides us with:
~
[ 0.834575 -0.068194 ] Eigenvalues: A ~ 0.8 A =
-0.011896 0.565417 A = 0.6 2
'il' [ -0.568066 -0.259609 ]
-0.288067 0.772964
c = [- 1. 5041 70
-0.560574
- 71 -
-0.505194]
1.105450
Realizations {!,!,£} and {!,!,~} are equivalent in the sense of Definition 15.
Realization{!,!,f} delivers exactly the same !(z) as the realization {!'!'£}.
The great advantage of the s.v.d. realization is its uniqueness, and an easy
test for the system order.
-72-
3. Identification of the structure of the multivariable dynamical
systems.
The whole chapter will be based on the definitions 13 and 14, and on
structural properties of dynamical models for MIMO systems as described
in chapter 2. The task of the structural identification it is to determin
the suitable "comple~ity " of the chosen model. Thus in view of definitions
13 and 14 it will be determination of the order -r , or ( and) the minimal
dimension - n of the considered system ( MIMO ).
Such a posing of the problem is possible only for some strictly conceptual
systems having both a finite order and a finite dimension. In the real ,
noisy systems identification, however, one cannot search for any exact rand
n , because due to the noise these are systems of infinite order and dimension.
The only goal which we can aim at is to find a reasonably simple, and resulting
in good outputs of the model, a finite dimensional approximation of the real
system - i.e.to find finite estimates of the rand n.
3.1. Estimation of structural invariants - Guidorzi's method
The Guidorzi's method is based on the canonically observable form
of the state equations - see 2.2.1 , relations (56),(57),(58),(59),(60),(61),(62).
According to Guidorzi, the structural identification is defined as:
Definition 16 The structural identification of a multivariable system it is
the determination of the set of integers VI •••• V ,( Kronecker q
invariants) defining the structure of the couple ( !, ~ ) from
input-output relations, without the intermediate construction of
a parametric model.
-73-
Thus the Guidorzi's method requires the input-output description of the
dynamical system, which would use the Kronecker invariants.
(159)
where
X.T(k) [ y 1 (k) , y 2 (k) , ..• , yq(k)l
.!!.T(k) [ u 1 (k) , u2
(k) , ••. , uq (k) 1
~(z) [ f" ,., ... f""'] Q(z) = [ j" 'd ... 1""'1
Pql(Z) Pqq(Z) qql(Z) qqp(Z)
From the j-th component of the canonically observable form it can be written:
( x(k).d~f x.(k) ) 1 1
v. 1+1) r
v. 1+2) J-
= y. (k) J
zy. (k) J
2 Z y. (k)
J
T - zb - (Vl+ ••• +V. 1+I)u(k) r -
...............................................................
V. ) J
... + v. I)u(k) -r -
v. 2 b T (k) - z J - _ (v 1 + ... + v. + 1 ).!!. J-I
Thus the canonically observable state equation can be rewritten as:
(160)
(161 )
where
::. (z)
max i
1 •••••••• a z •••••••• a 'VI-I
z •••••••• 0 o 1 ...... 0 Oz •••••• 0 · ........ . · ........ . o ........ 1 • •••••••• z
, 'v -I O ••••••• z q
( v. ) L
-74-
w =
1(z) =
o 4 ••••••••••••• 0 ••• 0 ••• O. 0
!?.r 0 .•....•......•....••. 0
T ' T b. O I .... b l 0 ....... 0 O. 0 -Vl- -
o •••.•..•••...•.•••.•.••. 0
bT ~-'V + 1 0 •••••••••••••••• 0
q
'T b I •••• -<1-
T ' b -<1-V +1
q
I
zl T
o ....... 0
Finally the relation between ~(k) and ~(k) is found to be:
Comparing coefficients of (159) and (162) we obtain
p .. (z) LL
p .. (z) LJ
v. L = z
- a.. z LJ , v, .
LJ
v .. -1 LJ
and for q .. (z) it is necessary to construct a matrix ~ = M B LJ
( 162)
( 163)
(164)
where
'" B
M =
-75-
-a ll ,2 -a ll ,3 - ...
-a I I , V 1
I
-aql ,2 ..• -aql,v I • • q
-a
-a
o
o o
q 1, V I o .... ~ ............. 0
o •...•....••....... 0
and comparing coefficients we get:
q .. (z) 1J + •.. +
II, VI
...
•• , +
-a lq ,2 .•. -alq,VI . q
o
-a· . . . . . . . . . .. 0 I q, vI o q
o .......•.......•. 0
-a qq,vql
.........
J ••••••••••••••••••
(165)
(166)
From relations (162),(163),(159) it is seen that the set of indices
V , v, .•. , V can be deduced by inspection or from the knowladge of A or 1 2 q
of K(z); also from the parametric standpoint! and !(z) are equivalent.
Matrix C can be directly written if the Kronecker invariants are known. See
also Guidorzi(1973),Bonivento c,Guidorzi R(1971), Bonivento c,Guidorzi R (1972).
Guidorzi considers the matrix of input-output data given by:
-76-
YI (k) Y I (k+ I) Y (k) Y q (k+ I) ul
(k) ... u (k) '" q p
Y I (k+ I) YI (k+2) Y (k+ I) q Yq (k+2) ul
(k+I) .. u (k+I) •• p
Y I (k+N) Y I (k+N+ I ) •. Y (k+N) q yq(k+N+I) .• ul
(k+N) •• u (k+N) •• p
= [2:.1 (k) 2:.1 (k+ I) ... 2:.1 (k+N) , 2:.2 (k) I.2 (k+ I) .. 'lq (k) ... ~I (k) •. • ~(k) ... ] (167)
From relations (162),(161) and(166) the following relation between y.(k) and -1
u.(k) can be derived: -1
v . q S1
Y (k+v ) = ~ ~ a .. y.(k+j-I) + s s . 1 • I S1,] 1
1= J=
and V = \) ss s
v
!:-E i= 1 j = 1
a(V + •• • +v I+j)ui (k+j-I) I s-
Relation (168) shows clearly the linear dependence between present and
former samples of outputs and between present outputs and former inputs.
If we look closely to the relation (167), this property may very easy be
forecasted. Each subsystem in(167) is the Hankel matrix and relation (168)
has the form of the realizability criterion. It first is due to Guidorzi
(1971),that this property has been discovered letting determination of
VI ••••••••• Vq by selecting of nonsingular matrices being products of
these output-input vectors :
(168)
L. (y.) -1 -J
- i elements (169)
L.(u.) = [u.(k) ..• u.(k+i-I)] -1 -J -J -J
- i elements (170)
Constructi~g following! and ~ matrices:
R (8 1,8Z, ... ,8 ) p+q
-77-
(171)
(17Z)
d im{ S (81
, ••• , 8 )} q+p (81+ 8Z+ ••. + 8q+p )x(8
1 + 8Z + ••• +8q+p )
(173)
To complete the structural identification we need to built up a sequence
of increasing dimension matrices :
l(Z,I,I, ••. ,I); l(Z,Z,I, ••. ,I); •.• ; l(Z,Z, ••• ,Z); •.•
and selecting nonsingular ones, find indices v. while a singular matrix is 1
found. Suppose S(ll,lZ, ..• ,l ) is a singular matrix and let 1. is the index - q+p 1
increased by one , with respect to previous nonsingu1ar matrix in the sequence.
Then v. = 1.-1 and v .. = 1. for j = I,Z, ••• ,q for i ; j • This ends the 1 1 1J J
structural identification.
The Guidorzi's method it is the first coherent approach to the structural
and parametric identification. For 11 really" multi variable systems with a high
number of inputs and outputs, however, the selecting procedure for S matrices
increases complexity of already complicated algorithm. In noisy cases the
structural identification produces higher dimensions then the realizations
based for example on the Hankel model or on partial minimal realizations - see
Tether(1970), Anderson(1977),Roman,Bullock(1975).
Also for noisy cases the Guidorzi's method requires a priori knowladge of
a covariance matrix of the noise. In noisy cases the structural ( and parametric)
identification , according to Guiclorzi, must be performed on the base of
following relations :
Y!"(k) = y. (k) + dey. (k)) J J J
(174)
-78-
'* u. (k) J
u. (k) + d(u. (k» J J
(175)
where 1*) are for the noisy signals.
Assuming that the covariance matrix of the noise vector
N (d) = prob lim..l..( !!, !!,T) (176) N->oo N
it is possible to find
and
... R
T R + n
s .. = (t)T(~*)
(177 )
(178)
Thus for the structural identification we need the II compensated estimates "
of Rand S Assuming that input and output noises are II zero mean " white
noises and that ~(d) - the estimate of ~(d) can be found, the compensated
estimate of S is
(179)
Discussion. The main difficulty for application of this method lies in the fact ,
that practically ~ will never be a singular matrix and the
structural identification will become very unreliable and tedious , unless
the additional constraints are imposed on the problem. It also excludes the
existence of another essumption about the noise nature which were previosly
made.
-79-
3.2. Order tests based on the" innovation approach" - Tse-Weinert's
order test.
where
Tse and Weinert (1975) started from the following model
~(k+l) = ~~(k) + ~(k)
Z(k) = £~(k) + ~(k) (180)
~(k) and !(k) are zero-mean Gaussian noises
with covariances:
E{_W(k)_w(j)T} - W 8 . - kJ
It means that there is considered a multivariable dynamical system for which the
input signal is stabilized. and we observe deviations from the steady state
caused by the state noise - w(k) • Unknown parameters are {x .n.A.C.W.V.D.}= {y}. o -----
The objective is to estimate {r}. using output data {!}N= {Z(I).Z(2) •...• Z(N) }.
As it is seen, a part of the objective is to estimate the system dimen8ion 0,
( called by tse and Weinert - the order of the MIMO system ) and only this part
of the algorithm will be discussed here. The set of parameters {y} is quite
extended and requires a large amount of data. As it is pointed out by Tse and
Weinert. {y} may not be identifiable and there fore the innovation reprezentation
is proposed :
supposing that
-x(k+ I , k) - is the predicted state vector ( the conditional mean at time -
k+l. given the estimate ~(k) at time k )
~(k+ I) - corrected state vector at time k+1
K - steady state Kalman filter gain
we get
-80-
-x(k+llk) = ~ ~(k) - one step prediction equation (181 )
~(k+l) = ~(k+llk) + !(L(k+l) - £~(k+llk» - the correction eq. ( 182)
L(k) = £ ~(klk-I) + v(k) - output equation (183)
~(k) = L(k) - £~(klk-I) - innovation equation ( or " zero mean (184)
innovation process)
From equations (181) ~ (184) we get
~(k+llk) A x(klk-I) + ~!~(k) ( 185)
y(k) = £ ~(klk-I) + ~(k) (186)
The set of equations (185) and (186) is called the innovation model of the dynamical
multivariable 8Y8tem. ( see references to chapter 2.3.3. of this report)
Additional assumptions are
A - is stable
(~,£) - is observable
(~,!) - is controllable
9. - is the unknown covariance matrix of ~(k)
B = A K - is the optimum gain
-dim {~(klk-I) } - is finite but unknown
The canonically observable form of (J8,;) and (186) is applied. If vi again
denote the Kronecker indices such that
q
LVi = n
i= I (187)
-81-
from the observability matrix and from the definition of the Kronecker indices
there follows already described the observable canonical form (56),(57),(58)-(62),
and this implies an existence of a unique set of {a .. k} such that for i = 1,2, •• q 1J.
T vi c. A -1 -
T c. -1
= i v.-I ~~ j=1 k=O
T k a .. k c. A 1J, -J-
c: is the i th row of C matrix. -1
if
if
Introducing the covariance matrix of states - S , -ot
v. > 1
o
v. = 0 1
- - - -r T T T { ~(k+I' k)l(k+llk)} = ~{~(klk-I)~\k(k-I)}~ + ~h:.(k)~ (k)}! +
( 188)
(189)
(190)
+ M ;(klk-I)l(k) }!T + !{~(k);h\k-1)}~T (191)
Taking expectations of both sides of (191) we get
S = A S AT -x --x- + ( 192)
for ~(k) was assumed to be a zero-mean noise
Defining
(193)
and considering
X.<k) £ ~(klk-I) (194)
-82-
finally we get:
!(O) = c s CT + Q (195) --oc-
!(a) = C Aa- 1S ( 196)
s = A S CT + !Q ( 197) --oc-
denoting as r .. (a) the ij th element in _R(a) , and as s. the j th column ~ ~
in S , using (188) from (195),(196) we get :
r .. (v. + , ) 1J 1
where T = 1,2, ...
v. 1 T A ' A'- = c. S.
-1- - -J
~~ 1=1 k=O
i-I v -I
2: ± 1=1 k=O
Basing on (196) and (198) we get:
r .. (v.+,) 1J 1
Thus for i=1
defining
T !.I
v -1 ~.J-. ~ ~ ail,k r 1j (k+,) 1=1 k=O
v -I ± all,k rlj(k+T) k=O
T k ,-1 a' l k c1 A As. 1, - - - -J
T k ,-I a' l k c1 A A s. 1., - - - -J
v. > 0 1
v. = 0 1
T = 1,2,3, •...
v.> 0 1
(198)
V.= 0 1
( 199)
(200)
-83-
rlj(l) r lj (2) r Ij (k)
r lj (2) r Ij (3) r lj (k+l)
!.I (k) (20 I)
r I j (k) r lj (2k-l)
Remark:
This is worth a notice, that rlj(cr) are the Markov parameters of the I st
subsystem in the canonical repre.entation, treated as the multi-input/single-
output subsystem. Thus (200) is merely the peaZiaabiZity aPitepion for the
first subsystem, and (201) is the Hankel matrix of the first subsystem. In such
a case the Hankel model can be applied :
(202)
Remark:
The Tse-Weinert's order test is nothing but a Hankel matrix determinant test
which will be described for a more general case later in this chapter.
If d I (k) def I det !.I (k) I =
t d I (k) > 0 for k = J,2,3, .•. v1 (203)
d I (k) 0 for k > vI
thus if !(cr) were exactly known, vI could be found by testing dl(k) for
k=I,2,3, .•• until dl(i) = 0 and then vl=i-1 ,which completes the order test.
Since only L! J N is avaliable ,!(cr) must be estimated the following way :
!(cr) = -N-
N-cr ~ ~(k+cr) ~T(k) k=1
(204)
-84-
~(cr) is a strongly consistent estimate of ~(cr) for a stable A and N+oo
Replacing !I(k) by !I(k) • we can perform the order test.
If the first sharp decrease in dICk) occurs at time k = k* • then vI is
chosen as vI = k*-I ( in the original paper by Tse - Weinert (1975) it is
vI = k* • but there is a different meaning of what is the" point where the
sharpest decrease occurs II In Tse and weinert's work it is always one point
before the decreased value of dICk) occured). ~
d,l"')
I --- -,---, ____ + __ ...l __
, ' , I I , , ,
I , , , , I I
-:... -=--..:-..:.=:::-=.-_ -:. T"_-=-=, -:.. __ ~ 1 :z. 3 ... I
Behaviour of the dICk)
6
The estimate of ~I is found from
~I
According to the order test criterion
k* = 4 => v = 3 1
(according to Tse-Weinert
k* = v = 3 ) 1
(Z05 )
(206)
~l is a strongly consistent estimate of ~1 ' provided vI = vI .For i=2.3, ... q,
v. and S.are computed in analogous manner. For i=Z. T = 1.2 •... (VI+VZ) 1 -1
E..2
.!2(k)
-85-
: r 2 ' (I) ... r2' (k) • J J • :r2 ,(2) ... r 2 ,(k+l) ___________________________ + __ 1 __________ 1 ________ _
r1j(v1+l)
r1j
(v1+2)
r 1j (2v 1)
r 1j (2v t 1)
• • • • · ' • • •
• • • • r 1j (2v
1+k-l) :r
2j(v1+k) •• r
2j(v
1+2k-l)
and again for the estimate .!2(k) of !2(k) we get:
and v2 is found testing d2
(k).
Remarks:
(207)
(208)
The method by Tse and Weinert is a combination of the innovation approach to
the state space with the canonical reprezentation of the innovation model,
which decomposes the original model into q interconnected subsystems having
a structure quaranteeing the complete observability of the j th subsystem
from the j th output component. Each of such subsystems is driven by the white
noise. The stochastic realization theory is applied to determin orders of
subsystems ( orders understood as dimensions)
The method is more convincing then the Guidorzi's method and delivers
very good results of the structural test.
The concept of the stochastic minimal realization of the sequence of the output
autocovariance coefficients was proposed by Rissanen(1974).
-86-
3.3. Order tests proposed for the transfer matrix model of the MIMO system
Furuta's approach.
The main idea of the Furuta's approach is to identify coefficients
appearing in the transfer matrix of the considered system, assuming a certain
degree of the Common Denominator of the transfer matrix, This identification
is performed employing the criterion function J.
J =
where
-t. i=1
r .u(k-i) -1-
(209)
~(k) = ~(k) + ~(k) - (p x 1) dimensional input measurement (210)
vector.
2(k) deterministic input (p x 1)
!'.!(k) stochastic input (p x 1)
E (~(k» = 0 (211 )
~(k) = x,(k) + !!.(k) - (q x 1) dimensional output measurement (212)
vector
x,(k) - deterministic output (q x 1 )
!!.(k) - stochastic output (q x 1 )
E (!!. (k» = 0 E(!!.(k)!!.T(t» = RI c (213) -- kt
E(~(k)!!.T(t» 0 (214)
Furuta assumes further the following model of the transfer function matrix
!.(z ,~)
Q(z,~)
r. -1
T ~il
-87-
----'-_ [ Il z -I +
Q(z,~)
+ ••• +
(q X p) matrix ~ .. 1]
(q x I) vector
(215)
which corresponds with (209). Using (215) and (209) the vector of parameters
is identified minimizing J with respect to ~.
The parameter estimation procedure is described in Furuta(1973). Assuming
the transfer function matrix has already been identified, the nonminimal
realization of the transfer function matrix can be found as :
~(k+1 ) = ~(k) + £':: (k)
Rn (216) I.(k) = ~(k) ~(k) E
where 0 •..... 0 -a I
R,:;. !~ I -I
F = G = b 0
0 .... 0 I -a I II
1-
H = [Q " .. . ,Q, .!.J
This is a very well known Frobenius canonical form ( see for ex. Hajdasinski(1976»,
and it is seen Q(z,~) - is the anihilating polynomial of F. From this form
Ho and Kalman(1966) extracted a controllable and observable realization,
-88-
called the minimal ~ealization. It is based on the Hankel matrix, which can be
derived as the product of the controllability and observability matrices.
w
(217)
H = [vT w1 -n --
(218)
Furuta introduces the concept of ~ - practical controllability and
observability, which gives a solution to the order test and further to the
£ - minimal realization :
Definition 17 The state x of the system (216) is said to be ~- practically
controllable ( observable ) if ~ is element in the space
spanned by the eigenvectors of ~T(yyT) corresponding to the eigenvalues
larger then ~,W and V denote controllability and observability matrices.
Remark It is easily seen that seeking for eigenvalues of the ~~T(~ ~T)
matrix is equivalent to the diagonal decomposition of the HT matrix, ...., which is only slightly different from the singular value decomposition of the
Hankel matrix.
Definition 18 The system represented by the states which are ~- practically
controllable and observable is said to be ~- practically
minimal realization.
The order test is made by comparison of eigenvalues of the ~~T , and
truncation of the selection matrix consisting of normal eigenvectors of
~ ~T, such that consists only of these eigenvectors which correspond to
eigenvalues larger than c.
-89-
w.(i =I, ..• ,n ) - normal eigenvectors of W WT -1 0
If £- is chosen as
v > £ > vn +1 n 0 0
where
n 0
2: v. 1
i= 1 -= n
L: v. 1
i=1
v.( i = I, ••• ,n) - eigenvalues of W wT 1
(219)
(220)
(221)
then the order test is considered as compl.eted, and £ - minimal realization
of the system is found as :
{ F -0
Remark
G -0
H = H s } -0
(222)
This refers to the notion of the numepiaal pank of the Hankel matrix
which will be discussed while considering the 8inguZaP value deaompo-
8ition of the Hankel matPix.
The Furuta's method is rather inefficient and incorporates an unnecessary step
of the transfer matrix identification with arbitrarily chosen ~. This method,
however, pointed out the way to the more advanced methods of the approximate
minimal realization and to the eigenvalue problem in determination of the system
order.
-90-
3.4. Miscellaneous order tests. The pattern recognition method by
Thiga and Gough.
The approach presented by Thiga and Gough is direct in the sense the
model parameters are not estimated at each 'step. The test is based on
a measure of the linear dependence(op independence) of features displayed by
each model opdeP.
This method is totally empirical and assumes identification of the order
(understood as a degree of the transfer function denominator) of individual
subsystems, such that the final system is described in form of the transfer
function matrix.
G = [gij 1 i=1,2, ... ,q j = 1,2, ... ,p (223)
where g .. correspond to the linear differential or difference equations 1J
n. m n.> m i=I,2 ... q
L: L: 1
d.ky. (t) c j J1.U j (t) 1 1
k=O J1. = 0 d ik " o j=1 ,2 ... p
and d ik and c j J1. are differential operators
or
y. (k) 1
9.=0
b. o u.(k-£) -J'" J
J1.=0
a. o y.(k-£) 1", L
k 0,1,2, ... , N
The order test is completed when all subsequent n. are found. 1
(224)
(225)
Authors proposed to solve the order recognition problem by testing a single-
valued function of pattePn, which will reflect a decision surface f(x) = £,
-91-
where E - is the treshold of pattern recognition. For the sake of the order
test for a noisy system, such a function must serve two following tasks :
I. help to spread the cluster of the patterns in the two classes further
apart in the pattern space.
2. reduce the dimensionality of the pattern space by combining dimensions
The input-output cross-correlation function is proposed as a one for the
training of the system :
R (k) uy =
T
M ± j=1
u(j) y(j + k) (226)
Authors are reporting quite a succes in the order discrimination during an
experiment carried on for a set of II representative systems " chosen from
amongst the vast possible combinations of characteristic roots of low-pass
filters, up to seventh order. The machine was learned to extract from such
patterns purported characteristics of a given system order and type of roots.
This type of approach was tested with different types of processes proving
its usefulness.
3.5. Akaike's FPE ( final prediction error) and AIC ( Akaike's maximum Information
Criterion) as order tests for MIMO systems.
Akaike has proposed two new approaches to the order determination and
consequently parameter estimation. These two methods - FPE and AlC are
assymPtoticall~equivalent in final results, as the distribution of the equation
error (for the AR model - considered as the prediction error ) converges to the
Gaussian one.
These two methods employ much more statistical properties of given measurements,
than some 11 a priori II assumptions about its nature . However one general
-92-
assumption, which in practical cases almost always holds, is the assymptotic
Gausseness of the time series samples • To get some more insight into these
two methods, let us study some important passages in their derivations, however
no claim for completeness is being made.
3.5.1. Statistical predictor identification - Final Prediction Error
Approach.
Let us consider first a single input-single output system and assume that the
output of this system is a stationary and ergodic process Y(n). In practice
Y(n) is given as a function of the recent values of Y(n) and the the structure
or the parameter of the function is determined. There is considered the situation
where the structure is identified using an observation of a process X(n) and,
using the structure, the prediction is made with another process Y(n) which is
independent of X(n), but with one and the same statistical property as X(n).
The FPE is defined as the mean square prediction error
FPE of X(n) - 2 = E {(X(n) - X(n)) } (227)
for X(n) being the predictor of X(n)
When the process X(n) is stationary and the predictor Y(n) of Y(n) is linear
and given by
M
Y(n) = ~ ~(m) Y(n-m) + ~(O)
m=1
where ~(m) is a function of X(n)
Using (227) and (228) we have
FPE of ~(n) = a2(M) +
M ~ ~=O
(228)
(229)
where
(i(M)
-93-
M
E {(yen) - L: "M(m) Y(n-m)
m=1
2 - "M(O) ) }
where "M(m) denotes the member of the set of parameters {a(m)} giving.
in the sense of mean squares, the best linear predictor ie.
M
min E {(yen) - ~ a(m) Y(n-m) - a(0»2}
I a (m) ( m=1
ll"M(m) ~(m) - "M(m) m 0.1.2 .... M
and
VM+ 1 (i.m) = E(Y(n-i)Y(n-m» i.m 1 ,2 , .•• M
VM+1(0.m) VM+ 1 (m. 0) = E(Y(n» m 1,2, ••. M
VM+1(0.0) =
(230)
(231 )
(232)
(233)
(234 )
In the relation (229) all components containing expectations of products
of noncorrelated quantities are neglected.
It is seen that FPE is composed of two components: prediction for a given M
a2(M) • and the second due to statistical deviation of ~(m) from "M(m).
Behaviour of the FPE .namely decrease of a2(M) for increasing M and increase
of the second term for increasing M. suggests that there exists an optimum
for a certain M. which is nothing but the order of the autoregressive model(228). 'TPE
x "'F-'\>E. :E x
. * G'l(IM)
0--0 '[~~-
z..
-94-
This idea is worked out further in Akaike(1970). Following this idea we get:
M
X(n) = L a(m) X(n-m) + a(O) + c(n)
m=1
(235 )
where c(n) are the samples of the" white noise" • uniformly distributed and
E(£(n)) o (236)
If there is a collection of data avaliable. { X(n) ; n = -M+l. -M+2 •••• N}.
the parameter ~(m) is defined as the least squares estimate of a(m) ( ~(m)
is the parameter of the predictor (228) and we are going to find it basing
on observations of X(n)).
Defining
where
C (m.i) xx =--N
n=1
N
x = m
N L. X(n-m)
n=1
M
C (m.i) a..(m) xx M = -- L N n=1
M
(237)
m=O,I,2, •.• M (238)
=> L. Cxx(m.i) ~(m) M M
_1_~ ( L ~(m)X(n-m) - ~(m) Xm) ( X(n-i) - Xi )
N n=1 m=1 m=1
M M
L C (m.i) a..(m) xx M 1 '"" -= -.L-, (X(n) N
n=1
(239)
m=1
-95-
where
M N
L 2: ~(m)X(n-m) m=1 n=1
(240) N
Thus for N large enough we can write :
M
L e (m,"') ~(m) = e (0,"') xx xx (241 )
m=1
{ '" = 1,2,3, ... ,M for
m = 0, I ,2 , •• • ,M
according to (237) and (238). Relation (241) can be rewritten in the form
e (I , I) e (2, I ) xx xx
e (1,2) e (2,2) xx xx
e (I,M) e (2,M) xx xx
or
if e 1S nonsingular -xx
~= e -Ie -xx -M
e (M,I) xx
e (M,2) xx
e (M,M) xx
if e is singular or ill conditioned -xx
~= + c e -xx -M
~(I)
~(2)
~(M)
e (0,1) xx
e (0,2) xx
e (O,M) xx
where + C - is the Moore-Penrose pseudo inverse of C . -xx -xx
(242)
(243)
(244 )
(245)
-96-
The" zero" coefficient ~(O) is estimated basing on (235)
M
~(O) Xo - .L. ~(m) Xm m=1
Following the definition given previously - (228) using (246), we get
M
yen) =2: m=1
~(m) ( Y(n-m) - Xm ) - x o
the relation for the predictor yen) of yen).
Assuming that yen) is generated by relation
M
yen) L a Yen - m) + a + 6(n) m 0
m=1
where
6(n) has exactly the same statistical properties
M
as E(n), we get
M
(246)
(247)
(248)
yen) - Yopt(n) = 6 (n) - L lI~(m)y(n-m) - ( lIXo - L ~(m)lIXm )
for a m
m=1 m=1
~(m) are coefficients of the optimal predictor Yopt(n).
(249)
Relation (249) is somewhat abstract and refers to statistical properties of
the optimal predictor. It is assumed here,the process yen) is stationary and
ergodic. thus defining the following:
lI"M(m) ~(m)
yen) yen) - E(Y(n» (250)
lIX~ X~ - E(X(n»
-97-
basing on (247) and(248) ,(249) we get two relations
M
yen) - Y (n) opt
M
o(n) - (L. M
~(m)Y(n-m) -~ ~(m)Y(n-m) - 2. ~(m)E(Y(n-m) m= I m= I m=1
M
+ L ~(m)E(Y(n-m)) ) - (X - E(X(n» ) + o
m=1
M
+ Z ~(m)( Xm - E(X(n»
m= I
from (249)
M
yen) - yen) = o(n) - L ~(m) (Y(n-m) - Xm) -
m=1
M
ao = Xo - L ~(m)Xm m=1
thus
M ~
yen) - yen) 8(n) -~ ( ~(m) - am)Y(n-m) =
m=1
M
= o(n) - Z « ~(m) - am(m» + ( ~(m) - am» Y(n-m) =
m=1
M
= o(n) - L. lI~(m) Y(n-m) +
m=1
M
2. (~(m) - am) Y(n-m)
mel
a Y(n-m) + a m 0
(251 )
(252 )
(253)
(254)
From (251) and (254) we see that ( yen) - yen»~ will statistically be equal
( yen) - Y t(n» if : op
+
M z:. ( "M(m)
m=1
? - a ) Y(n-m) =
m
-98-
M
L (~(m) - "M(m) ) y(n-m)
m=1 M
( Xo - E(X(n))) + ~ ~(m)(Xm - E(X(n)) )
m=1 (255)
From (255) it is seen that statistically left and right sides are equivalent
if am ~ "M(m) , thus (249) is proven. Taking into account independence of
yen) of 6"M(m) and 6X ~ , we get :
M M
FPE of yen) E( yen) - yen) )2 = (52 + L Z E(6"M(m)6"M(~)Rxx(~-m) +
m=1 ~=I
where
+ E( 6X o
M
2: ~(m)6Xm)2 m=1
R (~- m) xx
2 E( X(n-~)X(n-m) ) - (E( X(n) )
(256)
(257)
For the sake of numerical solution we have to be interested in assymptotic
properties of the FPE, which hopefully can be a lot simpler to handle than
statistical evaluation of all subsequent quantities of the relation (256).
For this sake quantities 6Xo and 6"M(m) have been introduced.
For the asymptotic evaluation of the FPE we will need the following theorem
Theorem 9: Under the assumption of the stationarity and ergodicity of X(n),
the limit distribution iN 6X =.(J;i1 (X - E(X(n)) ) and o 0
IN' 6"M(m) =.fN1 ( ~(m) - "M(m)) for m - 1,2, ••. M,
when N tends to infinity, is (M+I) - dimensional Gaussian with zero mean and
the varianae matrix :
-I !lr.!
where
and
M
1 ... L "M(m)
m=1
-99-
!M - (M x M) matrix of R(i,m)
o - denotes a zero column vector
R (i - m) xx
( this theorem is a special case of the limit theorem as presented in the
book of Anderson T. w. (1971) ).
From the ergodicity of the process it is clear that C (i,m) converges to xx
R (i- m) as N+ 00 , with probability one. Thus !. is a consistent estimate xx -M
of ~ ( a vector of parameters of the optimal predictor) with convergence
with probability one.
From (244) we have
- (i) ( -I
C (O,i) (258) = C (m, i» ( ~ -xx -xx
- (i) ~(i) ... (C (m,i) )-I( C (i» (259) ~ -xx -EX
where N
fsx(i) N- I 2: e(n)( X(n-i) - Xi ) (260)
n=1
1I~(t) = ( C (m, i) )-I(C (t) ) (261) -xx -ex
From the consistency of C (m,i) it is concluded that that the limit distribution xx
of iN lIKo and iN'1I"M is identical to that {N'lIKo and -iN''\;ICex
- 2 Now,instead of taking the expectation of ( yen) - yen»~ as it was done in (256),
the conditional expectation of (Y(n) - Yen) )2 for a given X(n) is considered.
The last is denoted by
E{ ( yen) - yen) /1 X(n) }
-100-
From the independency of Y(n) and X(n) it comes
M M
- 2 a 2
+ """ """ E{(Y(n) - Y(n)) / X(n) } = ~ ~ m=1 ~=I
M
+ (Mo - L ~(m)Mm)2 (262)
m=1
Again in the limit the difference between-{N'Mo and{N'Mm are stochastically
vanishing ( m= 1,2, ... ,M) and Akaike (1970) demonstrates that:
N { E( ( Y(n) - Y(n) )2/ X(n) ) - a2 } (263)
has a limit distribution with expectation equal to (M+I)a2 , thus
N { ( FPE )M - a2 } =(M+I)a
2 (264)
(265)
where (FPE)M stands for an asymptotic evaluation of FPE.
Now remains to find an apropriate estimate of a2 • From the ergodicity of X(n)
it is concluded that
S(M) c (0,0) xx
M
-2: ~=I
a..(~)c (O,~) 1'I xx (266)
is a consistent estimate of a2 • It is shown further in Akaike(1970) that
the expression
M + I )-IS(M) ( I - N (267)
is the best estimate of a2 while N+oo. Thus the final Akaike's FPE criterion is
-101-
(268)
The order M is chosen which minimizes(FPE)M' thus completing the order test.
For the sake of MIMO models identification. the order test can be performed in
the very similar way. The only assumption to be made is that the multivariable
system is described by the following autoregressive model :
where
M
!(j) = L A Y(j-m) -- +
m=1
y(j)
!(j) = y(j) !;!(j)
A -0
+ ~(j )
Y I (j)
Y2(j)
y q (j)
A (p+q) x (p+q) matrix of parameters -m
!;!(j)
~o - (p+q) x vector of initial conditions
-
!!:(j) - (p+q) x I random vector satjsfying the relations
E{!!:(j) } = Q.
E{ ~(j)!T(j_m) } = 0 for m >1
o. S Jm -
ul (j)
u2 (j)
u (j) p
Given a set of observed data (!(j) ; j a 1.2 ..... N ). where Yi(j) ( i
will in general denote a i th component of !(j).
I. Define Y.(j) ( i = 1.2 •••• k. k=p+q)( j = 1.2 •••• N ) by 1
Y i (j) Y i (j)
(269)
(270)
1.2 ... q+p)
(271)
where
y. = 1
1 N
-102-
II. For 01 = 0,1,2, •.. L , where L is maximally allowable order of the model
(272)
and should be generally kept below N/5k, define the(i,t) th element of
(k x k) matrix
C (i,t) -01 N
C as: -01
N-m
L )\ (j+m»)\ (j)
j=1
(273)
III. Basing on (270)(271)(272) and definition of ~ the sample covariance matrix
of residual error can be computed by the following recursive formulae:
M
~(M) = s;, L A (M) CT
-01 -m 01=1
where A (M) can be evaluated using a recursive algorithm. -01
~(M)
!l. (M)
Q(M)
~(M)
A (M+ I) -01
M
~+I - 2: A (M) C -01 -M+I-m
01=1 M
C - 2: B (M) C -0 -m -m
01=1
~(M) {g(M)}-1
{~(M) T}{~{M)}-I
m i,2, ..• ,M
(274 )
(275)
(276)
(277 )
(278)
(279)
-103-
~m(M+I) = Q(M) for m = M+I
B (M+I) -m ~m(M) - ~(M) ~+I_m(M) for m = 1.2 •...• M
B (M+ I) -m
§,(O)
~(M) for m M+I
_Q(O) = C -0
(280)
(281 )
(282)
(283)
IV. Subsequently the order test must be performed according to the (FPE)M criterion.
which in this case takes the form
FPC(M) = 1\ ~qxq (M) II (I + M~+ I ) q. ( I -Mk+1
N (284)
where ~qxq(M) denotes the qxq submatrix in the upper left corner of §,(M).
Adopting the value of M which gives the minimum of FPE(M) for ( M = 1.2 .... L).
as the order of the model (269). the necessary matrices of coefficients can be
found by relations (274) - (283) • n . II - denotes a determinant .
This completes the order identification in terms of Akaike's FPE(M) for MIMO
systems.
-104-
3.5.2 Akaike's maximum Information Criterion approach
Akaike has proposed a new approach to the order determination and
parameter estimation , based on the FPE statistics which was a simplified
concept of the broader concept of AIC ( Akaike's Information Criterion-
(1972),(1975) ). The AIC criterion is built on the extension of the maximum
likelihood principle , very well known and broadly applied everywhere,
where the computing time factor does not playa main role. A brief and elegant
explanation of the classical maximum likelihood method, the reader can find
in Eykhoff(1974).
Considering y being the output vector and the components of L random
variables {y(I), ... ,y(k)} (for 5150 systems for the time beeing), the
joint probabiZity function for L will be :
p(y(I),oo.,y(k); b ) = p( l; b ) (285)
where
k L depends on ~ , the vector of parameters.
This is our "a priori" knowladge; a posteriori knowledge encompases values
of the random variable as measured. From this, an estimate ~ of the b can
be determined.
To distinguish it from relation(285), the joint probability function
k of L and ~ is called the likelihood function and denoted as L
L{ y(I),oo.,y(k); ~} (286)
Fer the sake of convenience,lnL is usually considered and the logarithmic
function , as the monotonic function, has the maximum at the same value of
~ as L.( Eykhoff(1974». This value of ~ can be obtained by solving:
-105-
(287)
8 - is called a maximum likelihood estimate of ~. In the classical theory
of estimation it is usually assumed that observations in ~k are independent
so that
k k p{~ ;~ } L p(y(i) ;~) (288)
i=1
and k k n L{~ ;f } L(y(i) ;f) (289)
i=1
k k
R,nL{~ ;f} L R,nL(y(i) ;f) (290)
i=1
To discuss properties of the Akaike's method it is necessairy to discuss
some properties of the maximum likelihood method.
A
Let us define a bias ~(£) in the estimator ~, . . k
gLven the observat10ns ~ :
6(,,-J" ~-El~('i~)I\2) ~ \e- \&l'i<)\,('2.I<.JI1)d';!~ " and the error covariance matrix !(b) of the estimator f2
In general, it is not possible to compute ei ther L'l (~) or ~(b).
(291)
( 2.92)
However, for
any unbiased estimator, we have the following Cramer-Rao lower bound.
-<
R(~) > ~~. (~) (2.93)
-106-
1~d"1~-E U~,L"pll{'~)\~J ~
~ E lU~ lnpl'i\ £) 1~b \.\'\pl'i"i E) T'\ ~} (2.'l~)
is the Fisher information matrix.
Relation (293) is equivalent to
which also means that every diagonal element of !(b) must be no smaller than -,
the corresponding element of 1~' (Ill
Thus the Cramer-Rao lower bound provides a lower bound on the accuracy to which
any component of b can be estimated. The technical assumptions required in the
derivation of the lower bound do not include any assumptions of linearity or
Gaussianess. Thus the bound is well-suited for nonlinear problems such as the
parameter identification problem for dynamical systems. It is also worth noticing
that6(b) and !(b) and 6 k(b) depend on b. ":1..
Some asymptotic properties of the maximum likelihood estimate will also be very
interesting. The asymptotic properties are concerned with the limiting
behaviour as the number of observations becomes infinite. The first assumption
to be made is the identifiability condition.
(2%)
This assumption means that no two parameters lead to observations with identical
probabilistic behaviour. It is obvious that if the identifiability condition is
violated for some pair ~1' ~, of parameters, then ~, and ~, cannot be dis-
tinguished no matter how many observations are made.
Assuming independent observations, identifiability and additional technical
assumptions, the assymptotic results are as follows:
-107-
I. Consistency
( 2.~7)
2. ASymptotic unbiasedness
3. ASymptotic normality
4. Asymptotic efficiency
In other words, as the number of processed observations becomes infinite, the
maximum likelihood estimate converges to the true value of ~, and the A
parameter estimate error is asymptotically normally distributed (~- ~) I
wi th covariance matrix d-t (9.) , so that the Cramer-Rao lower bound is
asymptotically tight.
The independence assumption implies an additive form for the information matrix
(2. 9 B)
is the information matrix for a single observation.
In terms of the asymptotic covariance matrix, we see that:
(2.~9)
-108-
This was introductory information. Let us return to the AlC-criterion. /'.
Given a set of estimates ~ of the vector of parameters ~ of a probability
distribution with density function k p(Z ;~), we will choose the one which will
give the maximum of the expectation from the tn (logarithm) of the likelihood
function, which will be by the definition:
l300)
where ~ - is the random variable following the distribution with the density A
function p (y;~) and is independent of 0 . For the purpose of discrimination between the two probability distributions
with density functions p.(y) (i = 0,1), all the necessary information is 1
contained in the likelihood ratio T(y) = Thus Akaike suggests,
instead of the classical maximum likelihood principle, maximization of
information theoretic quantity, which is given by definition:
Relation (301) is very well known as the Kullback-Leibler's mean value of
information for discrimination k A k'
between p{Z I~) and p{y I~) and can be
interpreted as the distance between the two distributions.
EW(~l~\~l)= ~ ~('iI~)lnp(~10)cl.y -
- ) 1"('-2.12 )lnpl'i I~)cli ~
('Oo~)
-109-
Following the concept of the measure of information, Akaike defines the
"mean amount of information to be discriminated pel' obsel'Vation" by
IIp,,po,cp) = fqJ( \'1(Y)) Poly)dy ,3(3) J \'0('1) --
where ~ must be "properly" ch-;'sen, and by d1. demonstrates the measure with respect
to which p.(y) are defined. Considering a parametric situation, where 1 _
probability densities are specified by a set of parameters b in the form
, The quantity defined by (303) will be denoted !(~,~,~) with p (1.) = p(1.I~)
, and po(I,) p(.ll /2.). To find the most "proper" (suitable) form of the
A
function, Akaike analysed the sensitivity of I(~,~~) to the derivation from A
/'J to b.
It is assumed that either p(.ll ~) and~(r) are the regular functions.
l?,04 )
('~os)
-110-
and assuming validity differentiation unde~ integral sign as well as that
• which implies that
In the same way it can be shown that
t '!.08)
which gives
Comparing (309) with (298) we see that integral
.L 1c:ly .. t{ l'o) ll!.'1\ - d~ll.'M-f_l~ LA --
D" (h - -is the <l,m)th element of Fisher's information matrix.
-111-
Summarizing we get:
" Relation (311) shows that~(I) must be different from zero if ll~ I~.~)
ought to be sensitive to small variations of b. Also the relative sensitivity
of I(b,~,(f» is high when l4?l<l I is large. -The possible forms ofq)(r) are e.g. - q)l~)
loge r, (r-I) and r
To restrict further the form of ~(r), the increase of information is considered,
by increase of N independent observations of Y.
For this caSe
l'!>it)
thus
-112-
" From (313) it is seen that IN(~'~'¢) is not very much influenced by the
increase of information. It can be also seen from evaluation of (313) from
(312) that the only quantity concerned with a final result is
t.J
d n ptl.Jli)l~) 3bL
This last relation shows very clearly that taking into account the In from
the density function p(y(i) ~), (314) will be fulfilled in a natural way:
and thus this observation suggests the choice of~(r) = lnr for the definition
of the amount of information, - this simply leads to the Kullback-Leibler
definition of information.
Remark: Any other definition ofcp(r) will be useful only ifq? is not vanishing.
For the purpose of the estimation Akaike proposes the following loss
and the risk
tlQ.\ ~) " E Z Wl~~)~ functions, which are based on the Kullback-Leibler formula. Akaike pestulates
that when N independent realizations y(i) (i = 1,2, ••.• N) of! are available,
(-2) times the sample mean of the ~n-likelihood, will be a consistent estimate /'
ofW(b,[':,), -- \-l I'
(-L) L. 9.M. ( pll.Jl'j
• Q» ) c=< \ ~l~U),~)
-113-
" thus, at least for large values N, the value~ which will give the maximum of
I-l " LiM (e l '1Li
" f:) ) i>' p('1L\I,~)
'" will nearly minimize W(~,~).
The whole idea of Akaike is to, instead of considering a single estimate of ~,
to consider estimates corresponding to various possible restrictions of the
" distribution. This whole idea can be simply realized by comparing R(~.~ or t\ .-
W(~,~) if possible, for various ~ 's and taking the one with the minimum of
" ..-R(~,~) or W(~,~) as a final choice.
This approach may be viewed as a natural extension of the classical maximum
ZikeZihood principZe. The only problem in applying this extended principle in
" '" practical situations is how to get a reliable estimate of R(~,~) and W(~,~)
'" " Akaike gives a procedure which enables finding estimates of !(~.~) and W(~~)
which are used later for estimation of the order of the model and the vector ,...
of parameters ~.
Assuming bk
(k = 0,1,2, ••.. L) - components of the vector~. Akaike is looking A
for 0>k (k = 0, I ,2, ...•. L) being maximum likelihood estimates of bk
. Considering
the situation where the results y(i) (i = 1,2, ..•.. N) are obtained as
independent N observations, ~k will be the value of bk which gives the maximum \J
of the likelihood function fl p(y(i)lbk).
i=1
Thus Akaike suggests that
( ~2.0)
-114-
A "
as an estimate of W~, f!> k). In Akaike' s paper (1972) it is precisely proven
that the
is the good estimate of E tW(~ ,~,,))(where ~ is the estimate of f':J k according to
Euclidean norm), at least in cases where N is sufficiently large and Land k
are relatively large integers. Since we are only concerned with finding out
'" the f':> k which will give the minimum of r(!!>l (\>k)' we have only to compute:
( '!.2.z.)
('l>B)
Relation (322) is the essence of the Akaike's method.
3.5.3. Concluding remarks
As it was proven by Soderstrom (1977) Akaike's FPC criterion and Akaike's AIC
criterion are asymptotically equivalent. However, for the MIMO systems the
evidence of equivalence of FPC(M) and AIC(M) is not proven. From a practical
point of view it would seem that FPC(M) should be singular and less time
consuming. Soderstrom (1977) once again points out that it is not the case
and in view of his work FPE(M) and AIC(M) are equivalent for sufficiently large
observation samples.
-115-
3.6. Structural identification based on the Hankel model
3.6.1. Behaviour of the error function with respect to the number of Markov-parameters in the Hankel model
The error function for the Hankel model of the multivariable dynamical system
is defined as:
(324)
where W is the gain scaling matrix which is positive definite. Quantities
Y and Yare calculated according to the Hankel model (see chapter I).
It can be easily shown that the error function V (s), where "s" is the number -w
• AT T of different Markov-parameters encountered ln Y = N S , rapidly decreases
- """1Il
in value when the model reaches the proper order i.e.
Entier(~ + I)
where r is the estimate of the order r. Then "s" is also the number of
(325)
different Markov-parameters contained in the Hr matrix. Behaviour of the error
function is not always clear. Sometimes it is difficult, at first sight, to
decide for an optimal s. Then the difference ratio
K I Vw(i+l) - Vw(i + 2)1
/Vw(i) - Vw(i + I) 1
gives the clearer picture of changes.
for i = (s-2),(s-I), s, ....
(326)
Difficulties will occur in cases with a relatively high noise power and in the
case when eigenvalues of the state matrix of the identified system are close
to each other. Another possibility, in cases where the order of the system is
smaller than 3~ a direct decrease rate of the error function can be used.
D I1V (k)
w V (k) - V (k + I) w w
(327)
11k
-116 -
This order test may be compared with the error function order test for 5I50
sytems as presented by A. J. W. van den Boom. A.W.M. van den Enden (1973).
and more details can be found in A. Hajdasifiski (1979).
3.6.2. Behaviour of the determinant of the Hr ~rT matrix
This order test is based on the property. that the rank of the Hankel matrix
Hr is equal to n if
n r"}. ____ _
min (P.q)
(which means that when contructing the sequence of matrices {~r} for r = 1.2.3 •••.
and checking rank {~r} for r = 1.2 ••••.•• the proper r is already found while
for
n r} ____ _ + rank {~r} = n). (328)
min (P.q)
For purely deterministic systems having finite realization. we may find this way
a real nand r. Increasing the Hankel matrix dimension as long as singularity
is not detected. we find the rank n. the r can be deduced from structural
properties of the ~r.
In cases where q 1 p, it is necessary either to check all minors in subsequent
Hankel matrices. or (which was found to be easier) to check the singularity of
the HrHrT
matrix.
Talking about the rank of the matrix, we understand this as the "numerical
rank" defined as follows:
Definition 19 Let N(n) be a set of ordinal numbers {1.2.3 ••.•• } and R(O) a
set of reals. The (k x k ) matrix Hk of the rank min(k x k ). has a "numerical q p p q
rank" (s.o.n) with respect to the spectral norm if: 2
-n = inf {rank~: II~ - ~II 2'i d. rank ~ IS N(n)
~ = sup { 0 :U~ - ~ II 2 ~ 0 :::;. rank ~ ~ n} cSeR(cS) (329)
where the
-117-
spectral norm of the Hk matrix
/I~1J2 = 1I~~ae
sup ~ x n e
x - any (k x I) vector 6Rkp p
II' lie - Euclidean norm
is:
The method again incorporates observation of changes in the behaviour of
T det {~r} or {~r~r}. The rapid decrease in the value of the det{~r} or
det {~r~rT} indicates a proper r. Then for the chosen r, a test for the
dimensionality of the state space must be performed.
(330)
Usually this order test is a very convincing one, and when applied together with
the test for the error function behaviour, it appears to be a very efficient one.
In cases where this order test would be confusing, it is again desireable to
look for the ratio test:
dedH.} det{~~T} K
-J, K (331 ) = or =
dedH. I}I -1+ I ded~+I~+IT}1 For more details see: Haj das iiiski (1980) , Isidori (1972).
3.6.3 Singular value decomposition of the Hankel matrix
The singular value decomposition of the Hankel matrix is a very efficient order
test which can be combined with the new realization algorithm (A. Hajdasiiiski,
A.A.H. Damen (1979». The whole procedure is based on the following:
Having exact Markov-parameters, it is always possible to find k > r - the system
order for which (see relation (150)
D - is the n x n diagonal matrix. (332)
are
-118-
D = diag. (0 ,0 , ..... 0) 1 2 n
o. for i 1
1,2, •••• n are called singular values
U - is the (k~ x n) matrix consisting of n eigenvectors of the
~~T Le. !!J!kT
= I --n
v - is the (kp x n) matrix consisting of n eigenvectors of the
T . T, V D2VT
!4c~, 1.e. ~ ~ =
vTv .. I . a. ={>\ , where A. for i = I ,2, . .. n -n' 1 1
eigenvalues of ~T!4c or !!J!kT. In the ideal case, the singular value
decomposition performed on the ,!!k, delivers the Itn" - the system dimension equal
to the number of nonzero singular values.
By a simple deduction the r can be found from structural properties of the Hankel
matrix.
In the noisy cases an easy test can be performed to discriminate between which
singular values are substantial and which can be neglected by comparing their
rate of decrease. The decision concerning which singular values can be
neglected will depend on the accuracy we will impose on our model.
Two types of approach can be proposed:
I. The relative error of the least squares fit on the Hankel matrix.
Defining the Euclidean norm of the matrix as:
2
\I !4c 1\ e = (333)
we can prove the following theorem: (Kam J.J. van der, Damen A.A.H. (1978),
Golub G.H., Reinsch C. (1970».
Theorem 10:
-119-
Given a s.v.d. for a (k x k ) matrix Hk: q p
D = diag. (0,,02 ••••• 0 min (k x k ) q p
the (kq x kp) matrix ~ of the rank ~ ~ min (kq
x kp
) and such that
1\ ~ - ~ \I e' is given the following:
u
where
U - contains the first q columns of U -~ ;J
v - contains the first D columns of V -g ;J
D - diag. (01.02 •••••• 0 .. \ -3 ~
(334)
Remark: Thus setting the smallest (min (k .k ) - s) singular values to zero q p
in the s.v.d •• through the relation (17). we obtain the best. in the least
squares sense. approximation of the ~ matrix. being of a smaller rank than
~.
To decide which singular values can be neglected the absolute error criterion
min(k .k )
z= q P2
O. J
o < £ « 1
or the relative error criterionr' __ ~~ __ ~~ __ -' min(k .k ) 1
2::: QP2
... 1 O. 1=~+ 1
min(k .k ) ~ q p
L-:. 0.> i=I L
~t o < 8 < 1
(335)
(336)
-120-
may be applied.
II. The numerical rank approach
Let us start with the following Lemma.
Lemma I: Let 01) O2 ~ ., ~ 0min(k ,k ) be the nonzero singular value of the q p
matrix~, then ~ has a numerical rank (E,o,n)2 iff:
(337)
The definition of the numerical rank and relations (329),(330) together with
Lemma 1 provide another criterion of the absolute error of the approximate
realization based on~. Assuming the upper bound - ° and lower bound - E of
the absolute error, through relations (337) and (334) ,we obtain the least
square optimal approximation of the ~ matrix - ~, where
rank ~ = numerical rank {~} = (E,o,n)2 (338)
Assuming that 2k Markov-parameters are estimated, it is possible to construct
the following Hankel and shifted Hankel matrices - ~ and O~. Performing
the s.v.d. there also is found a vector of singular values
where s = min(k ,k). The relative error of the least squares fit or the q p
numerical rank absolute error (337) decides which singular values may be
neglected~ In such a way the estimate fi of the system dimension is determined.
Example 7. Let us compa~e the last three order tests for the system having the
following properties:
0.2 0.0 0.0 0.0 -1.0
.!(k+l) = .!(k) T ~(k) 0.0 0.4 0.0 1.0 1.0
0.0. 0.0 0.8 1.0 -1.0
Z{k) = [ 1.0
0.0 -1.0J ~(k)
0.0 1.0 0.0
, , , , , , , , , , , , , , , , , ,
-121-
for this dynamical system r = 2, n = 3. The output of the dynamical system
is affected by the noise which is generated from the white gaussian noise by
the filter:
!.F, (k + I) = I o. I
0.0
E.(k) /
1 .0
0.0
0.0
!.F,(k) 0.7
0.0/ !.F,(k)
1.0
+
/
1.0
0.0
0.0 I I .0
• F,(k) -
Thus the measured output Zm(k) = Z(k) + E.(k).
The intensity of the simulated noise - n(k) is 10% of the output signal
amplitude - Z(k).
Ideal and estimated Markov-parameters are the following:
M ~1 ~? ~1 ~" ~5 -0
Ideal Markov -1.0 0.0 0.8 0.6 1-0.64 0.6 -0.512 O. -0.41 0.4( 1-0.33 0.32
parameters 1.0 1.0 0.4 0.4 0.16 O. I 0.064 0.01 0.026 0.02 0.01 0.01
Estimated 1-0.99 0.0 0.79 0.59 1-0.61 0.5 -0.47 0.4 1-0.35 0.38 0.25 0.28
Markov param. 1.0 1.0 .403 0.4 p.162 0.1' ~.07 0.07 p.02S 0.03 .015 0.01
and values bf the V as the function of Markov param. number - s, are: w
" , , s , "T - I , ,
V =tr(Y-Y) (Y-Y) , , , , W ,
2 , 3 ,
4 5 , 6 , , , , , , , , , , ,
I , I , , , , , , , Estim. Mark. par.:0.50606 :0.26831 :0.15562 :0.09177 :0.04948 :0.00326 , , , ,
I I , , , , , , , , Ideal Mark. par. :0.3855 :0.2011 : 0.0602 :0.0422 :0.0271 :0.0000
, , , , , , , , , , , , , , ,
-122-
O,S ------
1
)t---- -)I ideal M, par. ------)(
1\ O,?>
-----~--\--o C> estim. M. par.
0,2. 1 \1 ----------~ -----1-----t'~ ---------~----+--~-~-- -------r=----...j---- ~*- : ---== -= =- =. - =-= -=-- F ==- -=----~ -=-~- ~ #-==-~ -=:.
0,1
s
At first sight there seems to be some confusion as to which s should be chosen.
One may choose s = 2 or s = 3. For example for:
r Entier s 1 ) s = 2; = (- + 2 2
s = 3; r = Entier (~ + 1) = 2 2
for r = 2, n = 3 is sufficient to provide a good approximation of the original
system.
Performing the determinant test on the H matrix we get: -r
4.0 O,~
I I I I I I I I I I I
o,oOsst ____ L ______ _ QOOI __ ---1 __ _
1 2. !>
-123-
we have exactly the same answer: r = 2, fi = 3.
For the s.v.d. order test we choose the ~. matrix and we get the following set
of singular values:
al = 2.7707; a2 =
-3 8.1052.10 ;
-I -2 1.7185; a,· 3.5826.10 ; a. = 3.709.10 ;
a6 = 3.8196.10-3; a7 = 2.8425.10-3 ; a. = 1.0788.10-3 •
It is seen that:
a, > 0.1 > 0.05 > a.
thus the choice fi = 3 assures the numerical rank of the ~. matrix equal to:
(E,o,fi)2 = (0.05; 0.1; 3)2
and provides the relative error with respect to the Euclidean norm
tt~.-~Ile
U ~. II e
= 0.03829
The system order r > fi min(p,q)., thus it is the smallest integer fulfilling
r> 1.5. And again, we have as the result of the order test r = 2, which is
sufficient to provide fi = 3.
It is demonstrated then, that all three order tests provide us with the same
result r = 2 and fi = 3.
Conclusions
There exists already a quite extended class of order tests. Certainly this
choice of order tests mentioned here is one of many possible choices. However,
it shows quite well the mainstream in the methodology of model building.
The Guidorzi's order test has been applied for many practical problems
providing quite satisfactory results.
-124-
However, in many cases it was possible to find lower order approximations of
investigated systems. Such a comparison between the Guidorzi ' s method,
Ho-Kalman's algorithm and the partial minimal realization by Tether (Tether A.
(1970)) was made by A. Krause (1976), showing that it was always possible,
for noisy dat~, to find a lower order model for Tether's and Ho-Kalman's methods,
whereas for Guidorzi's methods a higher order model was found.
Tse-Weinert's methods, being a special case of the determinant test for the
Hankel matrix, gives very good results. Examples can be found in an original
publication by Tse and Weinert, (1975).
Furuta's approach is rather unnecessarilly complicated, but leading to quite
good results for low dimensional model.
Akaike's methods have already been discussed. They are very interesting from
a theoretical point of view. However, in practical applications demand a large
amount of data.
Order tests proposed for the Hankel model are rather simple and efficient.
Especially good results are with use of the s.v.d. method. This is, however,
the beginning of research in this field and many unexpected events may occur
during application of these methods in practice.
4. Multivariable system identification
There has already been much said about identification and parameter estimation.
However, the general definition of the identification has not yet been given.
According to Zadeh (1962), identification is the determination, on the basis of
input and output, of a system within a specified class of systems, to which the
system under test is equivalent.
-125-
This short definition covers a huge range of knowledge about the modelling and
model building basing an input/output data. A good survey of problems
concerning modelling and identification of single input - single output systems
has been written by Fasal K.H.,and Jorgel H.P. as the theoretical lecture for
the 5th IFAC Symposium on Identification and Parameter Estimation - Darmstadt
1979. For there is no substantial difference in general classification of
MlMO and SISO systems it is fair to quote this tutorial lecture as a good
reference.
Of our particular interest, there is an expePimentaZ anaZysis of a process,
which in general is considered just as the "identification". If the structure
of the model is known in advance, or at least can be estimated (see the previous
chapter), parametPic methods can be used for identification of the mathematical
model. In cases where the structure cannot he estimated non-parametric
techniques have to be applied.
Among the various goals of identification the below mentioned are most
frequently quoted:
deeper insight into the nature of the process
prediction of the dynamical behaviour of the process
verification of theoretical models
computation (or better estimation) of non-measurable variables
design of the controller
optimization of the-process.
Most of all, our interest will be attached to discrete time modeZs which will
be split into stochastic modeZs and detenninistic modeZs. However, the
deterministic models, as purely conceptual ones, usually serve as the prelim-
inary information about the process.
A survey of methods for identification of multivariable systems has recently /
been written by Niederlinski A. and Hajdasifiski A. (1979). This paper discusses
-126-
multi variable systems structures, parametric and non-parametric identification
techniques for mu1tivariab1e systems.
In this report we will present only the outline of a few of the most important
techniques, which are related to already discussed order tests. Also more
attention will be given to the Markov-parameter approach.
It should also be noted that the parametric identification of canonical models
by Guidorzi and Tse-Weinert has already been presented in chapters 3.1 and 3.2,
(however briefly), the Akaike's information criterion approach has also been
presented in quite SOme detail. Thus the following algorithms, as in the
illustrative examples, will be discussed further;
Tether's minimal partial realization algorithm
Gerth's algorithm
The Approximate Gauss-Markov Scheme with the s.v.d. Minimal Realization
algorithm
4.1. The Tether's minimal partial realization algorithm
The problem of minimal partial realization is formulated as follows:
1. Given an infinite sequence of Markov-parameters {M.},.which does not 1
have a finite dimensional realization, it is desired to find such a finite
dimensional realization so that its first No Markov-parameters are
correspondingly equal to the first No Markov parameters of the first
sequence, or
2. Given a finite sequence of Markov-parameters M , M, ..•. M __ I' find a -0 - ~o-
finite - dimensional realization whose first No Markov-parameters are
correspondingly equal to the Markov-parameters of the given finite sequence.
-127-
The second alternative is much more suited to the purpose of identification
and gives a powerful tool for handling the noisy case. It does not, however,
incorporate any stochastic considerations, assuming that only some extraction
of the Markov-parameters from the noise corrupted systems is prior to the
minimal partial realization procedure. The following two definitions are
useful:
Definition 20: {~,!,~} is said to be a partial realization of order{Nolof the
sequence {M.}, if an only if M. = C AiS holds for i = 0,1,2, -1 -l. - --
N - 1. o
Definition 21: {~,!,~} is said to be a partial minimal realization of order
{No)if and only if the dimension of A is minimal among all other {~;!;~I}
satisfying definition 20.
Tether (1970) proved that for every finite series of Markov-parameters {M.}, -1
i=O,J,2, .•.. No 1 there exists an extension sequence {~o+l""'}' for
which a completely controllable and observable partial realization exists via
the Ho-Kalman's algorithm.
It can also be demonstrated that among all possible partial realizations must
exist at least one minimal partial realization which is unique if and only if
the extent ion sequence is unique. Defining
M !:!.1 ........ !:!.r,' -~l N
-0
I (339)
Mi ~
~2.' ••••••• ~,
!:!.r,-I .•...•.•.... !:!.r,' + N - 2
-128-
it is possible to prove that given a finite sequence of r x m matrices
{M , M1 , •••••• M. I} satisfying ~ - ~o-
rank ~l ,N rank ~'+I ,N rank ~',N+I
for some N,N' such that N'+ N = N the extension of the sequence o
(340)
{M , M' ....... M.. I} to {M ••••• M.. I' •••• M k I} with 0 ... k < '" for which --0 - ~o- -0 ~o- ~o+ -
rank !!(m' ,m) = rank ~l ,N
where ro 1+ m = N + k, is unique. o
This result proven by Tether (1970) leads to the following reaZizabiZity
criterion:
(341 )
Let {~, ~l, •••.•. ~O-I} be an arbitrary finite sequence of q x p real matrices.
let H. . for i + j ... N be a corresponding Hankel matrix. Then a minimal partial 1; J 0
realization {A, !, ~} given by the Ho-Kalman algorithm is unique and realizes
the sequence up to and including the Noth term if and only if there exists
positive integers Nl and N such that:
I. N' + N = N o
2. rank ~lN rank ~'+I ,N rank ~l ,N+I
(342)
(343)
If the realizability criterion is satisfied then the Ho-Kalman algorithm can be
applied with
n = rank ~l ,N
using for the Ho-Kalman algorithm ~'N and a~' N' , The resulting minimal partial realization is unique because the extension
(344)
k sequence ~k = ~!! for k No,No + 1, generated by the realization satisfies
rank .!!--l + :...~ j, N+i = rank~, ,N (345)
for all i,j ?O
-129-
If 1 and 2 are not satisfied, then a minimal partial realization - if it exists -
may not be unique, for in order to use the Ho-Kalman algorithm new matrices
{~o' .•...• ~o-I} must be found until
rank ~1+I,M = rank ~I,M+1 = rank ~l M , (346)
where Ml + M = Po These matrices can be partially or completely arbitrary.
since (~,!,~) are functions of the sequence {~o .••• ~O-I} they may also not
be unique.
One of the most important results of Tether (1970) is the determination of a
lower bound for the minimal partial realization dimension. This is formulated
as the following Lemma.
Lemma 2 Let (~,!,~) be a partial realization where first No Markov-parameters
are equal to the given sequence {M , Ml , ••••• M I}' Then the dimension of --0 - ~o-
the minimal partial realization satisfies the following inequality:
No n ~ n(No) = L
j=1
rank H. N 1 . J' 0+ -J
No
2: j=1
rank H. N • -.J, O-J
This lower bound can be achieved for a "suitably chosen extension!!. This
(347)
"suitable choice lt is the subject of the fundamental theorem of minimal partial
realization given by Tether (1970). In order to state this theorem, it is
necessary to formulate one more definition and one more lemma.
Definition 22: Let NI(No) equal the first integer such that every row of the
block row [~l (No) ...... M ] is linearly dependent on the rows of the 410-1
Hankel matrix ~l(No), No-NI(No)' let N(No) equal the first integer such that
every column of the block column [~(NO) .....• ~0_11T is linearly dependent
on the columns of the Hankel matrix ~O-N(No),N(NO)'
Lemma 3 Let n(No), NI(No) and N(No) be integers defined previously. Then
any extension {~o'~o+1 ....• } of {~o, ~l ••••• ~o-I} whose realization
-130-
achieves the minimal lower bound n(No) for its dimension also satisfies
rank ~(No),N(No) = rank ~l(No)+I,N(No) =
= rank ~l(No),N(No)+1 (348)
for that extension.
From this Lemma follows the Minimal Partial Realization Theorem, which is central
in Tether's algorithm:
Theorem II Let {~, ~1 •••••• ~0~1} be a fixed partial sequence of q x p
matrices with real coefficients and let n(No),N(No) and N1 (No) be integers
defined previously. Then:
I. n(No) is the dimension of the minimal partial realization
2. N(No) and N1 (No) are the smallest integers such that (348) holds
simultaneously for all minimal extensions
3. There is a minimal extension of order P(No) = N(No) + N1 (No) for which
n(No) is the dimension of the realization computed by the Ho-Kalman
algorithm, but which in general is not unique.
4. Every extension which is fixed up to P(No) is uniquely determined thereafter.
It must be stressed that Tether's algorithm does not explicitly recognize the
stochastic nature of experimentally determined Markov-parameters. Nevertheless
it proves useful for real-life noisy and incomplete data because it can
generate an approximate model which agrees with the data. A number of successful
applications of this algorithm have been described (see Rossen-Lapidus (1972),
Hajdu (1969), Roman (1975)).
4.2. Gerth's algorithm
Tether's algorithm was assuming the availability of Markov-parameters. However,
they must be estimated. Therefore, an asymptotically unbiased and efficient
estimation of Markov-parameters is an essential step for the successful
identification of multivariable systems via the realization theory.
-131-
Gerth (1971,1972) proposed a multistage procedure for the initial estimation
of Markov-parameters and further refinement of the finite series {M.} in a way -:L
that assures the linear dependence of the extension series. It works on a
noise-corrupted data set of input-output pairs, assumes a ppeseZeated degree r
of the minimal polynominal and applies the Ho-Kalman algorithm to determine
a realization.
Gerth's algorithm consists of two separate steps:
I. Estimation of the finite series of Markov-parameters {N.} for i c O.I.Z •••• k -1
2. Determination of the minimal polynominal coefficients ai and matrices M. --:L
for i = 0.1 ••••• r - I. which are as "close as possible" to N .• for the -1
)
same i and specify an infinite realizable sequence via the relation
r
Z i.:~
"'i'M' • <\;1 _ )-t. (349)
with ~) = -a . being the coefficients of the minimal polynominal with a = I. ~1 r
The Gerth's algorithm assumes r - the order equal n - the dimension of the
realization. Assuming further that:
1. initial conditions are zero
2. ncises corrupting the signals are white and zero mean
3. number of samples is sufficiently large
and using the Hankel model and the least-squares method. the algorithm results
in the estimate of (k + I) Markov-parameters:
N (350)
-132-
where NT ~ l~' ~" ...... ~1 and matrices; and Yare defined by
yT = [~(~), ~(~ + I), .•••• ~(~ + m)l
i(i) =[~ (i), ~ (i) ••.•.. y (id " q
S ""1Il
~(£ - I) u(£ + m - I)
~(~ - k - I) ••• u(£ + m - k - I)
~T(i) = [u, (i), u2 (i) .••.• up(i)]
(351 )
(352)
Using an input excitation {u.} = {I,O,O •••.• } for each of the p x q partial >
systems and using the Hankel model (8), the excitation matrices for these partial
systems will be
T '0 -UJ"
w .. J> (353)
o
for j = 1,2, ••••• q, £ = J, ••••• p, where Wj~ is the individual weighting factor
for every partial system.
For every partial system the following Hankel matrix is constructed from the
elements of {~-i} for i = 0, I, ..... k.
N '0 -o,jJV
. ~r-I,j£
N '0 ...... N '0 -jlJ" k-r oJ "
(354)
-133-
where N .. , is the jR,-th element of N .. The vector containing elements of 1,JN -~
{N.} for i = r, r + I, '" k is defined as: -1
N ." N I ., ....... Nk ., r,]J<. r+ ,]N ,]JV (355)
The estimation of the minimal polynominal coefficients is also performed using
the least-squares method and the following loss function.
obtained from (349), with the coefficients vector
and
h T --r
H
-a , I
vT = [v T ,V T ....... ~pT]
-11 -12 ."
T -u diag[T TTl ""'-'11 ' -U11.'····· -uqp
The minimization of a gives:
(356)
(357)
(358)
(359)
(360)
(361)
It can be proven that the least-squares estimation of the minimal polynominal
coefficients for a given degree r is unique if and only if rank ~ = r. Next
the finite sequence {M.}, for i = 0,1, .•••. r-I, must be determined using again -1
least-squares estimation for computing.
min m. , -J"
k
2: i=o
[ y(i).,(h ,M) - Y(i).,(N)12
J" --r - J" - (362)
-134-
for j = 1,2, •••• p, R. = 1,2, •••• q, where y(i).,(h M) is the output sequ~nce J'" -r-
generated for the jR.-th partial system by T ., for the already determined -UJ'"
h but unknown {M.}, and y(i).,(N) - is the output sequence generated for the -r -J.. Jx. -
same partial system and the same excitation, but for known {N.}. The solution -1.
to (362) is given by:
where
with
(G T T T. GT)-I G T T T , ., n., - -U-UJ'- - - -UJ "'-u -J '"
nT
., =[N ." N ." ..... Nk .,] - J'- o,J'" I oJ'- oJ'"
G
E ~ 0 I - I
== ---I r-I I
I -r-I I
I
k-r+1 1 •••••• R e - -r
~l ~T =[ 0 .... 0 .... 1. ... 0]
r-th position
(363)
(364)
(365)
(366)
(367)
(368)
If all T ., are nonsingular, the sequence {M.}, i = O,I ••••• r-l, with a given -UJ'" -,
vector h is always unique and can be used to determine the minimal realization -r
using the Ho-Kalman's algorithm.
Extensions of Gerth's idea can be found in papers by Hajdasifiski (1976,1978).
4.3. The approximate Gauss-Markov scheme with the singular value decomposition minim.al =ealization algorithm
Using the Hankel model (8) and relations (351) (352), defining the noise vector
ET = [.~(O, ~(2.+ I)
~T(i) = [ e1 (i) e
2 (i)
(369)
(370)
-135-
and
(371 )
the block vector containing estimates of the first k + 1 Markov-parameters
(372)
the block vector containing the remaining Markov-parameters and the matrix
of initial conditions ~ , the Hankel
~( -k-2) ............. ~( -k+m-2)
~( -k-3) ............. u( -k+m-3) ... ~(-I )
o
o •....•............• u,l-iJ
model, for the noisy system will be following:
Y
Assuming that input samples {u.} and the noise samples {e.} are mutually -1. -1
uncorrelated stationary processes and that E{~(i)} = 0, an estimate of the
(373)
(374)
first k + 1 Markov-parameters can be found minimizing the error function Vw
(see Hajdasinski (1976)(1978)(1979), Niederlinski-Hajdasinski (1978)).
where
Y S T N -1tl -
(375)
(376)
is the estimate of the Y. Thus applying the well known formalism, an expression
for a minimal trace of Vw is found:
= 0;
N (377)
-136-
Expressing N in terms of E
N (378)
it is seen that the term (5 W 5 T)-I 5 WE is the bias of the estimate (377). -m---m -m--
This expression depends on properties of the noise E and asymptotically vanishes
when E{e(i)} o and there is no correlation between samples of E and 5 . - """1ll
Also for the zero initial conditions the second term in the expression (377)
vanishes. In case E{!} f 0, for this type of model i.e. (374), it is very
easy to estimate the bias of the estimate - see Hajdasifiski (1978).
Considering the following expression:
as an accuracy criterion, it follows that:
E{(~ - ~)(~ -
Substituting E{E ET} = -I Rand W = R we get:
see also Hajdasifi.ski (1976 ,1978,1979)
-I Thus choosing ~ =! , this results in an efficient estimate of the Markov-
parameters, since it is easy to show that:
E{(M - N)(M - N)T} - - - - R
(379)
(380)
(381 )
(382)
This way of weighting resembles very much the classical Gauss-Markov estimation
for 5ISO systems.
Defining
T n
-137-
n. T = [e.(t), e.(9.+I), ...... e.(9.+m)]
-1 1 1 1
the noise covariance matrix will take the form:
T = E{E, E, }
It is seen that the trace of g is the following sum of squares:
m+1 trg 2:
i= I j = I
(383)
(384)
(385)
(386)
Writing the R matrix in an explicit form it can be seen that the trace of g
is equal to the trace of R. (Hajdasifiski (1978».
trg trR (387)
Treating the! matrix as the covariance matrix of a hypothetical sinie-input
single-output noise filter it is possible to reconstruct this noise and its
covariance matrix. This "composite" noise model is but a mathematical fiction,
having no strictly physical interpretation. According to (386) and (387),
minimization of the (379), which appears in the minimal tr!, results in the
minimal trace of the noise covariance matrix, attaining in this way the main
goal of an efficient estimation.
If all noises corrupting a multi variable system are stationary, the R matrix
has also a very simple structure:
R =
S 2
s 2 1
S 2 m
S 2 S 2 •••• 1 2
S 2 •••••••••
s2 m-I
S 2 S 2 m-I m
S 2 rn-2
S 2 m-I
(388)
-138-
In practice the R matrix must be estimated using a finite number of samples,
taking as 52 and 52 m
m + 1 q S2 = L ~
-2(.) (389) e i J m +
j i = 1
m-k+1 q
52 = 2: L. e. (j) e. (j+k) (390) 1 1
n m-k+1 j = 1 i = 1
where e.(.) are standing for estimates of e.(.). 1 1
Thus only the first part of the ~ matrix, being the estimate of !, can be
computed with a sufficient accuracy, for in an explicit method there are only
m + 1 input-output and residual error samples available. Higher ordered elements
in R will be increasingly less accurate. As it is demonstrated by Hajdasifiski
(1978), the quality of the ~ matrix estimation plays a key role in the convergence
of Markov-parameters estimates. The realization theory once again helps to get
improved results, providing meaDS to reconstruct the composite noise covariance
matrix. For a detailed derivation of this result the reader is referred to
Hajdasifiski (1978).
Treating E as the set of specially arranged samples of a one-dimensional noise
with the covariance matrix!, it can be assumed that this noise is generated
by a colouring filter from a hypothetical white noise. In such a caSe an
estimate R of R is given by the following expression:
-139-
R ~ee(o)
~ ee(l)
'I' ee (I ) . • • •• ~ ee (m)
~ ee(m-I)
~ ~ ee(m).............. ee(o)
where E is the estimate of the multivariable noise, calculated using the
residual error of the L.S. estimation of Markov-parameters.
(391 )
Assuming that the noise e(k) is generated from s(k) (white noise) by a moving-
average filter,
e(k) -I
+ C(z ) s(k) = (C a
(392)
a very interesting decomposition of the R matrix can be found. From (391) it
follows
with
Since
for
e = £~
~ = ~ (k) ~ (k + I) ....• ~ (k + T [T T T
[ T T T
~ = ~ (k-v) ~ (k-v+l) ..••• ~ (k
C v-I C v-2 C 0 ...... 0 0
C = 0 C v-I C C ...... 0
1 0
o ..........•..... C I ••••••••• C v- 0
E{ii T} C cTa2
E{~~T} = a2 1
(393)
(394)
(395)
(396)
(397)
(398)
Then with accuracy to a constant factor the composite noise covariance matrix
is
R T
aC C (399)
-140-
However, for the Gauss-Markov estimation it is only necessary that the
weighting matrix W is similar to the covariance matrix R.
w (400)
Knowing a finite number of initial elements in the R matrix and assuming
they are sufficiently exact, it will be possible to find, via the realization
theory, the remaining elements and reconstruct the full rank covariance matrix
R.
A comparison of three methods for Markov-parameters estimation is presented by
Hajdasifiski (1978). Sequential algorithms are also available at present. A
general background can be found in Hajdasifiski (1976). Another approach to
the identification with Markov-parameters is presented by Rissanen,Kailath
(1972h Anderson, Brasch, Lopresti (1975).
Having identified Markov-parameters {M.}, the minimal realization algorithm ~
with the s.v.d. of the Hankel matrix was proposed by Hajdasifiski and Damen (1979).
Application of the last is straighforward using results derived in the chapter
3.4.2. - (150) - (158). For the numerical example see Example 6.
- 141 -
References
Aasnaes, H.B. and T. Kailath (1973) AN INNOVATION APPROACH TO LEAST-SQUARES ESTIMATION - Part VII: Some applications of vector autoregressive-moving average models. IEEE Trans. Autom. Control, Vol. AC-18(1973), p. 601-607.
Ackermann, J.E. and R.S. Bucy (1971) CANONICAL MINIMAL REALIZATION OF A MATRIX OF IMPULSE RESPONSE SEQUENCES. Inf. & Control, Vol. 19(1971), p. 224-231.
Ackermann, J.E. (1972) ON PARTIAL REALIZATIONS. IEEE Trans. Autom. Control, Vol. AC-17(1972), p. 381.
Akaike, H. (1970) STATISTICAL PREDICTOR IDENTIFICATION. Ann. Inst. Statist. Math. (Tokyo), Vol. 22(1970), p. 203-217.
Akaike, H. (1971) AUTOREGRESSIVE MODEL FITTING FOR CONTROL. Ann. Inst. Statist. Math. (Tokyo), Vol. 23(1971), p. 163-180.
Akaike, H. (1972) USE OF AN INFORMATION THEORETIC QUANTITY FOR STATISTICAL MODEL IDENTIFICATION. In: Proc. 5th Hawaii Int. Conf. on System Sciences, Honolulu, 11-13 Jan. 1972. Ed. by A. Lew. North Hollywood, Calif.: Western Periodicals, 1972. P. 249-250-.
Akaike, H. (1973) INFORMATION THEORY AND AN EXTENSION OF THE MAXIMUM LIKELIHOOD PRINCIPLE. In: 2nd Int. Symp. on Information Theory, Tsahkadsor, Armenia, USSR, 2-8 Sept. 1971. Ed. by B.N. Petro v and F. Csaki. Budapest: Akad8miai Kiad6, 1973. P. 267-281.
Akaike, H. (1974a) STOCHASTIC THEORY OF MINIMAL REALIZATION. IEEE Trans. Autom. Control, Vol. AC-19(1974), p. 667-674.
Akaike, H. (1974b) A NEW LOOK AT THE STATISTICAL MODEL IDENTIFICATION. IEEE Trans. Autom. Control, Vol. AC-19(1974), p. 716-723.
Akaike, H. (1975) MARKOVIAN REPRESENTATIONS OF STOCHASTIC PROCESSES BY CANONICAL VARIABLES. SIAM J. Control, Vol. 13(1975), p. 162-173.
Anderson, B.D.O. (1977) AN APPROACH TO MULTIVARIABLE SYSTEM IDENTIFICATION. Automatica, Vol. 13(1977), p. 401-408.
Anderson, 5.H., F.M. Brasch, Jr. and P.V. Lopresti (1975) THE SEQUENTIAL CONSTRUCTION OF MINIMAL PARTIAL REALIZATIONS FROM FINITE INPUT-OUTPUT DATA. SIAM J. Control, Vol. 13(1975), p. 552-571.
Anderson, T.W. (1971) TH~ STATISTICAL ANALYSIS OF TIME SERIES. New York: Wiley, 1971. Wiley Series in Probability and Mathematical Statistics
- 142 -
Andree, R.V. (1951) COMPUTATION OF THE INVERSE OF A MATRIX. Amer. Math. Monthly, Vol. 58(1951), p. 87-92.
Audley, D.R. and W.J. Rugh (1973a) ON THE H-MATRIX SYSTEM REPRESENTATION. IEEE Trans. Autom. Control, Vol. AC-18(1973), p. 235-242.
Audley, D.R. and W.J. ~ (1973b) A GENERAL FOUNDATION FOR LINEAR SYSTEM REALIZATION. In: Proc. 7th Annual Princeton Conf. on Information Sciences and Systems, Princeton, 22-23 March 1973. Princeton, N.J.: Department of Electrical Engineering, Princeton University, 1973. P. 223-227.
Barraud, A. and P. de Larminat (1973) MINIMAL REALIZATION AND APPROXIMATION OF LINEAR SYSTEMS FROM NORMAL OPERATING RECORDS. In: Identification and System Parameter Estimation. Proc. 3rd IFAC Symp., The Hague/Delft, 12-15 June 1973. Part 2. Ed. by P. Eykhoff. Amsterdam: North-Holland, 1973. P. 953-956.
Bingulac, S.P. and M. Djorovic (1973) ESTIMATION OF STATE VARIABLE MODELS USING INPUT-OUTPUT DATA. In: Identification and System Parameter Estimation. Proc. 3rd IFAC Symp., The Hague/Delft, 12-15 June 1973. Part 2. Ed. by P. Eykhoff. Amsterdam: North-Holland, 1973. P. 995-1004.
Bingulac, S.P. (1976) IDENTIFICATION OF MULTIVARIABLE DYNAMIC SYSTEMS. In: Identification and System Parameter Estimation. Proc. 4th IFAC Symp., Tbilisi, USSR, 21-27 Sept. 1976. Part 3. Edited by N.S. Ra.jbman. Amsterdam: North-Holland, 1978. P. 1979-1988.
Birkhoff, G. and S. MacLane (1965) A SURVEY OF MODERN ALGEBRA. 3rd ed. New York: Macmillan, 1965.
nonivento, G. and R. Guidorzi (1971) CANONICAL INPUT-OUTPUT DESCRIPTION OF LINEAR MULTI VARIABLE SYSTEMS. Ricerche di Automatica, Vol. 2( 1971), p. 72-83.
Bonivcnto, C. and R. Guidorzi (1972) APPLICATION OF AN IDENTIFICATION METHOD TO DISTILLATION COLUMNS. In: Proe. 6th A~~ual Princeton Conf. on Information Sciences and Systems, Princeton, 23-24 March 1972. Princeton, N.J.: Department of Electrical Engineering, Princeton University, 1972. P. 185-189.
Bruni, C., A. =sidori and A. Ruberti (1969) THE ORDE1~ OF LINEAR CONTINUOUS SYSTEMS DESCRIBED BY THE IMPULSE RESPONSE ~~TRIX. Alta Frequenza, Vol. 38(1969), p. 108-116, No.2 (Feb.), English edition No.1.
- 143 -
Budin, M.A. (1971) MINIMAL REALIZATION OF DISCRETE LINEAR SYSTEMS FROM INPUT-OUTPUT OBSERVATIONS. IEEE Trans. Autom. Control, Vol. AC-16(1971), p. 395-401.
Budin, M.A. (1972a) A NEW APPROACH TO SYSTEM IDENTIFICATION AND STATE ESTIMATION. IEEE Trans. Syst., Man & Cybern., Vol. SMC-2(1972), p. 396-402.
Budin, M.A. (1972b) CANONICAL FORMS FOR MULTI-INPUTjMULTI-OUTPUT MINIMAL SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-17(1972), p. 554.
Candy, J.V., M.E. Warren and T.E. Bullock (1978) REALIZATION OF INVARIANT SYSTEM DESCRIPTIONS FROM INFINITE MARKOV SEQUENCES. IEEE Trans. Autom. Control, Vol. AC-23(1978), p. 93-96.
Chen, C.T., C.A. Desoer and A. Niederlinski (1966) SIMPLIFIED CONDITIONS FOR CONTROLLABILITY AND OBSERVALITY OF LINEAR TIME-INVARIANT SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-11(1966), p. 613-614.
Denham, M.J. (1974) CANONICAL FORMS FOR THE I9ENTIFICATION OF MULTIVARIABLE LINEAR SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-19(1974), p. 646-656.
DeRusso, P.M., R.J. Roy and Ch.M. Close (1965) STATE VARIABLES FOR ENGINEERS. New York: Wiley, 1965.
Dickinson, B.W., T. Kailath and M. Morf (1974) CANONICAL MATRIX FRACTION AND STATE-SPACE DESCRIPTIONS FOR DETERMINISTIC AND STOCHASTIC LINEAR SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-19 (1974), p. 656-666.
Dickinson, B.W., M. Morf and T. Kailath (1974) A MINIMAL REALIZATION ALGORITHM FOR MATRIX SEQu&~CES. IEEE Trans. Autom. Control, Vol. AC-19(1974), p. 31-38.
Eykhoff, P. (1974) SYSTEM IDENTIFICATION: Parameter and state estimation. London: Wiley, 1974.
Falb, P.L. (1967). INFINITE-DIMwNSIONAL FILTERING: The Kalman-Bucy filter in Hilbert space. Inf. <I: Control, Vol. 11(1967), p. 102-137.
- 144 -
Fasol, K.H. and H.P. Jorgl (1979) MODELLING AND IDENTIFICATION. In: Tutorials on System Identification. Presented at the 5th IFAC-Symposium on Identification and System Parameter Estimation, Darmstadt, 24-28 Sept. 1979. Ed. by R. Isermann. Darmstadt: Institut fUr Regelungstechnik, Technische Hochschule Darmstadt, 1979. P. 1-13.
Faurre, P. and J.P. Marmorat (1969) UN ALGORITHME DE REALISATION STOCHASTIQUE. C.R. Acad. Sci. Paris Ser. A, Vol. 268(1969), p. 978-981.
Faurre, P. (1970) IDENTIFICATION PAR MINIMISATION D'UNE REPRESENTATION MARKOVIENNE DE PROCESSUS ALEATOIRE. In: Symposium on Optimization, Nice, 29 .June -5 July 1969. Fn. by A.V. Balakrishnan et all Berlin: Springer, 1970. (Lecture Notes in Mathematics, Vol. 132.) P. 85-107.
Faurre, P.L. (1976) STOCHASTIC REALIZATION ALGORITHMS. In: System Identification: Advances and case stUdies. Ed. by R.K. Mehra and D.G. Lainiotis. New York: Academic Press, 1976. (Mathematics in Science and Engineering, Vol. 126.) P. 1-25.
Furuta, K. (1973) AN APPLICATION OF REALIZATION THEORY TO IDENTIFICATION OF MULTI VARIABLE PROCESS. In: Identification and System Parameter Estimation. Proc. 3rd IFAC Symp., The Hague/Delft, 12-15 June 197~. P~t 2. Ed. by P. Eykhoff. Amsterdam: North-Holland, 1973. P. 939-942.
Gantmacher, F.R. (1959) THE THEORY OF MATRICES. Vol. 1 and 2. New York: Chelsea, 1959. Original Russian ed.: Teorija matric 1, 2. Moskva 1954.
Gerth, W. (1971) ON THE CONSTRUCTION OF LOW DRDER LINEAR STATE-VARIABLE MODELS FROM MEASURED DATA. In: Proc. 2nd IFAC Symp. on Multivariable Technical Control Systems, Dusseldorf, 11-13 Oct. 1971. Vol. 1. Ed. by H. Schwarz. Amsterdam: North-Holland, 1971. Paper 1.2.1 (15 pages).
Gerth, W. (1 972) ZUR IDENTIFIKATION UND MINIMALREALISIERUNG VON MEHRGROSSENVBERTRAGUNGSSYSTEMEN DURCH MARKOVPARAMETER. Ph.D. Diss. Technische Universitat Hannover, 1972.
Golub, G.H. and C. Reinsch (1970) SINGULAR VALUE DECOMPOSITION AND LEAST SQUARES SOLUTIONS. Numerische Mathematik, Vol. 14(1970), p. 403-420.
Gevers, M.R. and T. Kailath (1973) AN INNOVATIO~;S APPROACH TO LEAST-SQUARES ESTIMATION - Part VI: Discrete-time i~nov&tions and recursive estimation. IEEE Trons. Acto" .. Control, Vol. AC-18(1973), p. 588-600.
Gopinath, B. (1569) ON THE IDENTIFICATION OF LINEAR TIME-INVARIANT SYSTEMS FROM INPUT-OUTPUT DATA. Bell Syst. Tech. J., Vol. 48(1969), p. 1101-1113.
- 145 -
Guidorzi, R.P. (1973) CANONICAL STRUCTURES IN THE IDENTIFICATION OF MULTI VARIABLE SYSTEMS. In: Identification and System Parameter Estimation. Proc. 3rd IFAC Symp., The Hague/Delft, 12-15 June 1973. Part 2. Ed. by P. Eykhoff. Amsterdam: North-Holland, 1973. P. 943-952.
Gupta, R.D. and F.W. Fairman (1973) AN EXPLICIT APPROACH TO MINIMAL REALIZATION FROM SYSTEM MARKOV PARAMETERS. IEEE Trans. Autom. Control, Vol. AC-18(1973), p. 52-54.
Ha,jdasinski, A.K. (1976) A MARKOV PARAMETER APPROACH TO IDENTIFICATION OF MULTIVARIABLE DYNAMICAL SYSTEMS. Eindhoven: Group Measurement and Control, Department of Electrical Engineering, Eindhoven University of Technology, 1976. Technical Report.
Ha,jdasiftski, A.K. (1978) THE GAUSS-MARKOV APPROXIMATED SCHEME FOR IDENTIFICATION OF MULTIVARIABLE DYNAMICAL SYSTEMS VIA THE REALIZATION THEORY: An explicit approach. Eindhoven: Department of Electrical Engineering, Eindhoven University of Technology, 1978. TH-Report 78-E-88
Hajdasinski, A.K. and A.A.H. Damen (1979) REALIZATION OF THE MARKOV PARAMETER SEQUENCES USING THE SINGULAR VALUE DECOMPOSITION OF THE HANKEL MATRIX. Eindhoven: Department of Electrical Engineering, Eindhoven University of Technology, 1979. TH-Report 79-E-95
Ha,jdasinski, A.K. (1979) IDENTIFICATION OF MULTIVARIABLE NOISE CORRUPTED SYSTEMS WITH THE AID OF THE REALIZATION THEORY. In: Preprints 5th IFAC Symp. on Identification and System Parameter Estimation, Darmstadt, 24-28 Sept. 1979. Vol. 1. Ed. by R. Isermann. Oxford: Pergamon, 1979. P. 437-444.
Hajdasinski, A.K. (1980) ESTIMATION OF THE ORDER OF MULTIVARIABLE DYNAMICAL SYSTEMS. Journal A, Vol. 21(1980), p. 21-32.
Hajdu, L.P. (ed.) (1969) N.EW DEVELOPMENTS IN SYSTEM STATE ESTIMATION AND ANOMALY DETECTION. Palo Alto, California: Systems Control, Inc., 1969. Contract 14-03-79968 (Final Report)
Ho, B.L. and R.E. Kalman (1966) EFFECTIVE CONSTRUCTION OF LINEAR STATE-VARIABLE MODELS FROM INPUT-OUTPUT FUNCTIONS. Regelungstechnik, Vol. 14(1966), p. 545-548.
Is ermann , R. (1977) DIGITALE REGELSYSTEME. Berlin: Springer, 1977.
- 146 -
Isidori, A. (1972) ALGORITHMS FOR ORDER EVALUATION IN LINEAR SYSTEM THEORY. Ricerche di Automatica, Vol. 3(1972), p. 78-96.
Jazwinski, A.H. (1970) STOCHASTIC PROCESSES AND FILTERING THEORY. New York: Academic Press, 1970. Mathematics in Science and Engineering, Vol. 64.
Kailath, T. (1968) AN INNOVATIONS APPROACH TO LEAST-SQUARES ESTIMATION - Part I: Linear filtering in additive white noise. IEEE Trans. Autom. Control, Vol. AC-13(1968), p. 646-655.
Kailath, T. and P. Frost (1968) AN INNOVATIONS APPROACH TO LEAST-SQUARES ESTIMATION - Part II: Linear smoothing in additive white noise. IEEE Trans. Autom. Control, Vol. AC-13(1968), p. 655-660.
Kailath, T. (1970) THE INNOVATIONS APPROACH TO DETECTION AND ESTIMATION THEORY. Proc. IEEE, Vol. 58(1970), p. 680-695.
Kailath, T. and R.A. Geesey (1971) AN INNOVATIONS APPROACH TO LEAST-SQUARES ESTIMATION - Part IV: Recursive estimation given lumped covariance functions. IEEE Trans. Autom. Control, Vol. AC-16(1971), p. 720-727.
Kailath, T. and R.A. Geesey (1973) AN INNOVATIONS APPROACH TO LEAST-SQUARES ESTIMATION - Part V: Innovations representations and recursive estimation in colored noise. IEEE Trans. Autom. Control, Vol. AC-18(1973), p. 435-453.
Kalman, R.E. (1960) ON THE GENERAL THEORY OF CONTROL SYSTEMS. In: Automatic and Remote Control. Proc. 1st Int. Congress of the IFAC, Moscow, 27 June-7 July 1960. Vol. 1. Ed. by J.F. Coales. London: Butterworth, 1961. P. 481-492.
Kalman, H.E. and R.S. Bucy (1961) NEW HESULTS IN LINEAR FILTERING AND PHEDIGTION THEORY. Trans. ASME Ser. D. J. Basic Engrg., Vol. 83(1961), p. 95-108.
Kalman, R.E. (1963) MATHEMATICAL DESCRIPTION OF LINEAR DYNAMICAL SYSTEMS. J. SIAM Cor:trol, Vol. 1 (1962/63), p. 152-192.
Kalman, R.S. (1967) ALGEBRAIC ASPECTS OF THE THEORY OF DYNAMICAL SYSTEMS. In: Differedial Equations and Dynamical Systems. Ed. by J.K. Hale and J.P. LaSalle. New York: Academic Press, 1967. P. 133-146.
- 147 -
Kalman, R.E., P.L. Falb and M.A. Arbib (1969) TOPICS IN MATHEMATICAL SYSTEM THEORY. New York: McGraw-Hill, 1969. International Series in Pure and Applied Mathematics.
Krause, A. (1976) METHODS FOR IDENTIFICATION OF MULTIVARIABLE LINEAR SYSTEMS. M.Sc. Thesis. Gliwice Polytechnic Institute, 1976. (In Polish)
Kuo, B.C. (1975) AUTOMATIC CONTHOL SYSTEMS. Jrd ed. Englewood Cliffs, ~.J.: Prentice-Hall, 1975. P. 156-157.
Kuo, B.C. (1970) DISCRETE-DATA CONTROL SYSTEMS. Englewood Cliffs, N.J.: Prentice-Hall, 1970.
Lal, M., H. Singh and R. Parthasarathy (1975) A MINIMAL CANONICAl REAlIZATION ALGORITHM FOR IMPULSE RESPONSE MATRIX USING MOMENTS. Proc. IEEE, Vol. 63(1975), p. 538-540.
Luenberger, D.G. (1966) OBSERVERS FOR MULTI VARIABLE SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-11(1966), p. 190-197.
Luenberger, D.G. (1967) CANONICAL fORMS FOR LINEAR MULTIVARIABLE SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-12(1967), p. 290-293.
MacFarlane, A.G.J. (1970) RETURN-DIFFERENCE AND RETURN-RATIO MATRICES AND THEIR USE IN ANAlYSIS AND DESIGN OF MULTIVARIABLE FEEDBACK CONTROL SYSTEMS. Proc. Inst. Electr. Eng. (London), Vol. 117(1970), p. 2037-2049.
Mayne, D.Q. (1968) COMPUTATIONAl PROCEDURE FOR THE MINIMAL REALISATION OF TRANSFER-FUNCTION MATRICES. Proc. Inst. Electr. Eng. (London), Vol. 115(1968), p. 1363.
Mayne, D.Q. (1972a) PARAMl,TRIZATION AND IDENTIFICATION OF' LINEAR MULTI VARIABLE SYSTEMS. In: Stability of Stochastic Dynamical Systems. Proc. of the Int. Symp., University of Warwick, 10-14 July, 1972. Ed. by R.F. Curtain. Berlin: Springer, 1972. Lecture Notes in Mathematics, Vol. 294. P. 56-61.
Mayne, D.Q. (1972b) A CANONICAL MODEL FOR IDENTIFICATION OF MULTIVARIABLE LINEAR SYSTEMS. IEF£ Trans. Autom. Control, Vol. AC-17(1972), p. 728-729.
Mehra, H.I<:. (1971) ON-LINE IDENTIFICATION OF LINEAR DYNAMIC SYSTEMS WITH APPLICATIONS TO KALMAN FILTERING. IEEE Trans. Aut am. Control, Vol. AC-16(1971), p. 12-21.
- 148 -
Mehra, R.K. (1972) AN INNOVATIONS APPROACH TO MAXIMUM LIKELIHOOD IDENTIFICATION OF LINEAR AND NONLINEAR DYNAMIC SYSTEMS. In: Froc. 5th Hawaii Int. Coni'. on System Sciences, Honolulu, 11-13 Jan. 1972.~Ed. by A. ~. North Hollywood, Calif.: Western Periodicals, 1972. P. 252-254.
Niederlinski, A. (1970) ANALYSIS AND DESIGN OF 2-VARIABLE INTERACTING CONTROL SYSTEMS USING INVERSE POLAR PLOTS. Proc. Inst. Electr. Eng. (London), Vol. 117(1970), p. 20%-2059.
Niederlinski, A. (1971a) DIE VERALLGEMEINERUNG DER ZIEGLER-NICHOLSSCHEN REGELN FUR qEKOPPELTE ZWEI~ UND MEHRFACHREGELUNGEN. Regelungstechnik und Frozess-Datenverarbeitung, Vol. 19(1971), p. 291-295.
Niederlinski, A. (1971b) SOME NEW MEASURES OF INTERACTION FOR LINEAR MULTIVARIABLE PLANTS AND THE APPLICATIONS. In: Froc. 2nd IFAC Symp. on Multivariable Technical Control Systems, Dusseldorf, 11-13 Oct. 1971. Vol. 1. Ed. by H. Schwarz. Amsterdam: North-Holland, 1971. Paper 1.3.5 (13 pages).
Niederlinski, A. (1971c) TWO-VARIABLE DISTILLATION CONTROL: Decouple or not decouple. AIChE J., Vol.17(1971), p. 1261-1263.
Niederlinski, A. (1971d) A HEURISTIC APPROACH TO THE DESIGN OF LINEAR MULTIVARIABLE INTERACTING CONTROL SYSTEMS. Automatica, Vol. 7(1971), p. 691-701.
Niederlinski, A. (1972) CONDITION FOR STRUCTURAL MONOTONIC INSTABILITY OF INTERACTING LINEAR MULTI VARIABLE CONTROL SYSTEMS WITH INTEGRATING CONTROLLERS. Proc. Inst. Electr. Eng. (London), Vol. 119(1972), p. 255-257.
Niederlinski, A. (1974) UKbADY WIELOWYMIAROWE AUTOMATYKI (Multivariable Control Systems). Warszawa: Wydawnictwo Naukowo-Techniczne, 1974. (In Polish)
Niederlinski, A. and A.K. Hajdasiriski (1979) MULTIVARIABLE SYSTEM IDENTIFICATION - A SURVEY. In: Freprints 5th IFAC Symp. on Identification and System Parameter Estimation, Darmstadt, 24-28 Sept. 1979. Vol. 1. Ed. by R. Isermann. Oxford: Pergamon, 1979. P. 43-76.
Ostrowski, A.M. (1952) NOTE ON BOUNDS FOR DETERMINANTS WITH DOMINANT PRINCIPAL DIAGONAL. Froc. Amer. Math. Soc., Vol. 3(1952), p. 26-30.
Paul, C.R. and Yen-Long Kuo (1971) CONTROLLABILITY AND OBSERVALITY OF MULTIVARIABLE SYSTEMS. IFEE Trans. Autom. Control, Vol. AC-16(1971), p. 207-209.
- 149 -
Popov, V.M. (1972) INVAI(lMT m:~3CIUrTlON OF lINl';JIH, TIMe:-INVAlUANT CONTHOLLABU: ~YSTEMS. SIAM J. Control, Vol. 10(1972), p. 252-264.
Rijnsdorp, J.E. (1961) SINGLE AND MULTIPLE LOOP CONTROLS. In: Handbook of Automation, Computation, and Control. Vol. 3. Ed. by E.M. Grabbe, S. Ramo and E.D. Wooldridge. New York: Wiley, 1961. Chapter 10 (42 pages).
Ri,insdorp, J .E. (1965) INTERACTION IN TWO-VARIABLE CONTROL SYSTEMS FOR DISTILLATION COLUMNS, I and II. Automatica, Vol. 3(1965/66), p. 15-52.
Rijnsdorp, J.E. (1971) MULTIVARIABLE SYSTEMS IN PROCESS CONTROL. In: Proc. 2nd IFAC Symp. on Multivariable Technical Systems, Dusseldorf, 11-13 Oct. 1971. Vol. 4. Ed. by H. Schwarz. Amsterdam: North-Holland, 1971. P. 133-161.
Rissanen, J. and T. Kailath (1972) PARTIAL REALIZATION OF RANDOM SYSTEMS. Automatica, Vol. 8(1972), p. 389-396.
Rissanen, J. (1974) BASIS OF INVARIANTS AND CANONICAL FORMS FOR LINEAR DYNAMIC SYSTEMS. Automatica, Vol. 10(1974), p. 175-182.
Roman, J.R. and T.E. Bullock (1975) MINIMAL PARTIAL REALIZATIONS IN A CANONICAL FORM. IEEE Trans. Autom. Control, Vol. AC-20(1975), p. 529-533.
Rosenbrock, H.H. (1970) STATE-SPACE AND MULTI VARIABLE THEORY. London: Thomas Nelson, 1970. Studies in Dynamical Systems
Rossen, R.H. and L. Lapidus (1972) MINIMUM REALIZATIONS AND SYSTEM MODELLING: Part I: Pundamental theory and algorithm. Part II: Theoretical and numerical extensions. AIChE J., Vol. 18( 1972), p. 673-684 (part I) and 881-892 (part II).
Rozsa, P. and N.K. Sinha (1974) EFFICIENT ALGORITHM FOR IRREDUCIBLE REALIZATION OF A RATIONAL MATRIX. Int. J. Control, Vol. 20(1974), p. 739-751.
Rozsa, P. and N.K. Sinha (1975) MINIMAL REALIZATION OF A TRANSFER FUNCTION MATRIX IN CANONICAL FORMS. Int. J. Control, Vol. 21(1975), p. 273-284.
Schwarz, n. (1971) MEHRFACHREGELUNGEN. Band 2. Berlin: Springer, 1971.
Shamash, Y. (1975) MODEL REDUCTION USING MINIMAL REALIZATION ALGORITHMS. In: Proc. 9th Convention of Electrical and Electronics Engineers in Israel, Tel-Aviv 22-24 April 1975. New York: IEEE, 1975. Paper A-3-4 (7 pages). '
- 150 -
Shieh, L.S., Y.J. Wei and R. Yates (1975) MINIMAL REALIZATIONS OF TRANSFER-FUNCTION MATRICES BY MEANS OF MATRIX CONTINUED FRACTIONS. Int. J. Control, Vol. 22(1975), p. 851-859.
Silverman, L.M. and H.E. Meadows (1969) EQUIVALENT REALIZATIONS OF LINEAR SYSTEMS. SIAM J. Appl. Math., Vol. 17(1969), p. 393-408.
Silverman, L.M. (1971) REALIZATION OF LINEAR DYNAMICAL SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-16(1971), p. 554-567.
Sinha, N.K. (1975) MINIMAL REALIZATION OF TRANSFER FUNCTION MATRICES: A comparative study of different methods. Int. J. Control, Vol. 22(1975), p. 627-639.
Sinha, N.K. and A. Sen (1976) ON LINE IDENTIFICATION OF MULTIVARIABLE SYSTEMS. In: Identification and System Parameter Estimation. Proc. 4th IFAC Symp., Tbilisi, USSR, 21-27 Sept. 1976. Part 3. Edited by N.S. Ra.jbman. Amsterdam: NorthHolland, 1978. P. 1501-1510.
Soderstrom, T. (1977) ON MODEL STRUCTURE TESTING IN SYSTEM IDENTIFICATION. Int. J. Control, Vol. 26(1977), p. 1-18.
Tether, A.J. (1970) CONSTRUCTION OF MINIMAL LINEAR STATE-VARIABLE MODELS FROM FINITE INPUT-OUTPUT DATA. IEEE Trans. Autom. Control, Vol. AC-15(1970), p. 427-436.
Tham, Q.C. (1976) A MINIMAL POLYNOMINAL FROM THE MARKOV PARAMETERS OF SYSTEM. IEEE Trans. Autom. Control, Vol. AC-21(1976), p. 133-134.
Thiga, R.S.A. and N.E. Gough (1974) ARTIFICIAL INTELLIGENCE APPLIED TO THE DISCRIMINATION OF THE ORDER OF MULTI VARIABLE LINEAR SYSTEMS. Int. J. Control, Vol. 20(1974), p. 961-969.
Tse, E. and H.L. Weinert (1975) STRUCTURE DETERMINATION AND PARAMETER IDENTIFICATION FOR MULTIVARIABLE STOCHASTIC LINEAR SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-20(1975), p. 603-613.
van den Boom, A.J.W. and A.W.M. van den Enden (1973) THE DETERMINATION OF THE ORDER OF PROCESS- AND NOISE DYNAMICS. In: Identification and System Parameter Estimation. Froc. 3rd IFAC Symp., The Hague/Delft, 12-15 June 1973. Part 2. Ed. by P. EYkhoff. Amsterdam: North-Holland, 1973. P. 929-938.
- lSI -
van der Karn, J.J. and A.A.H. Darnen (1978) OBSERVABILITY OF EI"ECTRICAL HEART ACTIVITY STUDIED WITH THE SINGULAR VALUE DECOMPOSITION. Eindhoven: Department of Electrical Engineering, Eindhoven University of Technology, 1978. TH-Report 78-E-81
Wolovich, W.A. (1974) LINEAR MULTI VARIABLE SYSTEMS. Berlin: Springer, 1974. Applied Mathematical Sciences, Vol. 11.
Zadeh, L.A. (1962) FROM CIRCUIT THEORY TO SYSTEM THEORY. Proc. Inst. Radio Engrs (New York), Vol. 50(1962), p. 856-865.
Zadeh, L.A. and Ch.A. Desoer (1963) LINEAR SYSTEM THEORY: The state space approach. New York: McGraw-Hill, 1963. McGraw-Hill Series in System Science
Zadeh, L.A. and E. Polak (1969) SYSTEM THEORY. New York: McGraw-Hill, 1969. Inter-University Electronics Series, Vol. 8.
Zeiger, H.P. and A.J. McEwen (1974) APPROXIMATE LINEAR REALIZATIONS OF GIVEN DIMENSION VIA HO'S ALGORITHM. IEEE Trans. Autom. Control, Vol. AC-19(1974), p. 153.
EINDHOVEN UNIVERSITY OF TECHNOLOGY THE NETHERLANDS DEPARTMENT OF ELECTRICAL ENGINEERING
Reports:
93) Duin, C.A. van DIPOLE SCATTERING OF ELECTROMAGNETIC WAVES PROPAGATION THROUGH A RAIN MEDIUM. TH-Report 79-E-93. 1979. ISBN 90-6144-093-9
94) Kuijper, A.H. de and L.K.J. Vandamme CHARTS OF SPATIAL NOISE DISTRIBUTION IN PLANAR RESISTORS WITH FINITE CONTACTS. TH-Report 79-E-94. 1979. ISBN 90-6144-094-7
95) Hajdasinski, A.K. and A.A.H. Damen
96)
REALIZATION OF THE MARKOV PARAMETER SEQUENCES USING THE SINGULAR VALUE DECOMPOSITION OF THE HANKEL MATRIX. TH-Report 79-E-95. 1979. ISBN 90-6144-095-5
Stefanov. B. ELECTRON MOMENTUM TRANSFER CROSS-SECTION IN CESIUM AND RELATED CALCULATIONS OF THE LOCAL PARAMETERS OF Cs + Ar MHD PLASMAS. TH-Report 79-E-96. 1979. ISBN 90-6144-096-3
97) Worm, S.C.J. RADIATION PATTERNS OF CIRCULAR APERTURES WITH PRESCRIBED SIDELOBE LEVELS. TH-Report 79-E-97. 1979. ISBN 90-6144-097-1
98) Kroezen, P.H.C. A SERIES REPRESENTATION METHOD FOR THE FAR FIELD OF AN OFFSET REFLECTOR ANTENNA. TH-Report 79-E-98. 1979. ISBN 90-6144-098-X
99) Koonen, A.M.J. ERROR PROBABILITY IN DIGITAL FIBER OPTIC COMMUNICATION SYSTEMS. TH-Report 79-E-99. 1979. ISBN 90-6144-099-8
100) Naidu, M.S. STUDIES ON THE DECAY OF SURFACE CHARGES ON DIELECTRICS. TH-Report 79-E-I00. 1979. ISBN 90-6144-100-5
101) Verstappen, H.L. A SHAPED CYLINDRICAL DOUBLE-REFLECTOR SYSTEM FOR A BROADCAST-SATELLITE ANTENNA. TH-Report 79-E-I0l. 1979. ISBN 90-6144-101-3
102) Etten, W.C. van THE THEORY OF NONLINEAR DISCRETE-TIME SYSTEMS AND ITS APPLICATION TO THE EQUALIZATION OF NONLINEAR DIGITAL COMMUNICATION CHANNELS. TH-Report 79-E-I02. 1979. ISBN 90-6144-102-1
103) Roer, Th.G. van de ANALYTICAL THEORY OF PUNCH-THROUGH DIODES. TH-Report 79-E-I03. 1979. ISBN 90-6144-103-x
104) Herben. M.H.A.J. DESIGNING A CONTOURED BEAM ANTENNA. TH-Report 79-E-104. 1979. ISBN 90-6144-104-8
EINDHOVEN UNIVERSITY OF TECHNOLOGY THE NETHERLANDS DEPARTMENT OF ELECTRICAL ENGINEERING
Reports:
105) Videc, M.F. STRALINGSVERSCHIJNSELEN IN PLASMA'S EN BEWEGENDE MEDIA: Een geometrischoptische en een golfzonebenadering. TH-Report 80-E-105. 1980. ISBN 90-6144-105-6
106) Hajdasinski, A.K. LINEAR MULTIVARIABLE SYSTEMS: Preliminary problems in mathematical description, modelling and identification. TH-Report 80-E-106. 1980. ISBN 90-6144-106-4
107) Heuvel, W.M.C. van den CURRENT CHOPPING IN SF6. TH-Report BO-E-l07. 19BO. ISBN 90-6144-107-2
108) Etten, W.C. van and T.M. Lammers TRANSMISSION OF FM-MODULATED AUDIOSIGNALS IN THE B7.5 - lOB MHz BROADCAST BAND OVER A FIBER OPTIC SYSTEM. TH-Report 80-E-l08. 19BO. ISBN 90-6144-108-0