8/3/2019 Kalman Filters for Nonlinear Systems
1/16
This article was downloaded by: [University of South Florida]On: 21 September 2011, At: 14:21Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House37-41 Mortimer Street, London W1T 3JH, UK
International Journal of ControlPublication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/tcon20
Kalman filters for non-linear systems: a comparison of
performanceTine Lefebvre
a, Herman Bruyninckx
a& Joris De Schutter
a
aKatholieke Universiteit Leuven, Dept. of Mechanical Eng., Division P.M.A., Celestijnenlaa
300B, B-3001 Heverlee, Belgiumb
Katholieke Universiteit Leuven, Dept. of Mechanical Eng., Division P.M.A., Celestijnenlaa
300B, B-3001 Heverlee, Belgium E-mail: [email protected]
Available online: 19 Feb 2007
To cite th is article: Tine Lefebvre , Herman Bruyninckx & Joris De Schutter (2004): Kalman filters for non-linear systems: a
comparison of performance, International Journal of Control, 77:7, 639-653
To link to this article: http://dx.doi.org/10.1080/00207170410001704998
PLEASE SCROLL DOWN FOR ARTICLE
Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions
This article may be used for research, teaching and private study purposes. Any substantial or systematic
reproduction, re-distribution, re-selling, loan, sub-licensing, systematic supply or distribution in any form toanyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation that the contentswill be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses shouldbe independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims,proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly inconnection with or arising out of the use of this material.
http://dx.doi.org/10.1080/00207170410001704998http://www.tandfonline.com/page/terms-and-conditionshttp://dx.doi.org/10.1080/00207170410001704998http://www.tandfonline.com/loi/tcon208/3/2019 Kalman Filters for Nonlinear Systems
2/16
Kalman filters for non-linear systems: a comparison of performance
TINE LEFEBVREy*, HERMAN BRUYNINCKXy and JORIS DE SCHUTTERy
The Kalman filter is a well-known recursive state estimator for linear systems. In practice, the algorithm is often usedfor non-linear systems by linearizing the systems process and measurement models. Different ways of linearizing themodels lead to different filters. In some applications, these Kalman filter variants seem to perform well, while for otherapplications they are useless. When choosing a filter for a new application, the literature gives us little to rely on. Thispaper tries to bridge the gap between the theoretical derivation of a Kalman filter variant and its performance in practicewhen applied to a non-linear system, by providing an application-independent analysis of the performances of thecommon Kalman filter variants.
This paper separates performance evaluation of Kalman filters into (i) consistency, and (ii) information content of theestimates; and it separates the filter structure into (i) the process update step, and (ii) the measurement update step. Thisdecomposition provides the insights supporting an objective and systematic evaluation of the appropriateness ofa particular Kalman filter variant in a particular application.
1. Introduction
During recent decades, many research areas
looked into the matter of on-line state estimation. Theuncertainty on the state value varies over time due to
the changes in the system state (the process updates) and
due to the information in the measurements (the meas-
urement updates). The uncertainty can be represented in
different ways, e.g. by intervals or fuzzy sets.
In Bayesian estimation (Bayes 1763, Laplace
1812), a state estimate is represented by a probability
density function (pdf ). Fast analytical update algo-
rithms require the pdf to be an analytical function
of a limited number of time-varying parameters,
which is only true for some systems. A well-known
example is systems with linear process and measure-ment models and with additive Gaussian uncertain-
ties. The pdf is then a Gaussian distribution, which is
fully determined by its mean vector and covariance
matrix. This mean and covariance are updated ana-
lytically with the Kalman filter (KF) algorithm
(Kalman 1960, Sorenson 1985).
For most non-linear systems, the pdf cannot be
written as an analytical function with time-varying
parameters. In order to have a computationally
interesting update algorithm, the KF is used as an
approximation. This is achieved by linearization of
the process and measurement models of the system.
It also means that the true pdf is approximated
by a Gaussian distribution. Different ways of lineari-
zation (different KF variants) lead to different
results.This paper describes (i) how the common KF
variants differ in their linearization of the process and
measurement models; (ii) how they take the linearization
errors into account; and (iii) how the quality of their
state estimates depends on the previous two choices.
The studied algorithms are:
1. The extended Kalman filter (EKF) (Gelb et al.
1974, Maybeck 1982, Bar-Shalom and Li 1993,
Tanizaki 1996);
2. The iterated extended Kalman filter (IEKF) (Gelb
et al. 1974, Maybeck 1982, Bar-Shalom and Li
1993, Tanizaki 1996);
3. The linear regression Kalman filter (LRKF)
(Lefebvre et al. 2002). This filter comprises the
central difference filter (CDF) (Schei 1997),
the first-order divided difference filter (DD1)
(Nrgaard et al. 2000a, b) and the unscented
Kalman filter (UKF) (Uhlmann 1995, Julier and
Uhlmann 1996, 2001, Julier et al. 2000).
The paper gives the following new insights:
1. The quality of the estimates of the KF variants
can be expressed by two criteria, i.e. the consis-
tency and the information content of the estimates(defined in } 3). This paper relates the consistency
and information content of the estimates to
(i) how the linearization is performed and
(ii) how the linearization errors are taken into
account.
2. Although the filters use similar linearization
techniques for the linearization of the process
and measurement models, there can be
International Journal of Control ISSN 00207179 print/ISSN 13665820 online # 2004 Taylor & Francis Ltdhttp://www.tandf.co.uk/journals
DOI: 10.1080/00207170410001704998
INT. J. CONTROL, 10 MAY 2004, VOL. 77, NO. 7, 639653
Received 1 September 2003. Revised and accepted 1 April2004.
* Author for correspondence. e-mail: [email protected]
y Katholieke Universiteit Leuven, Dept. of MechanicalEng., Division P.M.A., Celestijnenlaan 300B, B-3001Heverlee, Belgium.
8/3/2019 Kalman Filters for Nonlinear Systems
3/16
a substantial difference in their performance for
both updates:
(a) for the linearization of the process update
(} 4), which describes the evolution of the
state, the state estimate and its uncertainty
are the only available information;
(b) the measurement update (} 5), on the other
hand, describes the fusion of the information
in the state estimate with the information in
the new measurement. Hence, in this update
also the new measurement is available and
can be used to linearize the measurement
model.
Therefore, it can be interesting to use different
filters for both updates.
3. Two new insights on the performance of specific
KF variants are: (i) the IEKF measurement
update outperforms the EKF and LRKF updates
if the stateor at least the part of it that causes
the non-linearity in the measurement modelisinstantaneously fully observable (} 5.2); and
(ii) for large uncertainties on the state estimate,
the LRKF measurement update yields consistent
but non-informative state estimates (} 5.3).
These insights are obtained because:
1. This paper describes all filter algorithms as the
application of the basic KF algorithm to linear-
ized process and measurement models. The differ-
ence between the KF variants is situated in the
choice of linearization and the compensation for
the linearization errors. In previous work this
linearization was not always recognized, e.g. the
UKF is originally derived as a filter which does
not linearize the models.
2. The analysis clarifies how some filters automati-
cally adapt their compensation for linearization
errors, while other filters have constant (devel-
oper-determined) error compensation.
3. Additionally, the paper compares the filter per-
formances separately for process updates and
measurement updates instead of their overall
performance when they are both combined. In
the existing literature, the performances of the
KF variants are often compared by interpretingthe estimation results for a specific application
after executing a large number of process and
measurement update steps.
The analysis starts from a general formulation of
non-linear process and measurement models, making the
results application independent. The obtained insights
are important for all researchers and developers who
want to apply a KF variant to a specific application.
They lead to a systematic choice of a filter, where pre-
viously the choice was mainly made based on success in
similar applications or based on trial and error.
Examples of 2D systems are provided. The models
are chosen such that they provide a clear graphical
demonstration of the discussed effects. For the measure-
ment update, the filters performances are system
dependent, hence, in that case several models are usedfor illustration.
2. The Kalman filter algorithm
2.1. The (linear) Kalman filter
The Kalman filter (KF) (Kalman 1960, Sorenson
1985) is a special case of Bayesian filtering theory. It
applies to the estimation of a state x if the state space
description of the estimation problem has linear process
and measurement equations subject to additive
Gaussian uncertainty
xk Fk1xk1 bk1 Ck1qp, k1 1
zk Hkxk dk Ekqm, k: 2
z is the measurement vector. The subscripts k and k 1
indicate the time step. F, b, C, H, d and E are (possibly
non-linear) functions of the system input. qp denotes the
process uncertainty, being a random vector sequence
with zero mean and known covariance matrix Q. qm is
the measurement uncertainty and is a random vector
sequence with zero mean and known covariance matrix
R; qp and qm are mutually uncorrelated and uncorre-
lated between sampling timesy. Furthermore, assume a
Gaussian prior pdf px0 with mean xx0j0 and covariance
matrix P0j0.
For this system, the pdfsz pxkjZZk1 and pxkjZZk
are also Gaussian distributions. The filtering formulas
can be expressed as analytical functions calculating the
mean vector xx and covariance matrix P of these pdfs
xxkjk1 EpxkjZZk1xk
Fk1xxk1jk1 bk1 3
Pkjk1 Epxk
jZZk1
xk xxkjk1 xk xxkjk1 Th i
Fk1Pk1jk1FTk1 Ck1Qk1C
Tk1 4
y Correlated uncertainties can be dealt with by augmentingthe state vector, this is the original formulation of the KF(Kalman 1960). Expressed in this new state vector, theprocess and measurement models are of the form (1) and (2)with uncorrelated uncertainties.
zpxkjZZj denotes the pdf of the state x at time k, given the
measurements ZZj fzz1, . . . , zzjg up to time j.
640 T. Lefebvre et al.
8/3/2019 Kalman Filters for Nonlinear Systems
4/16
xxkjk EpxkjZZkxk
xxkjk1 Kkgk 5
Pkjk EpxkjZZkxk xxkjk
xk xxkjk Th i
Inn KkHk Pkjk1 6
where
gk zzk Hkxxkjk1 dk 7
Sk EkRkETk HkPkjk1H
Tk 8
Kk Pkjk1HTk S
1k : 9
g is called the innovation, its covariance is S. K is the
Kalman gain. Equations (3)(4) are referred to as the
process update, equations (5)(9) as the measurement
update. xxkjk1 is called the predicted state estimate and
xxkjk the updated state estimate. If no measurement zzk is
available at a certain time step k, then equations (5)(9)reduce to xxkjk xxkjk1 and Pkjk Pkjk1.
2.2. Kalman filters for non-linear systems
The KF algorithm is often applied to systems with
non-linear process and measurement modelsy
xk fk1xk1 Ck1qp, k1 10
zk hkxk Ekqm, k 11
by linearization
xk Fk1xk1 bk1 q
p, k1 Ck1qp, k1 12
zk Hkxk dk qm, k Ekqm, k: 13
The difference between these models and models (1) and
(2) is the presence of the terms qp and qm representing
the linearization errors. The additional uncertainty on
the linearized models due to these linearization errors
is modelled by the covariance matrices Q and R.
Unfortunately, applying the KFz (3)(9) to systems
with non-linear process and/or measurement models
leads to non-optimal estimates and covariance matrices.
Different ways of linearizing the process and measure-
ment models, i.e. different choices for F, b, Q, H, d
and R, yield other results. This paper aims at making
an objective comparison of the performances of the
commonly used linearizations (KF variants).
3. Consistency and information content of the
estimates
The KF variants for non-linear systems calculate
an estimate xxkji and covariance matrix Pkji for a pdf
which is non-Gaussian. The performance of these KFs
depends on how representative the Gaussian pdf with
mean xxkji and covariance Pkji is for the (unknown) pdf
pxkjZZi. Figure 1 shows a non-Gaussian pdf pxkjZZi
and three possible Gaussian approximations p1xkjZZi,
p2xkjZZi and p3xkjZZi. Intuitively we feel that p1xkjZZi
is a good approximation because the same values of xare probable. Similarly p3xkjZZi is not a good approxi-
mation because a lot of probable values for x of the
original distribution have a probability density of
approximately zero in p3xkjZZi. Finally pdf p2xkjZZi
is more uncertain than pxkjZZi because a larger
domain of x values is uncertain.
These intuitive reflexions are formulated in two
criteria: the consistency and the information content of
the state estimate. The consistency of the state estimate
is a necessary condition for a filter to be acceptable. The
information content of the state estimates defines an
ordering between all consistent filters.
3.1. The consistency of the state estimate
A state estimate xxkji with covariance matrix Pkji is
called consistent if
EpxkjZZi
xk xxkji
xk xxkji Th i
Pkji: 14
For consistent results, the matrix Pkji is equal to or
larger than the expected squared deviation with respect
to the estimate xxkji under the (unknown) distribution
pxkjZZi. The mean and covariance of pdfs p1xkjZZi
and p2xkjZZi in figure 1 obey equation (14). Pdf
p3
xk
jZZi, on the other hand, is inconsistent.
Inconsistency of the calculated state estimate xxkjiand covariance matrix Pkji (divergence of the filter) is
the most encountered problem with the KF variants.
In this case, Pkji is too small and no longer represents a
reliable measure for the uncertainty on xxkji. Even more,
once an inconsistent state estimate is met, the sub-
sequent state estimates are also inconsistent. This is
because the filter believes the inconsistent state estimate
to be more accurate than it is in reality and hence it
y Models which are non-linear functions of the
uncertainties qp and qm, can be dealt with by augmenting thestate vector with the uncertainities. Expressed in this new statevector, the process and measurement models are of the form(10) and (11).
z Ck1qp, k1 of equation (1) corresponds to
qp, k1 Ck1qp, k1 of equation (12); hence instead of
Ck1Qk1CTk1 in equation (4), Q
k1 Ck1Qk1C
Tk1 is used.
Ekqm, k of equation (2) corresponds to qm, k Ekqm, k of
equation (13); hence instead of EkRkETk in equation (9),
Rk EkRkETk is used.
Kalman filters for non-linear systems 641
8/3/2019 Kalman Filters for Nonlinear Systems
5/16
attaches too much weight to this state estimate when
processing new measurements.
Testing for inconsistency is done by consistency
tests such as tests on the sum of a number of normalized
innovation squared values (SNIS) (Willsky 1976, Bar-
Shalom and Li 1993).
3.2. The information content of the state estimate
The calculated covariance matrix Pkji indicates how
uncertain the state estimate xxkji is: a large covariance
matrix indicates an inaccurate (and little useful) state
estimate; the smaller the covariance matrix, the larger
the information content of the state estimate. For
example both pdfs p1xkjZZi and p2xkjZZi of figure 1
are consistent with pxkjZZi, however, p1xkjZZi has a
smaller variance, hence it is more informative than
p2xkjZZi (a smaller domain of x values is probable).
The most informative, consistent approximation is the
Gaussian with the same mean and covariance as the
original distribution, i.e. p1xkjZZi for the example.
There is a trade-offbetween consistent and informa-
tive state estimates: inconsistency can be avoided by
making Pkji artificially larger (see equation (14)).
However, making Pkji too large, i.e. larger than neces-
sary for consistency, corresponds to losing information
about the actual accuracy of the state estimate.
The different KF variants linearize the process and
the measurement models in the uncertainty region
around the state estimate. Consistent estimates are
obtained by adding process and measurement uncer-
tainty on the linearized models to compensate for the
linearization errors. In order for the estimates to be
informative, (i) the linearization errors need to be as
small as possible; and (ii) the extra uncertainty on the
linearized models should not be larger than necessary
to compensate for these errors. The following sections
describe how the extended Kalman filter, the iterated
extended Kalman filter and the linear regression
Kalman filter differ in their linearization of the process
and measurement models; how they take the lineariza-
tion errors into account; and how the qualityy of the
state estimates, expressed in terms of consistency and
information content, depends on these two choices.
4. Non-linear process models
This section contains a comparison between the
process updates of the different KF variants when deal-
ing with a non-linear process model (10) with lineariza-tion (12). The KF variants differ by their choice of Fk1,
y The more non-linear the behaviour of the process ormeasurement model in the uncertainty region around thestate estimate, the more pronounced the difference in qualityperformance (consistency and information content of the stateestimates) between the KF variants.
Figure 1. Non-Gaussian pdfpxkjZZi with three Gaussian approximations p1xkjZZi, p2xkjZZi and p3xkjZZi. p1xkjZZi and
p2xkjZZi are consistent, p3xkjZZi is inconsistent. p1xkjZZi is more informative than p2xkjZZi.
642 T. Lefebvre et al.
8/3/2019 Kalman Filters for Nonlinear Systems
6/16
bk1 and Qk1. After linearization, they all use process
update equations (3) and (4) to update the state estimate
and its uncertainty.
Section 4.1 describes the linearization of the process
model by the EKF and IEKF, } 4.2 by the LRKF. The
formulas are summarized in table 1. Section 4.4 presents
some examples.
4.1. The (iterated) extended Kalman filterThe EKF and the IEKF linearizey the process model
by a first-order Taylor series around the updated state
estimate xxk1jk1
Fk1 @fk1@x
xxk1jk1
15
bk1 fk1xxk1jk1 Fk1xxk1jk1: 16
The basic (I)EKF algorithms do not take the lineariza-
tion errors into account (n is the dimension of the state
vector x)
Q
k1 0nn: 17
This leads to inconsistent state estimates when these
errors cannot be neglected.
4.2. The linear regression Kalman filter
The linear regression Kalman filter (LRKF) uses
the function values of r regression points Xjk1jk1 in
state space to model the behaviour of the process
function in the uncertainty region around the updated
state estimate xxk1jk1. The regression points are chosen
such that their mean and covariance matrix equal the
state estimate^xxk1jk1 and its covariance matrixPk1jk1. The CDF, DD1 and UKF filters correspond
to specific choices. The function values of the regression
points are
Xjkjk1 fk1X
jk1jk1: 18
The LRKF algorithm uses a linearized process
function (12) where Fk1, bk1 and Qk1 are obtained
by statistical linear regression through the
Xjk1jk1, X
jkjk1 points, j 1, . . . , r; i.e. the deviations
ej between the function values in Xjk1jk1 for the
non-linear and the linearized function are minimized in
least-squares sense
ej Xjkjk1 FX
jk1jk1 b 19
Fk1, bk1 arg minF, b
Xrj1
eTj ej: 20
The sample covariance of the deviations ej
Qk1 1
r
Xrj1
ejeT
j 21
gives an idea of the magnitude of the linearization errors
in the uncertainty region around xxk1jk1.
Intuitively we feel that when enoughz regression
points are taken, the state estimates of the LRKF proc-
ess update are consistent and informative. They are
consistent because Qk1 gives a well-founded approxi-mation of the linearization errors (equation (21)). They
are informative because the linearized model is a good
approximation of the process model in the uncertainty
region around xxk1jk1 (equations (19) and (20)).
4.3. Extra process uncertainty
In all of the presented filters, the user can decide to
add extra process uncertainty Qk1 (or to multiply the
calculated covariance matrix Pkjk1 by a fading factor
larger than 1 (Bar-Shalom and Li 1993)). This is useful if
the basic filter algorithm is not consistent. For example,
this is the case for the (I)EKF or for an LRKF with anumber of regression points too limited to capture the
y The EKF and IEKF only differ in their measurementupdate (} } 5.1 and 5.2).
z This depends on the non-linearity of the model in theuncertainty region around the state estimate. A possibleapproach is to increase the number of regression points untilthe resulting linearization (with error covariance) does notchange any more. Of course, because the true pdf isunknown, it is not possible to guarantee that the iterationhas converged to a set of regression points representative forthe model behaviour in the uncertainty region.
Fk1 bk1 Qk1
EKF@fk1@x
xxk1jk1
fk1xxk1jk1 Fk1xxk1jk1 0nn
IEKF@fk1@x
xxk1jk1
fk1xxk1jk1 Fk1xxk1jk1 0nn
LRKF arg minF, bPr
j1 eT
j ej argminF, bPr
j1 eT
j ej1r
Prj1 eje
Tj
Table 1. Summary of the linearization of the process model by the extended Kalman filter (EKF),the iterated extended Kalman filter (IEKF) and the linear regression Kalman filter (LRKF).
Kalman filters for non-linear systems 643
8/3/2019 Kalman Filters for Nonlinear Systems
7/16
non-linear behaviour of the process model in the uncer-
tainty region around xxk1jk1.
For a particular problem, values for Qk1 that result
in consistent and informative state estimates are
obtained by off-line tuning or on-line parameter learning
(adaptive filtering, Mehra 1972). In many practical cases
consistency is assured by taking the added uncertainty
too large, e.g. by taking a constant Q over time which
compensates for decreasing linearization errors. This,
however, results in less informative estimates.
4.4. Examples
The different process updates are illustrated by a
simple 2D non-linear process model (xi denotes the
ith element of x)
xk1 xk11 2
xk2 xk11 3xk12)
22
with no process uncertainty: qp, k1 02 1. xk1
depends non-linearly on xk1. The process update of
xk2 is linear. The updated state estimate and its uncer-
tainty at time step k 1 are
xxk1jk1 10
15
" #; Pk1jk1
36 0
0 3600
" #: 23
4.4.1. Monte Carlo simulation. The mean value and
the covariance of the true pdf pxkjZZk1 are calculated
with a (computationally expensive) Monte Carlo
simulation based on a Gaussian pdf pxk1jZZk1 with
mean^xxk1jk1 and covariance matrix Pk1jk1. Theresults of this computation are used to illustrate the
(in) consistency and information content of the state
estimates of the different KF variants. The mean and
covariance matrix of the pxkjZZk1 calculated by the
Monte Carlo algorithm are
xxkjk1 136
55
" #; Pkjk1
16 9 94 721
721 32 4 36
" #: 24
4.4.2. (I)EKF. Figure 2 shows the updated and (I)EKF
predicted state estimates and their uncertainty ellipsesy.
The dotted line is the uncertainty ellipse of the distri-bution obtained by Monte Carlo simulation. The IEKF
y The uncertainty ellipsoid
xk xxkjiTP
1kjixk xxkji 1 25
is a graphical representation of the uncertainity on the stateestimate xxkji. Starting from the point xxkji, the distance to theellipse in a direction is a measure for the uncertainty on xxkji inthat direction.
Figure 2. Non-linear process model. Uncertainty ellipses for the updated state estimate at k 1 (dashed line), for the (I)EKFpredicted state estimate (full line) and the Monte Carlo uncertainty ellipse (dotted line). The predicted state estimateis inconsistent due to the neglected linearization errors: the uncertainty ellipse of the IEKF predicted estimate is shiftedwith respect to the Monte Carlo uncertainty ellipse and is somewhat smaller.
644 T. Lefebvre et al.
8/3/2019 Kalman Filters for Nonlinear Systems
8/16
state prediction and its covariance matrix are
xxkjk1 100
55
!; Pkjk1
14 400 720
720 32 4 36
!: 26
Due to the neglected linearization errors, the state esti-
mate is inconsistent: the covariance Pkjk1 is smaller
than the covariance calculated by the Monte Carlo
simulation for the first state component x1 which
had a non-linear process update. For consistent results
this covariance should even be larger because the IEKF
estimate xxkjk1 differs from the mean of the pdf
pxkjZZk1, calculated by the Monte Carlo simulation.
4.4.3. LRKF. Figure 3 shows the Xjk1jk1 points (top
figure) and the Xjkjk1 points (bottom figure), the
updated state estimate (top) and the predicted state
Figure 3. Non-linear process model. Uncertainty ellipses for the updated state estimate at k 1 (dashed line, top figure), for theLRKF predicted state estimate (full line, bottom figure), and Monte Carlo uncertainty ellipse (dotted line which coincideswith the full line, bottom figure). The LRKF predicted state estimate is consistent and informative: its uncertainty ellipse
coincides with the Monte Carlo uncertainty ellipse.
Kalman filters for non-linear systems 645
8/3/2019 Kalman Filters for Nonlinear Systems
9/16
estimate (bottom) and their uncertainty ellipses for
the LRKF. The Xjk1jk1 points are chosen with the
UKF algorithm of Julier and Uhlmann (1996)
where 3 n 1. This corresponds to choosing six
regression points, including two times the point xxk1jk1.
The uncertainty ellipse obtained by Monte Carlo simu-
lation coincides with the final uncertainty ellipse of
the LRKF predicted state estimate (bottom figure).
This indicates consistent and informative results. The
LRKF predicted state estimate and its covariancematrix are
xxkjk1 136
55
!; Pkjk1
16 992 720
720 32 4 36
!: 27
4.5. Conclusion: the process update
The LRKF performs better than the (I)EKF when
dealing with non-linear process functions:
1. The LRKF linearizes the function based on its
behaviour in the uncertainty region around the
updated state estimate. The (I)EKF on the other
hand only uses the function evaluation and itsJacobian in this state estimate.
2. The LRKF deals with linearization errors in a
theoretically founded way (provided that enough
regression points are chosen). The (I)EKF on the
other hand needs trial and error for each particu-
lar example to obtain good values for the covari-
ance matrix representing the linearization errors.
3. Unlike the (I)EKF, the LRKF does not need the
function Jacobian. This is an advantage where
this Jacobian is difficult to obtain or non-existing
(e.g. for discontinuous process functions).
5. Non-linear measurement models
The previous section contains a comparison between
the (I)EKF and LRKF process updates; this section
focuses on their measurement updates for a non-linear
measurement model (11) with linearization (13). The
EKF, IEKF and LRKF choose Hk, dk and Rk in a
different way. After linearization they use the KF
update equations (5)(9).
The linearization of the measurement model by the
IEKF (} 5.2) takes the measurement into account; the
EKF (} 5.1) and LRKF (} 5.3) linearize the measure-
ment model based only on the predicted state estimate
and its uncertainty. For the latter filters, the lineariza-
tion errors (Rk) are larger, especially when the measure-
ment function is quite non-linear in the uncertainty
region around the predicted state estimate. A large
uncertainty on the linearized measurement model Rk
EkRkETk (due to a large uncertainty on the state esti-mate) results in throwing away the greater part of the
information of the possibly very accurate measurement.
The different linearization formulas are summarized in
table 2. Section 5.5 presents some examples.
5.1. The extended Kalman filter
The EKF linearizes the measurement model around
the predicted state estimate xxkjk1
Hk @hk@x xxkjk1 28
dk hkxxkjk1 Hkxxkjk1: 29
The basic EKF algorithm does not take the linearization
errors into account
Rk 0mm 30
where m is the dimension of the measurement vector zk.
If the measurement model is non-linear in the uncer-
tainty region around the predicted state estimate, the
linearization errors are not negligible. This means that
the linearized measurement model does not reflect the
relation between the true state value and the measure-ment. For instance, the true state value is far fromy the
linearized measurement model. After processing the
measured value, given the linear measurement model
and the measurement uncertainty, the state is believed
y Far from (and close to) must be understood as: thedeviation of the true state with respect to the linearizedmeasurement model is not justified (is justified) by the
measurement uncertainty ~RRk ~EEk ~RRkETk .
Hk dk Rk
EKF @hk
@x
xxkjk1hkxxkjk1 Hkxxkjk1 0mm
IEKF @hk@x
xxkjkhkxxkjk Hkxxkjk 0mm
LRKF minH, dPr
j1 eT
j ej minH, dPr
j1 eT
j ej1r
Prj1 eje
Tj
Table 2. Summary of the linearization of the measurement model by the extended Kalman filter(EKF), the iterated extended Kalman filter (IEKF) and the linear regression Kalman filter (LRKF).
646 T. Lefebvre et al.
8/3/2019 Kalman Filters for Nonlinear Systems
10/16
to be in a region which does not include the true state
estimate, i.e. the updated state estimate is inconsistent.
5.2. The iterated extended Kalman filter
The EKF of the previous section linearizes the
measurement model around the predicted state estimate.
The IEKF tries to do better by linearizing the measure-
ment model around the updated state estimate
Hk @hk
@x
xxkjk
31
dk hkxxkjk Hkxxkjk: 32
This is achieved by iteration: the filter first linearizes
the model around a value xx0kjk (often taken equal to
the predicted state estimate xxkjk1) and calculates the
updated state estimate. Then, the filter linearizes the
model around the newly obtained estimate xx1kjk and cal-
culates a new updated state estimate (based on xxkjk1,
Pkjk1 and the new linearized model). This process
is iterated until a state estimate xxikjk is found for which
xxikjk is close to xx
i1kjk . The state estimate xxkjk and uncer-
tainty Pkjk are calculated starting from the state estimate
xxkjk1 with its uncertainty Pkjk1 and the measurement
model linearized around xxikjk.
Like the EKF algorithm, the basic IEKF algorithm
does not take the linearization errors into account
Rk 0mm: 33
If the measurement model is non-linear in the uncer-
tainty region around the updated state estimate xxkjk,
state estimates will be inconsistent. In case of a meas-urement model that instantaneously fully observes the
state (or at least the part of the state that causes the
non-linearities in the measurement model), the lineariza-
tion errors will be smally in the uncertainty region
around xxkjk. The true state estimate is then close to
the linearized measurement function and the updated
state estimate is consistent. The result is also informa-
tive because no uncertainty due to linearization errors
needs to be added.
5.3. The linear regression Kalman filter
The LRKF evaluates the measurement function in r
regression points Xjkjk1 in the uncertainty region
around the predicted state estimate xxkjk1. The Xjkjk1
are chosen such that their mean and covariance matrix
are equal to the predicted state estimate xxkjk1 and its
covariance Pkjk1. The CDF, DD1 and UKF filters
correspond to specific choices. The function values
of the regression points through the non-linear func-
tion are
Zjk hkX
jkjk1: 34
The LRKF algorithm uses a linearized measurement
function (13) where Hk, dk and Rk are obtained by
statistical linear regression through the points
Xjkjk1, Z
jk, j 1, . . . , r. The statistical linear regres-
sion is such that the deviations ej between the non-linear
and the linearized function in the regression points are
minimized in least-squares sense
ej Zjk HX
jkjk1 d
35
Hk, dk arg minH, d
Xrj1
eTj ej: 36
The sample covariance matrix of the deviations ej givesan idea of the magnitude of the linearization errors
Rk 1
r
Xrj1
ejeT
j : 37
Intuitively we feel that when enough regression
points (Xjkjk1, Z
jk) are taken the state estimates of
the LRKF measurement update are consistent because
Rk gives a well-founded approximation of the lineariza-
tion errors (equation (37)). However, if the measurement
model is highly non-linear in the uncertainty region
around xxkjk1, the (Xjkjk1, Z
jk) points deviate substan-
tially from a hyperplane. This results in a large R
k andnon-informative updated state estimates (see the
example in } 5.5).
5.4. Extra measurement uncertainty
In order to make the state estimates consistent, the
user can tune an inconsistent filter by adding extra
measurement uncertainty Rk.
Only off-line tuning or on-line parameter learning
can lead to a good value for Rk for a particular prob-
lem. In many practical cases consistency is assured by
choosing the added uncertainty too large, e.g. by taking
a constantR
over time which compensates for decreas-
ing linearization errors. This reduces the information
content of the results.
5.5. Examples
5.5.1. First example. The comparison between the dif-
ferent measurement updates is illustrated with the meas-
urement function zk h1xk m, k
h1xk xk1 2 xk2
2 38
y This assumes that the iterations lead to an accurate xxikjk.
The linearizations are started around a freely choosen xx0kjk. In
order to assure quick and correct iteration, (part of) this valuecan be chosen based on the measurement information if thisinformation is more accurate than the predicted state estimate.
Kalman filters for non-linear systems 647
8/3/2019 Kalman Filters for Nonlinear Systems
11/16
where
xk 15
20
" #
is the true value and
^xxkjk1
10
15" #
is the predicted state estimate with covariance matrix
Pkjk1 36 0
0 3600
" #:
The processed measurement is zzk 630 and the meas-
urement covariance is Rk 400.
5.5.2. Second example. To illustrate the consistency of
the state estimate of an IEKF when the measurement
observes the state completely, a second example is used.
The measurement function is
zk hxk qm, k h1xk qm, k1
h2xk qm, k2
" #39
with
h1xk xk1 2 xk2
2
h2xk 3 xk2 2=xk1
)40
where
xk 15
20
" #
is the true value and
xxkjk1 10
15
" #
is the predicted state estimate with covariance matrix
Pkjk1 36 0
0 3600
" #:
The processed measurement and the measurement
covariance matrix are
zzk 630
85" #; Rk 400 0
0 400" #: 41
In all figures, the true state value xk is plotted; if
this value is far outside the uncertainty ellipse of a
state estimate, the corresponding estimate is inconsis-
tent. Because the measurement is accurate and the
initial estimate is not, the uncertainty on the state esti-
mate should drop considerably when the measurement
is processed. The updated state estimate is not informa-
tive if this is not the case.
5.5.3. EKF. Figure 4 shows the state estimates, uncer-
tainty ellipses and measurement functions for the EKF
applied to the first example (equation (38)). The true
measurement function is non-linear. xk is the true
value of the state, and is close to this function. The
linearization around the uncertain predicted state esti-
mate is not a good approximation of the function
around the true state value: the true state value is farfrom the linearized measurement function. The result-
ing updated state estimate
xxkjk 10
25
" #; Pkjk
36 24
24 16
" #42
is inconsistent.
5.5.4. IEKF. Figure 5 shows the measurement function,
the linearized measurement function around the point
xxikjk, the true state value xk and the state estimates for
the IEKF applied to the first example (equation (38)).
The measurement model does not fully observe the state.
This results in an uncertain updated state estimate xxikjkaround which the filter linearizes the measurement func-
tion. As was the case for the EKF, the linearization
errors are not negligible and the true value is far
from the linearized measurement function. The updated
state estimate
xxkjk 10
23
" #; Pkjk
36 16
16 7:0
" #43
is inconsistent.
If however the measurement model fully observes the
state, the IEKF updated state estimate is accurately
known; hence, the linearization errors are small andthe true state value is close to the linearized measure-
ment function. In this case, the updated state estimate is
consistent. Figure 6 shows the measurement function,
the linearized measurement function, the true state
value xk, the state estimates and the uncertainty ellipses
for the IEKF applied to the second example (equations
(39) and (40)). The updated state estimate and covari-
ance matrix
xxkjk 14
21
" #; Pkjk
2:6 1:7
1:7 1:3
" #44
are consistent and informative due to the small, ignored,linearization errors.
5.5.5. LRKF. An LRKF is run on the first example
(equation (38)). The Xjkjk1 points are chosen with the
UKF algorithm with 3 n 1 (Julier and Uhlmann
1996). This corresponds to choosing six regression
points, including two times the point xxkjk1. Figures 7
and 8 show the non-linear measurement function, the
Xjkjk1-points and the linearization. The predicted
648 T. Lefebvre et al.
8/3/2019 Kalman Filters for Nonlinear Systems
12/16
Figure 5. Non-linear measurement model zz h1xk that does not observe the full state, and its IEKF linearization around xxkjk(dotted lines). The true state xk is far from this linearization, leading to an inconsistent state estimate xxkjk (uncertaintyellipse in full line).
Figure 4. Non-linear measurement model zz h1xk and EKF linearization around xxkjk1 (dotted lines). The true state xk is farfrom this linearization and the obtained state estimate xxkjk (uncertainty ellipse in full line) is inconsistent.
Kalman filters for non-linear systems 649
8/3/2019 Kalman Filters for Nonlinear Systems
13/16
Figure 6. Non-linear measurement model zz hxk that fully observes the state, and its IEKF linearization around xxkjk (dottedlines). The true state xk is close to this linearization, leading to a consistent state estimate (uncertainty ellipse in full line).
Figure 7. Non-linear measurement model zz h1x and LRKF linearization. The linearization errors are large.
650 T. Lefebvre et al.
8/3/2019 Kalman Filters for Nonlinear Systems
14/16
state estimate is uncertain, hence the Xjkjk1-points are
widespread. Rk is large (Rk 2:6 10
7) due to the large
deviations between the (Xjkjk1, Z
jk) points and the
linearized measurement function (see figure 8). The
updated state estimate and its covariance matrix are
xxkjk 10
2:6
" #; Pkjk
36 0
0 3600
" #: 45
Figure 9 shows the Xjkjk1 points, the measurement
function, the LRKF linearized measurement function,
the true state value xk, the state estimates and the
uncertainty ellipses. The updated state estimate is
consistent. However, it can hardly be called an improve-
ment over the previous state estimate (Pkjk % Pkjk1).
The information in the measurement is neglected due
to the high measurement uncertainty Rk EkRkETk
on the linearized function.
Figure 9. Non-linear measurement model zz h1x and LRKF linearization (dotted lines). The large linearization errors result
in a large measurement uncertainty Rk EkRkETk . The updated state estimate (uncertainty ellipse in full line) is consistent
but non-informative.
Figure 8. Figure 7 seen from another angle.
Kalman filters for non-linear systems 651
8/3/2019 Kalman Filters for Nonlinear Systems
15/16
Note that some kind of iterated LRKF (similar to
the iterated EKF) would not solve this problem: the
updated state estimate xxkjk and its covariance matrix
Pkjk are more or less the same as the predicted state
estimate xxkjk1 and its covariance matrix Pkjk1.
Hence, the regression points and the linearization
would be approximately the same after iteration.
5.6. Conclusion: the measurement update
Measurements which fully observe the part of the state
that makes the model non-linear, are best processed by the
IEKF. In this case (and assuming that the algorithm
iterates to a good linearization pointy), the IEKF lin-
earization errors are negligible.
In the other cases, none of the presented filters out-
performs the others. A filter should be chosen for each
specific application: the LRKF makes an estimate of its
linearization errors (Rk), the EKF and IEKF on the
other hand require off-line tuning or on-line parameter
learning of R
kto yield consistent state estimates.
Because the IEKF additionally takes the measurement
into account when linearizing the measurement model,
its linearization errors are smaller than those of the
EKF and LRKF. This means that once a well-tuned
IEKF is available, the state estimates it returns can
be far more informative than those of the LRKF or a
well-tuned EKF.
Finally, note that the LRKF does not use the
Jacobian of the measurement function, which makes
it possible to process discontinuous measurement
functions.
6. Conclusions
This paper gives insight into the advantages and draw-
backs of the extended Kalman filter (EKF), the iterated
extended Kalman filter (IEKF) and the linear regression
Kalman filter (LRKF). These insights are a result of the
distinct analysis approach taken in this paper:
1. The paper describes all filter algorithms as the
application of the basic KF algorithm to linear-
ized process and measurement models. The differ-
ence between the KF variants is situated in the
choice of linearization and the compensation for
the linearization errors. In previous work thislinearization was not always recognized, e.g. the
UKF is originally derived as a filter which does
not linearize the models.
2. The analysis clarifies how some filters automati-
cally adapt their compensation for linearization
errors, while other filters have constant (devel-
oper-determined) error compensation.
3. The quality of the state estimates is expressed by
two criteria: the consistency and the information
content of the estimates. This paper relates the
consistency and information content of the esti-mates to (i) how the linearization is performed
and (ii) how the linearization errors are taken
into account. The understanding of the lineariza-
tion processes allows us to make a well-founded
choice of filter for a specific application.
4. The performance of the different filters is com-
pared for the process and measurement updates
separately, because a good performance for one
of these updates does not necessarily mean a good
performance for the other update. This makes it
interesting in some cases to use different filters
for both updates.For process updates the LRKF performs better
than the other mentioned KF variants because
(i) the LRKF linearizes the process model based
on its behaviour in the uncertainty region around
the updated state estimate. The (I)EKF on the
other hand only uses the function evaluation
and its Jacobian in this state estimate; and
(ii) the LRKF deals with linearization errors in a
theoretically founded way, provided that enough
regression points are chosen. The (I)EKF on the
other hand needs trial and error for each particu-
lar application to obtain a good covariancematrix representing the linearization errors.
The IEKF is the best way to handle non-linear
measurement models that fully observe the part
of the state that makes the measurement model
non-linear. In the other cases, none of the pres-
ented filters outperforms the others: the LRKF
makes an estimation of the linearization errors;
the EKF and IEKF on the other hand require
extensive off-line tuning or on-line parameter
learning in order to yield consistent state esti-
mates. However, unlike the EKF and LRKF,
the IEKF additionally uses the measurement
value in order to linearize the measurementmodel. Hence, its linearization errors are smaller
and once a well-tuned IEKF is available, the state
estimates it returns can be far more informative
than those of the LRKF or a well-tuned EKF.
The insights described in this paper are important
for all researchers and developers who want to apply a
KF variant to a specific application. They lead to a
systematic choice of a filter, where previously the choice
y This assumes that the iterations lead to an accurate xxikjk.
The linearizations are started around a freely choosen xx0kjk. In
order to assure quick and correct iteration, (part of) this valuecan be chosen based on the measurement information if thisinformation is more accurate than the predicted state estimate.
652 T. Lefebvre et al.
8/3/2019 Kalman Filters for Nonlinear Systems
16/16
was mainly made based on success in similar applica-
tions or based on trial and error. Further work should
report on practical applications using these insights
and an effort should be made to analyse and include
future KF algorithms in this framework.
Acknowledgements
T. Lefebvre is a Postdoctoral Fellow of the Fund
for Scientific ResearchFlanders (FWO) in Belgium.
Financial support by the Belgian Programme on
Inter-University Attraction Poles initiated by the
Belgian StatePrime Ministers OfficeScience
Policy Programme (IUAP), and by K. U. Leuvens
Concerted Research Action GOA/99/04 are gratefully
acknowledged.
References
Bar-Shalom, Y., and Li, X., 1993, Estimation and Tracking:Principles, Techniques and Software (Boston, London:
Artech House).Bayes, T., 1763, An essay towards solving a problem in the
doctrine of chances. Philosophical Transactions of the RoyalSociety of London, 53, 370418. Reprinted in Biometrika, 45,293315, 1958.
Gelb , A., Kasper, J., Nash , R., Price, C., and Sutherland,A., 1974, Applied Optimal Estimation (Cambridge, MA:MIT Press).
Julier, S., and Uhlmann, J., 1996, A general method forapproximating nonlinear transformations of probabilitydistributions. Technical report, Robotics Research Group,Department of Engineering Science, University of Oxford.
Julier, S., and Uhlmann, J., 2001, Data fusion in non-linearsystems. In D. Hall and J. Llinas (Eds) Handbook ofMultisensor Data Fusion (New York: CRC Press), Chapter
13.
Julier, S., Uhlmann, J., and Durrant-Whyte, H., 2000,A new method for the transformation of means and covari-
ances in filters and estimators. IEEE Transactions on
Automatic Control, 45(3), 477482.Kalman, R. E., 1960, A new approach to linear filtering and
prediction problems. Transactions of the ASME, Journal ofBasic Engineering, 82, 3445.
Laplace, P.-S., 1812, Theorie Analytique des Probabilites
(Paris: Courcier Imprimeur).Lefebvre, T., Bruyninckx, H., and De Schutter, J., 2002,
Comment on A new method for the nonlinear transforma-tion of means and covariances in filters and estimators.
IEEE Transactions on Automatic Control, 47(8), 14061408.
Maybeck, P., 1982, Stochastic Models, Estimation, andControl, Volume 2 (New York: Academic Press).
Mehra, R., 1972, Approaches to adaptive filtering. IEEETransactions on Automatic Control, 17(5), 693698.
Nrgaard, M., Poulsen, N., and Ravn , O., 2000a,Advances in derivative-free state estimation for nonlinear
systems. Technical Report IMM-REP-1998-15 (revisededition), Technical University of Denmark, Denmark.
Nrgaard, M., Poulsen, N., and Ravn , O., 2000b, Newdevelopments in state estimations for nonlinear systems.
Automatica, 36(11), 16271638.Schei, T., 1997, A finite-difference method for linearisation
in nonlinear estimation algorithms. Automatica, 33(11),20532058.
Sorenson, H. W., 1985, Kalman Filtering: Theory andApplication (New York, NY: IEEE Press).
Tanizaki, H., 1996, Nonlinear Filters: Estimation andApplicationsSecond, Revised and Enlarged Edition(Berlin-Heidelberg: Springer-Verlag).
Uhlmann, J., 1995, Dynamic map building and localization:new theoretical foundations. PhD thesis, University of
Oxford, UK.Willsky, A., 1976, A survey of design methods for failure
detection in dynamic systems. Automatica, 12, 601611.
Kalman filters for non-linear systems 653