7/22/2019 Simo Sarkka (Applications of Bayesian Filtering and Smoothing) http://slidepdf.com/reader/full/simo-sarkka-applications-of-bayesian-filtering-and-smoothing 1/16 This PDF version is made available for personal use. The copyright in all material rests with the author (Simo S¨ arkk ¨ a). Commercial reproduction is prohibited, except as authorised by the author and publisher. 1 What are Bayesian filtering and smoothing? The term optimal filtering traditionally refers to a class of methods that can be used for estimating the state of a time-varying system which is indi- rectly observed through noisy measurements. The term optimal in this con- text refers to statistical optimality. Bayesian filtering refers to the Bayesian way of formulating optimal filtering. In this book we use these terms inter- changeably and always mean Bayesian filtering. In optimal, Bayesian, and Bayesian optimal filtering the state of the sys- tem refers to the collection of dynamic variables such as position, veloc- ity, orientation, and angular velocity, which fully describe the system. The noise in the measurements means that they are uncertain; even if we knew the true system state the measurements would not be deterministic func- tions of the state, but would have a distribution of possible values. The time evolution of the state is modeled as a dynamic system which is perturbed by a certain process noise. This noise is used for modeling the uncertainties in the system dynamics. In most cases the system is not truly stochastic, but stochasticity is used for representing the model uncertainties. Bayesian smoothing (or optimal smoothing) is often considered to be a class of methods within the field of Bayesian filtering. While Bayesian filters in their basic form only compute estimates of the current state of the system given the history of measurements, Bayesian smoothers can be used to reconstruct states that happened before the current time. Although the term smoothing is sometimes used in a more general sense for methods which generate a smooth (as opposed to rough) representation of data, in the context of Bayesian filtering the term (Bayesian) smoothing has this more definite meaning. 1.1 Applications of Bayesian filtering and smoothing Phenomena which can be modeled as time-varying systems of the above type are very common in engineering applications. This kind of model 1
16
Embed
Simo Sarkka (Applications of Bayesian Filtering and Smoothing)
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
7/22/2019 Simo Sarkka (Applications of Bayesian Filtering and Smoothing)
This PDF version is made available for personal use. The copyright in all material rests with the author (Simo Sarkk a). Commercial
reproduction is prohibited, except as authorised by the author and publisher.
1
What are Bayesian filtering and smoothing?
The term optimal filtering traditionally refers to a class of methods that
can be used for estimating the state of a time-varying system which is indi-
rectly observed through noisy measurements. The term optimal in this con-
text refers to statistical optimality. Bayesian filtering refers to the Bayesianway of formulating optimal filtering. In this book we use these terms inter-
changeably and always mean Bayesian filtering.
In optimal, Bayesian, and Bayesian optimal filtering the state of the sys-
tem refers to the collection of dynamic variables such as position, veloc-
ity, orientation, and angular velocity, which fully describe the system. The
noise in the measurements means that they are uncertain; even if we knew
the true system state the measurements would not be deterministic func-
tions of the state, but would have a distribution of possible values. The time
evolution of the state is modeled as a dynamic system which is perturbed
by a certain process noise. This noise is used for modeling the uncertainties
in the system dynamics. In most cases the system is not truly stochastic, butstochasticity is used for representing the model uncertainties.
Bayesian smoothing (or optimal smoothing) is often considered to be
a class of methods within the field of Bayesian filtering. While Bayesian
filters in their basic form only compute estimates of the current state of
the system given the history of measurements, Bayesian smoothers can be
used to reconstruct states that happened before the current time. Although
the term smoothing is sometimes used in a more general sense for methods
which generate a smooth (as opposed to rough) representation of data, in
the context of Bayesian filtering the term (Bayesian) smoothing has this
more definite meaning.
1.1 Applications of Bayesian filtering and smoothing
Phenomena which can be modeled as time-varying systems of the above
type are very common in engineering applications. This kind of model
1
7/22/2019 Simo Sarkka (Applications of Bayesian Filtering and Smoothing)
This PDF version is made available for personal use. The copyright in all material rests with the author (Simo Sarkk a). Commercial
reproduction is prohibited, except as authorised by the author and publisher.
2 What are Bayesian filtering and smoothing?
can be found, for example, in navigation, aerospace engineering, space en-
gineering, remote surveillance, telecommunications, physics, audio signalprocessing, control engineering, finance, and many other fields. Examples
of such applications are the following.
Global positioning system (GPS) (Kaplan, 1996) is a widely used satel-
lite navigation system, where the GPS receiver unit measures arrival
times of signals from several GPS satellites and computes its position
based on these measurements (see Figure 1.1). The GPS receiver typi-
cally uses an extended Kalman filter (EKF) or some other optimal filter-
ing algorithm1 for computing the current position and velocity such that
the measurements and the assumed dynamics (laws of physics) are taken
into account. Also the ephemeris information, which is the satellite ref-
erence information transmitted from the satellites to the GPS receivers,is typically generated using optimal filters.
Figure 1.1 In the GPS system, the measurements are time delaysof satellite signals and the optimal filter (e.g., extended Kalmanfilter, EKF) computes the position and the accurate time.
Target tracking (Bar-Shalom et al., 2001; Crassidis and Junkins, 2004;
Challa et al., 2011) refers to the methodology where a set of sensors
such as active or passive radars, radio frequency sensors, acoustic arrays,
1 Strictly speaking, the EKF is only an approximate optimal filtering algorithm, because it
uses a Taylor series based Gaussian approximation to the non-Gaussian optimal filtering
solution.
7/22/2019 Simo Sarkka (Applications of Bayesian Filtering and Smoothing)
This PDF version is made available for personal use. The copyright in all material rests with the author (Simo Sarkk a). Commercial
reproduction is prohibited, except as authorised by the author and publisher.
1.1 Applications of Bayesian filtering and smoothing 3
infrared sensors, and other types of sensors are used for determining
the position and velocity of a remote target (see Figure 1.2). When thistracking is done continuously in time, the dynamics of the target and
measurements from the different sensors are most naturally combined
using an optimal filter or smoother. The target in this (single) target
tracking case can be, for example, a robot, a satellite, a car or an airplane.
Figure 1.2 In target tracking, a sensor (e.g., radar) generatesmeasurements (e.g., angle and distance measurements) of thetarget, and the purpose is to determine the target trajectory.
Multiple target tracking (Bar-Shalom and Li, 1995; Blackman and
Popoli, 1999; Stone et al., 1999; Sarkk a et al., 2007b) systems are used
for remote surveillance in the cases where there are multiple targets
moving at the same time in the same geographical area (see Figure 1.3).
This introduces the concept of data association (which measurement
was from which target?) and the problem of estimating the number of
targets. Multiple target tracking systems are typically used in remote
surveillance for military purposes, but their civil applications are, for
example, monitoring of car tunnels, automatic alarm systems, and
people tracking in buildings.
Inertial navigation (Titterton and Weston, 1997; Grewal et al., 2001)
uses inertial sensors such as accelerometers and gyroscopes for comput-ing the position and velocity of a device such as a car, an airplane, or
a missile. When the inaccuracies in sensor measurements are taken into
7/22/2019 Simo Sarkka (Applications of Bayesian Filtering and Smoothing)
This PDF version is made available for personal use. The copyright in all material rests with the author (Simo Sarkk a). Commercial
reproduction is prohibited, except as authorised by the author and publisher.
4 What are Bayesian filtering and smoothing?
Figure 1.3 In multiple target tracking the data associationproblem has to be solved, because it is impossible to knowwithout any additional information which target produced whichmeasurement.
account the natural way of computing the estimates is by using an op-
timal filter or smoother. Also, in sensor calibration, which is typically
done in a time-varying environment, optimal filters and smoothers can
be applied.
Integrated inertial navigation (Grewal et al., 2001; Bar-Shalom et al.,
2001) combines the good sides of unbiased but inaccurate sensors, suchas altimeters and landmark trackers, and biased but locally accurate in-
ertial sensors. A combination of these different sources of information
is most naturally performed using an optimal filter such as the extended
Kalman filter. This kind of approach was used, for example, in the guid-
ance system of the Apollo 11 lunar module (Eagle), which landed on the
moon in 1969.
GPS/INS navigation (Grewal et al., 2001; Bar-Shalom et al., 2001) is a
form of integrated inertial navigation where the inertial navigation sys-
tem (INS) is combined with a GPS receiver unit. In a GPS/INS naviga-
tion system the short term fluctuations of the GPS can be compensated
by the inertial sensors and the inertial sensor biases can be compensated
by the GPS receiver. An additional advantage of this approach is that
it is possible to temporarily switch to pure inertial navigation when the
GPS receiver is unable to compute its position (i.e., has no fix) for some
reason. This happens, for example, indoors, in tunnels and in other cases
7/22/2019 Simo Sarkka (Applications of Bayesian Filtering and Smoothing)
This PDF version is made available for personal use. The copyright in all material rests with the author (Simo S arkk a). Commercial
reproduction is prohibited, except as authorised by the author and publisher.
1.3 Optimal filtering and smoothing as Bayesian inference 9
0 5 10 15
−0.25
−0.2
−0.15
−0.1
−0.05
0
0.05
0.1
0.15
0.2
0.25
Time step k
R e s o n a t o r p o s i t i o n x k
Signal
Measurement
Figure 1.6 An example of time series, which models adiscrete-time resonator. The actual resonator state (signal) ishidden and only observed through the noisy measurements.
noisy measurements fy1; y2; : : :g as illustrated in Figure 1.5. An example
of this kind of time series is shown in Figure 1.6. The process shown is a
noisy resonator with a known angular velocity. The state xk D .xk Pxk/T
is two dimensional and consists of the position of the resonator xk and
its time derivative Pxk. The measurements yk are scalar observations of the
resonator position (signal) and they are corrupted by measurement noise.The purpose of the statistical inversion at hand is to estimate the hid-
den states x0WT D fx0; : : : ; xT g from the observed measurements y1WT D
fy1; : : : ; yT g, which means that in the Bayesian sense we want to compute
the joint posterior distribution of all the states given all the measurements.
In principle, this can be done by a straightforward application of Bayes’
rule
p.x0WT j y1WT / D p.y1WT j x0WT / p.x0WT /
p.y1WT / ; (1.1)
where
p.x0WT /; is the prior distribution defined by the dynamic model, p.y1WT j x0WT / is the likelihood model for the measurements,
I · ~
7/22/2019 Simo Sarkka (Applications of Bayesian Filtering and Smoothing)
This PDF version is made available for personal use. The copyright in all material rests with the author (Simo Sarkk a). Commercial
reproduction is prohibited, except as authorised by the author and publisher.
1.3 Optimal filtering and smoothing as Bayesian inference 11
Because computing the full joint distribution of the states at all time steps is
computationally very inefficient and unnecessary in real-time applications,in Bayesian filtering and smoothing the following marginal distributions
are considered instead (see Figure 1.7).
Filtering distributions computed by the Bayesian filter are the marginal
distributions of the current state xk given the current and previous mea-
surements y1Wk D fy1; : : : ; ykg:
p.xk j y1Wk/; k D 1; : : : ; T : (1.4)
The result of applying the Bayesian filter to the resonator time series in
Figure 1.6 is shown in Figure 1.8.
Prediction distributions which can be computed with the prediction step
of the Bayesian filter are the marginal distributions of the future statexkCn, n steps after the current time step:
p.xkCn j y1Wk/; k D 1; : : : ; T; n D 1; 2 ; : : : : (1.5)
Smoothing distributions computed by the Bayesian smoother are the
marginal distributions of the state xk given a certain interval y1WT D
fy1; : : : ; yT g of measurements with T > k:
p.xk j y1WT /; k D 1; : : : ; T : (1.6)
The result of applying the Bayesian smoother to the resonator time series
is shown in Figure 1.9.
Figure 1.7 State estimation problems can be divided into optimalprediction, filtering, and smoothing depending on the time span of the measurements available with respect to the time of theestimated state.
7/22/2019 Simo Sarkka (Applications of Bayesian Filtering and Smoothing)
This PDF version is made available for personal use. The copyright in all material rests with the author (Simo S arkk a). Commercial
reproduction is prohibited, except as authorised by the author and publisher.
12 What are Bayesian filtering and smoothing?
0 5 10 15
−0.25
−0.2
−0.15
−0.1
−0.05
0
0.05
0.1
0.15
0.2
0.25
Time step k
R e s o n a t o r p o s i t i o n x k
Signal
Measurement
Filter Estimate
95% Quantile
Figure 1.8 The result of computing the filtering distributions forthe discrete-time resonator model. The estimates are the means of the filtering distributions and the quantiles are the 95% quantilesof the filtering distributions.
Computing filtering, prediction, and smoothing distributions require only
a constant number of computations per time step, and thus the problem of
processing arbitrarily long time series is solved.
1.4 Algorithms for Bayesian filtering and smoothing
There exist a few classes of filtering and smoothing problems which have
closed form solutions.
The Kalman filter (KF) is a closed form solution to the linear Gaussian
filtering problem. Due to linear Gaussian model assumptions the poste-
rior distribution is exactly Gaussian and no numerical approximations
are needed.
The Rauch–Tung–Striebel smoother (RTSS) is the corresponding closed
form smoother for linear Gaussian state space models.
Grid filters and smoothers are solutions to Markov models with finite
state spaces.
·· ~ t ~ t H H~ ~
7/22/2019 Simo Sarkka (Applications of Bayesian Filtering and Smoothing)
This PDF version is made available for personal use. The copyright in all material rests with the author (Simo S arkk a). Commercial
reproduction is prohibited, except as authorised by the author and publisher.
1.4 Algorithms for Bayesian filtering and smoothing 13
0 5 10 15
−0.25
−0.2
−0.15
−0.1
−0.05
0
0.05
0.1
0.15
0.2
0.25
Time step k
R e s o n a t o r p o s i t i o n x k
Signal
Measurement
Smoother Estimate
95% Quantile
Figure 1.9 The result of computing the smoothing distributionsfor the discrete-time resonator model. The estimates are themeans of the smoothing distributions and the quantiles are the95% quantiles of the smoothing distributions.
But because the Bayesian optimal filtering and smoothing equations are
generally computationally intractable, many kinds of numerical approxi-
mation methods have been developed, for example:
The extended Kalman filter (EKF) approximates the non-linear and non-
Gaussian measurement and dynamic models by linearization, that is,
by forming a Taylor series expansion at the nominal (or maximum a
posteriori, MAP) solution. This results in a Gaussian approximation to
the filtering distribution.
The extended Rauch–Tung–Striebel smoother (ERTSS) is the approxi-
mate non-linear smoothing algorithm corresponding to EKF.
The unscented Kalman filter (UKF) approximates the propagation
of densities through the non-linearities of measurement and noise
processes using the unscented transform. This also results in a Gaussian
approximation.
The unscented Rauch–Tung–Striebel smoother (URTSS) is the approxi-
mate non-linear smoothing algorithm corresponding to UKF. Sequential Monte Carlo methods or particle filters and smoothers repre-
sent the posterior distribution as a weighted set of Monte Carlo samples.
0
tr t t1 _
7/22/2019 Simo Sarkka (Applications of Bayesian Filtering and Smoothing)