Linear Filters
Linear Filters
denote a bivariate time series with zero mean.
Let
Tt
y
x
t
t :
The time series {yt : t T} is said to be constructed from {xt : t T} by means of a Linear Filter.
sstst xay
Suppose that the time series {yt : t T} is constructed as follows:
X t YtOutput Process
Input Process
LinearFilter
Schematic Diagram of a Linear Filter
hyy
s sxxss sshaa
'' '
dfeae xxs
sis
hi
2
dfAe xxhi
2
The autocovariance function of the filtered series
Thus the spectral density of the time series
{yt : t T} is:
xxxx
s
sisyy fAfeaf
22
Comment A:
is called the Transfer function of the linear filter.
is called the Gain of the filter while
is called the Phase Shift of the filter.
s
siseaA
A
Aarg
Also httxy yxEh
sxxs sha
dfAe xxhi
Thus cross spectrum of the bivariate time series
Tt
y
x
t
t :
is:
xx
sxx
sisxy fAfeaf
Definition:
= Squared Coherency function
jjii
ij
ij ff
fK
2
2
12 ijKNote:
Comment B:
= Squared Coherency function.
yyxx
xy
xy ff
fK
2
2
1 2
22
xxxx
xx
fAf
fA
if {yt : t T} is constructed from {xt : t T} by means of a linear filter
Linear Filterswith additive noise at the output
denote a bivariate time series with zero mean.
Let
t = ..., -2, -1, 0, 1, 2, ...
Tt
y
x
t
t :
Suppose that the time series {yt : t T} is constructed as follows:
ts
stst vxay
The noise {vt : t T} is independent of the series {xt : t T} (may be white)
X t YtOutput Process
Input Process
LinearFilter
Schematic Diagram of a Linear Filter with Noise
N t Additive Noise at Output
t
httyy yyEh
hsshaa vvs s
xxss
'' '
dfedfeae vvhi
xxs
sis
hi
2
The autocovariance function of the filtered series with added noise
continuing
hyy
dffAe vvxxhi 2
s
siseaA where
Thus the spectral density of the time series
{yt : t T} is:
vvxxyy ffAf 2
Also httxy yxEh
sxxs sha
dfea xxs
shis
dfAe xxhi
Thus cross spectrum of the bivariate time series
Tt
y
x
t
t :
is:
xx
sxx
sisxy fAfeaf
Thus
= Squared Coherency function.
yyxx
xy
xy ff
fK
2
2
vvxxxx
xx
ffAf
fA
2
22
1
1
1
1
2
xx
vv
fA
f
Noise to Signal Ratio
Box-Jenkins Parametric Modelling of a Linear Filter
Consider the Linear Filter of the time series {Xt : t T}:
ts
stst XBaXaY
0
0s
ss BaBawhere
0s
sis
i eaeaA and
= the Transfer function of the filter.
{at : t T} is called the impulse response function of the filter since if X0 =1and Xt = 0 for t ≠ 0, then :
ts
stst aXaY
0
for t T
Linear
Filter
Xt at
Also Note:
01
01
ssts
sststtt XaXaYYY
0
1
ss stst XXa
0ssts Xa
Hence {Yt} and {Xt} are related by the same Linear Filter.
ts
stst XBaXaY
0
Definition The Linear Filter
is said to be stable if :
0s
ss BaBa
converges for all |B| ≤1.
Discrete Dynamic Models:
Many physical systems whose output is represented by Y(t) are modeled by the following differential equation:
tYdt
d
dt
d
dt
dI
r
r
r
...
2
2
21
tX
dt
d
dt
d
dt
dI
s
s
s...2
2
21
Where X(t) is the forcing function.
If X and Y are measured at discrete times this equation can be replaced by:
tr
r YI ...221
bts
s XI ...221
where = I-B denotes the differencing operator.
This equation can in turn be represented with the operator B.
tr
r YBBBI ...221
bts
s XBBBI ...2210
tbs
s XBBBBI ...2210
or tt XBYB
sbs
bbb BBBBB ...22
110
where
This equation can also be written in the form as a Linear filter as
Stability: It can easily be shown that this filter is stable if the roots of (x) = 0 lie outside the unit circle.
ttt XBaXBBY 1
Determining the Impulse Response function from the Parameters of
the Filter:
Now
BBBa 1
or BBaB
Hence
...... 2210
221 BaBaIaBBBI r
r
sbs
bbb BBBB ...22
110
Equating coefficients results in the following conclusions:
aj = 0 for j < b.
aj - 1aj-1 - 2aj-2-...- r aj-r= j
or aj = 1j-1 + 2aj-2+...+ r aj-r+ j
for b ≤ j ≤ b+s.
and aj - 1aj-1 - 2aj-2-...- r aj-r= 0
or aj = 1aj-1 + 2aj-2+...+ r aj-r for j > b+s.
Thus the coefficients of the transfer function,
a0, a1, a2,... ,
satisfy the following properties
1) b zeroes a0, a1, a2,..., ab-1
2) No pattern for the next s-r+1 values
ab, ab+1, ab+2,..., ab+s-r
3) The remaining values
ab+s-r+1, ab+s-r+2, ab+s-r+3,...
follow the pattern of an rth order difference equation
aj = 1aj-1 + 2aj-2+...+ r aj-r
Example r =1, s=2, b=3, 1 = a0 = a1 = a2 = 0
a3 = a2 + 0 = 0
a4 = a3 + 1 = 0 + 1
a5 = a4 + 2
= [0 + 1] + 2
= 2w0 + 1 + 2
aj = aj-1 for j ≥ 6.
Transfer function {at}
0 1 2 3 5 6 7 8
Exponentially decaying Transfer Function
s-r+1b zeroes
as
Identification of the Box-Jenkins Transfer
Model with r=2
Recall the solution to the second order difference equation
aj = 1aj-1 + 2aj-2
follows the following patterns:
1) Mixture of exponentials if the roots of
1 - 1x - 2x2 = 0 are real.2) Damped Cosine wave if the roots to
1 - 1x - 2x2 = 0 are complex.
These are the patterns of the Impulse Response function one looks for when identifying b,r and s.
Estimation of the Impulse Response function, aj
(without pre-whitening).
Suppose that {Yt : t T} and {Xt : t T}are weakly stationary time series satisfying the following equation:
ts
stst NXaY
0
Also assume that {Nt : t T} is a weakly stationary "noise" time series, uncorrelated with
{Xt : t T}. Then
t
sshtsthttXY NXaXEYXEh
0
Suppose that for s > M, as = 0. Then a0, a1, ... ,aM can be found solving the following equations:
tts
shtts NXEXXEa
0
0s
xxs sha
If the Cross autocovariance function, XY(h), and the Autocovariance function, XX(h), are unknown they can be replaced by their sample estimates CXY(h) and CXX(h), yeilding estimates of the impluse response function
01
1011
100
10
10
10
xxMxxxxxy
xxMxxxxxy
xxMxxxxxy
aMaMaM
Maaa
Maaa
Maaa ˆ,ˆˆ ,1,0
In matrix notation this set of linear equations can be written:
Mxxxxxx
xxxxxx
xxxxxx
xy
xy
xy
a
a
a
MM
M
M
M
1
0
01
101
10
1
0
If the Cross autocovariance function, XY(h), and the Autocovariance function, XX(h), are unknown they can be replaced by their sample estimates CXY(h) and CXX(h), yeilding estimates of the impluse response function
0ˆ1ˆˆ
1ˆ0ˆ1ˆ1
ˆ1ˆ0ˆ0
10
10
10
xxMxxxxxy
xxMxxxxxy
xxMxxxxxy
CaMCaMCaMC
MCaCaCaC
MCaCaCaC
Maaa ˆ,ˆˆ ,1,0
Estimation of the Impulse Response function, aj
(with pre-whitening).
Suppose that {Yt : t T} and {Xt : t T}are weakly stationary time series satisfying the following equation:
ts
stst NXaY
0
Also assume that {Nt : t T} is a weakly stationary "noise" time series, uncorrelated with
{Xt : t T}.
In addition assume that {Xt : t T}, the weakly stationary time series has been identified as an ARMA(p,q) series, estimated and found to satisfy the following equation:
(B)Xt = (B)ut
where {ut : t T} is a white noise time series. Then
[(B)]-1(B)Xt = ut
transforms the Time series {Xt : t T} into the white noise time series{ut : t T}.
This process is called Pre-whitening the Input series.
Applying this transformation to the Output series {Yt : t T} yeilds:
tYBB )()]([ 1
ts
sts NBBXBBa )()]([)()]([ 1
0
1
ts
stst nuay
0
or
tt YBBy )()]([ 1
tt NBBn )()]([ 1
where
and
In this case the equations for the impulse response function - a0, a1, ... ,aM - become (assuming that for s > M, as = 0):
0
01
00
1
0
uuMuy
uuuy
uuuy
aM
a
a
0
ˆ and 0
or uu
uyk
uu
uyk C
kCa
ka
Summary
Identification and Estimation of
Box-Jenkins transfer model
To identify the series we need to determine
b, r and s.
The first step is to compute
1. the ACF’s and the cross CF’s
Cxx(h) and Cxy(h)
2. Estimate the impulse response function using
0ˆ1ˆˆ
1ˆ0ˆ1ˆ1
ˆ1ˆ0ˆ0
10
10
10
xxMxxxxxy
xxMxxxxxy
xxMxxxxxy
CaMCaMCaMC
MCaCaCaC
MCaCaCaC
The Impulse response function {at}
0 1 2 3 5 6 7 8
Exponentially decaying Transfer Function
s-r+1b zeroes
as
b s- r + 1 Pattern of an rth order difference equation
3. Determine the value of b, r and s from the pattern of the impulse response function
3. Determine preliminary estimates of the Box-Jenkins transfer function parameters using:i. for j > b+s. .
aj = 1aj-1 + 2aj-2+...+ r aj-r
ii. for b ≤ j ≤ b+s
aj = 1j-1 + 2aj-2+...+ r aj-r+ j
4. Determine preliminary estimates of the ARMA parameters of the input time series {xt}
5. Determine preliminary estimates of the ARIMA parameters of the noise time series {t}
1 bt t ty B B B x
bt t tB y B B x B
1 1t t r t ry y y
1 1t b t b s t b sx x x 1 1 1 1t t t
1 1 1 1t t t r t r t b t b s t b sy y y x x x 1 1 1 1t t
Maximum Likelihood estimation of the parameters of
the Box-Jenkins Transfer function model
The Box- Jenkins model is written
1 bt tB B B x
t ta B x 0
t s t s ts
y a x
21 2where ... r
rB I B B B 2
0 1 2and ... ssB I B B B
The parameters of the model are:
1 2 0 1 2 , ,..., and , , ,...,r s In addition
1. the ARMA parameters of the input series {xt}
2. The ARIMA parameters of the noise series {t}
The model for the noise {t}series can be written
1
t tB B u
21 2where ... p
pB I B B B
21 2and ... q
qB I B B B
Given starting values for {yt}, {xt}, and tu
0 0 0, and x y u
and the parameters of the transfer function model and the noise model , and δ ω β α
We can calculate successively: 0 0 0, , , , ,t tu u δ ω β α x y u
The maximum likelihood estimates are the values ˆ ˆˆ ˆ, and δ ω β α
that minimize: 2
0 , , tt
S u δ ω β α
Fitting a transfer function model
Example: Monthly Sales (Y) and Monthly Advertising expenditures
Mon Adver Sales Mon Adver Sales Mon Adver Sales Mon Adver Sales Mon Adver Sales
1 78.54 1315 51 102.18 1763 101 147.35 2143 151 81.03 1400 201 120.08 916
2 134.82 1304 52 87.95 1840 102 118.26 1909 152 84.61 1348 202 116.6 1024
3 106.38 1507 53 81.08 1679 103 112.15 2106 153 120.19 1658 203 121.53 1441
4 105.31 1580 54 93.78 1756 104 123.8 1997 154 65 1641 204 92.48 1601
5 122.18 1662 55 127.12 1766 105 96.46 2452 155 89.86 1512 205 145.82 1996
6 120.54 2044 56 126.21 1730 106 147.96 2298 156 91.53 1466 206 117.38 2109
7 153.02 1940 57 118.74 1498 107 104.87 2089 157 89 1782 207 135.48 2158
8 154.58 2015 58 109.54 1597 108 109.97 2200 158 80.6 1597 208 119.35 1854
9 163.61 2080 59 134.59 2019 109 112.95 1924 159 89.29 1553 209 138.61 2260
10 146.6 2070 60 107.67 2261 110 75.54 2354 160 58.2 1576 210 122.75 2215
11 157.49 2544 61 119.26 2110 111 94.58 2029 161 71.36 1603 211 102.4 2312
12 150.65 2598 62 115.73 2074 112 91.04 2205 162 71.83 1556 212 115.84 2248
13 155.82 2913 63 107.76 2219 113 59.15 1900 163 73.04 1554 213 119.65 2394
14 134.51 2758 64 102.35 2172 114 67.83 1647 164 70.84 1275 214 106.22 2316
15 145.09 2855 65 87 2136 115 93.02 1671 165 82.82 1214 215 103.06 1962
16 93.54 2671 66 92.78 1978 116 78.14 1623 166 60.55 1263 216 97 2104
17 189.59 2755 67 83.73 2020 117 70.19 1295 167 89.51 1344 217 78.26 2018
18 135.28 2632 68 64.12 1977 118 91.23 1312 168 52.83 1270 218 78.22 2014
19 165.66 2559 69 85.27 1671 119 61.48 1372 169 21.57 1437 219 56.17 1876
20 164 2105 70 111.13 1631 120 60.77 1437 170 34.75 1229 220 42.87 1861
21 119.11 2849 71 115.95 1606 121 69.03 1446 171 56.88 1342 221 58.93 1600
22 156.23 2559 72 107.55 1303 122 50.16 1418 172 36.69 1254 222 59.8 1413
23 133.32 2787 73 107.99 1449 123 48.42 1340 173 65.53 727 223 66.4 1185
24 134.64 2998 74 104.28 1721 124 65.86 1069 174 45.23 641 224 82.62 977
25 100.12 2459 75 103.91 2016 125 64.19 1248 175 47.97 804 225 53.57 905
26 112.62 2659 76 76.51 1896 126 40.46 1035 176 53.66 735 226 64.77 1103
27 113.36 2454 77 79.89 2023 127 48.73 976 177 22.47 1011 227 73.24 1131
28 104.49 2538 78 44.64 1863 128 45.87 1049 178 77.5 937 228 78.88 1334
The Data
Using SAS
Available in the Arts computer lab
The Start up window for SAS
To import data - Choose File -> Import data
The following window appears
Browse for the file to be imported
Identify the file in SAS
The next screen (not important) click Finish
The finishing screen
• You can now run analysis by typing code into the Edit window or selecting the analysis form the menu
• To fit a transfer function model we need to identify the model– Determine the order of differencing to achieve
Stationarity– Determine the value of b, r and s.
• To determine the degree of differencing we look at ACF’s and PACF’s for various order of differencing
To produce the ACF, PACF – type the following commands into the Editor window- Press Run button
• To identify the transfer function model we need to estimate the impulse response function using:
0ˆ1ˆˆ
1ˆ0ˆ1ˆ1
ˆ1ˆ0ˆ0
10
10
10
xxMxxxxxy
xxMxxxxxy
xxMxxxxxy
CaMCaMCaMC
MCaCaCaC
MCaCaCaC
• For this we need the ACF of the input series and the cross ACF of the input with the output
To produce the Cross correlation function – type the following commands into the Editor window
• the impulse response function using can be determined using some other package (i.e. Excel)
-2
0
2
4
6
8
10
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
b = 4
r,s = 1
To Estimate the transfer function model – type the following commands into the Editor window
To estimate the following model
1 bt t ty B B B x
21 2where ... r
rB I B B B
20 1 2and ... s
sB I B B B
Use
input=( b $ ( -lags ) / ( -lags) x)
In SAS
20 1 2and ... s
sB B B B
The Output
The Output