-
8/8/2019 Risk Lab 2002
1/58
Hierarchical Mixtures ofHierarchical Mixtures ofARARModelsModels
forfor FinancialFinancial Time Series AnalysisTime Series Analysis
Carmen Vidal(1)
& Alberto Surez(1,2)
(1) Computer Science Dpt., Escuela Politcnica Superior
(2) Risklab Madrid
Universidad Autnoma de Madrid (Spain)
-
8/8/2019 Risk Lab 2002
2/58
2
Time series of assets are highly irregularIf market efficiency hypothesis is correct they are
also unpredictable.
Time series of assets are non-stationaryThey are usually transformed in log-returns, or, forshort periods of time, in relative returns
Asset returns exhibit deveations from normalityLeptokurtic: Heavy tails
Heteroskedastic: Volatility clustering
Financial time seriesFinancial time series
-
8/8/2019 Risk Lab 2002
3/58
3
Modelling finacial time-series is not easy
Natural sciences Not reproducible
Underlying model?
Inductive / statistical learning Small data sets Complex data
Non-linear
Non-stationarity
Non-gaussian
Heteroskedastic 0 500 1000 1500 2000 2500 302000
4000
6000
8000
10000
12000
14000
Financial time seriesFinancial time series modellingmodelling/ analysis/ analysis
-
8/8/2019 Risk Lab 2002
4/58
4
Two stylized facts (Two stylized facts (Timo TersvirtaTimo Tersvirta))
Returns exhibit two empirically
observed features:
Correlations
Short term for the returnsMedium term for absolute
values of returns
LeptokurtosisHeavy tails
Extreme events
-
8/8/2019 Risk Lab 2002
5/58
5
An example: IBEX35An example: IBEX35
-
8/8/2019 Risk Lab 2002
6/58
6
Daily returns: IBEX35 (5 years)Daily returns: IBEX35 (5 years)
-
8/8/2019 Risk Lab 2002
7/58
7
DailyDaily--returns distributionreturns distribution
-
8/8/2019 Risk Lab 2002
8/58
8
BlackBlack--ScholesScholes theorytheory
In theory: Markets are efficientAbsence of arbitrage opportunities.
No systematic trends.
Very short term memory.
Model: Black-ScholesLog of daily returns of an asset are distributed
according to a normal distribution.Two parameters:
Risk free interest rate.
Volatility [ free parameter]
-
8/8/2019 Risk Lab 2002
9/58
9
Advantages
Simple minimal model with only one free
parameter, the volatility. Goodpricing accuracy forat-the-money
options.
Analyticpricing formulas for simple
derivatives.
Drawbacks:
Incorrect pricing formulas for:
Deep in-the-money or out-of-the-money
Short-term (less than a month) orptions
Options on underlying with very low orvery high volatility.
This is reflected in the fact that impliedvolatility is not constant [Volatility smile]
Is BlackIs Black--Scholes a good model?Scholes a good model?
80 85 90 95 100 105 110.24
0.242
0.244
0.246
0.248
0.25
0.252
0.254
0.256
0.258
Volatility smile(European call)
Im
pliedvolati
lity
Strike
-
8/8/2019 Risk Lab 2002
10/58
10
In practice markets are
Not efficient: Memory effects (short/long term?).
Very unpredictable (at least sometimes)Extreme events are more frequent than what the Black-Scholes models
predicts.
Occurrence ofcrashes.
Changes in economic paradigm.Market friction: Transaction costs, lack of liquidity, dividends,
etc.
Heteroskedasticity + heavy tails
Need more sophisticated model
Parametric models: Generalizations of Blak-Scholes.
Non-parametric models: Neural networks, Mixture models
Beyond BlackBeyond Black--ScholesScholes
-
8/8/2019 Risk Lab 2002
11/58
-
8/8/2019 Risk Lab 2002
12/58
12
-
8/8/2019 Risk Lab 2002
13/58
13
Failure of normal model: Heavy tails
-
8/8/2019 Risk Lab 2002
14/58
14
-
8/8/2019 Risk Lab 2002
15/58
15
Empirical evidence for leptokurtosisEmpirical evidence for leptokurtosis
Volatility smiles and smirksBlack-Scholes is insufficient to
account for time evolution ofunderlying.
Incremented risk
Multiplicative factor in marketRisk estimates (Basel Accord1988, 1996 ammendment)
80 85 90 95 100 105 110 115 1200.205
0.21
0.215
0.22
0.225
0.23
0.235
0.24
-
8/8/2019 Risk Lab 2002
16/58
16
Time series analysisTime series analysis
Consider the time series
Time series analysis
Forecasting
Classification
Modelling
These problems are closely related to each other:
Tt21 X,,X,,X,X
);;( tF ,X,XX 1ttdt + =);( tFClass ,X,X 1tt =
);|( tP
,X,XX 1ttdt +
);|(
;);(
t
dtt
P
F
,X,X
,X,XX
1ttdt
1ttdt
+
++ +=
-
8/8/2019 Risk Lab 2002
17/58
17
Time series prediction: a Learning viewTime series prediction: a Learning view
Network model for time-series prediction
Learning
device
1tX
2tX
ptX
tX
1
-
8/8/2019 Risk Lab 2002
18/58
18
Tasks in time series analysisTasks in time series analysis
Obtaining data:
Selection of attributes: Choose relevant indicators
Data collectionDiscrete data: Grouping /averaging in time window
Continuous data: Importance of sampling frequency
Preprocessing data
Clean data : Missing data, outliersNormalization of data
Eliminate trends /seasonality: Handle a-priori info explicit /
Stationary data.11
11 log;;
t
t
t
tttt
X
X
X
XXXX
( )
minmax
minmax2;;XX
XXX
iq
medianXX ttt
+
-
8/8/2019 Risk Lab 2002
19/58
19
Parametric / nonParametric / non--parametric data analysisparametric data analysis ParametricFormulate (restrictive) hypothesis dependent on a set of parameters
Find parameters by data-driven optimization [training set]
Sensitivity analysis
Uncertainty in estimated parameters
Robustness
Validation of models [test set]
Non-ParametricConsider a family of universal approximants
Fix architecture / parameters by data-driven optimization [training set]Sensitivity analysis
Robustness
Uncertainty
Intelligibility
Validation of models [test set]
-
8/8/2019 Risk Lab 2002
20/58
20
Classical models in timeClassical models in time--seriesseries
Consider the time series
The series exhibits randomness.
The process is covariance-stationary when:
Mean is time independent
Autocovariance is independent of time-translations
Ttt XXXXXX ,,,,,, 1210
( )( )[ ] =+ tt XXE
[ ] =tXE
-
8/8/2019 Risk Lab 2002
21/58
21
AutorregressiveAutorregressive+Moving average models+Moving average models
Autorregressive model for a time-series
Vectors of delayed values:
The systematic term reflects trends.
The innovations are uncorrelated noise.
Maximization of the likelihood function yields estimatesof the model parameters.
tu
[ ][ ] ][
][
21
][
21][
mttt
m
t
mtttm
t
uuu
XXX
+
+
=
=
u
X
);,( ][][ qtp
tt f uXX =
t
q
t
p
tt ufX += );,(][][ uX
-
8/8/2019 Risk Lab 2002
22/58
22
Autoregressive (Autoregressive (feedforwardfeedforward) MLP) MLP
;;1 1
)1(0
)1()2(
= =
+=
J
j
j
D
d
jjdtjdjt cwxwfwx
)1(
20w
)1(
10w1
1tx
)1(
JDw
Input layerHidden layer(s)
Output layer)2(
1
w
)( tx)2(2w
)2(Jw
Sigmoidal (logistic)xe
xf
=1
1)(
xx
xx
ee
eexf
+
=)(
Hyperbolic tangent:
2tx
Dtx
-
8/8/2019 Risk Lab 2002
23/58
23
ARMA(p,q) MLPARMA(p,q) MLP
11
ARw
Input layerHidden layer(s)
Output layer)2(
1w
)( tx)2(2w
)2(
Jw
delay
delay
delay
1
1tx
2tx
ptx
1 tx2 tx
qtx +
_qtu
( ) ;
1
1 1
++
+=
=
= =
p
d
jdtdtMAjd
J
j
p
d
dt
AR
jdjt
xxw
xwfwx
2
MAw
-
8/8/2019 Risk Lab 2002
24/58
24
Mixture modelMixture model
st
t
X
X
1
tX
2 t
1
2
2
1
st
t
MODEL 1
MODEL 2
MODEL J
GATING
NETWORK
gJ
g2
g1
-
8/8/2019 Risk Lab 2002
25/58
25
Gating NetworkGating Network
1 tX
2
tX
rtX
1
1h
2h
1Jh
-c1
1
ar-1
a1
+=
=
i
r
k
ktitii cXaXbh1
1
11exp
Probabilities
=
=
==
+
=1
11
1
1)1(21;
1
J
jjJJ
j
j
i
i
ggJ,,i
h
hg
i hi l i
-
8/8/2019 Risk Lab 2002
26/58
26
Hierarchical mixturesHierarchical mixtures
MODEL 1 MODEL 2
MODEL 3
2
11|212
11|111
3Model
;2Model
;1Model
=
=
1
1|1
2
2|1
12
1
1
1
1111
1
1
1
1111
1
1
exp1
exp
=
++
+
=
=
=
cXaXb
cXaXb
r
k
ktkt
r
k
ktkt
1|11|2
2
1
1
1212
2
1
1
1212
1|1
1
exp1
exp
=
++
+
=
=
=
cXaXb
cXaXb
r
k
ktkt
r
k
ktkt
Input = Vector of Delayed values
-
8/8/2019 Risk Lab 2002
27/58
27
Mixture ofMixture ofGaussiansGaussians for tfor t--independentindependentpdfpdf
Empirical sample
Model pdf
Two steps:Toss a K-sided loaded dice to choose component.
Extract value from the selected model.
Advantages:Close to the normal world.
Accounts for leptokurtosis of empirical unconditional
distributions in finance.
),;(N)(
1
kk
K
k
k xpxP =
=
NXXX
,, 21
-
8/8/2019 Risk Lab 2002
28/58
28
-
8/8/2019 Risk Lab 2002
29/58
29
-
8/8/2019 Risk Lab 2002
30/58
30
Mixture ofMixture ofGaussiansGaussians
Intuition:
Implicitly market forecasts are made in terms of scenarios.
Each of these scenarios is characterized by an expected returnand a volatility.
Markets assign a different probability to each scenario.
Dynamical picture?
Direct time aggregation of the process yields a normal model (byCentral Limit Theorem).
It is possible to construct a discontinuous jump process
maintaining the mixture form. Not realistic.
-
8/8/2019 Risk Lab 2002
31/58
31
Mixture of AR processesMixture of AR processes
Mixtures of Gaussians + autorregressive dynamics InIn: Vector of delays (Used in gating network + AR models)OutOut: Next value in time series
No hierarchy Tree hierarchy
-
8/8/2019 Risk Lab 2002
32/58
32
Synthetic dataSynthetic data: E: Example 1xample 1
10 8 6 4 2 0 2 4 6 80
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Contribucion de cada experto
E3E1E2
Histogram (unconditional pdf)
10 8 6 4 2 0 2 4 6 80
50
100
150
200
250
Time series generated by a hierarchical mixture of 3 AR(1)
experts
Expert contributions
-
8/8/2019 Risk Lab 2002
33/58
33
Model 1 fitModel 1 fit
Fitting to a mixture of 2 AR(1) experts(wrong type of model!)
Contributions Histogram Percentile plot
10 8 6 4 2 0 2 4 6 8 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Contribucion de cada experto
g1g2
15 10 5 0 5 100
20
40
60
80
100
120
140
10 8 6 4 2 0 2 4 6 815
10
5
0
5
10
X Quantiles
Y
Quantiles
0.46450-18009-17967
ECM TestK-S TestLL TestLL Train
-
8/8/2019 Risk Lab 2002
34/58
34
Model 2 fitModel 2 fit
Fitting to a mixture of 3 AR(1) experts(learnable model)
Contributions Histogram Percentile plot
0.31640.9666-16755-16675
ECM TestK-S TestLL TestLL Train
10 8 6 4 2 0 2 4 6 8 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Contribucion de cada experto
E3E1E2
10 8 6 4 2 0 2 4 6 80
20
40
60
80
100
120
140
160
180
200
10 8 6 4 2 0 2 4 6 810
8
6
4
2
0
2
4
6
8
X Quantiles
Y
Quantiles
-
8/8/2019 Risk Lab 2002
35/58
35
AR(1) fit for Ibex35AR(1) fit for Ibex35 (1200 +712 days)(1200 +712 days)
-
8/8/2019 Risk Lab 2002
36/58
36
AR(1) fit for Ibex35AR(1) fit for Ibex35 (1200 +712 days)(1200 +712 days)
-
8/8/2019 Risk Lab 2002
37/58
37
MIX 2 AR(1) fit for Ibex35MIX 2 AR(1) fit for Ibex35
-
8/8/2019 Risk Lab 2002
38/58
38
MIX 3 AR(1) fit for Ibex35MIX 3 AR(1) fit for Ibex35
-
8/8/2019 Risk Lab 2002
39/58
39
Hierarchical MIX 3 AR(1) fit for Ibex35Hierarchical MIX 3 AR(1) fit for Ibex35
-
8/8/2019 Risk Lab 2002
40/58
40
ConclusionsConclusions
and perspectivesand perspectives
MixturesMixtures of AR(1) models improveimprove the results of single
AR(1) models in financial returns time series. Mixtures ofMixtures of2 / 3 experts2 / 3 experts seem to be sufficientsufficient to
model leptokurtosis and dynamics.
The introduction ofhierarchyhierarchy in the structure of themixture may significantly improve statistical descriptionof financial time series data.
To do: Heteroskedasticity
Calibration of models to market
Mi f ARCHMi t f ARCH
-
8/8/2019 Risk Lab 2002
41/58
41
Mixture of ARCH processesMixture of ARCH processes
MixARCH
The model for the residuals is
The quantities are assumed to beN(0,1)
)(
),(
][
][
][
][
][
][
i
r
ti
i
m
tit
gyprobabilitwith
tuX
,,,,
X
X += +
)(][)(
)()(
][][
22][
][][
tut
Zttu
qiiii
tii
+=
=
+
tZ
Mi t f GARCHMi t f GARCH
-
8/8/2019 Risk Lab 2002
42/58
42
Mixture of GARCH processesMixture of GARCH processes
MixGARCH
The model for the residuals is
The quantities are assumed to beN(0,1)
),(
),(
][
][
][
][
][
][
i
r
ti
i
m
tit
gyprobabilitwith
tuX
X
X += +
)(][)(][)(
)()(
][ ][2][ ][22 ][
][][
ttut
Zttu
piiqiiii
tii
++=
=
++
tZ
AR(1) / ARCH(1) f IBEX35AR(1) / ARCH(1) f IBEX35
-
8/8/2019 Risk Lab 2002
43/58
43
AR(1) / ARCH(1) for IBEX35AR(1) / ARCH(1) for IBEX35
The maximum-likelihood fit of the time-series
IBEX35 yields the model
The quantities are assumed to follow a N(0,1)
distribution.tZ
( )2212
1
1129.01118.09097.0
1129.0
+=
+=
ttt
tttt
XX
ZXX
R id l l ti ARCH(1)R id l l ti ARCH(1)
-
8/8/2019 Risk Lab 2002
44/58
44
Residual correlations: ARCH(1)Residual correlations: ARCH(1)
-
8/8/2019 Risk Lab 2002
45/58
45
Normality hypothesis: ARCH(1)
KS Test = 0.12
-6 -4 -2 0 2 4 6
-6
-4
-2
0
2
4
6
X Quantile s
YQ
uantiles
-4 -3 -2 -1 0 1 2 3 4 5
0
50
100
150
200
MIXARCH for IBEX35MIXARCH for IBEX35
-
8/8/2019 Risk Lab 2002
46/58
46
MIXARCH for IBEX35MIXARCH for IBEX35
The mixture model is
The probabilities for the mixture are
( )
( )2212
1
2
21
2
1
1380.003821.06820.0
1380.02Model
0559.01976.02194.2
0559.01Model
+=
+=
+=
+=
ttt
tttt
ttt
tttt
XX
ZXX
XX
ZXX
{ }
)(1)(
;)5155.2(6839.0exp1
1)(
1]1[1]2[
1
1]1[
=
+=
tt
t
t
XgXg
XXg
Residual correlations: MIXARCH
-
8/8/2019 Risk Lab 2002
47/58
47
Residual correlations: MIXARCH
-
8/8/2019 Risk Lab 2002
48/58
48
Normality hypothesis: MixARCH(1)
KS Test = 0.83
-3 -2 -1 0 1 2 30
20
40
60
80
100
120
140
160
-6 -4 -2 0 2 4 6-6
-4
-2
0
2
4
6
X Quantile s
YQuantiles
MIXARCH Model fit
-
8/8/2019 Risk Lab 2002
49/58
49
MIXARCH Model fit
AR(1) / GARCH(1 1) for IBEX35AR(1) / GARCH(1 1) for IBEX35
-
8/8/2019 Risk Lab 2002
50/58
50
AR(1) / GARCH(1,1) for IBEX35AR(1) / GARCH(1,1) for IBEX35
The maximum-likelihood fit of the time-series
IBEX35 yields the model
The quantities are assumed to follow a N(0,1)
distribution.tZ
( )2
1
2
21
2
1
8733.0
1358.00755.00527.0
1358.0
++=
+=
t
ttt
tttt
XX
ZXX
Residual correlations: GARCH
-
8/8/2019 Risk Lab 2002
51/58
51
Residual correlations: GARCH
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
1
Magnitude
Autocorrelations of residuals
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
1
Delay
Magnitude
Autocorrelations of abs(residuals)
-
8/8/2019 Risk Lab 2002
52/58
52
Normality hypothesis: GARCH(1,1)
-4 -2 0 2 4 60
50
100
150
200
250
KS Test = 0.56
-6 -4 -2 0 2 4 6
-6
-4
-2
0
2
4
6
X Quantiles
YQuantiles
Test Data
-
8/8/2019 Risk Lab 2002
53/58
53
Test Data
-5 0 5-6
-4
-2
0
2
4
6
X Quantiles
YQuantiles
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
1
Magnitude
Autocorrelations of residuals
0 5 10 15 20 25 30
00.2
0.4
0.6
0.8
1
Delay
M
agnitude
Autocorrelations of abs(residuals)
-6 -4 -2 0 2 4 6
0
10
20
30
40
50
60
70
80
90
100 200 300 400 500 6000
1
2
3
Time
Vola
tility
KS = 0.33
MIXGARCH for IBEX35MIXGARCH for IBEX35
-
8/8/2019 Risk Lab 2002
54/58
54
MIXGARCH for IBEX35MIXGARCH for IBEX35
The mixture model is
The probabilities for the mixture are
( )
( ) 2 12
21
2
1
2
1
2
21
2
1
0285.03314.00000.06230.2
3314.02Model
8937.01255.00778.00156.0
1255.01Model
++=
+=
++=
+=
tttt
tttt
tttt
tttt
XX
ZXX
XX
ZXX
{ }
)(1)(
;)8710.4(0.5418exp1
1)(
1]1[1]2[
1
1]1[
=
+=
tt
t
t
XgXg
XXg
Residual correlations: MIXGARCH
-
8/8/2019 Risk Lab 2002
55/58
55
Residual correlations: MIXGARCH
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
1
Magnitud
e
Autocorrelations of residuals
0 5 10 15 20 25 300
0.2
0.4
0.6
0.8
1
Delay
Magnitude
Autocorrelations of abs(residuals)
N li h h i MIXGARCH
-
8/8/2019 Risk Lab 2002
56/58
56
Normality hypothesis: MIXGARCH
-6 -4 -2 0 2 4 6
-6
-4
-2
0
2
4
6
X Quantiles
YQ
uantiles
-3 -2 -1 0 1 2 3
0
20
40
60
80
10 0
12 0
14 0
16 0
KS test = 0.95
MIXGARCH Model fit
-
8/8/2019 Risk Lab 2002
57/58
57
MIXGARCH Model fit
200 400 600 800 1000 12000
1
2
Time
Volatility
200 400 600 800 1000 12000
0.2
0.4
0.6
0.8
Entro
py
200 400 600 800 1000 12000
0.2
0.4
0.6
0.8
Pro
ba
bilities
Model 1Model 2
Test Data
-
8/8/2019 Risk Lab 2002
58/58
58-6 -4 -2 0 2 4 6
-6
-4
-2
0
2
4
6
X Quantiles
YQuantiles
100 200 300 400 500 6000
1
2
3
Time
Volatility
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
1
Magnitude
Autocorrelations of residuals
0 5 10 15 20 25 30
0
0.2
0.4
0.6
0.8
1
Delay
Magnitude
Autocorrelations of abs(residuals)
-6 -4 -2 0 2 4 60
10
20
30
40
50
60
70
80
KS = 0.25
