Adaptive Stress Testing

www.riskcommons.org | www.fna.fi Alan Laubsch, [email protected] | Kimmo Soramaki, [email protected]

Adaptive Stress TestingHarnessing Network Intelligence in Stress Testing

and Reverse Stress Testing

ERM Symposium, Chicago April 24 2013

2 www.riskcommons.org www.riskcommons.org | www.fna.fi Alan Laubsch, [email protected] | Kimmo Soramaki , [email protected] 2

www.riskcommons.org | www.fna.fi Alan Laubsch, [email protected] | Kimmo Soramaki , [email protected] 2

Agenda

1. A Framework for Adaptive Stress Testing

2. Signal or Noise?

3. Introducing StressGrades™

4. Network Approaches to Stress Testing

5. Summary and Conclusions


Scenarios are continually emerging and evolving

• Integrate interdisciplinary perspectives

Source: infomous.com/

http://infomous.com/

http://infomous.com/


Implication of Complexity: “Sense and Respond”

Dynamic Steering: continual feedback


Seek to understand systemic fault lines…

• …and how is your portfolio is positioned relative to fault lines.

• Major challenge: disaster myopia (see “Why Banks Failed the

Stress Tests” by A. Haldane, 2009)

Earthquake activity vs Nuclear power plants

Source: http://googlemapsmania.blogspot.com/2011/03/nuclear-power-plants-earthquake.html

http://googlemapsmania.blogspot.com/2011/03/nuclear-power-plants-earthquake.html



I. Macro: identify structural risks (potential risks)• Stress Library based on Thought Leaders (Innovators)

• Awareness of systemic cycles, in particular credit and asset bubbles

• Financial or economic imbalances (e.g., capital flows, consumption vs. saving)

• Examples: Shiller – (a) tech bubble (2000) and (b) housing bubble (2005)

II. Micro: monitor potential precipitating events (visible risks)• Focus on short term market movements, especially outliers and regime

shifts

• Early Warning: identify amplification mechanisms and critical (tipping) points

• Examples: vol spike in (a) tech stocks and (b) US mortgage securities & financials

Adaptive Stress Testing Framework



Designing an Adaptive Stress Library

Source: Wikipedia; see Geoffrey Moore’s “Crossing the Chasm” (1999)

• Diffusion of ideas and innovation follow a predictable course after

a tipping point is crossed

Two key perspectives for stress testing

1. Stress Library: Innovators

2. Early Warning: Early Adopters

Innovators:

Roubini, Rosenberg, Shiller, Rogoff, Reinhart, Ferguson, …

Key early adoption signals:

- Outliers clustering, vol spikes, super-exponential trends

- Adoption by hedge funds and broker dealers.


US Financials Case Study

Financial Meltdown (“Roubini”) scenario escalates from ’07 and peaks March ’09 and then declines… inverse Financial Recovery scenario emerges

2-Ja

n-02

9-Fe

b-02

19-M

ar-0

2

26-A

pr-0

2

3-Ju

n-02

11-Ju

l-02

18-A

ug-0

2

25-S

ep-0

2

2-Nov

-02

10-D

ec-0

2

17-Ja

n-03

24-F

eb-0

3

3-Apr

-03

11-M

ay-0

3

18-Ju

n-03

26-Ju

l-03

2-Se

p-03

10-O

ct-0

3

17-N

ov-0

3

25-D

ec-0

3

1-Fe

b-04

10-M

ar-0

4

17-A

pr-0

4

25-M

ay-0

4

2-Ju

l-04

9-Aug

-04

16-S

ep-0

4

24-O

ct-0

4

1-Dec

-04

8-Ja

n-05

15-F

eb-0

5

25-M

ar-0

5

2-May

-05

9-Ju

n-05

17-Ju

l-05

24-A

ug-0

5

1-Oct

-05

8-Nov

-05

16-D

ec-0

5

23-Ja

n-06

-20.0%

-15.0%

-10.0%

-5.0%

0.0%

5.0%

10.0%

15.0%

20.0%Daily 99% VaR Backtest (.94 decay, Student t)

Feb 27 ‘07 outlier

Source: Alan Laubsch, “Equities as Collateral In U.S. Securities Lending Transactions”,

The RMA Executive Committee on Securities Lending & RiskMetrics, March 2011

March 6 Market bottom

June 1 Market peaks

Escalating vol bear marketDeclining vol bull

mkt

Chart: U.S. Financials “death star pulse”


Tipping Point Dynamics require early detection and action

• Limited window of opportunity for exerting control

• What are early warning signals of a phase transition?

Source: “Building A Reputation Risk Management Capability”, Diermeier & Loeb, 2011

Invisible/Potential Visible & amplifying



Agenda


2. Signal or Noise?






Exogenous vs Endogenous Crises

• Nassim Taleb’s “Black Swan” claims that crises arise from

unknowable events that cannot quantified or predicted • Historical examples: Eisenhower heart attack; Lincoln & Kennedy

assassinations; asteroid impact or flood basalt eruptions resulting in mass extinctions; 911;

• Didier Sornette’s “Dragon King” thesis holds that most

financial crises are endogenous in nature and can be

diagnosed in advance, can be quantified, and have some

predictability• Examples of endogenous crises in history: rise of Fascism; rise of

dictators (Hitler, Mao);’29 Great Depression, ’87 Black Monday, ’89 Japan Bubble; ’01 Tech Bubble; GFC; current ecological crisis

• Endogenous structural risk combined with exogenous

precipitating event is common (e.g., forest fire)Source: Alan Laubsch “Integrated Risk Management - Early Overview”, RiskMetrics



Phase transitions can result from amplifying feedback

• Super-exponential instability and change characterizes phase transitions

See: http://www.er.ethz.ch/presentations/Endo_Exo_Oxford_17Jan08.pdf

Source: Sornette et al., Endogenous versus Exogenous Origins of Crises (2008)

http://www.er.ethz.ch/presentations/Endo_Exo_Oxford_17Jan08.pdf



Subprime CDO volatilities spiked 7 & 4 months before the meltdown

-100.0%

-80.0%

-60.0%

-40.0%

-20.0%

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

120.0%

7/1

9/2

00

6

8/1

9/2

00

6

9/1

9/2

00

6

10

/19

/20

06

11

/19

/20

06

12

/19

/20

06

1/1

9/2

00

7

2/1

9/2

00

7

3/1

9/2

00

7

4/1

9/2

00

7

5/1

9/2

00

7

6/1

9/2

00

7

7/1

9/2

00

7

8/1

9/2

00

7

9/1

9/2

00

7

10

/19

/20

07

11

/19

/20

07

12

/19

/20

07

1/1

9/2

00

8

2/1

9/2

00

8

3/1

9/2

00

8

4/1

9/2

00

8

5/1

9/2

00

8

6/1

9/2

00

8

Date

Sp

rea

d C

ha

ng

e

300%+ increase in vol from Dec 12 to 21 '06

357% vol spike on Feb 23 '07

RM 2006 99% VaR bands vs 2006-1 AAA spread changes

One major outlier, a 12 sd move on Feb 23 '07, the day after the $10.5bn HSBC loss announcement

Backtesting summary: 2.4% upside excessions0.81% downside excessions

Major ratings agencies initiate reviews and/or downgrades week of July 9 '07

Source: Alan Laubsch “Subprime Risk Management Lessons”, RiskMetrics

GS exits

subprime



The Dec ’06 and Feb ’07 spikes in volatility can be seen as tremors (foreshocks) that cascaded into a major earthquake

'2006-1 AAA'

0

50

100

150

200

250

300

350

400

450

1/19

/200

6

2/19

/200

6

3/19

/200

6

4/19

/200

6

5/19

/200

6

6/19

/200

6

7/19

/200

6

8/19

/200

6

9/19

/200

6

10/1

9/20

06

11/1

9/20

06

12/1

9/20

06

1/19

/200

7

2/19

/200

7

3/19

/200

7

4/19

/200

7

5/19

/200

7

6/19

/200

7

7/19

/200

7

8/19

/200

7

9/19

/200

7

10/1

9/20

07

11/1

9/20

07

12/1

9/20

07

1/19

/200

8

2/19

/200

8

3/19

/200

8

4/19

/200

8

5/19

/200

8

6/19

/200

8

The first tremor (vol up 300% Dec 12-21)

Feb 23 '07, first major outlier, 350% vol increase in 1 day, 12sd move

June 20 '07, ML tries to liquidate Bear Subprime CDO's

Absolute Spread Levels

Major ratings agencies initiate reviews and/or downgrades week of July 9 '07

bp's

• Absolute spread moves were small, but rate of change was super-exponential. Parallels to

failure and rupture process in material science (pressure to break point)



Feb 27 ‘07 DJIA outlier marks the beginning of a phase transition with increasing waves of volatility

• Increasing amplitude of volatility is a telltale sign of endogenous crises

-5.00%

-4.00%

-3.00%

-2.00%

-1.00%

0.00%

1.00%

2.00%

3.00%

4.00%

5.00%

1/23

/200

6

3/23

/200

6

5/23

/200

6

7/23

/200

6

9/23

/200

6

11/2

3/2

006

1/23

/200

7

3/23

/200

7

5/23

/200

7

7/23

/200

7

9/23

/200

7

11/2

3/2

007

1/23

/200

8

3/23

/200

8

5/23

/200

8

DJIA daily returns vs 99% VaR bands (.94 decay, t dist)

Feb 27, 6th biggest outlier in DJIA history 4 days after largest spike in subprime

spreads

Source: Alan Laubsch “Integrated Risk Management - Early Overview”, RiskMetrics


Gold Early Warning


Let’s look more closely at Outliers


Downside outlier Clustering Escalates from Oct 2012



Agenda


2. Signal or Noise?






Typical stress testing processes generate much data, but not necessarily intelligence

“We run over 180 stress scenarios against each of our counterparties

on a daily basis. But we don’t know what to do with the information” –

Risk manager at global bank

Key questions:

• With overwhelming amount of data, which scenarios to focus on?• …given market conditions (systemic)

• …given our portfolio exposures (specific)


DStress

PStress

Figure: Student t distributions and Z-scores

We estimate DStress w/market implied vols (and correlations for multi factor scenarios) and PStress using a distributional assumption (e.g., Normal or Student

t).

StressGrades™ harness market intelligence highlight emerging risks

We define three components of StressGrades™:

1. PStress = Market Implied Probability of a Stress Scenario

2. DStress = Distance to stress scenario in standard deviations (z-score)

3. StressQ = Quantile (percentile) historical rank of stress scenario (e.g.,

StressQ = .82 implies stress levels have exceeded current levels 18% of the

time)



Stress Grades™ provide early warning and can be backtested

S&P 500 Case Study: • Since 1987, the biggest one day drop in the S&P 500 was a 9.6% fall on

Dec 1 ’08, which we use to calibrate and backtest our StressGrade scenario.

• DStress escalates from -24sd to -2sd before Dec ‘08 drop. Regime shift warning Feb‘07



S&P500 cont’d: Super-exponential increase in PStress: 170x on Feb 27 then another 1300x before Dec 1 ’08

• Note log scale on the PStress Chart below (right scale)



ETF analysis reveals systemic risk early warning signals

• Implied Probability of Stress Event (PStress) for major ETFs shows

super-exponential escalation during GFC

Log

Scale

Source: Alan Laubsch, “Introduction to StressGrades©”, riskcommons.org, 2011



Super-exponential increase in PStress preceded market crash…

• … and broad declines in PStress from peak levels signaled market

recovery

Source: Alan Laubsch, “Introduction to StressGrades©”, riskcommons.org, 2011



Highlight escalating and large risks in StressGrades™ HeatMap

Mouse over to get

more information

about each scenario

StressGrade™ (PStress)

% c

hange in S

tresG

rade

0 100 200 300 400

10%

-10%

100%

1000%

2. PIGS Inflationary Bust

- StressGrade 385 up 550%

0%

Highest Priority: Escalating and High PStress

Early Warning: Moderate PStress but escalating



Outlier Analysis can identify regime shifts

Rank % Move in PStress Scenario [each hyperlinked]

1 780% to 389 PStress Sovereign default

2. 690% to 355 PStress Deflationary bust

….

23. 55% to 80 PStress Gold spike

• Sample Outlier Analysis: 5% threshhold

StressGradiealyss• 23 of 80 scenarios were outliers

• 6 Outliers Average over 12 months



Network Graphs allow visualization of interrelationships

• Potential to integrate stress themes into interactive network

graphs and play movie of changing correlation and volatility

dynamics over time



Agenda


2. Signal or Noise?


4. Network approaches to Stress Testing




DStress Network

How are asset stresses coordinated?

We calculate the Euclidean distance

between pairwise series of daily DStress

values.

Keep only most important links that form

the backbone dependencies, i.e. present a

data reduction.

Size of node scales with risk as defined by

average DStress during the period: Large

node, high risk. Small node, low risk.

The network shows us the coordination of

stress among the assets in a portfolio.

Jan 20 - April 19 2013

http://www.fna.fi/demos/erm/dstress-tree.html





Stress Testing a Portfolio - Opening up the Black Box

Partial correlation measures the

degree of association between two

random variables, controlling for other

variables

Network of statistically significant

partial correlations of dailyt returns for a

wide set ETFs during 2009-2013

• link = partial correlation

• green node = positive return

• red node = negative return

• node size scales with absolute return

We can use the partial correlations to

undestand linkages within a standard

portfolio stress test model



Calculating Partial Correlation

• We build regression models for daily returns of e.g. Oil and Gold based on all

other assets of interest and look at the correlation of their model residuals

(i.e. what is left unexplained by the other factors) -> Partial correlation

Model 1: Regress Gold on all other assets except Oil Model 2: Regress Oil on all other assets except Gold

• Gold residuals = vector of differences between observed Gold values and

values predicted by Model 1

• Oil residuals = vector of differences between observed Oil values and values

predicted by Model 2

• Partial correlation between Oil and Gold is the correlation between Oil

residuals and Gold residuals

32



The Network for an Oil shock

http://www.fna.fi/demos/erm/cascade-oil-01.html





Shocking multiple nodes

• We use multivariate percentiles (based on the multivariate normal

distribution) to simultaneously shock Financials, German Stocks and

Gold

• First we estimate the mean and covariance matrix of these three

asset returns from theobserved data.

• Then, for the first percentile, we find the schocks x, y, and z such

that the joint probability P(XLF < x AND EWG < y AND GLD < z) =

0.01 and the marginal probabilities are equal, i.e., P(XLF < x) =

P(EWG < y) = P(GLD < z)

• A similar calculation finds the 99th percentile.



The Network for Multiple Shocks

http://www.fna.fi/demos/erm/cascade-three-01.html





Is it correct?

The test:

• We develop a model where we use the network structure to

estimate many small models (some of which are based on

estimates)

• We see how well cascading predictions works by predicting values

for a out of sample data set whose values are known.

• We compare results to a normal linear model

• Result: Predictions based on partial correlation network are as good

for single asset shock, and just slightly worse for multiple asset

shock

-> The partial correlations do open up the model

and provide more insights into asset dynamics



Agenda


2. Signal or Noise?


4. Network approaches to Stress Testing




Summary: sense and respond to emerging risks

• Use algorithms and visualization techniques to detect signals

amidst noise (e.g., super-exponential rates of change)

• Prioritize attention to relevant macro fault lines and specific

portfolio vulnerabilities

Anticipate

Most of the focus at most companies is on what’s directly ahead. The leaders lack “peripheral vision.” This can leave your company vulnerable to rivals who detect and act on ambiguous signals.

- 6 Habits of True Strategic Thinkers, Paul Schoemaker, Mar 20, 2012

http://www.inc.com/paul-schoemaker/6-Habits-of-Strategic-Thinkers.html





Summary: Architect stress tests to adapt to market intelligence

• As early warning signals are observed:1. Focus on affected systemic fault lines (and related nodes)

2. Assign higher probability of stress

3. Apply more severe stress scenarios

• Proactive response is essential1. War game scenarios to better understand potential impacts and

consequences over time, and practice playing out various permutations of scenarios across the enterprise

2. Take advantage of calm periods to reduce concentration risks, increase capital and liquidity buffers. Get prepared to weather more severe storms ahead.



Conclusions

• Adaptive stress testing practices• Experiment: explore emerging vulnerabilities and seek

to uncover risk concentrations

• Learn: intelligent feedback loops: market signals and subjective perspectives (scenarios)

• Practice: play through various scenario permutations

• Early detection and adaptation is crucial for

systemic risks

• Harness market intelligence to prioritize

attention

“The future is already here — it's just not very evenly distributed.” William Gibson

Adaptive Stress Testing

Economy & Finance

alan laubsch

kimmo soramaki

stress testing5

stress testsby

stress testing1

adaptive stress testing2

adaptive stress librarysource

early warning signals