Recent Trends in Macro Stress Testing: Elements of good Practice by Charles Augustine Abuka Director, Financial Stability Department BANK OF UGANDA COMESA.

Recent Trends in Macro Stress Testing: Elements of good Practice

by

Charles Augustine AbukaDirector, Financial Stability Department

BANK OF UGANDA

COMESA Course on Forward Looking Financial Stability Reports

October 31, 2012

KSMS, Nairobi, Kenya

Outline

• Micro and Macro stress testing• Issues to note about macro stress

testing• Recent trends in macro stress testing• Elements of good practice• The way forward

2

INTRODUCTION

3

I. Introduction

• What is stress testing?– Risk management tool that is used to evaluate

the impact on a firm of a specific event and/or movement in a set of financial variables (BIS, 2005).

– It is a process to identify and manage situations that could cause extraordinary losses (Jorion, 2007).

– It is a what if scenario that takes the world as given but assumes a major change in one or more variables in order to see what effect this would have on various indicators (IMF, 2003).

4

I. Introduction

• What is the goal of stress testing?– The goal of stress testing models is to measure how a financial

system will respond to negative shocks and to trace the effect of common exposures and interlinkages.

• What should stress testing do?– It should serve to provide forward looking assessments of risk,– Overcome limitations of models and historical data,– Support internal and external communication,– Feed into capital and liquidity planning procedures,– Inform the setting of a financial institutions risk tolerance,– Facilitate the development of risk mitigation or contingency

plans across a range of stress conditions.

• Refer to Principles for sound stress testing practices and supervision, BCBS (2009).

5

I. Introduction

• Micro stress testing– Micro stress testing is designed to test the

performance of individual portfolios or the stability of individual institutions

– Any stress test whether micro or macro has four elements:• The risk exposures subjected to stress,• The scenario that defines the (exogenous) shocks

that stress those exposures,• The model that maps those shocks onto an

outcome (or impact),• The measure of the outcome.

6

I. Introduction

• Micro stress testing– Stress testing supplements other risk management

tools.– Provides forward looking assessments of risks.– Facilitates development of mitigation/contingency plans– BCBS Principles for Sound stress testing Practices&

Supervision:• 15 principles for banks:

– Use of stress testing and integration in risk governance,– Stress test methodology and scenario selection,– Specific focus (risk mitigation techniques, complex products, pipeline

and warehousing risks, reputational risk, leveraged counterparties).

• 6 principles for supervisors:– Regular stress testing, corrective action, assessment of

scope/severity, Pillar 2 common scenarios, systemic vulnerabilities.

7

I. Introduction

• What is macro stress testing?– Macro stress testing is designed to stress the financial system

as a whole or sub-sets thereof, rather than micro stress testing that is designed to stress individual institutions.

– Given the current technology macro stress tests are still ill-suited as early warning devices (tools for identifying vulnerabilities during tranquil times and for triggering remedial action).

– Macro stress tests are however quite effective as tools for crisis management and resolution.

– Macro stress tests can discipline our thinking about financial stability risks• They help to reconcile different perspectives of different

stakeholders• Foster better communication.• Help cross check performance of individual firms’ risk

models• Help to identify important data gaps.

8

I. Introduction

• What is a macro stress test?– In its broadest sense it is a technique to test the

stability of an entity or system under adverse conditions.

– Macro stress testing also has the objective of identifying vulnerabilities in tranquil times and provide a basis for addressing them i.e. act as an early warning tool.

– Macro stress testing also provides support to crisis management and resolution.

– In conducting stress tests, the bank supervision department should focus on microprudential aspects, while financial stability units evaluate the macroprudential aspects.

9

10

I. Introduction

• In organizing a stress testing exercise, a number of questions need to be answered:– Who is responsible for the stress tests?– How scenarios are designed and chosen?– How are assumptions (variables to be stressed)

examined and chosen?– What data is available?– How frequently should the stress tests be performed?– What aggregation/granularity level is selected?– How results are presented?– What actions supervisory agency/central bank takes?– Are results published or not?

Recent trends in macro stress testing

– The IMF started using macro stress tests as part of its Financial Stability Assessment Programs a decade ago. Central banks soon followed in using the methodology for assessing vulnerabilities.

– The recent global economic and financial crisis has encouraged the use of macro stress testing to support crisis management and resolution.

– The early stress testing models were very basic. They relied on equations linking aggregate profits and losses to macro developments.

– In 2006 Elsinger developed a model that integrated market risk, credit risk, interest rate risk and counterparty credit risk in the interbank sector.

11

Recent trends in macro stress testing

– In 2010 Drehmann et al was able to integrate credit and interest rate risk in the banking book. He was able to explore the impact on banks profits and losses of different investment behaviours by banks.

– The most comprehensive approach is the RAMSI, the risk assessment model of the Bank of England.

– Macroeconomic feed backs are the focus of work by Jacobsen et al 2005. They propose reduced form approach for Sweden that consists of aggregate vector autoregressive model (VAR) that included default frequencies of companies as a measure of financial stability.

– The new generation of dynamic stochastic general equilibrium (DSGE) that include the financial sector are likely to become useful for stress testing purposes.

– The biggest improvements in the area of stress testing have been in the treatment of liquidity risk.

12

Issues to note about macro stress testing

• Macro stress testing is a tool box, not a single tool.– The set of institutions and exposures assessed:

• Banking sector is the most common object of analysis but stress tests should cover other institutions as well,

• Individual as opposed to multi country tests,• Focus has tended to be on credit risks though tests

now cover market, trading book, income and counterparty risk as well as liquidity risk.

– The choice of scenarios:• Best practice calls for “severe yet plausible scenarios”

13

Issues to note about macro stress testing

• Macro stress testing is a tool box, not a single tool– The features of the model:

• The process may be top down or bottom up or a combination of the two,

• The initial component is a macro model that provides estimates of how the exogenous shocks affect the economy,

• More sophisticated tests seek to assess the size of various potential feed back effects.

– The measures of the outcome:• This captures the final impact of the shocks on the balance

sheets and income statements,• The most common metric are portfolio losses, capital and

liquidity adequacy.

14

Issues to note about macro stress testing

15

Issues to note about macro stress testing

– General observations about models for macro stress testing include:• Macro stress tests are still largely partial

equilibrium exercises. This frameworks do not allow for feedbacks.• The models are likely to be mis-specified

econometrically:– Non linearities for example are at the heart of

stress episodes and need to be taken into account.

– Structural breaks.– Degree of statistical confidence in the results.

16

Issues to note about macro stress testing

• Beware of macro stress tests as early warning devices:– Most macro stress tests carried out ahead of the

crisis failed to identify the vulnerabilities. They indicated that the “system was sound”.

– Stress tests turned out to be part of the problem rather than the solution.

– They lulled policy makers and market participants into a false sense of security.

– The current generation of models is a long way from providing a realistic picture of the dynamics of financial distress.

17

Issues to note about macro stress testing

• Beware of macro stress tests as early warning devices– The models are the antithesis of what financial instability is all

about (Borio and Dremann (2011)). The very essence of financial stability is that normal sized shocks cause the system to breakdown.

– Empirical evidence is inconsistent with the implicit assumption of macro stress tests that crises are a result of very large negative shocks.

– Financial crises usually do not begin after output has collapsed, but before it contracts significantly. This is shown in the graph that follows, which traces the evolution of real GDP, actual and forecasted, around 43 banking crises in 30 countries (top panels). Even average real property prices have not fallen substantially at that point (lower right hand panel) partly because credit growth is still well in positive territory (lower left-hand panel).

18

Issues to note about macro stress testing

19

Issues to note about macro stress testing

– Another key concept is the “paradox of financial instability (Borio and Dremann (2011)).

– The system looks strongest precisely when it is most vulnerable.• Credit growth and asset prices are usually strong,

leverage measured at market prices artificially low, profits and asset quality healthy, risk premia and volatilities unusually low precisely when risk is highest.

– All the above suggest macro stress testing still faces an up hill struggle.

20

Issues to note about macro stress testing

• Macro stress tests help crisis management and resolution:– They help to identify relevant scenarios– Help to identify how much capital should

be injected in the whole system to prevent a credit crunch.

– Help to weed out weak institutions from strong ones, with a view to resolving those that have no future.

21

Issues to note about macro stress testing

• There are additional benefits of macro stress testing:– Help to discipline and structure thinking about

financial stability among many parties involved• Macroeconomists, finance specialists, risk managers,

loan officers, prudential supervisors.

– Improve availability and use of valuable historical data (property prices, interbank exposures, credit registry data).

– It can improve banks’ own stress testing practices, validate their models and assess their risk management systems.

22

Issues to note about macro stress testing

• There are elements of good practice macro stress testing.– There must be a will to really stress the system.

• Scenarios should not be over constrained by historical experience

– Ensuring buy – in by all stakeholders:• This is a precondition for the commitment of the time and

resources on the part of stakeholders and for follow up.

– Have a clear follow up plan in line with the objective of the exercise:• Authorities should always entertain the option of taking targeted

action to strengthen the systems defences (liquidity and capital backstops).

• Follow up communication strategy for financial stability policy is important:– Risk spotting fatigue – frequent exercises may be counter productive.

23

THE WAY FORWARD

24

The way forward

• It is important to use more complimentary information from leading indicators of financial distress. This helps to constrain the limitations of macro stress tests as early warning devices.– Deviations of credit-to-GDP ratio and property prices from

historical trends.

• There is also scope to improve the universe of institutions subjected to macro stress tests.– Extend beyond banks, beyond national borders.

• More serious treatment of common (similar) exposures and bilateral interlinkages is required.

• It is important to achieve the right balance between top-down and bottom up approaches.

25

Conclusions and implications for COMESA Countries

26

IV. Conclusions and Implications for COMESA Countries

• Macro stress tests will increasingly become the core element of macroprudential frameworks.

• It is important to understand what stress tests can and cannot do• Expectations should not be set unrealistically high.• It appears for the moment that macro stress tests will be best

suited to crisis management and resolution.• Stress tests as currently set up have limited reliability for

identifying vulnerabilities in tranquil times.• They can help to improve the dialogue about financial stability

vulnerabilities, but must be properly interpreted.• They can help spot shortcomings in our models of systemic risk

and financial crises.• Macro stress testing in African Countries still has a long way to

go given the data and methodological gaps that exist. There are several directions for improvement.

27

28

IV. Conclusions and Implications for COMESA Countries

• Data collection should be expanded and enhanced. Data on real estate prices as well as better bank data, including cross-border and –sector exposures should be collected.

• Stress testing approaches and skills should be improved. A distinction should be drawn between stress testing for supervisory purposes and for macroprudential surveillance with the latter focused on checking the vulnerability of the financial sector to macro shocks, systemic events, and common or interlinkage-resulting exposures.

• Regarding macroprudential analysis and mitigation measures, greater and more analytical use should be made of financial soundness indicators. Studies should be made of available and feasible macroprudential tools for financial stability policy.

29

Looking for risk in all the wrong places: The limitations of modeling

• Models Deepen, but Don’t Widen, Understanding Risks

• Financial Models Can Be Invalidated by Changing Human Behavior.– “I can calculate the motion of heavenly

bodies but not the madness of people” – Isaac Newton.

• Quantifying (e.g. VAR) Can Anchor Expectations in the Wrong Place.

REFERENCES

30

References

Barnhill, T and L Schumacher (2011): “Modeling correlated systemic liquidity and solvency risk in a financial environment with incomplete information”, IMF Working Paper, WP/11/263.

Borio, C, M, Drehmann and K, Tsatsaronis (2012) “Stress – testing macro stress testing: Does it live up to expectations? Monetary and Economic Department, BIS Working Papers No 369

Drehmann, M, S Sorensen and M Stringa (2010): “The integrated impact of credit and interest rate risk on banks: A dynamic framework and stress testing application”, Journal of Banking & Finance, 34, pp 735–51.

Drehmann, M and N Tarashev (2011): “Systemic importance: Some simple indicators”, BIS Quarterly Review, March, pp 25–37.

Ong, L, R Maino and N Duma (2010): “Into the great unknown: Stress testing in weak data”, IMF Working Paper, WP/10/282.

Quagliarello, M (2009): Stress testing the banking system: Methodologies and applications, Cambridge University Press.

31