DEVELOPMENTS IN OPERATIONAL RISK MANAGEMENT Neil Brown Managing Director Global Head of Risk Management & Product Control 16 April 2003.

DEVELOPMENTS IN OPERATIONAL RISK MANAGEMENT

Neil BrownManaging DirectorGlobal Head of Risk Management & Product Control

16 April 2003

RISK AND CONSEQUENCES

“...only the foolhardy make choices based on the probability of an outcome without regard to its consequences....”

“...only the pathologically risk-averse make choices based on the consequences without considering the probability involved...”

Peter Bernstein

Slide 3

CONSULTATIVE PAPERS

CP140 (Insurers) – February 2003 (advance of Prudential Sourcebook in 2004)

CP142 (Asset Managers) – 2004 (parts into Prudential Sourcebook, parts into Senior Management, Systems & Controls)

Should reflect “common practices at prudently managed firms and that many firms already meet it”

Risk Identification / Risk Management / Risk Control

Slide 4

Nature of firm’s customers / products / activities / distribution

Design / implementation / operation of processes / systems

Risk Culture

HR management practices

Operating environment: political / legal / technological / market structure

CONSULTATIVE PAPERS – Risk Identification

Slide 5

People resourcing / training / succession planning

Systems IT platform – minor manual error to major systemic error

External BCP

Outsourcing external / internal – still need to manage

Fraud / Money Laundering

Legal interpretation / enforcement of contracts

Group Risks assessment of other parts of Group

CONSULTATIVE PAPERS – Risk Management

Slide 6

Improving Risk Culture

Corporate Governance - structure

Audit Trail / Evidence

Insurance ?

CONSULTATIVE PAPERS – Risk Controls

Slide 7

Establish specific accountability, policies & controls

Clearly document procedures and map process flows

Ensure segregation of duties

Ensure access controls to assets / data privacy

Ensure audit trails / evidence

Ensure continuity and disaster recovery

Review & approve control processes

OPERATIONAL RISK FRAMEWORK

Slide 8

OPERATIONAL RISK FRAMEWORK

Event / Loss database / Self assessment

“Quantification” of risk exposure?

Control identification / mapping

“Quantification” of mitigation / net exposure?

Identification of control improvements

Action tracking process

“ Make the important measurable and not the measurable important”.

Slide 10

KEY INPUTS TO OPRISK MANAGEMENT PROCESS

Building Blocks……

Risk Reviews

Business Process Mapping

Control Self Assessment

Internal and External audit reports

Errors and Breaches Report

Compliance Monitoring programme

MIS data

Slide 11

KEY DELIVERABLES

Risk reviews / Process Maps / CSA action items.

Investigation of major errors and breaches.

Oversight of audit / BCP / ISO

Resolution and/or escalation of issues.

Slide 12

MANAGEMENT REPORTING

Key Risk Indicator / Key Control Indicator Reporting

Control Improvement Plans

Loss Data Reporting

Audit Tracking

Other Management Reporting

Slide 13

Quantification of OpRisk is sufficient to mitigate it

Any data is better than no data

Well run firms will be more certain about the probability and severity of an OpRisk Loss

Massive losses require EVT to model them

Insurance is an alternative to measuring and managing OpRisk exposures

Quantification is still nascent, and is only part of the issue

Loss data is context dependent

Well run firms will suffer from small sample problem in modelling OpRisk losses

Massive losses build over time

— Improve controls

— Evaluate relevance of EVT

Insurance is potentially an additional mitigation

SOME “MYTHS” SURROUNDING OPERATIONAL RISK

Slide 14

Market Risk Credit Risk Operational Risk

Risk position

Quantifiable exposure

Yes Yes Difficult1

Exposure measure

Position; Risk sensitivityMoney lent; Potential

exposureDifficult – no ready position

equivalent available1

Completeness

Portfolio completeness

Known Known Unknown

Context dependency

Context dependency

Low Medium High

Data frequency High Medium Low1

Measurement & validation

Risk assessmentVAR; Stress testing;

Economic risk capital

Rating models; Loss models; Economic risk

capital

No industry consensus; top-down scenarios may be

useful

Accuracy Good Reasonable Low

TestingAdequate data for

backtestingBacktesting difficult to

perform over short termResults very difficult to test

over any time horizon

Usage issues Usage issues

Instability of underlying price volatility;

Correlation instability in stressed markets

Many issues: correlations, ratings through time, data

lumpy

Results could be misleading; distraction

effect; false reliance; lack of cause and effect; redundant systems

SummaryMarket risk models well established and proven

tools

Using models considered reasonable – but should be used with

care

Models appear flawed

COMPARING OPRISK WITH MARKET RISK AND CREDIT RISK

1 Unlikely other than for certain high frequency low loss events, eg. operations losses.

Slide 15

OPERATIONAL RISK MODELS

Gross Income

— Simple, cheap,transparent, no loss data required, verifiable

— Backward looking, not indicative of risk, penalise well-run firms

Full Scorecard Approach

— Understands processes, uses firm knowledge, uses historical data, incentivises

— Very costly, bureaucratic, subjective

EVT

— Relevant part of loss distribution

— Ignores most of distribution, large losses not one-off events, small sample problem choice of threshold (how rare is rare)?

Slide 16

OPERATIONAL RISK MODELS

Bayesian Networks

— Cause/effect and control become apparent, prior probabilities based on firm knowledge and experience, estimates easy to update, scenario analysis easy, simplifies complex processes, networks are firm specific

— Complexity (require strong documentation), interpretation of results requires expertise, costly and time consuming (versus benefit?)

Monte Carlo simulation

— Handles complex systems, produces appropriate loss distribution, can be dynamic, precision increased by increasing number of simulations

— Larger the system the slower the process, complexity leads to few really understanding a complex system, choice of events to populate distribution key (GIGO), costly and time consuming (versus benefit?)

Slide 17

EXTERNAL DATA

Useful

— For external risks

— For information on HOW an event can occur

— A reminder of relevance of OpRisk

Not Useful

— To augment a small data set

— For “any data are better than no data” argument

Slide 18

VALIDATION

Validation of OpRisk models is a major issue:

Current published approaches do not address the “completeness of portfolio” issue

Causes of large losses are generally complex, the result of several factors so ability to predict future large losses based on previous ones is reduced

– Much easier to predict for operations processing losses where, generally, few factors often cause loss

Context dependency issue: Lack of cause and effect– As yet no proven predicative link between past and future events

Lack of sufficient relevant data: System (firm, organization unit within firm) changes in character before adequate data is accumulated to validate a model

– Sufficient data only available for the high-frequency, low-impact loss events – But these events would not drive the capital charge

Slide 19

PRACTICAL ISSUES FROM USING OPRISK MODELS

Basel 2 proposed Basic and Standard approaches: Current approaches could be misleading: Current basic indicator and

standardized approaches base the OpRisk capital charge on a single indicator such as gross income

– In general, more profitable institutions have less OpRisk – can invest in good people, systems, training

– Eg. compare with airlines – more profitable airlines generally safer– Single indicators could lead to dysfunctional accounting practices and perverse incentives– Some evidence that OpRisk losses of the same magnitude happen to big and small firms

Proposed OpRisk quantification approaches: False reliance: attempting to summarize all OpRisk into single measure &

managing by analogy to market risk and credit risk could be misleading and dangerous

– May give impression of being in control to senior management/owners when in reality model generating misleading results

Misleading output: May cause senior management/owners to take actions that reduce OpRisk per the model, but not in reality – Actions may actually increase real risk

Lack of cause and effect: If the model does not predict all causes and effects accurately, incorrect management decisions could be the result

Distraction effect: Focus on quantification will divert important resources from other work

– Potentially reduces the focus on sound risk management practices (Pillars 2 and 3)

Slide 20

SUMMARY Encourage innovation of best practices

– Current state of thinking for both OpRisk measurement and OpRisk management still evolving

– Rules need to remain flexible to offer banks incentives to continue development in this area

OpRisks are highly context dependent & causes of large losses are generally complex

– The higher the context dependency the less the past will be a good indicator for the future

No evidence yet to suggest that OpRisk is amenable to measurement to same extent as market risk or credit risk. No validated models that link back to underlying risk drivers

– Many of the current approaches could create a false sense of security & distract resources from other work

– If models had been in place in the past, how many material adverse OpRisk events would have been prevented?

CS approach – Focus resources on shrinking those “holes”(1) Devote OpRisk resources into improving OpRisk management

practices and tools, rather than quantification(2) CS’s current Economic Risk Capital approach is to ensure

management awareness of OpRisk and to integrate into overall risk capital process

(3) Most areas will use blend of tools - no silver bullet - lots of old fashioned management of people, MIS, systems, controls, etc.

Slide 21