[email protected] Total Cost of Cyber (In)security – Integrating operational security metrics into business decision-making Russell Cameron.

[email protected]

Total Cost of Cyber (In)security –Integrating operational security metrics

into business decision-making

Russell Cameron Thomas

Principal, [email protected]

Mini-Metricon, February 5, 2007

San Francisco, CA

Mini-Metricon, San Francisco - Feb 5, 2007

Page 2

[email protected]

© 2007 Meritology. All Rights Reserved

Total Cost of Cyber (In)security

Purpose of this Talk

• To introduce a new approach

– Influence thought leaders,

academic research, and

professional practice

– Stimulate your thinking and

inspire hope

• Build productive bridges between business and IT

– Show how key concepts of

each can be made compatible

– Take a stand on what will work

and what won’t

• To get your feedback– Is this on the right rack? Is it

worth pursuing?

– Does it fit with other

approaches to security metrics?

• To recruit collaborators and advocates

• Non-purposes– Debate the devilish details

– Debate politics

– Debate acceptability in

“Mainstream” and “Late

Adopter” organizations

• It will take years, of course!


Page 3

[email protected]



The Challenge

• Problem: Disconnect between business decision-makers and security specialists regarding value and risk of InfoSec*

– “Security directors appear to be politically isolated within their companies”

– “They face a challenging search for allies when they need to gain support

from upper management for new security initiatives.”

– “Companies reported less alignment of security with long-range strategic

objectives of the firm.”

– “The results suggest that security remains a function that is mired in

operations in the eyes of senior executives.”

• Result: under-spending, over-spending, misallocation, burden-dumping, denial, and worse…

– Fighting the last war

– Failures of imagination

– Unintended consequences

* Conference Board Survey Oct. 2006: “Navigating Risk—The Business Case for Security”


Page 4

[email protected]



The Simplistic Approach is a “Blind Alley”ROSI*, ALE**, and variants

p(L|ei) Lii = 1

^i = incident types

n

Probability of loss given incident &

exposure

Expected loss value

Why a “blind alley”?

• Laplace’s Dream: “If only we had more data…”

• (see appendix)

Why a “blind alley”?

• Laplace’s Dream: “If only we had more data…”

• (see appendix)

V =

ROSI* = ∆V / I

* “Return on Security Investment”

** “Annualized Loss Expectancy”

Security “investment”

Loss of Economic Value

Example reference: “Calculated Risk - Guide to determining security ROI” - CSO Magazine - December 2002


Page 5

[email protected]



Two Viewpoints on Economic Risk

#1 “Rational Investor”(Capital Asset Pricing,Discounted Cash Flow)

#2 “Insurance Actuary”(Ruin Theory,“Iceberg Risk”)

“Ruin”

time

timechange in value

change in value

valu

eva

lue random walk with

“avalanches”

random walk

p(v)

p(v)

What matters:• Mean, variance• Fat part of the curve

When:• Quarterly EPS• Earnings volatility• Shorter time periods

99%

What matters:• Extreme events• Tail of the curve

When:• Credit rating• Solvency• Reserve funds• Longer time periods

Normaldistributions

“Fat Tailed”and skeweddistributions


Page 6

[email protected]



The Core Idea: Three Costs CategoriesA

nnua

l Pro

babi

lity

Total Cost of InfoSec

mean

1x 10x 100x 1,000x

1 2 3 4 5 6 7

“Budgeted” “Self-insurance” “Catastrophic”

(borrowed from “Value at Risk” concept in Financial Services Risk Management)

Idealized


Page 7

[email protected]



Budgeted Costs

• Q: What is the expected (average) impact of security-related costs on EPS and earnings volatility (+/– budget)?

• The rule: costs must already be in the budget* somewhere– Defined to fit the budget and spending approval processes

– Results in stable ratio-scale values

– Theoretically and practically sound

• Applies Activity-based Costing methods

• Compatible with accounting practice (GAAP)

• Fits discounted cash flow assumptions for multi-year analysis

– Good information available (in principle)

– Simple Arithmetic Tractable and simple to understand

– Composable across organization units and systems

• “If you are claiming cost reductions, show me whose budget I should cut. If you are claiming revenue increases, show me whose sales quota I should raise.” (Exec VP)

* Includes both operating and capital budgets, but excludes cyber insurance or reserves


Page 8

[email protected]



Calculating Budgeted Costs (1)

• Aggregate direct costs– Security staff, training, awareness, tools, services, technology, management, threat

monitoring, assessments, etc.– Direct cost of predictable and expected loss events and remediation w/ portfolio

effects

• Use cost driver models for indirect costs– Patch testing, installation, upgrades, etc.– Vendor support costs, 3rd party support– Help desk– New employee screening and hiring process– Indirect costs of predictable and expected loss events with portfolio effects

• Negotiate cost allocation rules for bundled and overhead costs– Infrastructure software and hardware costs– Application software– Internal IT development– Legal dept.

• Identify costs from unintended consequences and “business prevention”

– It’s a judgment call how best to account for these, but they will win credibility!

• If possible, use incremental cost analysis, not just total costs– Compare to a base case (e.g. a “barely legal” budget)


Page 9

[email protected]




Modeling indirect costs using cost drivers: e.g. Desktop/Laptop Incidents and Remediation

Cost #1: Provisioning

Cost #2:Help Desk

Method:

1. Identify cost drivers using security metrics combined with business operational metrics (e.g. number of new employees, turnover, etc.).

2. Aggregate and simplify where possible.

3. Only account for budgeted (forward-looking) costs. Use historical costs as a guide, if available.

Illustrative

Benefits:

• Simplicity – many fewer budget categories than incident types, scenarios, etc.

• Effectiveness – puts attention on the right levers

• Focus – most often, a few cost drivers dominate (80/20 rule).


Page 10

[email protected]




Modeling indirect costs using cost drivers: e.g. Indirect costs of predictable and expected loss events, with portfolio effects

Asset:Customer

DB

attacks, breaches, incidents

Damage, violations, etc.

Abstracted and

Aggregated

Exposure, given defenses RiskDrivers

Detection, remediation, etc..

Cost Categories:

• Staff (extra headcount)

• Customer Service (damage control)

• etc.

Benefits:

• Simpler calculations

• More robust to varying assumptions

Cost Drivers


Page 11

[email protected]



Decision Framework for Budgeted Costs Differential Analysis

#1 TotalBudgetedCosts vs. benchmarks

“Barely legal”Budget

Dir

ect

Ind

irec

t

“ Premium”Budget

CurrentBudget

Tim

e

Current

#4 Self-insurance CostImplications

Higher

Same

Lower

#3 Lifetime

#2 Budget Optimization


Page 12

[email protected]



Self-Insurance Cost

• Q: How much money would you put aside each year into a reserve fund* to avoid a serious decline in credit rating due to low-probability/high-impact losses?

• The rule: an actuarially-sound self-insurance premium, given…

– Budget-busting loss events • Severe outage, delay in a key new product, loss of major sales contract, etc.

• Material to quarterly EPS (> 1% )

– Extreme loss events (short of bankruptcy) that threaten credit rating, etc.• Long-lasting business interruption, executive fraud, earnings restatement,

regulatory action, punitive damages, etc.

– Interdependencies, correlations (“avalanche effects”), and portfolio effects– Parameters: Maximum risk threshold and time horizon set by top management– “Mark to Model” approach, calibrated by history & “wisdom of the crowds”

• A betting man’s judgment: “The race doesn’t always go to the swiftest, but that’s how you bet.”

*Analogous to the concept of Economic Capital in financial services


Page 13

[email protected]



Calculating Self-Insurance Cost (1)

99th Percentile threshold

1 2

Budget threshold

Self-insurance pool (“Value at Risk”)

3 Time period*

54 Fund solvency*Shape of the curve

Annual premium ≈ Pool ÷ (Time Period)Annual premium ≈ Pool ÷ (Time Period)

Estimation Parameters

* Policy decisions by top management

Modeling:

• Distribution curves from parameters

• Monte Carlo simulation of self-insurance pool with funding parameters, interest rates, etc. to calculate annual premium

• Dominated by largest losses 2

6 Interest rates

Cost distribution

curve

Magnitude of costs

(if time period is long enough)


Page 14

[email protected]



Calculating Self-Insurance Cost (2)

How: A Competitive Marketplace for Models

time

ConsensusEstimates

Prediction Markets

Delphi Technique

Qualitative Reasoning (e.g. Inference to the Best Explanation,

Reasoning about Uncertainty, etc.)

Bayesian Networks

Statistical analysis of

historical loss data

External data bases, benchmarks

parameter

Parameter values change with new information

Assessments,Scorecards

Simulations


Page 15

[email protected]



Ways to Make Self-Insurance Cost “Real”

• Link it to real cyber insurance policies

• Set up a real self-insurance fund via Finite Risk program* or tradable subordinated debt

• Use it as the “glue” for multi-firm “risk sharing” pools– Focused on information sharing and mutual assistance, with incentive instruments

• Link to performance management and incentive compensation– Subdivide Self-Insurance Cost into a “Risk Budget” for each org. unit, or– Use it as a “risk adjustment” factor for other performance metrics

• Create incentive instruments tied to self-insurance costs or cost drivers for…

– Security outsource vendors– Supply chain partners– Channel partners – Customers– Alliance partners

• Public disclosure– SEC filings, other regulatory filings– Stakeholder reports– Credit rating agencies– “Cap and Trade” markets *See appendix


Page 16

[email protected]



Catastrophic Costs

• Q: How much confidence should we have that the firm can survive InfoSec catastrophes?

• The rule: prioritized loss scenarios above a significance threshold that cover the space of possibilities.

– Use for business continuity preparation → agility and robustness

– Avoid failures of imagination and “fighting the last war”

– Root out unintended consequences

– Categorize and prioritize – don’t waste time on precision estimates

– Strategic scenario analysis, “war gaming”, etc.

– Focus on discovery, “out of the box”, and reframing

– Challenge conventional wisdom!

• “It’s not what we don’t know that will kill us. It’s what we know that ain’t so”.


Page 17

[email protected]



Risk Management Decisions

BudgetedCosts Self-insurance

Costs

CatastrophicCosts

Prudence Gambling


Page 18

[email protected]



A Simple Example – Earthquake Preparation

Probabilities Min Prep. Max Prep. Benefits

Quake 2%

No Quake 98%

Mod. | Quake 88% 94% 46% lower cost of moderate damage

Severe | Quake 10% 5% 50% reduction in probability of severe damage

Death | Quake 2% 1% 50% reduction in probability of death (catastrophe)

#1: Minimum Preparation #2: Maximum Preparation

Probability Cost ALE Probability Cost ALE

Preparation costs 98% $ 60 $ 59 98% $ 1,500 $ 1,470

Mod. Damage 1.76% $ 57,060 $ 1,004 1.88% $ 31,500 $ 592

Severe Damage 0.20% $ 500,060 $ 1,000 0.10% $ 501,500 $ 502

Death + Severe 0.04% $ 2,500,060 $ 1,000 0.02% $ 2,501,500 $ 500

ALE $ 3,063 ALE $ 3,064

Mean* $ 2,887 Mean* $ 3,087

Spend an extra $1,440 per year over 30 years for earthquake loss reduction?

ALE same for both Simple average says “no” to extra spending

*from Monte Carlo simulation


Page 19

[email protected]



Self-insurance Costs (1)

Minimum Preparation Loss DistributionMonte Carlo Simulation 1,000 points: 30 year time series

0

100

200

300

400

500

600

-$25

,621

$187

,655

$400

,931

$614

,207

$827

,483

$1,0

40,75

9

$1,2

54,03

5

$1,4

67,31

1

$1,6

80,58

7

$1,8

93,86

3

$2,1

07,13

9

$2,3

20,41

5

$2,5

33,69

1

$2,7

46,96

7

$2,9

60,24

3

Fre

qu

en

cy

99

% p

erc

en

tile

7 "catastrophes"

Self-insurance

$612,207($20,414 / yr)


Page 20

[email protected]



Self-insurance Costs (2)

Maximum Preparation Loss DistributionMonte Carlo Simulation 1,000 points: 30 year time series

0

100

200

300

400

500

600

700

-$25

,621

$187

,655

$400

,931

$614

,207

$827

,483

$1,0

40,75

9

$1,2

54,03

5

$1,4

67,31

1

$1,6

80,58

7

$1,8

93,86

3

$2,1

07,13

9

$2,3

20,41

5

$2,5

33,69

1

$2,7

46,96

7

$2,9

60,24

3

Fre

qu

en

cy

99

% p

erc

en

tile

8 "catastrophes"

Self-insurance

$529,598($19,093 / yr)

Total Cost ComparisonMax. Prep. vs. Min. Prep.

Budgeted $ (1,440)

Self-insurance $ 2,760

Annual Savings $ 1,320

Justifies extra

spending on

maximum preparation

Justifies extra

spending on

maximum preparation


Page 21

[email protected]



Needed: Self-insurance Decision Framework


Budgeted $ (1,440)

Self-insurance $ 2,760

Annual Savings $ 1,320


Budgeted $ (1,440)

Self-insurance INTEREST COST @ 10% $ 2,76

Annual Savings $ (1,164)

Which is more credible? Which leads to better decisions?

B. Self-borrowing

A. Like other insurance


Page 22

[email protected]



Summary of the Method

• Apply enterprise risk management methods

• Break InfoSec costs into three categories:– “Budgeted”

– “Self-insurance”

– “Catastrophic”

• Establish methods, targets, and decision processes for each category

– Appropriate to the information and uncertainty involved

– The nature of decisions that apply

– Link the categories

• Use operational metrics plus inference to model costs in each category, as appropriate

• Focus energy on continuous organization learning


Page 23

[email protected]



Next Steps

• Need more theoretical development and empirical testing– Esp. self-insurance concept, models, and decision rules.– Factoring in impact on revenue, market share, profitability (pricing power), and

reputation

• Need to standardize “Budgeted Costs” and map to InfoSec assessments and frameworks

• Need proofs-of-concept using real companies and real data

• Make it work politically– Enterprise Risk Managers = your new best friends– TQM and 6 Sigma Specialists = your allies– CFOs = Status excelsior sponsors– Neutralize or convert opposition (legal department, auditors, etc.)– Lead industries = Financial Services? Supply Chain? other?– Political change role model = Indian Gaming??

• Make it acceptable to the mainstream managers

• Q: is it sufficiently promising to continue pursuing?

[email protected]

Appendix

Russell Cameron Thomas

Principal, [email protected]

Mini-Metricon, February 5, 2007

San Francisco, CA


Page 25

[email protected]



Why Measuring the Value of InfoSec is Hard (1)

• Information security (InfoSec) should be seen* as a component of enterprise risk management.

– "Risk” is a forward-looking estimate of uncertain loss over a time period

(same as the timeframe for return on the assets).

– Must cope with all forms of uncertainty and ignorance that apply to actors,

assets, threats, vulnerabilities, and learning/adaptation over that timeframe.

• InfoSec is a repeating evolutionary game– Between threatening actors (incl. nature) and protecting actors (incl. nature)

– Each with an evolving capability set, which may be emergent, nascent,

and/or tacit.

– The terrain for the security game is threats, vulnerabilities, assets, etc.

– Thus, "security" is not a state of the system or the assets. It's how the

protecting actors define success in the game over time.

– Economics of repeating evolutionary games aren’t well understood yet.

They don’t fit existing static equilibrium investment models. They require

emergent, dynamic models, e.g. agent-based simulation

*From the viewpoint of business value


Page 26

[email protected]



Why Measuring the Value of InfoSec is Hard (2)

• InfoSec* is inextricably part of the cyber trust “fur ball”, including– Privacy

– Digital Rights

– Intellectual Property, brands, reputation, trade secrets

– Stakeholder disclosure

• … and physical security

• Historical loss data, even if copious and available, has limited use– The landscape changes too fast

– Low frequency / high impact events matter

– Unique events matter

• The business value of InfoSec isn’t just loss prevention – Value comes from the ability to support profitable risk taking

• e.g. Brakes, condoms

– Risk balancing is a reflexive process involving perceptions of risk and reward

• Varies dramatically by industry and sector– E.g. a bank vs. a rock quarry

*From the viewpoint of business value


Page 27

[email protected]



Blind Alleys and Dirt Roads

• “Blind Alleys” look good in concept, but won’t work by themselves

– Return on Investment (ROI), Net Present Value (NPV), Payback, etc.– Annualized Loss Expectancy (ALE)– Cyber insurance– Product liability and tort laws (“actual damages”)

• “Dirt Roads” work, but just barely– 2x2 or 3x3 matrix categorization of incident types or risks by frequency vs.

severity– Assessments using scoring and ranking systems– Balanced scorecards– Strategic scenario analysis and walkthroughs

• Are there any “Autobahn” approaches out there?– The null / “realist” hypothesis is “no”, assuming insurmountable problems– “Total Cost of (In)security” might be such an approach


Page 28

[email protected]



Why ALE is Dumb

• A Simple Case of Three Loss Event Categories*

– Firm Equity = $50 million; Annual Earnings = $5 million; ROE = 10%– Category A: “Common flood”

• 50% chance of $10,000 loss = $5,000 ALE

– Category B: “100 year flood”• 1.0% chance of $500,000 loss [10% of earnings, 1% of equity] = $5,000 ALE

• 26% chance of happening at least once in 30 years

– Category C: “10,000 year flood”• 0.01% chance of $50 million loss [100% of equity] = $5,000 ALE

• Reason 1: ALE math hides risk drivers– A+B+C = A+A+A = B+B+B = C+C+C = $15,000 ALE [1.5% of earnings]– Conflates simple random walks with random walks with avalanches

• “Three independent common risks = three independent catastrophic risks”

• Reason 2: Unreliable estimates of low probability events dominate – Lack of data + psychology means estimation errors for the tail are much higher

• 50% 55% chance for A $5,250 ALE

• 1.0% % chance for B $10,000 ALE (45% chance in 30 years!)

• 0.01% 0.05% chance for C $25,000 ALE

• $40,250 ALE (2.7 times larger!)*Pareto Distribution, k=1, min = 5,000


Page 29

[email protected]



Finite Risk Programs

Year 1

Fund established$$$ Operational

lossesInterest paid

Balance carry-forward

The insurance industry offers multi-year self-insurance plans that are commonly called finite risk insurance. The name arises from the fact that the risk transfer is very limited. Therefore, the insured will pay for most (or all) the losses

From: “Applying Insurance ModelingTechniques to Quantify OR”Dr Marcelo Cruz, RiskMaths, presented atGARP OR Seminar18-19 October 2001 London

time


Page 30

[email protected]



Ruin Theory applied to Finite Risk

n

jinn XR

1

Initial Finite Risk capital

Percentage of grossincome allocated against Finite Risk

Losses following a certain stochastic process

-40,000,000

-20,000,000

0

20,000,000

40,000,000

60,000,000

80,000,000

1

15 29 43 57 71 85 99

113

127

141

155

169

183

197

211

225

239

253

267

281

Finite Risk hedgingneeds

“ruin”

From: “Applying Insurance ModelingTechniques to Quantify OR”Dr Marcelo Cruz, RiskMaths, presented atGARP OR Seminar18-19 October 2001 London

[email protected] Total Cost of Cyber (In)security – Integrating operational security metrics into business decision-making Russell Cameron.

Documents

security slide

san francisco

security specialists

alignment of security

security directors

cost reductions

loss value

cyber insurance