Estimating Likelihood of Unlikely Events Dealing With Disasters Conference Northumbria University Newcastle 17-18 September 2015.

WILLIAM PILKINGTON, DPA,MPA,MA PUBLIC HEALTH OFFICER PUBLIC HEALTH AUTHORITY,

CABARRUS COUNTY, NORTH CAROLINA, USA

TIM STEPHENS, MPH CEO, MESH COALITION

INDIANAPOLIS, INDIANA, USA

Estimating Likelihood of Unlikely Events

Dealing With Disasters Conference

Northumbria University Newcastle

17-18 September 2015

“ARE WE SAFER”

Reality Check

“How much should we be willing to pay for a small reduction in probabilities that are already extremely low?”

According to Astronomer Alan Harris, “the lifetime probability that a resident of earth will die at the hands of terrorism is about one in 85,000, about the same probability of being killed by an asteroid or comet.”

Some Caveats Our Perspective Is Very Much American “We” Really Don’t Know How Much Risk Is

Reduced By Homeland Security Expenditures or How Many Terrorists Attacks Are Prevented

Resources are somewhat limited, but here are our recent publications and findings:

VALUE-BASED MODELS FOR SUSTAINING

EMERGENCY PREPAREDNESS CAPACITY

AND CAPABILITY IN THE UNITED STATES

A Paper Prepared and Presented to Institute of Medicine Forum on Medical and Public Health

Preparedness for Catastrophic

Events

http://iom.edu/~/media/Files/Activity%20Files/PublicHealth/MedPrep/v2Final%20white%20paper%20Preparedness%20FinancingJan14.pdf

Some Findings Need for specific measures of emergency

preparedness and response at both the national and community level

These specific measures should be instrumental in developing gap-analyses

Public investment should address the gaps and should include metrics to determine effectiveness and impact

Public-private coalitions have an important role in national and community preparedness and response

“The Need for A Value-Based Approach for Financing Public Health Preparedness and Response” (masters’s thesis, US Naval Postgraduate School, Center for Homeland Defense and Security, 2014), https://www.hsdl.org/?view&did=762427

Some Findings Biggest challenge in designing and funding

preparedness is deciding exactly how much to invest and determining impact of investments

One size does not fit all-some need for flexibility Focus on longer term evidence-based results

instead of one-time low impact investments Governments need to be more analytical in

establishing preparedness and response spending goals

An Economic Analysis and Approach for Health Care Preparedness in a Substate Region Benoit Stryckman, MA; Thomas L. Grace, MPM; Peter Schwarz, MBA; David Marcozzi, MD

http://www.phe.gov/ASPRBlog/Documents/brief-report.pdf

Some Findings Uncertainty, varying response expectations, recent declines

in preparedness funding for disaster preparedness, and continuing disasters make economic analyses increasingly important in health care preparedness.

The practical application of economic evaluation methods can help decision-makers better understand the value of preparedness and create financial mechanisms to ensure the sustainability of preparedness activities at the regional level.

Importantly, this economic analysis demonstrated a positive return on investment for the first 6 years of operations of the regional response team program.

Relevant Questions/Few Answers What is the probability of a terrorist attack on a

specific target? Is it not impractical to completely protects some

targets except by closing them down (tube, bridges) How does cost of protection compare with cost of

repair or replacement? What risk reductions have been achieved by security

spending? How many lives have been saved? What economic losses have been avoided?

What else could have been done with the money? What are the opportunity costs of say putting the money in education or social welfare programs?

All Value for Money Methodologies Are Different and the Differences Are Significant

Risk Assessment Analysis (RA) Cost-Benefit Analysis (CBA) Return On Investment (ROI) Cost-Effectiveness Analysis (CEA) Hazard Vulnerability Analysis (HVA)

Risk Assessment Threat Likelihood Establish Level of Acceptable Risk Establish Limits of How Much Threat Is Likely to

be Reduced by a Specific Security Intervention

Does Not Assume All Disasters Are Potentially Possible and therefore Must Be Addressed

Risk= (probability of event) X (losses sustained from event)

Cost-Benefit Analysis

On Other Hand-CBA Provides a Vehicle for Evaluating Competing Options

However, Calculating Benefits Is Not Easy, Especially with Respect to Terrorism Prevention Initiatives

Benefits(gained by new initiative) / Costs (new initiative) = Benefit-to Cost Ratio

Return On Investment

More Rigorous than CBA-Allows for Consideration of Time Value of Money

When Investments/Benefits Involve Future Years-Homeland Security-ROI Calculation Is Less Accurate-Improved Accuracy Requires Discounted ROI Calculation Method

ROI = (Gain from Investment – Cost)

Cost of Investment

Cost-Effectiveness Analysis (CEA) Expenses V. Measured Impact What is effective may not be cost

effective, but what is cost effective is always effective” (Maynard, Lancet, 1997)

Excellent Examples-Post Event Prophylaxis as Single Most Important Reducing Loss of Lives

Cost Effectiveness Ratio=Total Cost/Units of Effectiveness

Hazard Vulnerability Analysis (HVA) What Is Worth Protecting? How Much

Should Be Spent? Basic HVA Sequence

1-Probability and Impact of Hazard

Probability

2-Impact Ranked on Scale from Low/

Moderate/HighHVA Formula (Threat X Vulnerability) X Consequences = Risk

International Picture $1 Trillion Spent and Counting in USA on Homeland

Security and At Least $75 Billion in New Expenses Each Year in USA-Probably A Conservative Estimate

In UK, Dedicated Spending for “Counterterrorism, Intelligence, and Resilience” Increased from 923 Million Pounds in 2001-2002 to 2.5 Billion Pounds in 2007-2008 to 3.5 Billion Pounds in 2010-2011.

USA Spends About 2X as Much Per Capita Canada Spending Has Increased Each Year Since 2002

by Approx. US$1.9 Billion or C$2.1 Billion Australia Is Similar-About A$1.3 Billion, or US$1.1 Billion

“ A billion here and a billion there and pretty soon you are talking big money.”

What We Know: We Need A Shift in Thinking About Emergency Preparedness and Response From a Focus on Measuring Expenditures…to A

Focus on Outcome Measures From A Focus on Restricted Spending…to A Focus

on Supporting Innovation in Financing Preparedness and Response

From Large Fluctuations in Funding… to “Steady and Robust” Funding.

From A Focus on Investments…to A Focus on Benefits

Why We Know It

2004-9/11 Commission Called for Cost-Benefit and Risk Analytic Approaches to “assess the risks and cost-effectiveness of security measures.”

US Congressional Research Service Concluded, “Department of Homeland Security cannot answer central questions about the rate of return, as defined by the quantifiable and empirical risk reductions in DHS expenditures.”

Challenges to the Current Funding Model Finding the Right Level of Emergency

Preparedness Funding Agreement on Definition of “Prepared” Uncertainty About the Severity of An Event Public Attitudes…Zero Tolerance….Notion

that We Can Be Invulnerable to Another Attack

Current Model Derived from Past Mistakes, Politics

Some Problems Using A Value-Based Model Measuring Preparedness and Response1. Selecting Appropriate Range of Services

2. Catastrophic Emergency Disasters Are Relatively Rare

3. Is Cost of Mitigation for a Terrorism Event Disproportionate to the Risk from Other Natural Threats

4. Lack of Real Empirical Data Especially Probability Data

A Simple Equation But A Wicked Problem to Solve--Linking Funding to Threat, Vulnerability, or Consequences

Benefit=(probability of a successful attack) X (losses sustained in the successful attack) X (reduction in risk generated by security measures )

Risk

Event Probability X Losses Sustained In An Event

Probability Is Likelihood of a Natural or Man-made Disaster

Losses Include Fatalities and Other Damage Both Direct and Indirect

Example of Risk (Natural Disaster)

On the Average, Miami is Affected by A Hurricane Once Every Five Years. The Probability of a Hurricane in Miami in Any Given Year is 20%, or .20.

The Average Loss from a Hurricane Striking Miami is 26 deaths and Damages of $27M. Each Death is Valued at $6.5M for A Total Estimated Loss of $196M.

The Yearly Risk of A Hurricane in Miami is:

$196M X .20, or $39.2M

Benefit of Security Measures So, If Miami Implemented Security Measures that Reduced Risk for

Hurricane Damage by 30%, the Benefit of These Safety Measures Would Be

.20 (Probability) X $196M (Losses) X .30 (Reduction In Risk) , or $11.76M.

If The Cost of The Risk Reduction Measures Were Less Than $11.76M, The Benefits Would Outweigh the Costs, And The Measures Would Be Deemed Cost-Effective.

So Invest In Security Measures Up to $11.76M. This Provides

Benefits but Not Return on Investment Strategy. Here’s the Wicked Problem: There Are Other Threats to

Miami So How Does this Factor In? Which one creates the greatest ROI with most confidence of success?

ROI Application to Terrorism Example

Variation of Same Equation Can Be Used in A Break-Even Analysis to Calculate How Many Attacks Would Have to Take Place to Justify the Investment or Expenditure-Problem Is that Terrorist Attacks Are Rare So Estimating Probability Difficult

Probability of Successful Attack = Security Cost/

[(losses Sustained in the Successful Attack) X (Reduction in Risk)]

Assumptions $75B YR. Increased Expenditures Each Year for Homeland

Security (Probably Conservative) Probability of A Successful Attack .45 (this probability is

based on estimates alone and is likely very high) Avg. $100M Averaging All Successful Terrorists Attacks

Loss Sustained in Successful Attack

$75B / [$100M X .45] = 1667 Attacks Per Year

AT .45 PROBABILITY, WE WOULD NEED TO DETER, PREVENT, OR STOP 1667 ATTACKS PER YEAR-AVG. OF FOUR PER DAY AND AT .05 PROBABLITY (CLOSER TO REAL PROBABILITY), WE WOULD NEED TO DETER, PREVENT OR STOP 15,000 ATTACKS PER YEAR

$75B/ [$100M X .05] = 15,000 ATTACKS PER YEAR

Lives Saved Analysis

Value Life Saved (VLS) Is US$6.5M

AT INCREASED EXPENDITURES OF US$75B YR-THERE WOULD HAVE TO BE 11,500 LIVES SAVED PER YEAR FOR THE INCREASED EXPENDITURE TO BE COST-EFFECTIVE

Other Premises That Must Be Considered in Developing A Value-Based Approach Number of Terrorists Targets Infinite Number of Competent Terrorists Appears

Small Target Selection Random Probability for Any Specific Target Is Small Too Much Protection Moves Targets Most Targets Can Be Rebuilt in Short

Order No Target Can Be Completely Protected

Opportunity Cost Considerations What Else Could Be Done With $75B

1. Compensate Families

2. Rebuild Attacked Structures

3. Other ROIs Such As Investment in Education or Health Care Might Yield More Positive Results-Some Research Indicates The ROI on Innovations in Education or Mental Health Would Be Much Greater Than for Homeland Security

4. This is key. Where are there better ROI’s or at least the best ones in preparedness.

So, In Conclusion We Need Better Intelligence (Decision-Support

Systems) to Improve Probability Estimates We Need to Incorporate Value-Based Decision-

Making into Critical Security Financing Decisions-Maybe Leading to a Risk-Acceptance Criteria

Expenditures Designed to Lower Terrorism Risks Have Been Excessive Unless We Know How Many Terrorists Attacks We Have Prevented

Politicians Are Never Wrong