Top Banner
ESSA Technologies Overview of Decision Analysis with examples - Jan 10, 2002 How ESSA has successfully used Decision Analysis to overcome challenges in multi-objective resource management problems General overview January 10 2002 Developed by ESSA Technologies Ltd. David Marmorek, Calvin Peters, Ian Parnell, Clint Alexander
31

Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

Mar 26, 2015

Download

Documents

Joseph Powers
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

How ESSA has successfully used Decision Analysis to

overcome challenges in multi-objective resource management

problems

General overview

January 10 2002

Developed byESSA Technologies Ltd.

David Marmorek, Calvin Peters, Ian Parnell, Clint Alexander

Page 2: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Common challenges in resource management

• Getting stakeholder groups to agree on a course of action, given multiple values and objectives

• Getting scientists to agree on which uncertainties most critically affect management decisions, and what decisions are most robust to these uncertainties

• Evaluating the costs and benefits of adaptive management - is it worth it?

Page 3: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

How decision analysis can help with these challenges

• It provides a toolbox for handling multiple objectives / values, and analyzing tradeoffs among these objectives

• It systematically analyzes the impacts of uncertainties on decisions

• It can be used to evaluate the ability of Adaptive Management experiments to improve decisions

• It provides a helpful way to integrate many techniques employed by managers and scientists (i.e. models, interactive workshops, sensitivity analysis) into products that better clarify management decisions

Page 4: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Three examples

• Getting scientists to agree: PATH

• Getting stakeholders to agree: Cheakamus

• Evaluating adaptive management: Keenleyside

Page 5: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

PATH: Decision Context

• Multiple historical changes in Columbia and Snake River ecosystems and fisheries management practices

• Endangered species listings for Snake River salmon populations

• Multiple hypotheses and uncertainties held by different groups of scientists

• Duelling models representing these hypotheses and uncertainties

• Best management policies for species recovery?

Page 6: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

PATH: Washington State, US

Page 7: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Decision Analysis: 8 elements

1. List of alternative management actions

2. Management objectives composed of performance measures (to rank management actions)

3. Uncertain states of nature (different hypotheses)

4. Probabilities of those states (to account for uncertainty);

5. Model to calculate outcomes of each combination of management action and hypothesised state of nature;

6. Decision tree;

7. Rank actions based on expected value of the performance measures; and,

8. Sensitivity analyses.

Page 8: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Decision Analysis: Basic Elements

Module 3 -36MoF Adaptive Management Training Course

Action 1

Managementactions

Probabilities ofstates of nature

States of natureor hypotheses

Outcomes orconsequences

Action 2

P1

P2

P1

P2

Hypothesis 1

Hypothesis 2

Hypothesis 1

Hypothesis 2

C11

C12

C21

C22

Page 9: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

PATH Decision Tree

Page 10: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Benefits of decision analysis in PATH

• Allowed evaluation of multiple hypotheses for 14 uncertainties - scientists did not have to agree!

• Only 3 of these turned out to make a difference to the decision - created a common focus for AM, research

• Preferred actions were those which were most robust to the critical uncertainties (drawdown A3)

• Sensitivity analyses defined how much belief you would have to have in a given hypothesis to change decision

Page 11: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Recent Publications on PATH

• Marmorek, David R. and Calvin Peters. 2001. Finding a PATH towards scientific collaboration: insights from the Columbia River Basin. Conservation Ecology 5(2): 8. [online] URL: <http://www.consecol.org/vol5/iss2/art8>

• Deriso, R.B., Marmorek, D.R., and Parnell, I.J. 2001. Retrospective Patterns of Differential Mortality and Common Year Effects Experienced by Spring Chinook of the Columbia River. Can. J. Fish. Aquat. Sci. 58(12) 2419-2430 http://www.nrc.ca/cgi-bin/cisti/journals/rp/rp2_tocs_e?cjfas_cjfas12-01_58

• Peters, C.N. and Marmorek, D.R. 2001. Application of decision analysis to evaluate recovery actions for threatened Snake River spring and summer chinook salmon (Oncorhynchus tshawytscha). Can. J. Fish. Aquat. Sci. 58(12):2431-2446. <same web site as above>

• Peters, C.N., Marmorek, D.R., and Deriso, R.B. 2001. Application of decision analysis to evaluate recovery actions for threatened Snake River fall chinook salmon (Oncorhynchus tshawytscha). Can. J. Fish. Aquat. Sci. 58(12):2447-2458. <same web site as above>

Page 12: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Cheakamus WUP: Decision Context

• British Columbia Hydro, Water Use Planning: Stakeholder driven multi-objective consultation / decision process.

• No formal incorporation of uncertainty as for PATH

• Emphasis: values, objectives, performance measures, trade off analysis (DA steps 1, 2, 5 and 7).

• Used PrOACT approach (Smart Choices, Hammond et al 1999)

Page 13: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Cheakamus WUP: ProcessPrOACT Approach

Problem

Objectives

Alternatives

Consequences

Tradeoffs

Clear choice

Many choices

WUP Steps

Page 14: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Cheakamus WUP:Decision ProblemSelect operating alternatives for Daisy Lake Dam that:

1) recognize multiple water uses in the Cheakamus and Squamish Rivers, and

2) achieve a balance between competing interests and needs.

Page 15: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Cheakamus WUP:Objectives and PMsFundamental

ObjectivesPerformance Measures

Average power revenue ($M/yr)

Power production (GWh)1. Maximize economicreturns from powergeneration. Greenhouse Gas emission reductions (Ktonnes/yr)

2. Protect integrity ofSFN heritage sites andcultural values.

Flood and erosion risk to ancestral burial groundsand culturally important locations

Rafting (Avg. #days/yr)

Kayaking (Avg. #days/yr)3. Maximize physicalconditions / access forrecreation (kayaking,rafting, sportfishing). Sportfishing (Avg. #days/yr)

4. Minimize adverseeffects of flood events.

Flooding (# floods >450cms at Brackendale)

Anadromous rearing Habitat Availability (m2),

Resident rearing Habitat Availability (m2)

Anadromous Effective Spawning Area (m2),

5. Maximize wild fishpopulations

Adult Migration flows (Avg. #days <10CMS)

Anadromous Riffle Benthic Biomass (kg benthos),6. Maximize area andintegrity of aquaticecosystem Resident Riffle Benthic Biomass (kg benthos)

Power

First Nations

Recreation

Flooding

Fish

Aquatic Ecosystem

Page 16: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Cheakamus: WUP Alternatives

• Consultative Committee specifies operating alternatives for Hydro operations model (AMPL).

• Basic constraints: minimum flow at Brackendale gauge, minimum dam release.

• AMPL model produces 32 water years of flow data for these control points

• Flow data and other models used to calculate performance measures.

• Performance measures summarize consequences of alternatives for objectives.

Page 17: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Cheakamus WUP: ConsequencesF u n d a m e n t a l

O b j e c t i v e sP e r f o r m a n c e

M e a s u r e s

1 5 M i n 3 D a m 1 5 M i n 5 D a m 1 5 - 2 0 M i n 3 -7 D a m " H y b r i d "

2 0 M i n 7 D a m 1 0 D a m

1 . M a x i m i z e e c o n o m i c r e t u r n s f r o m p o w e r

g e n e r a t i o n . A v e r a g e p o w e r

r e v e n u e ( $ M / y r )3 5 . 6 3 4 . 8 3 4 . 3 3 2 . 3 3 1 . 8

2 . P r o t e c t i n t e g r i t y o f S F N h e r i t a g e s i t e s a n d c u l t u r a l

v a l u e s .

K a y a k i n g ( A v g . # d a y s / y r )

1 2 3 . 9 1 3 7 . 7 1 9 9 . 8 2 4 2 . 0 2 0 4 . 1

S p o r t f i s h i n g ( A v g . # d a y s / y r )

5 7 . 6 7 2 . 0 8 2 . 7 1 9 2 . 8 1 2 2 . 0

5 . M a x i m i z e w i l d f i s h p o p u l a t i o n s ( x 1 0 3 m 2 )

R U A R e s i d e n t H a b i t a t R a i n b o w P a r r 3 5 . 8 3 7 . 7 4 2 . 5 4 2 . 5 4 5 . 2

E f f e c t i v e S p a w n i n g A r e a C h u m 9 . 8 9 . 2 9 . 7 7 . 3 6 . 5

6 a . M a x i m i z e a r e a a n d i n t e g r i t y o f a q u a t i c

e c o s y s t e m

R e s i d e n t R i f f l e B e n t h i c B i o m a s s ( g

x 1 0 6 )3 . 4 3 . 5 2 . 9 2 . 9 3 . 0

P a r t l y c o n s i d e r e d b y F l o o d P M s , w i l l b e a d d r e s s e d i n f u t u r e i f n e c e s s a r y .

A l t e r n a t i v e s

3 . M a x i m i z e p h y s i c a l c o n d i t i o n s / a c c e s s f o r

r e c r e a t i o n ( k a y a k i n g , r a f t i n g , s p o r t f i s h i n g ) .

Page 18: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Tradeoffs (or not)Tradeoff: VOE vs. RB Parr

10Dam20Min7Dam

20Min3Dam 20Min

7Dam 15Min5Dam

15Min3Dam

5Dam

0

10000

20000

30000

40000

50000

31.00 32.00 33.00 34.00 35.00 36.00

VOE ($M/yr)

RB

Par

r H

abit

at A

vail

abil

ity

(m2)

Tradeoff: VOE vs. Chum Effective Spawning Area

15Min3Dam

5Dam

15Min5Dam

7Dam

20Min

20Min3Dam

20Min7Dam

10Dam

0

2000

4000

6000

8000

10000

12000

31.00 32.00 33.00 34.00 35.00 36.00

VOE ($M/yr)

Ch

um

Eff

. S

pw

n. A

rea

Win-Win

Win-Lose

Page 19: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Cheakamus WUP: Filtering

• Use PMs to Eliminate clearly inferior alternatives.

• Drop insensitive PMs (e.g., rafting).

• Drop Objectives that don’t help the decision (e.g., flooding).

• Tradeoff analysis: Even Swaps

• Elicit values behind decisions (e.g., rating exercises)

• Develop new alternatives to address concerns (e.g., chum spawning vs. rainbow trout rearing).

Page 20: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Keenleyside Problem Keenleyside Problem : Increased egg mortality from dam operation

Flow during spawningFlow during spawning

Flow during Flow during incubationincubation

stage

Proportion eggs in de-watered areaRiskRisk

Biological Biological flows too high reduce productive capacity, may drive population towards extinction

Economic Economic smaller flows may reduce de-watering mortality but reduce potential $ and operational flexibility

Page 21: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Problem IIProblem II: Uncertainty True whitefish recruitment dynamics?

No reliable baseline information

Alternative Hypotheses

-

5,000

10,000

15,000

20,000

25,000

0 5 10 15 20 25

Eggs Just Prior to Hatching (millions)

Age

4 W

hite

fish

Very Sensitive

Sensitive

Neutral

Insensitive

Very Insensitive

Given typicalegg mortality,

LARGE differences in abundance

associated with these curves

Page 22: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Stage 1 - Decision Analysis w current uncertainty

Columbia RiverFlows During

WhitefishSPAWNING

Kootenay RiverFlows During

WhitefishSPAWNING

Min. Columbia RiverFlows Prior to

WhitefishHATCHING

Egg-Age4RecruitmentRelationship

EggAbundance

Abundance4+ Recruits

50 kcfs Model

30 kcfs

20 kcfs

85 kcfs

80 kcfs

20 kcfs

15 kcfs

10 kcfs

55 kcfs

25 kcfs

20 kcfs

15 kcfs

85 kcfs

a3, b 3

a2, b 2

a1, b 1

a5, b 5

a4, b 4

ForegonePower

Revenues

......

ManagementActions States of Nature and their Probabilities Outcomes

Min. KootenayRiver Flows Prior

to WhitefishHATCHING

20 kcfs

15 kcfs

10 kcfs

55 kcfs

...

40 kcfs

45 kcfs

55 kcfs

60 kcfs

65 kcfs

70 kcfs

Natural variability in flow Uncertainty due to lackof understanding / data

Page 23: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Stage 1 Results: Current UncertaintyExpected adult N, year 50

30,000

35,000

40,000

45,000

50,000

20 30 40 45 50 55 60 65 70 80 85

HKD Spawning Q (kcfs)

N

Minimum desired

Base Case: Current Uncertainty

Whitefish recruitment dynamics: Current state of knowledge

0

0.2

0.4

H1(sensitive)

H2 H3 H4 H5(insensitive)

P

Objective:Maintain “least cost” whitefish population nearest to or greater than 45,000 adults

Page 24: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Stage 2 - Simulated learning from flow experiments and monitoring

Uses same model and uncertain components but...

Actions are now alternative experimental Actions are now alternative experimental flow regimes + monitoring programsflow regimes + monitoring programs

Assume a true relationship for population Assume a true relationship for population dynamics with process errordynamics with process error

Page 25: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

What would you change if you knew the “truth”?If population insensitive, then maximize power revenues (85 kcfs)

If population sensitive, then minimize biological risk (~60 kcfs)Expected adult N, year 50

25,000

30,000

35,000

40,000

45,000

50,000

20 30 40 45 50 55 60 65 70 80 85

HKD Spawning Q (kcfs)

N

Minimum desired

Current Uncertainty

Sensitive

Insensitive

10

5

2.5

7.5$Cnd mil

Max. potential power revenues (per yr)

Page 26: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Example Stage 2 Results: Good monitoring is critical for differentiating hypotheses; flow

manipulation had less effect than expected.

Flow manipulation

High Meas. Error

Low Meas. Error

High Meas. Error

Low Meas. Error

Constant 0.55 ($0.48) 0.88 ($1.55) 0.51 ($0.48) 0.74 ($1.55)Passive 0.60 ($1.23) 0.92 ($2.3) 0.57 ($1.23) 0.85 ($2.3)

Active 0.63 ($3.48) 0.92 ($4.55) 0.54 ($3.48) 0.85 ($4.55)

$ CDN MillionsBlue = things under AM practitioners controlRed = beyond AM practitioners control

Probability identify insensitive population (10-year experiments)

Low Nat Variability High Natural Variability

Natural Variability and Measurement Error

Page 27: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

AM can “pay for itself”

Flow manipulation

High Meas. Error

Low Meas. Error

High Meas. Error

Low Meas. Error

Constant $0.2 (2.4) $0.6 (2.6) $0.2 (2.4) $0.2 (7.7)

Passive $0.2 (6) $0.6 (3.8) $0.2 (6.15) $0.6 (3.8)

Active $0.6 (5.8) $0.6 (7.6) $0.2 (17.4) $0.6 (7.6)

$Cnd millionsNumbers in brackets = experimental pay-back interval in yearsBlue = things under AM practitioners controlRed = beyond AM practitioners control

I ncrease in annual power revenues from operating with experimental information (insensitive population only, 10-year experiments)

Low Nat Variability High Natural Variability

Natural Variability and Measurement Error

Page 28: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Is AM and monitoring worth it?Is AM and monitoring worth it?

“Yes” IfNew information leads to choice of a

different management action that better satisfies a particular objective,

or rigorously confirms that current

management action is appropriate.

Page 29: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

No definitive “yes/no”No definitive “yes/no”

Management objective(fish vs. power $)

Ability to do well designed experiments

Initial level of uncertainty in alternative hypotheses

Magnitude of natural variability in the system

What “truth” really is

Inherent sensitivity of best action to uncertainty

FactorUnder AM

practitioners controlCan evaluate implications using decision analysis?

Yes

Yes

Maybe

No

No (can’t know without doing the experiment)

No

Yes

Yes

Yes

Yes

Yes

Yes

Page 30: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

General ConclusionsGeneral Conclusions• Value of AM potentially large

• Whether to proceed depends on “the kind” of system you are in (i.e. previous factors)

• Decision Analysis is very helpful for evaluating these benefits

– Determine which uncertainties have strongest effect on choice of “best” management decision

– Decisions more robust to uncertainties (reduces risk - integrates broader range of possible outcomes included)

– Include new information as revised probabilities on hypotheses

Page 31: Overview of Decision Analysis with examples - Jan 10, 2002ESSA Technologies How ESSA has successfully used Decision Analysis to overcome challenges in.

ESSA TechnologiesOverview of Decision Analysis with examples - Jan 10, 2002

Decision Analysis - SummaryDecision Analysis - SummaryElement of DecisionAnalysis

PATH –scientist

consensus

Cheakamus –stakeholderconsensus

Keenleyside –AM evaluation

Actions

Objectives

Uncertainties

Probabilities

Model

Decision Tree

Rank Actions

Sensitivity Analyses