-
Ecological Indicators 26 (2013) 7686
Contents lists available at SciVerse ScienceDirect
Ecological Indicators
jo ur n al homep ag e: www.elsev ier .com
Multi-criteria decision analysis to select metrics sustain
M. Conve le, a Department ob Florida Climac Badger Techn eam,
(ERDC), Concord Department of Civil and Environmental Engineering,
Environmental and Water Quality Program, Massachusetts Institute of
Technology, Cambridge, MA, USAe Risk and Decision Science Team,
Environmental Laboratory, Engineer Research and Development Center
(ERDC), U.S. Army Corps of Engineers, Vicksburg, MS, USAf
Department of Engineering and Public Policy, Carnegie Mellon
University, Pittsburgh, PA, USA
a r t i c l
Article history:Received 28 FeReceived in reAccepted 8 Oc
Keywords:Multi-criteria EnvironmentaEcosystem
resMonitoringStakeholder p
1. Introdu
In the cmeasurablequantify thent life stagof Engineer
Corresponand Florida Cl32611-0570, U
E-mail add
1470-160X/$ http://dx.doi.o e i n f o
bruary 2012vised form 2 October 2012tober 2012
decision analysisl metricstoration
references, utility
a b s t r a c t
The selection of metrics for ecosystem restoration programs is
critical for improving the quality andutility of design and
monitoring programs, informing adaptive management actions, and
characterizingproject success. The metrics selection process, that
in practice is left to the subjective judgment of stake-holders, is
often complex and should simultaneously take into account
monitoring data, environmentalmodels, socio-economic
considerations, and stakeholder interests. With limited funding, it
is often verydifcult to balance the importance of multiple metrics,
often competing, intended to measure differentenvironmental,
social, and economic aspects of the system. To help restoration
planners and practition-ers develop the most useful and informative
design and monitoring programs, we propose the use ofmulti-criteria
decision analysis (MCDA) methods, broadly dened, to select optimal
ecosystem restora-tion metric sets. In this paper, we apply and
compare two MCDA methods, multi-attribute utility theory(MAUT), and
probabilistic multi-criteria acceptability analysis (ProMAA), for a
hypothetical river restora-tion case study involving multiple
stakeholders with competing interests. Overall, the MCDA results in
asystematic, quantitative, and transparent evaluation and
comparison of potential metrics that providesplanners and
practitioners with a clear basis for selecting the optimal set of
metrics to evaluate restorationalternatives and to inform
restoration design and monitoring. In our case study, the two MCDA
meth-ods provide comparable results in terms of selected metrics.
However, because ProMAA can considerprobability distributions for
weights and utility values of metrics for each criterion, it is
most likely thebest option for projects with highly uncertain data
and signicant stakeholder involvement. Despite theincrease in
complexity in the metrics selection process, MCDA improves upon the
current, commonly-used ad-hoc decision practice based on
consultations with stakeholders by applying and
presentingquantitative aggregation of data and judgment, thereby
increasing the effectiveness of environmentaldesign and monitoring
and the transparency of decision making in restoration
projects.
2012 Elsevier Ltd. All rights reserved.
ction
ontext of ecosystem restoration projects, metrics are system
properties that characterize the system ande impact of restoration
activities, possibly at differ-es of restorations (Allen et al.,
1997; U.S. Army Corpss, 1999; Nienhuis et al., 2002; Reichert et
al., 2007;
ding author at: Department of Agricultural and Biological
Engineeringimate Institute, Frazier Rogers Hall, Museum Road, PO
box 110570,SA. Tel.: +1 781 645 6070; fax: +1 352 392 4092.ress:
[email protected] (M. Convertino).
Seager et al., 2007; Martine and Cockeld, 2008; McKay et al.,
2011).Thoughtful, appropriate metrics selection is key to
effectively char-acterizing the system, selecting a restoration
strategy or a singlerestoration among a set of restoration
alternatives, and under-standing the effects of project actions on
the system (Ehrenfeld,2000). Appropriate, clearly dened metrics
should reduce uncer-tainty, increase knowledge of the system and
assess the usefulnessof applied restoration alternatives by
creating a targeted, effectivemeans of evaluation. The evaluation
of a restoration alternative canoccur both pre- and post-execution
(Holmes, 1991), and it is cer-tainly important considering the
variability of climate and otheranthropic factors (Palmer et al.,
2008). For example, a monitor-ing plan based on sound metrics can
demonstrate progress and
see front matter 2012 Elsevier Ltd. All rights
reserved.rg/10.1016/j.ecolind.2012.10.005able ecosystem
restorations
rtinoa,b,, K.M. Bakerc, J.T. Vogelc, C. Lud, B. Suedef
Agricultural and Biological Engineering, University of Florida,
Gainesville, FL, USAte Institute, c/o University of Florida,
Gainesville, FL, USAologies contractor with the U.S. Army Corps of
Engineers, Risk and Decision Science Td, MA, USA/ locate /eco l
ind
for design and monitoring of
I. Linkove,f
Environmental Laboratory, Engineer Research and Development
Center
-
M. Convertino et al. / Ecological Indicators 26 (2013) 7686
77
the degree to which objectives of a restoration are being met
toleadership, stakeholders, and future project sponsors, increase
thedepth and breadth of understanding about the effects of
ecosystemrestoration practices, contribute to expanding knowledge
aboutecosystemstive, efcienThom and Grootjans eations are trof
environmecosystem r
The comgives rise toExtensive lioptions, ofttem charac2006). For
eprovides fhabitat suplogical, geogothers (Thaonly be posa few
metrclearly indiproject goa
Metrics choice of ming multiplcommunicauating thestask that
rmethod. Thselection, indence, concsets, and AnDale and BeMoberg,
20we briey refer the remethodolog
The use sive and timmethod forselection
vistakeholdercomplexityjudgment amakes the d(Dale and B
Historicabeen previo(e.g. those wical charactinvolve simmetrics
oftcross-compinvestmenttion via thiwell-suitedmore famili
As a mofessional jumanagers magainst a sof metrics fand
time-ef
a more structured metrics selection method than best
professionaljudgment and historical precedence, but is generally
not adequateas a standalone method. Screening does not facilitate
formal consid-eration of a metrics utility within the total
collection of its metrics
mosijer
interehenlyticsele
edgeecis
nmen et alrisond wevers007)imprringtion
thete alte fo
e impbergst uswouformrove
e band uatintion
MCDer. Mles ice aria a
2004ple
tandarent2009erentsion006;
and techor eed tot actihis ptem
a h Riveosyst in als avelopectioic anA). R
anaparattudy., and guide management decisions on the most
effec-t, and cost-effective courses of action (Kondolf, 1995;
Wellman, 1996; U.S. Army Corps of Engineers, 1999;t al., 2002;
Rohde et al., 2004). The same consider-ue for design plans that aim
to change the congurationental systems at the initial or
intermediate steps ofestorations.plexity of ecological systems and
restoration objectives
a multitude of potential ecosystem monitoring metrics.sts of
monitoring metrics provide hundreds of potentialen with numerous
choices for just one specic ecosys-teristic (Thayer et al., 2005;
Faber-Langendoen et al.,xample, NOAAs Tools for Monitoring Coastal
Habitatsteen different metrics to monitor whether a mangroveports a
complex trophic structure alone, including bio-raphical,
hydrological, and chemical metrics as well as
yer et al., 2005). However, with limited funding, it maysible to
effectively measure, estimate, or otherwise useics, so it is
critical to select the metrics that can mostcate the state of the
system and changes in relation tols.selection is thus a challenging
process. The optimumetrics will depend on a number of factors
includ-
e project objectives, technical feasibility,
effectiveness,bility, and stakeholder preferences. Balancing and
eval-e factors with respect to each metric choice is a
difcultequires a comprehensive, practical metrics selectionere are
a number of commonly used methods for metricscluding best
professional judgment, historical prece-eptual modeling, screening
using established criteriaalytic Hierarchy Process models (AHP)
(Saathy, 1980;yeler, 2001; Niemeijer and de Groot, 2008; Linkov
and11; Convertino et al., 2012; Mexas et al., 2012). Heredescribe
only the most commonly used methods andader to more extended review
papers for additionaly (see for example Linkov and Moberg,
2011).
of best professional judgment (BPJ) is generally
inexpen-e-efcient and may be an appropriate metrics selection
small, well-understood projects. However, metricsa this method
may exclude or place bias on specic
values, and becomes exceedingly difcult as project increases.
Another weakness of both best professionalnd historical precedence
is lack of transparency, whichecision-making more difcult to
document and justifyeyeler, 2001; Niemeijer and de Groot, 2008).l
precedence constitutes selection of metrics that haveusly utilized
in similar ecosystem restoration programsith similar objectives,
with similar regional or ecolog-eristics, that respond to similar
disturbances, and/orilar stakeholders). Maintaining the use of
historicalen allows for easy comparison to baseline data andarison
among projects, and may involve lower initial
than developing new metrics. However, metrics selec-s method may
encourage project planners to overlook
and site-specic metrics in favor of less appropriate butar
metrics.re transparent alternative or supplement to best pro-dgment
and historical precedence, restoration projectay sometimes evaluate
or screen potential metrics
et of criteria to identify the most appropriate subsetor a given
project. Screening is relatively inexpensivecient, and criteria are
well-documented. Screening is
set, as(Niemetative compr
Ananative knowlof the denviroHuangcompamalizerank reet al.,
2
To monitoapplicascienceevaluaity valurelativand Mothe mowhich
well-into imptices. Wuseful in evalrestorajustifyconsidit enabers,
sinof criteSigrid,the comunderstranspet al., of diffof deciet al.,
2LinkovMCDAwork fdesignprojec
In tecosystion toBlackistic ecpresenMaterithe desame
sministProMAinationa comcase st criteria are meant to apply to
metrics individuallyand de Groot, 2008). In particular, there is no
a quanti-nal structure for determining whether a metrics set
issive.al Hierarchy Process is a controversial method for
alter-ction developed by Saaty, 1980. To the best of our
it was never used in selection of metrics as alternativesion
problem. However, AHP has been used in a variety oftal management
problems (Linkov and Moberg, 2011;., 2011). Because AHP is based on
a subjective pairwise
of criteria, rather than using value functions and nor-ights, it
has been criticized for its measurement scale,al, and transitivity
of preferences (Gass, 2005; Yatsalo.ove the efcacy of ecosystem
restoration design and
programs (Linkov and Moberg, 2011), we suggest the of MCDA, a
decision-making analysis based on decisionory (Keeney and Raiffa,
1976) that can quantitativelyernatives (i.e. metrics in our case)
based on their util-r stakeholders with respect to dened criteria,
and theortance of those criteria (Drechsler et al., 2003; Linkov,
2011). Applied correctly, MCDA methods will result ineful metric
set for evaluating stated project priorities,ld enable project
managers to make comprehensive,ed decisions, and allow researchers
and practitioners
and update the principles that guide restoration prac-elieve
that a formal MCDA-based method is largelyneeded for the selection
of metrics that can be usedg restoration alternatives or monitoring
alternatives ofs. Tsoutsos et al. (2009) provides several reasons
thatA for use in complex decisions with similar factors toCDA is
appropriate for complex decisions because: (a)ntegration of
interests and objectives of multiple play-ll of this information
can be accounted for in the formnd weight factors (Pohekar and
Ramachandran, 2004;; Loken, 2007; Tsoutsos et al., 2009); (b) it
deals with
xity of having multiple stakeholders by providing easilyable
outputs, and, by virtue of working systematically, is
and user-friendly (Georgopoulou et al., 1997; Tsoutsos); and (c)
it is well-documented and a large number
MCDA methods have been applied in a wide ranges (Phillis and
Andriantiatsaholiniana, 2001; Kaminaris
Diakoulaki and Karangelis, 2007; Tsoutsos et al., 2009; Moberg,
2011). Here, we expand the application ofniques by developing and
applying a MCDA frame-valuating and ranking ecosystem restoration
metrics
characterize the system and quantify the effects ofons.aper, we
introduce the framework for using MCDA forrestoration metrics
selection and illustrate its applica-ypothetical restoration case
study which we call ther Restoration Project. Our case study
resembles a real-tem because we consider all the components
typicallya river ecosystem. The paper is structured as follows.nd
Methods describe the hypothetical case study andment of the
components of the MCDA models. In then we introduce the theoretical
background of the deter-d stochastic multi criteria decision models
(MAUT andesults and Discussion present the results of the dom-lysis
and metric alternative rankings. We also provideive assessment of
both MCDA models applied to the
The Conclusions section discusses the benets and
-
78 M. Convertino et al. / Ecological Indicators 26 (2013)
7686
limitations of utilizing MCDA for metrics selection and the
mostappropriate circumstances in which to apply this new
methodol-ogy.
2. Materials and methods
This paper builds on metrics selection methodology proposed
byLinkov and Moberg (2011) and tailors it specically for
evaluationof ecosystem restoration monitoring metrics. Two MCDA
methods,MAUT and ProMAA, were utilized to rank metrics for
monitoringand evaluating a hypothetical river restoration project
in which thecoupling between human and natural systems is very
high.
2.1. Hypothetical aquatic ecological restoration
The Black River is a hypothetical perennial river with a
broadoodplain consisting of impermeable surfaces, cottonwood
forests,aquatic wetlands, bare soils, and the river network. Fig. 1
representsa schematic view of the hypothetical Black River and its
land covercategories. Over the past several decades, the river and
oodplainhave undergone signicant changes due to urbanization and
damconstruction. The cumulative effect of these stressors is the
disrup-tion of the original hydrologic regime, main stem
channelization,and reduced river-oodplain interaction, which has
increased reand ood hazards, reduced wildlife habitat quality and
quantity,decreased biodiversity, and facilitated encroachment of
harmfulexotic plants. In partnership with state authorities,
federal gov-ernment institutions are planning an ecosystem
restoration projectwith the goal of increasing ecosystem quality by
restoring the struc-ture and fun
A multi-NGOs, andhydrologistbled to setapproach fouate
restorRiver oodptrack systemdepended owell projec
a complex system with multiple objectives and stakeholders,
theteam chose to use MCDA methodology to guide their selection
ofthe optimal metric set.
2.2. Multi-criteria decision analysis (MCDA)
MCDA is a structured approach to decision-making that
quan-titatively evaluates alternatives, in this case, metrics,
based ondened project criteria, expert opinions, and stakeholder
prefer-ences (Linkov and Moberg, 2011; Wood et al., 2012). It
integratesa wide variety of information to evaluate project
alternatives andrank them based on their aggregated value with
respect to a setof criteria (Linkov and Moberg, 2011). It usually
consists of fourstages. The project team, incorporating expert and
stakeholderopinions, must dene: (1) the set of possible decision
alternatives(in this case, metric alternatives) to be evaluated and
ranked; (2)the criteria of the value tree that will inuence the
decision thatthese alternatives will be evaluated against; (3) the
importance ofeach criterion relative to the others or their weight
followed bya normalization of weights performed separately for each
order ofcriteria (criteria of order one, criteria of order two (or
sub-criteria),etc.); and (4) the value of each alternative with
respect to each cri-terion. Depending on the specic MCDA method,
(3) and (4) mayalso include uncertainty estimates.
In MCDA methods that incorporate utility theory (Keeney
andRaiffa, 1976) the values in (4) for each criterion are
transformed intoutility values according to utility functions for
each criterion. Utilityfunctions are expressions of stakeholder
preferences of alternativesas a function of each criterion (Keeney
and Raiffa, 1976) that are
asserect torat
andpproial al
withre caltern
withema
Fig. 1. Schem iver bidentied by t et al. A is the basiction of
the Black River oodplain ecosystem.agency (federal, state, and
local government, academia,
private consultants), multi-disciplinary (ecologists,s,
geologists, engineers, economists) team was assem-
objectives, develop a conceptual model, identify anr assessing
environmental benets, formulate and eval-ation alternatives to
address degradation of the Blacklain, and develop an effective
monitoring program to
changes and evaluate project success. The latter taskn selecting
the most appropriate metrics to assess howt objectives were being
met. As the project involves
usuallyor indi
Rescriteriawhile apotentnativesoftwathose anativethose r
atization of a hypothetical Black river. The main land cover
categories within the rhe river network and river basin boundaries
(dashed lines). The river basin in Settinn outlet.ssed by direct
methods (e.g. interviews (Keeney, 1977)),methods (e.g. serious
games (Braziunas, 2012)).ion planners and stakeholders should
determine the
the relative importance (weighting) of each criterion,priate
professionals and eld experts should create theternatives pool, and
determine the value of each alter-
respect to each criterion. Using this information, MCDAn be used
to rst eliminate dominated alternatives, oratives that are less
valuable than at least one other alter-
respect to every decision criterion, and then to rankining. The
rank is an ordinal number in the range [1,n],
asin (as described in Section 2.1). are represented. The river
basin is(2007) is considered as the hypothetical Black river in
this case study.
-
M. Convertino et al. / Ecological Indicators 26 (2013) 7686
79
where n is the number of metrics, assigned as a function of
thedecreasing utility. The higher the utility, the lower the
rank.
In the case of the Black River restoration, the MCDA analy-sis
was used to narrow down and rank an initially large set ofaquatic
ecoability to prto project othose objecthe people tered as sug
We utiliuation in CoSystem) (Yathe input dacomplex dutem
compothese systemware was dDecision Scand Mobergcase study iFor
further (2011).
The hypis twofold: to evaluateto which thproject impric
alternatintended beent orders ocriteria untcriteria andbut
generalpublic healtFrom the obcriteria andity of each pbest based
be the mostmeasuring
The resurepresentedshowing thdominated methods, thcriteria of
onormalizatisub-criteriacriteria but are normali
Chosen tives and shproject succinterest. Yetions shouland risks
o2002; ClarkForum, 201case study wlogical, geocriteria (Tabbecause
theria case by ecosystem 2012).
The general criteria selected give a good indication of the
stateof the environment and the environmental effects of
restorationmeasures. Some of these criteria included more detailed
sub-criteria such as recreation and maintenance under economy.
The
sub-criteria allows stakeholders to weight both the gen-tegory
(e.g., economy) and more detailed aspects that relateic stakeholder
concerns. Criteria formulation is an impor-ep in dening what is
important to project success and what
be considered in the decision making process. More
criteria,iteria, and criteria of higher order could have been
considered
project, but the current set gives an adequate indication ofte
of the system and progress toward objectives.e the taxonomy of
criteria and sub-criteria was established,ject team then developed
an initial, comprehensive set ofl aquatic ecosystem monitoring
metrics related to each crite-able 1). The set of metrics to be
selected should be as holistic
t of potential metrics for aquatic ecosystems, organized into
functional cat-Each category of metrics can be thought of as an
ecosystem service class.m services can be grouped in sustainability
classes (Environmental, Social,nomical). Each initial potential
metric is then considered dominated, non-ed, or equal to another
metric according to a Pareto domination analysis
2.2). The 43 metrics constitute an exhaustive list according to
the stake-(i.e. the authors in this case study) involved in the
hypothetical Black riverion.
Metrics
logical Water Table LevelSoil MoistureBankfull
DischargeHydroperiodFlooding Return Period 100-yearsFlooding
Frequency 1-year RunoffFlooding Frequency 2-year RunoffMinimum
Water FlowRiver Salinity
rphological Maximum Elevation GradientRiver Basin
ExtensionFloodplain ExtensionAverage Riparian WidthHacks
ExponentHillslope Stability FactorSediment Discharge
ical Species Area ExponentLocal Species RichnessRegional Species
RichnessPairwise Species Richness SimilaritySpecies
AbundanceAverage Landscape Connectivityp/A Patch RatioCanopy
EvapotranspirationHabitat Area SongbirdGeographic Range
SongbirdHabitat Area FishesNumber Invasive SpeciesMetapopulation
Risk
ical
mical
econom
system monitoring metric alternatives based on theirovide
information about system characteristics relatedbjectives (the
criteria) and stakeholder preferences fortives (the weights). We
consider stakeholders to be allhat have a stake in the
environmental problem consid-gested in Wood et al. (2012).zed the
MCDA software DECERNS-SDSS (Decision Eval-mplEx Risk Network
SystemsSpatial Decision Supporttsalo, 2011) to model the problem
space and analyzeta. The idea behind DECERNS-SDSS is that systems
aree to the high degree of interconnections among sys-nents and
because of the multiplicity of risks affectings (Linkov and Moberg,
2011; Yatsalo, 2011). The soft-
eveloped by Yatsalo (2011) supported by the Risk andience Team
of the U.S. Army Corps of Engineers (Linkov, 2011). A demonstration
version of the model and thes included in this paper in the
Supplementary Material.information we refer the reader to Linkov
and Moberg
othetical monitoring goal of the optimal set of metrics(i) to
select the best restoration alternative; and, (ii)
restoration project success by measuring the degreee intended
objectives have been achieved following thelementation period. To
begin the MCDA analysis of met-ives, we rst dened the set of
project objectives, ornets, in order to develop the model criteria
at differ-f the value tree (Table 1). In this case study we
developil the second order in the value tree; thus, we refer to
sub-criteria. Objectives vary from project to project,ly include
environmental, economic, socio-political, andh and safety
considerations (Linkov and Moberg, 2011).jectives of the
hypothetical project, we derived a set of
sub-criteria against which we later evaluated the util-otential
metric. The metric alternatives that performedon these criteria and
sub-criteria were considered to
useful in characterizing important project aspects andfulllment
of specic ecosystem restoration objectives.lts of the criteria and
alternative development are
in DECERNS-SDSS by a structured value tree (Fig. 2)e overall
objective, criteria, sub-criteria, and non-metric alternatives.
Note that, on the contrary of AHPe tree can be asymmetric (e.g. a
different number ofrder two for each criterion) and the assignment
andon of weights is performed separately for criteria and. In this
case, it is possible to compare the weights ofnot the weights of
criteria and sub-criteria because theyzed at their respective level
in tree.criteria depend on clear, well-dened project objec-ould be
comprehensive, including all aspects relating toess as well as any
additional system characteristics oft, we believe that a
sustainable paradigm for restora-d be adopted considering that
components, services,f ecosystems are highly interconnected (Pauly
et al.,, 2007; Bettencourt and Kaur, 2011; World Economic1;
National Academy of Sciences, 2012). Thus, in oure included
environmental (hydrological, geomorpho-
logical, ecological, biochemical), social, and economicle 1).
Health criteria have been considered separatelyy can be part of
both social and environmental crite-case depending on the drivers
of health issues in theanalyzed (Clark, 2007; National Academy of
Sciences,
use oferal cato spectant stshouldsub-crfor thisthe sta
Oncthe progenerarion (T
Table 1Initial lisegories. Ecosysteand Ecodominat(Sectionholders
restorat
Hydro
Geomo
Ecolog
Geolog
Bioche
Socio-
HealthGranulometric CurveConductivity
Total Maximum Daily LoadN %P %C %O %PHMicrobial
BiomassBioaccumulation Potential
ical Number Visits/YearNumber of Trails
Number of Epidemics last 10 years
-
80 M. Convertino et al. / Ecological Indicators 26 (2013)
7686
Fig. 2. Multi-Cdeletion of thegure is a screare in Figure S
as possible are conside
The potein the eldbecomes avnd it beneproject-speconsideringtion
of a spcan be also time.riteria Decision Analysis (MAUT and ProMAA)
model architecture for the hypothetical Bl dominated metrics (see
Table 1 for initial metrics and Table 2 for the analysis of
non-denshot of the model in DECERNSSDSS (Supplementary Material).
Dw and Uf are the crite1, S2 and S3 respectively). In the
theoretical formulation of the model (Sections 2.3 and 2
in order to guarantee that the most important metricsred.ntial
metrics pool can be adjusted as new information
of environmental and socio-economical managementailable or, in
some projects, restoration managers maycial to adjust the potential
metrics pool to account forcic objectives. This would certainly be
necessary when
very specic objectives such as increasing the popula-ecic
endangered species. Moreover the metrics pooladjusted when
project-specic objectives are shifted in
In practexperts aretial metric terminologscore, or pasented in
thknowledgementary Mhydroperio
Next, thand removack river case study. The value tree of the
MCDA model is shown afterominated metrics using the Pareto-based
domination analysis). Theria weights and partial utility score of
criteria and metrics (examples.4) Dw and Uf are indicated as wj and
Uj(ai) respectively.
ice, environmental practitioners, scientists and other then
consulted to evaluate the utility of each poten-in measuring
fulllment of each criterion. In MCDAy, this is to say that the
analysts assign each metric artial utility, for each criterion. In
the case study pre-is paper, the authors provided the scores using
their best
of aquatic ecosystem function and structure. In Supple-aterial
(Fig. S1) we report an example of scores for thed as a function of
each criterion.e team ran an MCDA domination analysis to identifye
metrics that were dominated by one or more others
-
M. Convertino et al. / Ecological Indicators 26 (2013) 7686
81
Table 2Pareto-based domination analysis. The table reports the
non-dominated (whitelines), dominated (dark gray lines), and
equivalent metrics (light gray lines). Equiv-alent metrics are
characterized by the same utility for the objective of the
decisionproblem that idominated alt(Sections 2.3 atable. The
initi
Metric Index
1 234
5 6
7 8
9
10 11 12
13 1415 16 1718 19
20
2122 23 24 25
26
27
28 29 30
31
32
33 34 35 36 37 38 39 40 41 42 43
(Table 2). D(had lower ria. These munder any wset of
nonranked.
Specically DECERNSDSS implements a Pareto dominancemethod
(Yatsalo, 2011). A feasible combination of metrics for a
col-lection of objectives is said to be Pareto dominated if there
does
nother feasible combination of metrics under which eachve is
(Em a muof wr the
prefs the Black river restoration. The overall utility, U(ai),
of the 25 non-ernatives is calculated by the MCDA methods (MAUT and
ProMAA)nd 2.4 respectively). The metric index is for a better
readability of theal list of metrics is in Table 1.
Metric name Domination
River Basin Extension Non-dominatedHacks Exponent
Non-dominated
exist aobjectiter offpurelycation
AfteholderAverage Riparian Width Non-dominatedFloodplain
Extension Dominated by River
Basin ExtensionLocal Species Richness
Non-dominatedMetapopulation Risk Dominated by
Number of InvasiveSpecies
Number of Invasive Species Non-dominatedHabitat-area Fishes
Dominated by Local
Species RichnessHabitat-area Songbird Dominated by Local
Species RichnessNumber of Trails Non-dominatedNumber of
Visits/Year Non-dominatedFlooding Frequency 1-year Runoff Equals
with
Flooding Frequency2-year Runoff
Flooding Frequency 2-year Runoff Non-dominatedWater Table Level
Non-dominatedSoil-moisture Non-dominatedBankfull Discharge
Non-dominatedHydroperiod Non-dominatedMinimum Water Flow
Non-dominatedFloodplain Return Period 100-years Dominated by
Flooding Frequency1-year Runoff
River Salinity Dominated byWater Table Level
Maximum Elevation Gradient Non-dominatedHillslope Stability
Factor Non-dominatedSediment Discharge Non-dominatedSpecies-area
Exponent Non-dominatedRegional Species Richness Dominated by
Local
Species RichnessPairwise Species Richness Similarity Dominated
by Local
Species RichnessSpecies Abundance Dominated by Local
Species RichnessGeographic Range Songbird Non-dominatedAverage
Landscape Connectivity Non-dominatedP/A Patch Ratio Dominated by
Local
Species RichnessCanopy Evapotranspiration Dominated by
Flooding Frequency1-year Runoff
Total Maximum Daily Load Dominated byMicrobial Biomass
N % Equals with P %P % Dominated by O %C % Dominated by N %O %
Non-dominatedPH Dominated by O %Microbial Biomass Non-dominatedD50
Non-dominatedGranulometric Curve Dominated by D50Conductivity
Non-dominatedBioaccumulation Potential Non-dominatedNumber of
Epidemics Last 10 Years Non-dominated
ominated metrics were those that were outperformedvalue scores)
by at least one other metric in all crite-etrics were eliminated as
they would not be selectedeighting scenario. The result of this
step was a smaller
-dominated metrics which were then analyzed and
importanceto which mtives transland sub-cricase study,titioners
anelicitation, ria and subaquatic ecowould be inences for
eacontaining tion of the Bhydrologicato observe t
For prefmore sensihighest wehabitat impogy, 0.17 fohealth,
0.11malized wedependent is critical tosuite of prefweights
repvalues of al(Keeney andby uncertaiis capable ois only
capaassigned totainty relatdo not consgroup of sta
The inpuand sub-crvalues; anddecision su2007) whicutility to
thranking wilbelow wheand ProMA
2.3. Multi-a
The Mupreferencesmetric altera given metproject objoverall
utilwere combaspects of thria of differe at least as well off and
some objective is strictly bet-merich and Deutz, 2006). Domination
comparison is alti-objective metric comparison that gives some
indi-
hich of two metric sets is closer to the Pareto front.
domination analysis, restoration manager and stake-erences need to
be elicited to establish the relative
of each criterion and sub-criterion. The relative extentsanagers
and stakeholders value various project objec-ate to the relative
weights of the corresponding criteriateria. Because this
investigation utilized a hypothetical
we did not have access to a group of restoration prac-d
stakeholders. Instead of practitioner and stakeholderwe assigned
hypothetical weights to each of the crite--criteria, based on what
we believed were reasonablesystem restoration priorities. In
practice, stakeholdersterviewed or tested to determine their
relative prefer-ch criterion. Supplementary Material reports the
tablesthe weights of criteria and sub-criteria for the restora-lack
River (Figs. S2 and S3 respectively; the example ofl sub-criteria
is reported). In Figs. S2 and S3 it is possiblehe normalization of
the weights in a [0,1] range.erence weighting, we assumed the
stakeholders weretive to ecological problems and therefore assigned
theights to ecology, and to those criteria with the largestact. The
normalized weight values are: 0.22 for ecol-r geomorphology and
hydrology, 0.14 for economy and
for geology, and 0.05 for biochemistry. These nor-ights always
sum to one. These weights are highlyon which stakeholders views are
incorporated, so it
involve a variety of stakeholders to capture the fullerences for
project outcomes. In general the aggregatedresenting all the
stakeholders involved are the averagel the individual stakeholder
weights for each criterion
Raiffa, 1976). In reality these weights are characterizednty and
may vary in time. The MCDA method, ProMAA,f handling weight
uncertainty. On the contrary, MAUTble of evaluating the uncertainty
related to the utility
each alternative for each criterion and not the uncer-ed to the
stakeholders preferences (weights). Here weider the variability of
stakeholder preferences amongkeholders and in time.t data are
complete once the (1) taxonomy of criteria
iteria; (2) pool of potential metrics; (3) partial utility (4)
weights, are formulated. An MCDA is then run usingpport software
such as DECERNS-SDSS (Yatsalo et al.,h will rank the potential
metrics in terms of their overalle set of weighted project
objectives. Details of metricl be explained further in the specic
method sectionsre we report the analytical characterization of
MAUT,A methods used to rank metric alternatives.
ttribute utility theory (MAUT)
lti-Attribute Utility Theory (MAUT) resolves multiple and value
scores into an overall utility value for eachnative, enabling
comparison. In this case, the utility ofric for measuring fulllment
of a specic aspect of theective was treated as a partial utility.
To calculate anity for each potential metric, the partial utility
valuesined based on the relative importance of the componente
objectives to the stakeholders (the weighting of crite-nt order in
the value tree). MAUT considers uncertainty
-
82 M. Convertino et al. / Ecological Indicators 26 (2013)
7686
related to the utility assigned to each metric for each
criterion, butdoes not consider the uncertainty related to the
stakeholders pre-ferences (weights) (von Winterfeldt and Edwards,
1986; Belton andStewart, 2002a,b).
To calculet the set oand the setmethod (Keranked base
U(ai) = f (Uwhere Uj(aicj of any ordmay take oKeeney andtial
utilitiesis widely usof the overa
U(ai) = w1Uwith the no
m
j=1wj = 1,
where wj isterion cj inpartial utili
MAUT capartial utiland thus thbe consider
Despite is not univuse expecteVincke, 198Stewart, 202008).
Howcharacteristutility valueple metricsto describe terion.
Thisutilities forfunctions aimportant sRaiffa, 1976
For uncesidered for to 10 propoto each critsidered fromutility
of eaand the stathis paper (gist (B.S.), ato reduce thmay bring
i
2.4. Probab
Probabilan MCDA mvalues, for (Yatsalo, 20technical m
weighting uncertainties. The ProMAA algorithm utilizes the
prob-ability distributions of alternative scores and of weight
coefcientsfor assessing rank acceptability indices based on
pair-wise compar-ison of alternatives (Linkov and Moberg, 2011;
Yatsalo et al., 2007;
, 201l utili
p(w
p(wjolderrma
MAU weis andhe no
j)wj
ProMeria
of s in otivesre bales aaccep
indicnces004)
(Sik ),
Sik ink krms ). ThA is gs ar
set the
typi
1
wak
wkac
998)s, raA is
on the ailar thod), Preightlma f SMueirroxi
et al). Inof untivestionthetiilitylate the overall utility for
each metric alternative, rstf potential metrics alternatives be A =
{ai, i = 1, . . . , n}
of criteria be C = {cj, j = 1, . . . , m}. Using the MAUTeney
and Raiffa, 1976), each metric alternative, ai, isd its overall
utility, U(ai):
1(ai), . . . , Uj(ai)) (1)
) is the utility of alternative ai with respect to criterioner
in the value tree. The generic MAUT function (Eq. (1))n multiple
functional forms (Keeney and Raiffa, 1976;
Gregory, 2005). For this paper, we assume that the par- are
independent, and utilize the additive form, whiched for practical
MAUT applications. The functional formll utility of metric
alternative ai, U(ai), is:
1(ai) + + wjUj(ai), (2)rmalization condition,
wj > 0, (3)
a weighting factor representing the importance of cri- the
project. In DECERNSSDSS the weight wj and thety Uj(ai) are
indicated as Dw and Uf respectively (Fig. 2).n use distributions
for alternative utility scoring (i.e. theity, Uj(ai)), but can only
use point values for weightse uncertainty related to stakeholder
preferences cannoted.extensive use of the expected utility concept,
its useersally accepted, and other approaches that do notd utility
methods are often implemented (Brans and5; von Winterfeldt and
Edwards, 1986; Belton and02a,b; Figueira et al., 2005; Tervonen and
Figueira,ever, in metrics selection problems, like evaluating
siteics and restoration project success, we believe that the
of a metric, rather than the expected value of multi- for a
variety of scenarios and criteria, is the best waythe aspect of the
ecosystem corresponding to each cri-
is because same values of criteria can have different different
stakeholders. Thus, the elicitation of utilitynd the translation of
values to utilities is an extremelytep for selecting and evaluating
restoration (Keeney and).rtainty considerations, a normal
distribution was con-the utility weight value with average ranging
from 0rtionally to the importance of each metric with respecterion.
A standard deviation from 0.1 to 0.01 was con-
the average value according to the uncertainty in thech metric
in describing each criterion. The utility valuesndard deviations
have been assessed by the authors ofenvironmental engineers (M.C.,
K.B., and C.L.), a biolo-n ecotoxicologist (I.L.), and an ecologist
(J.V.)) in ordere subjective uncertainty and the bias that one
expert
nto the decision problem.
ilistic multi-criteria acceptability analysis (ProMAA)
istic Multi-criteria Acceptability Analysis (ProMAA) isethod
that can use distributions, rather than point
both weights and alternative utility scoring of criteria11;
Linkov and Moberg, 2011; and see DECERNS-SDSSanual), allowing the
user to account for both scoring and
Yatsalooveral
U(ai) =wherestakehThe nounlikefor theauthor0.05. T
m
j=1p(w
Theof critbilitiesmetricalternaMAA avariabRank abilityprefereet
al., 2
Pik = Pwherewith raai in teeventsProMARankincriteria
Forsum is
Pi =n
k=
whereet al., 1
ThuProMAand/orwhich
Siming me(SMAAand w(Lahdetions oand Figcal appYatsalomanuatance
alternarestora
Synprobab1; and see DECERNS-SDSS technical manual). Thus, thety
of metric alternative ai, U(ai), is:
1)w1U1(ai) + + p(wj)wjUj(ai), (4)) is the probability to observe
the weight wj expressings preference for criterion j of any order
in the value tree.
lization condition for the weights still holds; however,T,
ProMAA uses distributions instead of point values
ghts. In this case, the weights were determined by the to each
weight was assigned a standard deviation ofrmalization condition is
expressed analytically as:
= 1, wj > 0, 0 p(wj) 1, (5)
AA algorithm can also utilize probability distributionsutilities
and weight coefcients for assessing proba-likely rank events (where
events are associated withur decision problem) based on pairwise
comparison of
in an integrated scale. In this case realizations of Pro-sed on
numerical approximation of functions of randomnd numerical
assessment of integrals (Yatsalo, 2011).tability indices are the
output of ProMAA. Rank accept-es are probabilities that describe
the variety of different
resulting in a certain rank for an alternative (Lahdelma, and
can be expressed as Pik :
(6)
s the event characterized by the metric alternative ai and i, k
= 1 . . . n (i.e., k 1 alternatives are better thanof a given
criteria for a subset of a space of elementaryus, ranking or
screening metrics {ai, i = 1,. . .,n} withinbased on the analysis
of the matrix {Pik}, i,k = 1,. . .,n.e based on the weighted
overall score of ai against theC.aggregation of the indicated
probabilities, a weightedcally used:
cPik , (7)
are weights of relative importance of ranks (Lahdelma.nking or
screening alternatives {ai, i = 1,. . .,n} withinbased on the
analysis of the matrix {Pik}, i,k = 1,. . .,n,the holistic
acceptability indices Pi, i = 1,. . .,n, fromverage rank of
alternatives k can be assessed.to the more commonly used
probabilistic outrank-, and stochastic multi-objective
acceptability analysisoMAA accounts for uncertainty ranges in both
criteria
values in its calculation of rank acceptability indiceset al.,
2004). However, while software implementa-AA are based on Monte
Carlo simulations (Tervonena, 2008), ProMAA implementation is based
on numeri-mation of random variables (Linkov and Moberg, 2011;l.,
2007; Yatsalo, 2011; and see DECERNS-SDSS technical
this paper, we apply ProMAA to assess the impor-certainty in
stakeholder preferences in determining
ranking. In this case the alternatives are ecosystem
metrics.cally generated values can be assigned as standard
distributions to criteria and preferences; thus, Monte
-
M. Convertino et al. / Ecological Indicators 26 (2013) 7686
83
Carlo simulations sample these distributions. In the majority
ofcases stakeholder preferences are elicited from workshops or
usingother methodologies (for example serious games (Nesloa
andCooke, 2011)) and their value and distribution can vary
con-siderably. Criteria distributions are inferred from data or
fromstakeholder judgment. Thus, both preferences and criteria
utilitiesare strongly case-specic and there is no standardized
method-ology to gauge and assign their distributions. For further
technicaldetails of ProMAA we direct readers to Yatsalo (2011) and
to Linkovand Moberg (2011).
3. Results and discussion
The main objective of the paper was to provide
methodologicalframework for a quantitative, structured, scalable,
and transparentselection of metric alternatives. Metric selection
is an extremelyimportant aspect of environmental management;
however it isoften prone to high subjectivity that strongly affects
both the selec-tion and the evaluation of restoration projects. In
this paper, weaimed to illustrate the use of a quantitative,
structured and trans-parent metrics selection methodology, MCDA, to
rank potentialecosystem restoration metrics. The results of this
case study arediscussed below.
The initial analysis, the Pareto-based domination analysis,
elim-inated nearly half of the potential metrics based solely on
theirpartial utility with respect to each criterion. Specically,
thisresulted in the identication and elimination of 18
dominatedmetrics, narrowing the metric pool from the initial
comprehensiveset of 43 aquatic ecosystem restoration metrics shown
in Table 1to the 25 non-dominated metrics shown in Fig. 2. This
greatly sim-plies the decision and provides a clear and logical
justication forremoving dominated metrics independently of
stakeholder prefer-ences as they are sub-optimal under any set of
weights.
The results of the MAUT and ProMAA models, that is, the aver-age
utility sFig. 3. The
[0,1] for both methods. This type of visualization allows
analysts toeasily compare the utility of each metric as calculated
by MAUTand ProMAA. For MAUT, the utility is calculated using Eq.
(2),and for ProMAA using Equation 4. As many of the utility
scoresare similar to each other, this ranking is not intended to
explic-itly determine which metrics to use but it is an excellent
guidefor decision makers and clearly indicates that some metrics
aremore suitable than others (e.g. the metric local species
richnessis clearly more useful than geographic range of songbird).
ProMAAenhances the ecological metrics according to the stakeholders
pref-erence for ecological criteria, but overall the differences
betweenthe results of MAUT and ProMAA in terms of utilities of each
ofthe metrics (Eqs. (2) and (4), respectively) are negligible.
Despitethat the average utility is higher for ProMAA, the rank of
themetrics as a function of the utility score for ProMAA and MAUT
isvery similar.
The comparison of utilities from MAUT and ProMAA showsthat the
sensitivity of the selected metric set to the
stakeholderspreferences is low. This may be related to the small
uncertaintygiven to the weights for each criterion, or to the
existence ofan already well-dened set of metrics that were selected
by avariety of experts in different elds. Certainly large
variations instakeholder preferences that are much bigger than the
uncertaintyassigned to the weights in this case study (Section 2.4)
would causebigger variation in the utility of each metrics. Thus,
such variationsin weights need to be considered in real practice.
However, suchlarge variability of weights is observed when
different groups ofstakeholders are considered for the same
problem, or the samegroup of stakeholders is observed in time.
Here, we consider uncer-tainty that is small and related to the
assessment of stakeholderpreferences. The application of global
sensitivity and uncertaintyanalysis (Saltelli et al., 2008) in
utility-based MCDA methods forevaluating the effect of large
uncertainties is an ongoing effort andnot the purpose of this
paper.
e thow m
Fig. 3. Overall non-dProMAA respe AUT arank in a rangecore for
each of the metric alternatives, are shown inaverage utility score
is represented in the same range
Oncabout h
utility value calculated by the MAUT and ProMAA models. The
utility U(ai) of the ctively. The utility is ordered from the
highest to lowest value of utility according M
[1,25], where 25 is the number of non-dominated metrics.e
metrics ranking is formulated (Fig. 3) the decisionany metrics to
use should be a function of the available
ominated metrics is calculated using Equation 2 and 4 for MAUT
andnd it is used to rank each metric. The higher the utility, the
lower the
-
84 M. Convertino et al. / Ecological Indicators 26 (2013)
7686
Fig. 4. Probab n forcalculated usin the aindex (Eq. (7))
ssigne(Fig. 3). The low obabi
resources foconsider th
The predstakeholdercriterion arMetrics are(Eq. (6)) thaders
preferfor ProMAAtainty) and are also calmetrics is etribution
ofprobability This is usefuncertain inrank of metutility (Eq. rank
(Eq. (6in both modcompromisthat the utiparative ansets of metvalue
of infa metric in in this caseset arising prior to the
4. Conclus
Selectinand evaluatmanagemeupdating re
res dif
Groaseden inith mer harentncorreoveility distributions of the
rank of metrics calculated by ProMAA. The rank k is showg Eq. (6).
The overall rank of each metric alternative can be calculated by
considering. The ranks based on the holistic acceptability indices
are equivalent to the ranks aer the rank, the higher the overall
utility of a metric. In the gure the value of the pr
r monitoring those metrics. In this case study we do notis
aspect that is highly specic to each restoration.icted ranks of
ProMAA considering the uncertainty in
preferences and in the utility of each metric for eache
represented by the probability distributions in Fig. 4.
characterized by a probability distribution of their rankt is
the result of the combined uncertainty of stakehol-ences (for MAUT
the preferences are constant values,
preferences are a distribution that accounts for uncer-of
utility values of metrics for each criterion. These ranks
limitedoften aand dethose bare ofttems wthe othtranspwhile i
Mo
led rank acceptability indices. The ranking order ofstablished
considering the average value of the dis-
each rank (Eq. (7)). In Fig. 4 we show an example
ofdistributions of ranks for the six most valuable metrics.ul in
showing the reliability of rankings in the face ofputs and how
uncertainty affects metric ranking. The
rics from ProMAA determined after calculating metrics(4)) or
determined directly after calculating metrics)) is equivalent. This
suggests that ProMAA can be usedes (probabilistic utility and
outranking modes) withouting the metric selection process. However,
we believelity is more useful information than the rank for a
com-alysis of metrics utility. The difference in utility amongrics
or between two metrics can be interpreted as theormation (Keeney
and Raiffa, 1976) of a metric set or ofthe metric decision process.
The value of information is
dened as the increase in the overall utility of a metricfrom the
predicted additional information of a metric
metric selection.
ions and perspectives
g an appropriate and informative metrics set to monitore
ecosystem restoration projects is critical for informingnt
decisions of ecosystems, furthering the science, andstoration
practices. With myriad metric choices and
incorporatitative modeand environport more s
In this sMAA, to deselection foof potentiaproject, descreened
thnally appranked accoholder prefto each critter suited tothey
translMoreover, Pferences thlacking or intain. Howevdid not
shoranking of t
Comparin the introdence, concsets, and An the six highest ranked
metrics. The rank acceptability index, Pik , isveraged value of the
ranks after calculation of the holistic acceptabilityd to the
metrics after the utility calculated using Eq. (4) with ProMAAlity
for each rank (x-axis) is reported above each bar of the
histograms.
ources for monitoring, selecting the best metric set iscult task
(Noss, 1999; Dale and Beyeler, 2001; Niemeijerot, 2008). Current
metrics selection methods, such as
on best professional judgment or historical precedenceadequate
for decision-making involving complex sys-ultiple alternatives and
evaluation criteria. MCDA, on
and, provides decision makers with a tool to clearly andly
evaluate metric alternatives over a number of criteriaporating
stakeholder values and expert opinions.r, MCDA limits the
subjectivity in metric selection by
ng stakeholder preferences into structured and quanti-ls. By
simultaneously considering the social, economic,mental components,
the selected metrics aim to sup-ustainable restorations.tudy, we
applied the MCDA methods, MAUT and Pro-monstrate how MCDA can be
used to aid in metricsr aquatic restoration projects. We formulated
a setl metrics for the hypothetical Black River restorationveloped
a taxonomy of weighted project objectives,e initial metrics set to
remove dominated metrics, andlied MAUT and ProMAA to develop a list
of metricsrding to their importance as a function of a set of
stake-
erences and utility functions of each metric with respecterion
and sub-criterion. Utility-based methods are bet-
this type of analysis than value-based methods becauseate the
value of metrics into utility to stakeholders.roMAA can incorporate
uncertainty in stakeholder pre-
at can be useful when a real elicitation of preferences is cases
where stakeholder preferences are highly uncer-er, in our
hypothetical case study, MAUT and ProMAAw signicant differences in
the predicted utility andhe selected metrics.ed to the common
metrics selection methods presentedduction (best professional
judgment, historical prece-eptual modeling, screening using
established criteriaalytic Hierarchy Process (Saathy, 1980)) MCDA
is more
-
M. Convertino et al. / Ecological Indicators 26 (2013) 7686
85
comprehensive and inclusive, incorporating expert opinion ona
variety of subjects and stakeholder preferences from severalelds.
This method allows planners to simplify complex situationswith
varied and often conicting options, objectives, and opinions.New
and chevaluated. Iincluding wclearly justThe MCDA project manThe
quantitcompare ea
MCDA cby some liminput fromand more eods. Thoughcriteria andogy
is involbasing critestakeholderjudgment tterion. Smaalternative
input informalternativeswith respecmetrics werThis requirewith
respecthe certaint
Usually concerns, aholder invoto participais designednot
includeAnother methat will bebination of precedencethe original
An MCDholders areand/or highconsideratimanagemenated, for exathe
monitoruse screeninlist of metrous methodpractice of and monito
Acknowled
The authBenet Ana(http://cw-acknowledgSDSS availaedged for tthe
theoretand two antheir constr
project Decision and Risk Analysis Applications
EnvironmentalAssessment and Supply Chain Risks for his research at
the Riskand Decision Science Team, Engineering Research and
Develop-ment Center, U.S. Army Corps of Engineers, in Concord, MA,
USA.
as B, andhis rnt & . Armaterif thepons
dix A
plem in th0.00
nces
., Coviration
V., Steroach.V., Steroach.urt, LS, http.P., VMETHs, D.,
PhD rs/Da.C., 20://dx.dino, Mction ene://cw., Bey
catorski, D.,
lysis ow. Sur, M.,n extis. Ecold, J.G.or. Ecoch, Mciplesies,
htngendomer,etlan
e of W, J., Gre of th., 2005312.oulouoach
J. Opes, A.Ptal du
N., 1nce w.B., Ketal sc, 3578is, S., tricity
R.L., 1. Perf
R.L., Gctives
R.L., Re Tradanging information can also easily be integrated
andnterested parties can review components of the modeleights and
alternative scores, and decision makers canify management choices
according to model results.method for metrics selection thus
enables restorationagers to make systematic, and transparent
decisions.
ative results allow decision makers to clearly and easilych
alternative and select the optimal metric set.an be extremely
benecial, but it is also characterizeditations. Because it is so
comprehensive and it includes
a variety of stakeholders, it can be time consumingxpensive than
other, simpler metrics selection meth-, technically the project
team alone could specify the
weights, one of the cornerstones of MCDA methodol-ving
stakeholders in the decision process. This meansria and weighting
partially on preferences elicited froms. It also takes a signicant
amount of work and experto assign value scores to each alternative
for every cri-ll increases in the amount of evaluation criteria
andchoices translate to much larger increases in requiredation. The
number of evaluation criteria and metric
is limited because each alternative must be evaluatedt to each
criterion. For example, in this case study 25e evaluated with
respect to 7 criteria and 6 sub-criteria.d 325 expert judgments of
the value of each alternativet to each criterion as well as another
325 estimates ofy of those values.involving complex projects with
serious stakeholder
successful MCDA evaluation often depends on stake-lvement, and
is therefore limited by their willingnesste (Linkov and Moberg,
2011). Also, the methodology
to narrow down chosen metric alternatives and does guidance for
choosing the original larger metric set.thod must be used to
generate the metric alternatives
included in the MCDA analysis; perhaps one or a com-the
previously mentioned methods, such as historical
or best professional judgment, can be used to assemble metric
set.A technique should be used when many diverse stake-
interested in the project, and the situation is complex-prole
with several objectives and alternatives underon. It is also useful
with projects involving adaptivet as the situation can easily be
updated and reevalu-mple by new ecosystem restoration design,
consideringed metrics. Restoration managers may nd it useful tog
before or after MCDA techniques to generate an initial
ics. We believe that overall this study provides a
rigor-ological and computational advancement to the currentmetrics
a utility-based selection for restoration designring, and in
general for environmental management.
gements
ors acknowledge the support of the Environmentallysis Program of
the U.S. Army Corps of Engineers
environment.usace.army.mil/eba/). Dr. B. Yatsalo ised for making
a demonstration version of DECERNS-ble. D. Dokukin, and J.M.
Keisler are kindly acknowl-heir support with DECERNS-SDSS and for
the basis ofical framework respectively. The Editor, Dr. F.
Mulleronymous reviewers are gratefully acknowledged foructive
comments. M.C. acknowledges the funding of
C. Lu wnologywhen tronmethe U.Sthis mthose oother s
Appen
Supfound,2012.1
Refere
Allen, Eresto
Belton, App
Belton, App
BettencoPNA
Brans, JPRO
Braziunaties,pape
Clark, Whttp
ConvertSeletal B(http
Dale, V.Hindi
DiakoulaAnaRene
Drechslelatiosion
EhrenfelRest
EmmeriPrinStud
Faber-LaT., Cfor WOfc
FigueiraStat
Gass, S.I308
GeorgopapprEur.
Grootjancoas
Holmes,defe
Huang, Imen(19)
Kaminarelec
Keeney,Hum
Keeney,obje
Keeney,ValuSc and MSc student at Massachusetts Institute of
Tech- research intern at the Risk and Decision Science Groupesearch
was performed. C. Lu is currently at AMEC Envi-Infrastructure,
Oakland, CA. Permission was granted byy Corps of Engineers Chief of
Engineers to publish
al. The views and opinions expressed in this paper are
individual authors and not those of the U.S. Army, ororing
organizations.
. Supplementary data
entary data associated with this article can bee online version,
at http://dx.doi.org/10.1016/j.ecolind.5.
ngton, W.W., Falk, D.A., 1997. Developing the conceptual basis
for ecology. Restor. Ecol. 5 (4), 275276.wart, T., 2002a. Multiple
Criteria Decision Analysis: An Integrated
Kluwer Academic Publishers, Dordrecht.wart, T., 2002b. Multiple
Criteria Decision Analysis: An Integrated
Kluwer Academic Publishers, Boston..M.A., Kaur, J., 2011.
Evolution and structure of sustainability
science.://dx.doi.org/10.1073/pnas.1102712108.
incke, P., 1985. A preference ranking organisation method: theEE
method for MCDM. Manag. Sci. 31 (6), 647656.
2012. Decision-theoretic elicitation of generalized additive
utili-thesis, University of Toronto,
http://www.cs.toronto.edu/darius/rius Braziunas PhDThesis.pdf07.
Sustainability science: a room of its own. PNAS 104 (6),
17371738,oi.org/10.1073/pnas.0611291104.., Baker, K., Lu, C.,
Vogel, J., McKay, K., Linkov, I., 2012. Metricfor Ecosystem
Restoration, ERDC Technical Note, Environmen-ts Analysis Program,
US Army Corps of Engineering,
forthcoming-environment.usace.army.mil/eba/index.cfm).eler, S.C.,
2001. Challenges in the development and use of ecological. Ecol.
Ind. 1, 310.
Karangelis, F., 2007. Multi-Criteria Decision Analysis and
Cost-Benetf alternative scenarios for the power generation sector
in Greece.stain. Energy Rev. 11/4, 716727.
Karin, F., Ilkka, H., OHara, R.B., Wissel, C., 2003. Ranking
metapopu-nction risk: from patterns in data to conservation
management deci-. Appl. 13, 990998,
http://dx.doi.org/10.1890/1051-0761(2003)13., 2000. Dening the
limits of restoration: the need for realistic goals.l. 8 (1),
29.
., Deutz, A., 2006. Multicriteria Optimization and Decision
Making,, Algorithms and Case Studies, Leiden Institute of Advances
Computertp://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.162.694oen,
D., Rocchio, J., Schafale, M., Nordman, C., Pyne, M., Teague, J.,
Foti,
P., 2006. Ecological Integrity Assessment and Performance
Measuresd Mitigation. Final Report to US Environmental Protection
Agency etlands, Oceans and Watersheds, NatureServe, Arlington,
VA.
eco, S., Ehrgott, M. (Eds.), 2005. Multiple Criteria Decision
Analysis:e Art Surveys. Springer Science+Business Media, Boston,
USA.. Model world: the great debateMAUT versus AHP. Interfaces 35
(4),
, E., Lalas, D., Papagiannakis, L., 1997. A multicriteria
decision aidfor energy planning problems: the case of renewable
energy option.rat. Res. 103, 483500.., Geelen, H.W.T., Jansen,
A.J.M., Lammerts, E.J., 2002. Restoration ofne slacks in the
Netherlands. Hydrobiologia 478, 181203.991. Post-project appraisal
of conservation enhancements of oodorks. R&D Report 285/1/A,
National Rivers Authority, Reading UK.isler, J., Linkov, I., 2011.
Multi-criteria decision analysis in environ-iences: ten years of
applications and trends. Sci. Total Environ. 4093594,
http://dx.doi.org/10.1016/j.scitotenv.2011.06.022.Tsoutsos, T.,
Agoris, D., Machias, A., 2006. Assessing renewables-to-
systems: fuzzy expert system model. Energy Policy No 34,
13571366.977. The art of assessing multiattribute utility
functions. Organ. Behav.orm. 19 (August (2)), 267310.regory, R.S.,
2005. Selecting attributes to measure the achievement of. Operat.
Res. 53 (1), 111.aiffa, H., 1976. Decision with Multiple
Objectives: Preferences andeoffs. John Wiley & Sons, New
York.
-
86 M. Convertino et al. / Ecological Indicators 26 (2013)
7686
Kondolf, G.M., 1995. Five elements for effective evaluation of
stream restoration.Restor. Ecol. 3 (2), 133136.
Lahdelma, R., Hokkanen, J., Salminen, P., 1998. SMAAstochastic
multi-objectiveacceptability analysis. Eur. J. Operat. Res. 106,
137143.
Lahdelma, R., Makkonen, S., Salminen, P., 2004. Treating
dependent uncertaintiesin multi-criteria decision problems. In:
Rao, M.R., Puri, M.C. (Eds.), OperationalResearch and Its
Applications: Recent Trends, Vol. II, New Delhi. , pp. 427434.
Linkov, I., Moberg, E., 2011. Multi-Criteria Decision Analysis:
Environmental Appli-cations and Case Studies. CRC Press.
Loken, E., 2007. Use of multicriteria decision analysis methods
for energy planningproblems. Renew. Sustain. Energy Rev. 11,
15841595.
Martine, M., Cockeld, G., 2008. Managing trade-offs in landscape
restoration andrevegetation projects. Ecol. Appl. 18, 20412049,
http://dx.doi.org/10.1890/07-1328.1.
McKay S.K., Linkov I., Fischenich J.C., Miller S.J., Valverde
L.J. Jr., 2011. EcosystemRestoration Objectives and Metrics, USACE
ERDC Technical Note.
Mexas, M.P., Goncalves Quelhas, O.L., Gomes Coasta, L., 2012.
Prioritization of enter-prise resource planning systems criteria:
Focusing on construction industry. Int.J. Product. Econ.,
http://dx.doi.org/10.1016/j.ijpe.2012.05.025.
National Academy of Sciences. 2012. Ecosystem Services: Charting
a Pathto Sustainability, Interdisciplinary Research Team Summaries,
NationalAcademy of Sciences Press,
http://download.nap.edu/cart/download.cgi?&record
id=13331&free=1
Nesloa, R.E.J., Cooke, R.M., 2011. Modeling and validating
stakeholder pre-ferences with probabilistic inversion. Appl. Stoch.
Model. Bus. Ind.,http://dx.doi.org/10.1002/asmb.888.
Niemeijer, D., de Groot, R.S., 2008. A conceptual framework for
selecting environ-mental indicator sets. Ecol. Indicat. 8,
1425.
Nienhuis, P.H., Bakker, J.E., Grootjans, A.P., Gulati, R.D., De
Jonge, V.N., 2002. Thestate of the art of aquatic semi-aquatic
ecological restoration projects in theNetherlands. Hydrobiologia
478, 219233.
Noss, R.F., 1999. Assessing and monitoring forest biodiversity:
a suggested frame-work and indicators. Forest Ecol. Manage. 115,
135146.
Palmer, M.A., Reidy Liermann, C.A., Nilsson, C., Florke, M.,
Alcamo, J., Lake, P.S., Bond,N., 2008. Climate change and the
Worlds River Basins: anticipating managementoptions. Front. Ecol.
Environ. 6, 8189.
Pauly, D., Christensen, V., Guenette, S., Pitcher, T.J.,
Sumaila, U.R., Walters, C.J., Wat-son, R., Zeller, D., 2002.
Towards sustainability in world sheries. Nature 418,689695.
Phillis, Y.A., Andriantiatsaholiniana, L.A., 2001.
Sustainability: an ill-dened conceptand its assessment using fuzzy
logic. Ecol. Econ. 29, 235252.
Pohekar, S.D., Ramachandran, M., 2004. Application of
multi-criteria decision mak-ing to sustainable energy planninga
review. Renew. Sustain. Energy Rev. 8 (4),365381.
Reichert, P., Borsuk, M., Hostmann, M., Schweizer, S., Sprri,
C., Tockner, K., Truffer,B., 2007. Concepts of decision support for
river rehabilitation. Environ. Model.Software 22, 188201.
Rohde, S., Kienast, F., Brgi, M., 2004. Assessing the
restoration success of riverwidenings: a landscape approach.
Environ. Manage. 34, 574589.
Saathy, T.L., 1980. The Analytic Hierarchy Process: Planning,
Priority Setting,Resource Allocation. McGraw-Hill, New York.
Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni,
J., Gatelli, D., 2008. GlobalSensitivity Analysis: The Primer. John
Wiley & Sons Ltd.
Seager, T.P., Satterstrom, F.K., Linkov, I., Tuler, S.P., Kay,
R., 2007. Typological reviewof environmental performance metrics
(with illustrative examples for oil spillresponse). Integr.
Environ. Assess. Manag. 3 (3), 310321.
Settin, T., Botter, G., Rodriguez-Iturbe, I., Rinaldo, A., 2007.
Numerical studies onsoil moisture distributions in heterogeneous
catchments. Water Resour. Res.43, W05425.
Sigrid, S., 2004. Valuation for sustainable development the role
of multicriteriaevaluation. Vierteljahreshefte zur
Wirtschaftsforschung (Quart. J. Econ. Res.) 73(1), 110.
Tervonen, T., Figueira, J.R., 2008. A survey on stochastic
multicriteria acceptabilityanalysis methods. J. Multi-Criteria
Decision Anal. 15 (12), 114.
Thayer G.W., Merkey D.H., McTigue T.A., Burrows F.M., Salz R.J.,
Gayaldo P.F., 2005.Science based restoration monitoring of coastal
habitats. Decision AnalysisSeries, No. 23 Vol. 2. NOAA.
Thom, R.M., Wellman, K.F., 1996. Planning aquatic ecosystem
restoration monitoringprograms. Pacic Northwest National
Laboratory, Richland, Washington; nalreport prepared for US Army
Corps of Engineers, Institute for Water Resources,Alexandria, VA,
IWR Report 96-R-23.
Tsoutsos, T., Tsouchlaraki, A., Tsiropoulos, M., Serpetsidakis,
M., 2009. Visual impactevaluation of a wind park in a Greek island.
Appl. Energy 86 (5), 15871600.
U.S. Army Corps of Engineers (USACE). 1999. Civil Works
Ecosystem RestorationPolicy. ER 1165-2-501.
von Winterfeldt, D., Edwards, W., 1986. Decision Analysis and
Behavioral Research.Cambridge University Press, Cambridge.
Wood, M.D., Bostrom, A., Bridges, T., Linkov, I., 2012.
Cognitive mapping tools: reviewand risk management needs. Risk
Anal. 32 (8), 13331348.
World Economic Forum. 2011. Global Risks 2011 Sixth Edition: An
initiative ofthe Risk Response Network, World Economic Forum Press,
http://reports.weforum.org/global-risks-2011/
2011. DECERNSSDSS: Decision Evaluation in Complex Risk Network
Systems.www.decerns.com
Yatsalo, B.I., Gregory, A., Kiker, A., Kim, J., Bridges, T.S.,
Seager, T.P., Gardner, K., Sat-terstrom, F.K., Linkov, I., 2007.
Application of multicriteria decision analysis toolsto two
contaminated sediment case studies. Integr. Environ. Assess. Manag.
3(2), 223233.
Multi-criteria decision analysis to select metrics for design
and monitoring of sustainable ecosystem restorations1 Introduction2
Materials and methods2.1 Hypothetical aquatic ecological
restoration2.2 Multi-criteria decision analysis (MCDA)2.3
Multi-attribute utility theory (MAUT)2.4 Probabilistic
multi-criteria acceptability analysis (ProMAA)
3 Results and discussion4 Conclusions and
perspectivesAcknowledgementsAppendix A Supplementary dataAppendix A
Supplementary data