Fish-based methods for assessing Europeanrunning waters: a synthesis
S . S C H M U T Z
Department of Water, Atmosphere and Environment, Institute of Hydrobiology and Aquatic Ecosystem Management,BOKU – University of Natural Resources and Applied Life Sciences, Vienna, Austria
I . G . C O W X
Hull International Fisheries Institute, University of Hull, Hull, UK
G . H A I D V O G L
Department of Water, Athmosphere and Environment, Institute of Hydrobiology and Aquatic Ecosystem Management,
BOKU – University of Natural Resources and Applied Life Sciences, Vienna, Austria
D . P O N T
UMR CNRS Ecologie des Hydrosystemes Fluviaux, Universite Claude Bernard Lyon1, Villeurbanne, France
Abstract The European Union, Water Framework Directive (WFD) requires monitoring of riverine fish fauna.When the WFD came into force in 2000, most of the EU member states did not have fish-based assessmentmethods compliant to WFD requirements. Therefore, the objectives of FAME (http://fame.boku.ac.at), a projectunder the fifth R&D Framework Programme of the European Commission were to develop, evaluate andimplement a standardised fish-based method for assessing the ecological status of European running waters. Thispaper synthesises the outputs of FAME and defines future research needs. Two different methodologies were used:the so-called spatially based modelling and the site-specific modelling, the latter leading to the European FishIndex (EFI). The advantage of the EFI is that, despite being a single index, it is applicable to a wide range ofenvironmental conditions across Europe precluding the need for inter-calibration. The EFI will support the WFDtowards harmonised/standardised assessment and management of running waters in Europe, thus enablingcomparative analyses of the ecological status of running waters across Europe.
KEYWORDS : Europe, fish-based assessment, index of biotic integrity, Water Framework Directive.
Introduction
In the year 2000, the European Commission promul-gated the Water Framework Directive (WFD) (2000/60/EEC) to improve the environmental quality ofrivers, lakes and transitional waters. To achieve this,the WFD requires that all waters are classifiedaccording to five quality (from 1 – high status to 5 –bad status) classes depending on the degree of degra-
dation and bring all water bodies that have not beenheavily modified to at least �good ecological status�(Class 2) by the year 2015. Consequently, the WFDrequires standardised methods for assessing the eco-logical status of rivers, lakes and transitional waters,using fish amongst other biological quality elements.Within Europe, however, there was a lack of appro-priate methods when the WFD came into force, and inparticular standardised methods that could be used
Correspondence: Stefan Schmutz, Department of Water, Atmosphere and Environment, Institute of Hydrobiology and Aquatic Ecosystem
Management, BOKU – University of Natural Resources and Applied Life Sciences, Max Emanuel-Strasse 17, 1180 Vienna, Austria
(e-mail: [email protected])
Fisheries Management and Ecology, 2007, 14, 369–380
� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd doi: 10.1111/j.1365-2400.2007.00585.x
across member states of the EU. In an attempt toovercome this problem for rivers in relation to fish, theEU financed the FAME project (Development, Eval-uation and Implementation of a standardised Fish-based Assessment Method for the Ecological Status ofEuropean Rivers; http://fame.boku.ac.at) to developmethods compliant with the WFD that could be usedfor a wide range of European conditions.
This paper summarises the results of the FAMEproject presented in this dedicated issue, but specifi-cally compares the different approaches developed,discuss advantages and disadvantages, and to identifyadvances in the development of assessment methods.Major steps in method development, i.e. data require-ments, identification of reference conditions, quantify-ing human pressures, selection of metrics, indexcalibration, index validation and method implementa-tion (Fig. 1), are presented and recommendations formethod selection and implementation given. Finally,remaining limitations and future research needs areidentified.
The challenge in developing assessment methods isto distinguish between natural and human-inducedvariability in fish assemblages (Fausch, Lyons, Karr &Angermeier 1990), an issue that has been stronglyinfluenced by the concept of index of biotic integrity(IBI; Karr 1981). IBIs are based on the assumptionthat within a given entity the variability of fishcommunities is low enough to be able to distinguishbetween natural and human-induced variability.
Although IBIs have been developed worldwide (Roset,Grenouillet, Goffaux, Pont & Kestemont 2007),appropriate delineation of regions or river types is amajor issue in developing regional IBIs (Strange 1998)because of large-scale natural variability in fish com-munities.
Environmental descriptors are used to account forwithin-region variability (Roset et al. 2007). Stratifyingintraregional criteria by using only a single factordespite the great number of factors likely to controlfish communities (Matthews 1998) is clearly inade-quate to cope with natural variability. Moreover, inmost IBIs abundance-related metrics (i.e. functionalmetrics) are not adjusted at all (Oberdorff, Pont,Hugueny & Porcher 2002). For this reasons, classicalIBIs are only developed for a specific region or countryor for a given river type (e.g. coldwater streams).
Two different Europe-wide methods were devel-oped: the so-called spatially based modelling (SBM)and the site-specific modelling approach, the latterleading to the European Fish Index (EFI). The SBMcopes with the problem of natural variability bypredicting reference conditions of distinct fish assem-blage types. For each fish assemblage type an individ-ual method is developed. By contrast, the EFI ischaracterised by a single index that predicts referenceconditions for individual sites and is applicable to thewhole range of environments examined (Pont, Hugu-eny, Beier, Goffaux, Melcher, Noble, Rogers, Roset &Schmutz 2006; Pont, Hugueny & Rogers 2007). The
Fish PressureEnvironment
Basic data
Guild
Pressure criteria
Reference sites Impacted sites
Metric calculation
Metric values of reference sites
Metric values of impacted sites
Cluster analyses
Fish types
Discriminantanalysis
Predicted fish types
Discriminantanalysis
SBM status class
Regression analysis
Predicted reference metrics
Residuals calculation
EFI index
Scoring
EFI status class
Figure 1. Comparison of the spatially based (SBM) and site-specific (European Fish Index – EFI) methodologies.
S. SCHMUTZ ET AL.370
� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd
SBM approach was first applied to individual ecore-gions (SBM-ER: Ferreira, Caiola, Casals, Oliveira &Sostoa 2007; Grenouillet, Roset, Goffaux, Breine,Simoens, de Leeuw & Kestemont 2007; Noble, Cowx& Starkie 2007b; Melcher, Frangez, Haidvogl, Beier,Bohmer, Breine, Simoens, Caiola, de Sostoa, Ferreira,Oliveira, Grenouillet, Goffaux, de Leeuw, Noble,Roset & Verbickas 2007; Virbickas & Kesminas2007) and afterwards to all ecoregions at once (SBM-EU: Melcher, Schmutz, Haidvogl & Moder 2007).
Data requirements for developing Europeanmethods
A pre-requisite for developing assessment methods isthe availability of adequate data. Existing fish datacollected for various purposes were used to develop themodels. Data were collated using a dedicated database(FIDES) that covered >8000 sites in 12 Europeancountries and 17 ecoregions (Beier, Degerman, Mel-cher, Rogers & Wirlof 2007). This included humanpressure data that were used to identity referenceconditions and calibrate assessment methods. Data onnatural drivers, i.e. physiographic characteristics of therivers, were used to predict reference conditions ofdisturbed sites (Fig. 1).
Fish sampling
To avoid potential variation in the quality of fish dataarising from differences in sampling methods usedacross Europe, fish data were confined to thosecollected during the first run of electric fishing surveyscarried out according to CEN standards on electricfishing (CEN 2003; Beier et al. 2007). The onlydifference was that CEN (2003) requires a river lengthof 10–20 times the channel width to be sampled. Thiswas not always achieved, and a threshold of 100 m2 offished area was defined as the selection criterion for theEFI, to exclude sites with very low sampling effort.Sampling method and effort were included in thepredictive models of EFI to account for differences insampling technique environments (Pont et al. 2006,2007). This sampling definition was a trade-off betweendata quality and availability, and defined the frame-work for the metrics that could usefully be tested andincluded in the models (see metrics selection).
Data representativeness and quality
Irrespective of the selection criteria, data representa-tiveness varied within and among regions and coun-tries. Although Western and Northern countries were
reasonably well covered, Mediterranean and easternregions were underrepresented. In addition, only a fewvery large rivers, such as the Rhine, were included (deLeeuw, Buijse, Haidvogl, Lapinska, Noble, Repecka,Virbickas, Wisniewolski & Wolter 2007).
Species guild classification
Using ecological guilds instead of single species is oneof the core concepts in IBIs and allows humanpressures to be correlated with functional aspects oforganisms to enable the application of standardmethods across fish assemblages with varying taxo-nomic composition. Although many fish species ofEurope have been classified into ecological guilds,classifications between countries have not been har-monised. Consequently, the 123 principal Europeanfish species in the data set were classified into consis-tent ecological guilds (Noble, Cowx, Goffaux &Kestemont 2007a). The assessment methods developedused traditional guild classification, such as trophicguild, reproductive behaviour and habitat preferences,but also included guilds such as migration behaviour,longevity and population age structure (0+ vs olderfish) to improve the characterisation of essentialecological requirements.
Reference conditions and classificationof human pressure
The concept of IBIs is based on the assumption thatwithin a given entity the variability of fish communitiesis low enough to distinguish between natural andhuman-induced variability. To accomplish this, IBIsare commonly calibrated by pre-classifying humanpressure using expert judgement (Lyons, Wang &Simonson 1996) or real pressure data (Hughes, Howlin& Kaufmann 2004). Consequently, pressure data areused for two purposes: to identify reference conditionsand to calibrate indices. In FAME, reference condi-tions were developed using field data from onlyminimally or slightly impacted sites. Historical data,although considered valuable information, could notbe used at the European level because only a fewcountries (i.e. Austria, Germany, the Netherlands)provided complete data sets.
Defining human pressure
Twenty-four potential pressures to fish communitieswere identified based on lists from the Impressgroup(2003) and AQEM Consortium (2002) supplementedthrough literature on fish-pressure studies and expert
SYNTHESIS OF FISH-BASED ASSESSMENT METHODS 371
� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd
judgement (Degerman, Beier, Breine, Melcher, Quata-ert, Rogers, Roset & Simoens 2007). As monitoringprogrammes differ considerably among countries interms of parameters used and how they are assessed,expert judgement was used whenever pressure datawere lacking or inadequate. Initially, it was intended tocollect pressure data covering the whole range ofpotential types of impacts on fish, but, because oflimited availability of pressure data, only five keyphysical/chemical pressures were considered: (1)hydrology; (2) morphology; (3) nutrients; (4) toxicsand acidification at the site scale and (5) connectivity atthe river segment scale. Connectivity was used only forSBM methods as there was no significant correlationwith fish at the continental scale (Pont et al. 2006). Amean of the individual pressures was used to calibratethe fish indices. The pressure index based on the foursite pressures was poorly correlated with biologicalpressures (e.g. introduction of fish) and connectivity,demonstrating that not all human pressures are cov-ered by this index. Also computing a pressure index asa mean of four or five pressure variables implies thatall pressures contribute in the same way to globalpressure conditions and that there are no interactionsbetween individual pressures. For example, a physicalmorphology pressure of class 5 and a chemical toxicitypressure of class 1 would result in a pressure index of 3indicating moderate conditions, but it is known thatfish assemblages would be severely affected by thebad physical habitat conditions. Consequently, theresponse to individual and combined effects ofpressures should be analysed in future studies.
Analyses of pressure-specific response found onlymetrics based on omnivorous species reacted(increased) to continuum disruptions in the SBMmethods (Schmutz et al. 2007). Among the ecologicalguilds considered for the biological assessment, diad-romous and potamodromous fish species are particu-larly sensitive to continuum disruptions, as aninterruption of migrations severely affects these species(Northcote 1998). Apart from an insufficient descrip-tion of the pressure (dimensions and passage possibil-ities of the migration barrier were not considered), thismight also have been caused by missing informationabout the reference situation of diadromous species.
Defining reference conditions
According to the WFD, reference conditions may bedefined either spatially based or by modelling tech-niques using a reference network containing enoughsites of high status to provide an acceptable level ofconfidence. Reference conditions equal the high status
(class 1) of the five-tiered WFD classification schemeand are defined as no or only very minor anthropogenicalterations of the physicochemical and hydromorpho-logical conditions and no or only very minor evidence ofdistortion of the biological conditions (Annex II,WFD). Hence, both abiotic and biotic conditions haveto meet reference criteria.
In FAME, human pressures were first quantifiedand then these conditions were compared with thebiological criteria (metrics) and tested for differencesbetween unimpacted and impacted conditions. Lessthan 10% of FIDES sites represented pure referencesites, i.e. pressure variables equivalent to class 1. Thesesites mainly occurred in northern Europe (Sweden) andheadwaters, and were thus not representative for allriver types. Consequently, slightly impacted sites(Class 2) were combined with Class 1 sites to calibratethe models. This approach may seem contradictory tothe WFD reference principle, but the calibration dataset was only used to identify and predict fish assem-blage types in the spatially based models (SBM) and tocalibrate the reference models for the EFI. The finalindices used the full range of impairment gradient anddiscriminated between class 1 and 2. However, as aresult, the EFI is not able to discriminate statisticallybetween 1 and 2 (Pont et al. 2006). Nevertheless, theboundary between 2 and 3 is more important as itrepresents the limit for restoration action.
Natural variability of European fish assemblages
Fish assemblage types
The �spatially based approach� is the underlyingmethodological principle of the WFD for monitoringand managing surface waters. The concept is thatrivers can be classified into homogenous water bodies(i.e. river segments) using abiotic criteria. However,because the lower the biotic heterogeneity withinidentified types, the higher the accuracy of the IBI(Smogor & Angermeier 2001), biotic instead of abiotictypologies were used. In principle, two spatial dimen-sions, the zoogeography across Europe and the longi-tudinal pattern within each river, structure fishassemblages at the large scale. In the WFD, Illies�(1967, 1978) ecoregions are supposed to provide aspatial framework for river classification; however,they have not been evaluated for their ability todiscriminate among fish assemblages at a continentalscale. In addition, longitudinal zonation proposedmore than a century ago (Fritsch 1872), has not beenevaluated at larger spatial scales, such as ecoregions orwhole continents.
S. SCHMUTZ ET AL.372
� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd
In the SBM approach, 60 fish types were identifiedranging from two to eight types per ecoregion (Sch-mutz et al. 2007). Grouping these 60 ecoregional fishassemblage types at the European level reduced thenumber to 15 types (Melcher et al. 2007). Althoughthis represents a reduction in the number of typescompared with the abiotic classification of the WFD,the variability within those types was sufficiently low todevelop robust methods; the reason being the fishassemblages across Europe have comparable guildcomposition and are mainly structured along thelongitudinal gradient. At both the ecoregional andEuropean scales, the classical trout to bream zonationwas observed. Most environmental parameters foundto structure fish assemblages describe the longitudinalgradient of streams and rivers. About five environ-mental descriptors were used in each ecoregion (Sch-mutz et al. 2007) and seven at the European scale(Melcher et al. 2007) to discriminate among fish types(Table 1). Melcher et al. (2007) demonstrated thatthere are only five European Fish Assemblage Types –EFT – that are distinct for particular ecoregions andthat the majority (10 types) are found in severalecoregions. This indicates that Illies� ecoregions areimpractical for explaining European fish assemblagepatterns.Correct assignment of sites to fish types by discri-
minant function analyses using environmental descrip-tors ranged from 18% to 100% within ecoregions(Schmutz et al. 2007) and improved to 70% at the
European level (Melcher et al. 2007). In particular, theSBM-EU approach showed that the environmentaldescriptors used were adequate to explain and predictthe spatial variability of fish assemblages in Europe.This approach has the advantage that no a prioriexpert classification of fish assemblage types is neces-sary and that the definition and assignment of sites totypes follows a standardised procedure.
Predicting site-specific fish assemblages
Oberdorff, Pont, Hugueny & Chessel (2001) usedstatistical procedures to elaborate models that predictmetrics in relation to factors describing the naturalenvironmental variability of French rivers at both localand regional scales. This technique was used in FAMEto develop the EFI. The EFI employs 11 environmen-tal descriptors (Table 1) to predict reference conditionsfor each metric of a given site using multiple regressionmodels (Pont et al. 2006). In concordance with theSBM approach, longitudinal descriptors were prevail-ing. Only one explicit regional descriptor, the mainriver region – a river basin classification based onnative fish fauna similarities, accounted for regionaleffects.
The level of variability explained in the 10 referencemodels for each of the EFI metrics was between 31%and 48% (Pont et al. 2006). Equivalent variability (20–68%) was found by Oberdorff et al.(2002) for an indexdeveloped in France that used similar environmentalpredictors and Porter, Rosenfeld & Parkinson (2000)explained 73–90% of fish species distributions usingmap-based macrohabitat variables. Oakes, Gido,Falke, Olden & Brock (2005) correctly classifiedspecies presence/absence at a level of 81%; Pont,Hugueny & Oberdorff (2005) achieved a level of 71–92%. The higher level of explanation in these studiescompared with the EFI models might be because theEFI does not predict species absence/presence butfunctional metrics composed of several species. Theconcept of ecological guilds assumes similar functionalroles of species within a guild category. Therefore, onemight expect analogous response of guild members toenvironmental drivers. However, the results of the EFImodels show that there is still a high variability withinthe guilds that probably could be attributed to thedifferent ecological requirements of individual species.
Metric selection
Biotic indices employ biological criteria (metrics) thatrespond to human alterations of ecosystems; theselection of metrics depends on their reaction to
Table 1. Environmental descriptors used in FAME methods
Environmental descriptors SBM-ecoregion SBM-EU EFI
Main river region · ·Longitude ·Latitude ·Mean air temperature · · ·Mean air temperature in July ·Mean air temperature in January ·Geology · ·Altitude · · ·Slope · · ·Distance from source · · ·Wetted width · · ·Size of catchment · ·Stream order ·Conductivity ·Average depth ·Huet�s zonation ·Sampling strategy ·Sampling method ·Sampled area ·
SBM: spatially based models, EFI: European Fish Index.
SYNTHESIS OF FISH-BASED ASSESSMENT METHODS 373
� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd
human pressures (Karr 1981). The WFD requires theassessment of species composition, abundance, sensi-tive species and population structure.
Metrics are often selected on the basis of expertjudgment, without checking for empirical relationsbetween metrics and environmental variables (Hughes& Oberdorff 1998; Angermeier, Smogor & Stauffer2000; McCormick, Peck & Larsen 2000). However,Angermeier et al. (2000) advised against building IBIsonly with metrics exhibiting strong empirical relations,because such indices may lack the ecological founda-tion needed for detection of a broad range of humanimpacts. Mebane, Maret & Hughes (2003) suggestedthat both an ecological foundation and empiricalrelationship should underlie the construction of IBIs.Redundant metrics should also be excluded (Mebaneet al. 2003; Hughes et al. 2004).
In FAME, a multiple-step selection procedure wasapplied to ensure that only metrics responsive tohuman pressures were selected for the assessmentmethods. In the SBM, correlation analyses were usedto pre-select metrics. Final metrics were chosen bydiscriminant function analyses for each fish assemblagetype using the five-tiered global pressure index (Mel-cher et al. 2007; Schmutz et al. 2007). For the EFI,only metrics for which reference values could bepredicted by environmental descriptors were pre-selected and only metrics discriminating betweendifferent levels of human pressures were selected (Pontet al. 2006).
The main difference in types of metrics selectedbetween the FAME methods was that the SBM-ERmainly employed sentinel metrics that are based onindividual species, whereby SBM-EU and EFI mainlyor solely used functional metrics (Fig. 2). The largerthe scale that the methods were designed for (spatial
gradient from SBM-EU to SBM-ER and EFI), themore general the characteristics of the fish assem-blages, i.e. functional metrics, that were employed toovercome taxonomic differences among regions. Sen-tinel species metrics are specific to individual fishassemblage types and are known to react to humanpressure in a predictable way but might not be presentor react in a different way in other types or ecoregions.
Species richness is a classical metric in IBIs (Nobleet al. 2007a, b) and, according to the biodiversityconcept one of the most important metrics to assessecosystem degradation (Magurran 2004). However,species richness plays a minor role in all FAMEmethods. Nevertheless, most FAME methods achieveda fairly high level of accuracy without species richnessmetrics.
Migratory species metrics are new metrics of impor-tance to reflect continuum disruptions. Althoughneither SBM-EU nor EFI used continuum disruptionin the global pressure index to test metrics reaction,both methods employed diadromous and potamodro-mous species guilds, proving the general sensitivity ofmigratory species to human disturbance.
Longevity is an important species trait for bioas-sessment (Plafkin, Barbour, Porter, Gross & Hughes1989; Doledec, Statzner & Bournard 1999) and wasalso tested in FAME by distinguishing between long-and short-living species (Noble et al. 2007a, b), but theresponses were not clear. Longevity metrics were notselected for the EFI and in the SBM-EU the responseof short- and long-living species was inconsistentamong fish assemblage types.
Different metric modalities (number of species,percentage of species, density, relative density, bio-mass, etc.) were tested (Noble et al. 2007a). Densityand number of species were the metric modalities most
25
Historical metrics
Longevity
Migration
Overall composition
ToleranceFeeding
HabitatReproduction
Sentinel
%0 5 10 15 20
Figure 2. Comparison of metric types selected for different FAME
methods: h SBM-ER, spatially based methods at ecoregional level;
SBM-EU, spatially based methods at European level; EFI, European
Fish Index.
60
0+
Biomass
N species
Density
%0 10 20 30 40 50
Figure 3. Comparison of metric variants selected for different FAME
methods: h SBM-ER, spatially based methods at ecoregional level;
SBM-EU, spatially based methods at European level; EFI, European
Fish Index.
S. SCHMUTZ ET AL.374
� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd
often employed (Fig. 3). An estimation of density, interms of catch-per-unit-effort (single pass) (Beier et al.2007), seems to be essential to achieve a sufficient levelof accuracy. Data for metrics describing the absence/presence or density of 0+ (young of the year) werestrongly limited. These metrics were only selected inSBM-ER, where sufficient data were available.
Index computation and scoring
Scoring of metrics and indices involves basic assump-tions affecting IBI behaviour and results. The classicaltri-section method proposed by Karr (1981) splits therange of metric values in three equal parts assuming alinear response of fish to human pressures. In FAME,scoring the indices was based on the pre-classificationof pressure presuming that the thresholds defined forpressure classes were the same as for ecological statusclasses. Both approaches lack profound biologicalfoundation. In future, more effort should concentrateon defining thresholds based on biological criteria suchas irreversibility, recovery time and extinction risk.Traditionally, indices are computed by summing up
individual metric scores assuming equal contributionof individual metrics to index score (Karr 1981;Bramblett, Johnson, Zale & Heggem 2005). The EFIwas computed in this way as the sum of 10 metrics(Pont et al. 2007). The SBM approach used discrimi-nant function analyses to model and predict statusclass membership of any given site (Melcher et al.2007; Schmutz et al. 2007). Discriminant functionanalyses have also been used in other studies to findpressure variables that best differentiate among IBIclasses (e.g. Joy & Death 2002). The advantage of thisapproach is that metric combinations are selected thathave the highest probability in explaining biologicalresponses to human pressures. Metrics are weighted inthe discriminant function according to their individualcontribution to the overall pressure–index relationship.The disadvantage of discriminant function analysis isthe contribution of individual metrics is �hidden� in thediscriminant functions of the model because of itsmulti-dimensional nature.
Index accuracy
Uncertainties arise at all levels of IBI development:field sampling, guild classification, pressure quantifi-cation, definition of reference conditions, metricsselection, index computation and scoring. Tacklingthe entire potential source of uncertainty would requireerror estimates for all types of uncertainty sources andideally computing a cumulative error estimate. To
date, error estimates were only performed for distinctsteps of IBI development, e.g. sampling intensity(Peck, Lazorchak & Klemm 2003) or cross-validationof index with human pressure (Hughes et al. 2004;Bramblett et al. 2005).
A precise IBI should be insensitive to natural spatialvariability. The natural spatial variability was explic-itly integrated into the development of the FAMEmethods. Being able to predict EFTs with environ-mental descriptors using the SBM-EU approach dem-onstrated that among variability of EFTs is higherthan within variability. Therefore, it can be assumedthat natural variation within EFTs has marginal effecton the index results (Melcher et al. 2007). Naturalvariability was integrated into the EFI by establishingstatistical models describing metric responses to natu-ral environmental variability in the absence of anysignificant human disturbance.
As only a limited number of sites with repeatsamples or time series were available, the effects oftemporal variability on index stability could not betested explicitly. However, temporal variability isinherently included in the data set because the samplescover about 20 yr in the various ecoregions.
Method accuracy was evaluated by cross-validatingecological status with global pressure status (Quataert,Breine & Simoens 2007). This method identifies bothtype I errors (unimpacted sites that were misclassifiedas impacted sites) and type II errors (impacted sitesthat were classified as unimpacted). In a preciseassessment method, both types of error should beequal and low. Accuracy of SBM-ER methods basedon cross-validation with pre-classification variedbetween 47% and 98% (mean 81%) when contrastingcalibration data set (classes 1 and 2) with degradedsites (classes 3, 4 and 5) (Schmutz et al. 2007). For theSBM-EU approach, on average, 85% of all sitescoincided with the pressure pre-classification (Melcheret al. 2007). The EFI correctly classified 80% ofreference and disturbed sites (Pont et al. 2007). AllFAME methods were about the same level of accuracy(Fig. 4) but type II errors were generally higher. Mostregional methods have similar accuracy to the FAMEmethods (Quataert et al. 2007).
Index validation with independent data
Cross-validation through comparison of the IBI withan independent estimate of the ecological status can beused to evaluate the error of assessment methods, butthe problem of circularity arises when the samepressure variables are used for calibration and valida-tion. Using pressure variables not employed for
SYNTHESIS OF FISH-BASED ASSESSMENT METHODS 375
� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd
calibration might be a solution to avoid circularity, butpressure variables are usually inter-related. A moreappropriate approach is to validate IBIs with anindependent data set not used for its development. InFAME, depending on the method developed, severalvalidation approaches were applied. In the SBM-ERapproach, splitting the database into 60 fish typeshampered method validation with independent databecause too few samples remained. For the SBM-EUapproach, the data set was randomly split into two andused for model development and validation respec-tively (Melcher et al. 2007). For the EFI, the referencedata set was divided randomly into three subsets: one
set for model calibration, one for validation of metric-specific models, and one for the final index validation(Pont et al. 2006).
End-user perspective
Any method developed should not only be scientifi-cally sound but also be cost efficient, transferable todifferent river basins and regional conditions and easyto use. Throughout the development of the methodol-ogies, specific collaboration was sought with agencies(Applied Partners) responsible for the implementationof the WFD, to collect data, seek feedback on theapplicability of the methods, use of software anddissemination of outputs (Backx 2004).
A major concern of end-users was the potential highcosts of sampling associated with the methods. Thetotal budget for monitoring fish in rivers variedconsiderably between countries because of differencesin the number of expected samples, the number of sitessampled per day and cost of equipment (fixed costs)and personnel (variable costs) (Table 2; Backx 2004).A large proportion of the variation in costs arisesbecause of the different sampling strategies used. Somecountries use multiple-run depletion sampling, whereasothers perform only single-run sampling. This is alsotrue for collecting data on fish length and weight.
Applied Partners recommended that the CEN stan-dard on electric fishing (CEN 2003) should be evalu-ated with respect to sampling costs and efficiencyrelated to the EFI metrics and eventually should berevised. More analyses on spatial and temporal vari-ability of fish data are necessary. Information ondiadromous species which is underrepresented inelectric fishing samples should also be considered forfish-based assessment. There are still problems sam-
0
10
20
30
40
50
60
70
80
90
Existingmethods
%
Type I errorCorrectly classifiedType II error
EFI SBM-EU SBM-ER
Figure 4. Comparison of method accuracy (cross-validation with
pressure pre-classification) of FAME methods with existing regional
methods. SBM-ER, spatially based methods at ecoregional level; SBM-
EU, spatially based methods at European level; EFI, European Fish
Index.
Table 2. Expected number of electric fishing sampling sites per country and per 1000 km2 area for the WFD monitoring programme,
the maximum number of sites sampled per day and the sampling costs (adapted from Backx 2004)
Country
Total number
of samples
Samples per
1000 km2
Samples
per day
Fixed
costs (€)
Variable
costs (€)
Costs per
sample (€)
Austria 400 4.8 2 4750 96 000 1679
Belgium Flanders 200 14.8 5 14 500 60 500 375
France 675 1.2 2 41 700 97 900 2945
Germany 1250
Greece 180 1.4 2 59 000 5000 1837
Lithuania 80 1.2 3 2030 5820 1000
Poland 540 1.7 3 25 200 55 000 149
Portugal 80 0.9 4600 61 425 943
Sweden 252 0.6 2 5975 251 700 512
The Netherlands 120 2.9 15 17 242 151 58 520
UK 1333 5.4 2 821
Mean (median) 226 1.5 2 14 500 60 500 943
S. SCHMUTZ ET AL.376
� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd
pling large rivers in an efficient and cost-effectivemanner (de Leeuw et al. 2007). Furthermore, guide-lines are needed on how to sample catchments andwater bodies representative of the WFD. Finally,although the project provided easy to handle, publicdomain software for the EFI (FAME Consortium2004) enabling all potential end-users to use themethod, interpretation requires expertise in fish-basedassessment methods.End-user feedback was used to select the final
method for supporting the WFD based on a numberof criteria (Table 3). No relevant differences in accu-racy were found between FAME and existing regionalmethods (ERM) where they exist, but many countrieshad not developed methods or assessment was basedmainly on expert judgement that lack scientific foun-dation and statistical robustness, and therefore sufferspoor objectivity and reliability. However, as ERMsmainly follow the classical IBI approach they are moretraceable than model-based FAME methods.Although different in their conceptual framework, allmethods seem to be equally simple in their practicalapplicability, but, as most ERMs were developedbefore the WFD came into force, they are not fullyin accordance with the WFD (reference approach,types of metrics). Nevertheless, the FAME methodsalso need to be further developed (e.g. integration ofpopulation age-structure) to be fully compliant withWFD requirements. Although there was no thoroughassessment of cost efficiency, the EFI and SBM-EU areno more expensive than the ERMs, as both requireonly single-pass electric fishing data, whereas for someERMs quantitative data are necessary. Furthermore,ERMs have limited spatial applicability and need
intercalibration to produce comparable results. Theonly method to use the same criteria and standardsacross Europe is the EFI; thus, given the similar levelsof accuracy between methods, the great objectivity ofstatistical methods and the advantage of standardisedmethods, the EFI was the preferred method amongstend-users.
Conclusions
The FAME project produced assessment methods atthe regional and European scales. Spatially based (fishassemblage type-specific) and site-specific methods(EFI) were developed and compared, with the EFIrecommended for the WFD assessment. The advanta-ges of the EFI include:1 applicability for a wide range of environmentalconditions across Europe;2 representative of a single index that does not requireintercalibration among countries;3 precision is comparable with spatially based andERMs;4 incorporation of natural variability in the index bypredicting reference metrics values for individual sites;5 standardised assessment of impacted sites based onstatistical evidence of deviation from unimpactedconditions;6 calibration against an independent assessment ofpressure;7 only requires low-cost single-pass electric fishingsampling;8 easy to use with simple public domain softwareimplemented in MS Excel�.
The EFI represents a major step towards a standar-dised fish-based assessment of European runningwaters. However, there are still limitations that needattention.
Extending the EFI to regions and environments notadequately covered in FAME
Three particular regions were notably weak in theFAME programme:• Mediterranean river systems, which are character-ised by species-poor but very diverse communities thatrequire better definition of the natural environmentalvariability of the rivers, specific species guilds andmetrics adapted to the particular type of pressure andconsideration of the presence of a large number ofalien species.• Large floodplain rivers, which have high diversity ofwater bodies with respect to morphology and fishfauna. Large mid-channel sections and temporarily
Table 3. Comparison of the European Fish Index (EFI) (Pont et al.
2007) with the spatially based method developed at the European
level (SBM-EU) (Melcher et al. 2007) and existing regional methods
from the end-users perspective
Criteria EFI SBM-EU
Existing
methods
Accuracy + + +
Scientific foundation + + ±
Objectiveness + + ±
Reliability + + ±
Traceability (black box) ± ± +
Simplicity of use + + +
WFD accordance ± ± )Cost efficiency + + ±
Standardised applicability
(ecoregions, river types, pressures)
+ ± )
European standard + ± )
+, strong; ±, medium; ), low significance.
SYNTHESIS OF FISH-BASED ASSESSMENT METHODS 377
� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd
connected floodplain water bodies form an integralpart of these water systems and the complex fishcommunities require several complementary samplingmethods and indices to make robust assessment.• Central/Eastern Rivers, e.g. the River Danube,which have high species diversity and endemism, thatneed to be considered when extending the EFI toeastern countries and predicting reference conditions.
Improving the capacity of the EFI to detectpressures
To improve the response of the EFI to hydromor-phological pressures, more precise pressure data areneeded. It is necessary to distinguish more accuratelybetween different types or pressures, e.g. to splitmorphological alteration into effects of impoundmentand channelisation as both alter the morphologicalconditions but result in different biological responses.Furthermore, most European rivers are impacted bymultiple pressure types, and biological assessmentshould help to identify which pressures affect fish themost to prioritise restoration measures. Thus, a moreprecise classification of pressure types (e.g. minimumflow), a better quantification of pressure variables (e.g.ratio of observed minimum flow to natural minimumflow) and links to drivers acting on these pressures(e.g. hydropower production) are needed.
Improving the ecological guild classification ofEuropean fish species
The ecological requirements of European fish species arereasonably well known but about half the speciescovered by the FAME project could not be classifiedbecause of missing information. These are mainly rareand/or endemic species with restricted distribution, butsometimes represent more sensitive species groups.Research on the ecological requirements of these speciesis necessary to fill existing gaps and to improve assess-ment methods. Additional guilds should also be testedfor their ability to reflect human disturbance. Forexample, temperature preferences of species could beused to detect thermal disturbances. A more preciseclassification of guilds such as a distinction betweenlithophilic and litho-obligate reproductive guilds mightreflect substrate alterations caused by channelisation orother pressures. Similarly, the effects of continuitydisruptions may be better elucidated by historic, pres-ence/absence information on diadromous species that isusually readily available.More detailed analyses are alsoneeded to evaluate the significance of longevity inbioassessment. For example, combinations of longevity
with other ecological requirements such as tolerance orhabitat requirements might be promising. Populationstructuremetrics are required by theWFD andmight bevery important where the response of the fish species ismore subtle. This is particularly the case in streams andrivers with very low species diversity (one or two species)where �functional community metrics� fail to indicatehuman pressure.
Testing new environmental descriptors for referencemodels
New environmental descriptors should be tested toimprove reference models, e.g. precipitation, streampower, neighbourhood relationship and landscapevariables. GIS-derived Europe-wide catchment datamight provide a useful and cost-effective source ofstandardised data in future approaches.
Testing new metrics
Further efforts are necessary to quantify continuumdisruptions at different spatial scales better and torelate these disturbances to migratory requirements offish species. As diadromous species are extinct in manyEuropean rivers, historical information could be usedto reconstruct former occurrences.
Field sampling and representative monitoring
Further tests are necessary for standardisation andoptimisation of field sampling to ensure cost-efficiencyand consistency at the European scale. Standardisedsampling protocols have to be developed for largefloodplain rivers. Analysing spatial and temporalstability of the EFI should give an indication ofmonitoring requirements to guarantee representativeassessments of catchments. Tools for up-scaling site-specific assessment results to entire water bodies andcatchments have to be developed.
Developing tools for prioritising restorationmeasures
The EFI and future versions will support harmonisedassessment and management of running waters inEurope. Providing a standardised assessment tool willenable comparative analyses of the ecological status ofEuropean rivers. Restoration and rehabilitation mea-sures can be evaluated between countries to assess ifgood ecological status has been achieved. Therefore,the EFI could become an important tool to define andprioritise restoration measures.
S. SCHMUTZ ET AL.378
� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd
Acknowledgements
We would like to thank the European Commission forfunding the FAME project (EVK1-CT-2001-00094).Special thanks are due to the many institutions inEurope providing data for the FAME project.
References
Angermeier P.L., Smogor R.A. & Stauffer J.R. (2000)
Regional frameworks and candidate metrics for assessing
biotic integrity in mid-Atlantic highland streams.
Transactions of the American Fisheries Society 129, 962–
981.
AQEM Consortium (2002) AQEM Software. Available at:
http://www.aqem.de.
Backx J. (2004) Transfer into water management. Report of
the Applied Partners WP11/D18 – R&D project FAME
EVK1-CT-2001-00094. Available at: http://fame.boku.
ac.at/downloads/D18_AppliedPartners_TransferWM.pdf.
Beier U., Degerman E., Melcher A., Rogers C. & Wirlof H.
(2007) Processes of collating a European fisheries database
to meet the objectives of the European Union Water
Framework Directive. Fisheries Management and Ecology
14, 407–416.
Bramblett R.G., Johnson T.R., Zale A.V. & Heggem D.G.
(2005) Development and evaluation of a fish assemblage
index of biotic integrity for northwestern Great Plains
streams. Transactions of the American Fisheries Society
134, 624–640.
CEN (2003) Water quality – sampling of fish with electricity.
European standard – EN 14011:2003. Brussels: European
Committee for Standardization, 18 pp.
Degerman E., Beier U., Breine J., Melcher A., Quataert P.,
Rogers C., Roset N. & Simoens I. (2007) Classification and
assessment of degradation in European running waters.
Fisheries Management and Ecology 14, 417–426.
Doledec S., Statzner B. & Bournard M. (1999) Species traits
for future biomonitoring across ecoregions: patterns
along a human-impact river. Freshwater Biology 41, 737–
758.
FAME Consortium (2004) Development, Evaluation and
Implementation of a standardised Fish-based Assessment
Method for the Ecological Status of European Rivers
(FAME). Available at: http://fame.boku.ac.at.
Fausch K.D., Lyons J., Karr J.R. & Angermeier P.L.
(1990) Fish communities as indicators of environmental
degradation. American Fisheries Society Symposium 8,
123–144.
Ferreira T., Caiola N., Casals F., Oliveira J.M. & de Sostoa
A. (2007) Assessing perturbation of river fish communities
in the Iberian Ecoregion. Fisheries Management and
Ecology 14, 519–530.
Fritsch A.J. (1872) Die Wirbeltiere Bohmens Ein Verzeichnis
aller bisher in Bohmen beobachteten Saugetiere, Vogel,
Amphibien und Fische. Archiv fur naturwissenschaftliche
Landesdurchforschung von Bohmen 2, 1–152.
Grenouillet G., Roset N., Goffaux D., Breine J., Simoens
I., de Leeuw J.J. & Kestemont P. (2007) Fish assem-
blages in European Western Highlands and Western
Plains: a type-specific approach to assess ecological
quality of running waters. Fisheries Management and
Ecology 14, 509–517.
Hughes R.M. & Oberdorff T. (1998) Application of IBI
concepts and metrics to waters outside the United States.
In: T.P. Simon (ed.) Assessment Approaches for Estimating
Biological Integrity using Fish Assemblages. Boca Raton,
FL: Lewis Press, pp. 79–83.
Hughes R.M., Howlin S. & Kaufmann P.R. (2004) A bio-
integrity index (IBI) for coldwater streams of Western
Oregon and Washington. Transactions of the American
Fisheries Society 133, 1497–1515.
Illies J. (1967) Limnofauna europaea. Eine Zusammenstellung
aller die europaischen Binnengewasser bewohnenden
mehrzelligen Tierarten mit Angaben uber ihre Verbreitung
und Okologie. Stuttgart: Gustav Fischer Verlag, 474 pp.
Illies J. (1978) Limnofauna Europaea. A Check-list of the
Animal Inhabiting European Inland Waters, with an
Account of their Distribution and Ecology. Stuttgart:
Gustav Fischer Verlag, 241 pp.
Impressgroup (2003) Analysis of Pressures and Impacts.
Guidance Document No. 3. Common Implementation
Strategy for the Water Framework Directive (2000/60/
EC). Brussels: Office for Official publications of the Euro-
pean Communities, 86 pp.
Joy M.K. & Death R.G. (2002) Predictive modelling of
freshwater fish as a biomonitoring tool in New Zealand.
Freshwater Biology 47, 2261–2275.
Karr J.R. (1981) Assessment of biotic integrity using fish
communities. Fisheries 6, 21–27.
de Leeuw J.J., Buijse A.D., Haidvogl G., Lapinska M.,
Noble R., Repecka R., Virbickas T., Wisniewolski W. &
Wolter C. (2007) Challenges in developing fish-based
ecological assessment methods for large floodplain rivers.
Fisheries Management and Ecology 14, 483–494.
Lyons J., Wang L. & Simonson T.D. (1996) Development
and validation of an index of biotic integrity for coldwater
streams in Wisconsin. North American Journal of Fisheries
Management 16, 241–256.
Magurran A.E. (2004) Measuring Biological Diversity.
Oxford: Blackwell Science, 256 pp.
Matthews W.J. (1998) Patterns in Freshwater Fish Ecology.
New York: Chapman & Hall, 756 pp.
McCormick F.H., Peck D.V. & Larsen D.P. (2000)
Comparison of geographic classification schemes for
SYNTHESIS OF FISH-BASED ASSESSMENT METHODS 379
� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd
Mid-Atlantic stream fish assemblages. Annuals of the
American Benthological Society 19, 385–404.
Mebane C.A., Maret T.R. & Hughes R.M. (2003) An index
of biological integrity (IBI) for Pacific Northwest rivers.
Transactions of theAmerican Fisheries Society 132, 239–261.
Melcher A., Schmutz S., Haidvogl G. & Moder K. (2007)
Spatially based methods to assess the ecological status of
European fish assemblage types. Fisheries Management and
Ecology 14, 453–463.
Noble R.A.A., Cowx I.G., Goffaux D. & Kestemont P.
(2007a) Assessing the health of European rivers using
functional ecological guilds of fish communities: standar-
dising species classification and approaches to metric
selection. Fisheries Management and Ecology 14, 381–392.
Noble R.A.A., Cowx I.G. & Starkie A.S. (2007b) Develop-
ment of fish-based methods for the assessment of ecolog-
ical status in English and Welsh rivers. Fisheries
Management and Ecology 14, 495–508.
Northcote T.G. (1998) Migratory behaviour of fish and its
significance to movement through riverine fish passage
facilities. In: M. Jungwirth, S. Schmutz & S. Weiss (eds)
Fish Migration and Fish Bypasses. Oxford: Fishing News
Books, Blackwell Science, pp. 3–18.
Oakes R.M., Gido K.B., Falke J.A., Olden J.D. & Brock
B.L. (2005) Modelling of stream fishes in the Great Plains,
USA. Ecology of Freshwater Fish 14, 361–374.
Oberdorff T., Pont D., Hugueny B. & Chessel D. (2001) A
probabilistic model characterizing fish assemblages of
French rivers: a framework for environmental assessment.
Freshwater Biology 46, 399–415.
Oberdorff T., Pont D., Hugueny B. & Porcher J.P. (2002)
Development and validation of a fish-based index for the
assessment of �river health� in France. Freshwater Biology
47, 1720–1734.
Peck D.V., Lazorchak J.M. & Klemm D.J. (2003) Environ-
mental Monitoring and Assessment Program-Surface
Waters: Western Pilot Study Field Operations Manual for
Wadeable Streams. Corvallis, OR: US Environmental
Protection Agency, 295 pp.
Plafkin J.L., Barbour M.T., Porter K.D., Gross S.K. &
Hughes R.M. (1989) Rapid Bioassessment Protocols for
Use in Stream and Rivers: Benthic Macroinvertebrates and
Fish. EPA/444/4-89-011. Washington, DC: US Environ-
mental Protection Agency, 150 pp.
Pont D., Hugueny B. & Oberdorff T. (2005) Modelling
habitat requirement of European fishes: do species have
similar responses to local and regional environmental
constraints? Canadian Journal of Fisheries and Aquatic
Sciences 62, 163–173.
Pont D., Hugueny B., Beier U., Goffaux D., Melcher A.,
Noble R., Rogers C., Roset N. & Schmutz S. (2006)
Assessing river biotic condition at a continental scale: a
European approach using functional metrics and fish
assemblages. Journal of Applied Ecology 43, 70–80.
Pont D., Hugueny B. & Rogers C. (2007) Development of a
fish-based index for the assessment of river health in
Europe: the European Fish Index. Fisheries Management
and Ecology 14, 427–439.
Porter M.S., Rosenfeld J. & Parkinson E.A. (2000) Predictive
models of fish species distribution in the Blackwater
drainage, British Columbia. North American Journal of
Fisheries Management 20, 349–359.
Quataert P., Breine J. & Simoens I. (2007) Evaluation of the
European Fish Index: false-positive and false-negative
error rate to detect disturbance and consistency with
alternative fish indices. Fisheries Management and Ecology
14, 465–472.
Roset N., Grenouillet G., Goffaux D., Pont D. & Kestemont
P. (2007) A review of existing fish assemblage indicators
and methodologies. Fisheries Management and Ecology 14,
393–405.
Schmutz S., Melcher A., Frangez C., Haidvogl G., Beier U.,
Bohmer J., Breine J., Simoens I., Caiola N., de Sostoa A.,
Ferreira M.T., Oliveira J., Grenouillet G., Goffaux D., de
Leuuw J.J., Noble R.A.A., Roset N. & Verbickas T. (2007)
Spatially based methods to assess the ecological status of
riverine fish assemblages in European ecoregions. Fisheries
Management and Ecology 14, 441–452.
Smogor R.A. & Angermeier P.L. (2001) Determining a
regional framework for assessing biotic integrity of
Virginia streams. Transactions of the American Fisheries
Society 130, 18–35.
Strange R.M. (1998) Historical biogeography, ecology, and
fish distributions: conceptual issues for establishing IBI
criteria. In: T. P. Simon (ed.) Assessing the Sustainability
and Biological Integrity of Water Resources Using
Fish Communities. Boca Raton, FL: CRC Press LLC,
pp. 65–78.
Virbickas T. & Kesminas V. (2007) Development of fish-
based assessment method for the ecological status of rivers
in the Baltic region. Fisheries Management and Ecology 14,
531–539.
S. SCHMUTZ ET AL.380
� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd