Fish-based methods for assessing European running waters: a synthesis

Fish-based methods for assessing Europeanrunning waters: a synthesis

S . S C H M U T Z

Department of Water, Atmosphere and Environment, Institute of Hydrobiology and Aquatic Ecosystem Management,BOKU – University of Natural Resources and Applied Life Sciences, Vienna, Austria

I . G . C O W X

Hull International Fisheries Institute, University of Hull, Hull, UK

G . H A I D V O G L

Department of Water, Athmosphere and Environment, Institute of Hydrobiology and Aquatic Ecosystem Management,

BOKU – University of Natural Resources and Applied Life Sciences, Vienna, Austria

D . P O N T

UMR CNRS Ecologie des Hydrosystemes Fluviaux, Universite Claude Bernard Lyon1, Villeurbanne, France

Abstract The European Union, Water Framework Directive (WFD) requires monitoring of riverine fish fauna.When the WFD came into force in 2000, most of the EU member states did not have fish-based assessmentmethods compliant to WFD requirements. Therefore, the objectives of FAME (http://fame.boku.ac.at), a projectunder the fifth R&D Framework Programme of the European Commission were to develop, evaluate andimplement a standardised fish-based method for assessing the ecological status of European running waters. Thispaper synthesises the outputs of FAME and defines future research needs. Two different methodologies were used:the so-called spatially based modelling and the site-specific modelling, the latter leading to the European FishIndex (EFI). The advantage of the EFI is that, despite being a single index, it is applicable to a wide range ofenvironmental conditions across Europe precluding the need for inter-calibration. The EFI will support the WFDtowards harmonised/standardised assessment and management of running waters in Europe, thus enablingcomparative analyses of the ecological status of running waters across Europe.

KEYWORDS : Europe, fish-based assessment, index of biotic integrity, Water Framework Directive.

Introduction

In the year 2000, the European Commission promul-gated the Water Framework Directive (WFD) (2000/60/EEC) to improve the environmental quality ofrivers, lakes and transitional waters. To achieve this,the WFD requires that all waters are classifiedaccording to five quality (from 1 – high status to 5 –bad status) classes depending on the degree of degra-

dation and bring all water bodies that have not beenheavily modified to at least �good ecological status�(Class 2) by the year 2015. Consequently, the WFDrequires standardised methods for assessing the eco-logical status of rivers, lakes and transitional waters,using fish amongst other biological quality elements.Within Europe, however, there was a lack of appro-priate methods when the WFD came into force, and inparticular standardised methods that could be used

Correspondence: Stefan Schmutz, Department of Water, Atmosphere and Environment, Institute of Hydrobiology and Aquatic Ecosystem

Management, BOKU – University of Natural Resources and Applied Life Sciences, Max Emanuel-Strasse 17, 1180 Vienna, Austria

(e-mail: [email protected])

Fisheries Management and Ecology, 2007, 14, 369–380

� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd doi: 10.1111/j.1365-2400.2007.00585.x

across member states of the EU. In an attempt toovercome this problem for rivers in relation to fish, theEU financed the FAME project (Development, Eval-uation and Implementation of a standardised Fish-based Assessment Method for the Ecological Status ofEuropean Rivers; http://fame.boku.ac.at) to developmethods compliant with the WFD that could be usedfor a wide range of European conditions.

This paper summarises the results of the FAMEproject presented in this dedicated issue, but specifi-cally compares the different approaches developed,discuss advantages and disadvantages, and to identifyadvances in the development of assessment methods.Major steps in method development, i.e. data require-ments, identification of reference conditions, quantify-ing human pressures, selection of metrics, indexcalibration, index validation and method implementa-tion (Fig. 1), are presented and recommendations formethod selection and implementation given. Finally,remaining limitations and future research needs areidentified.

The challenge in developing assessment methods isto distinguish between natural and human-inducedvariability in fish assemblages (Fausch, Lyons, Karr &Angermeier 1990), an issue that has been stronglyinfluenced by the concept of index of biotic integrity(IBI; Karr 1981). IBIs are based on the assumptionthat within a given entity the variability of fishcommunities is low enough to be able to distinguishbetween natural and human-induced variability.

Although IBIs have been developed worldwide (Roset,Grenouillet, Goffaux, Pont & Kestemont 2007),appropriate delineation of regions or river types is amajor issue in developing regional IBIs (Strange 1998)because of large-scale natural variability in fish com-munities.

Environmental descriptors are used to account forwithin-region variability (Roset et al. 2007). Stratifyingintraregional criteria by using only a single factordespite the great number of factors likely to controlfish communities (Matthews 1998) is clearly inade-quate to cope with natural variability. Moreover, inmost IBIs abundance-related metrics (i.e. functionalmetrics) are not adjusted at all (Oberdorff, Pont,Hugueny & Porcher 2002). For this reasons, classicalIBIs are only developed for a specific region or countryor for a given river type (e.g. coldwater streams).

Two different Europe-wide methods were devel-oped: the so-called spatially based modelling (SBM)and the site-specific modelling approach, the latterleading to the European Fish Index (EFI). The SBMcopes with the problem of natural variability bypredicting reference conditions of distinct fish assem-blage types. For each fish assemblage type an individ-ual method is developed. By contrast, the EFI ischaracterised by a single index that predicts referenceconditions for individual sites and is applicable to thewhole range of environments examined (Pont, Hugu-eny, Beier, Goffaux, Melcher, Noble, Rogers, Roset &Schmutz 2006; Pont, Hugueny & Rogers 2007). The

Fish PressureEnvironment

Basic data

Guild

Pressure criteria

Reference sites Impacted sites

Metric calculation

Metric values of reference sites

Metric values of impacted sites

Cluster analyses

Fish types

Discriminantanalysis

Predicted fish types

Discriminantanalysis

SBM status class

Regression analysis

Predicted reference metrics

Residuals calculation

EFI index

Scoring

EFI status class

Figure 1. Comparison of the spatially based (SBM) and site-specific (European Fish Index – EFI) methodologies.

S. SCHMUTZ ET AL.370

� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd

SBM approach was first applied to individual ecore-gions (SBM-ER: Ferreira, Caiola, Casals, Oliveira &Sostoa 2007; Grenouillet, Roset, Goffaux, Breine,Simoens, de Leeuw & Kestemont 2007; Noble, Cowx& Starkie 2007b; Melcher, Frangez, Haidvogl, Beier,Bohmer, Breine, Simoens, Caiola, de Sostoa, Ferreira,Oliveira, Grenouillet, Goffaux, de Leeuw, Noble,Roset & Verbickas 2007; Virbickas & Kesminas2007) and afterwards to all ecoregions at once (SBM-EU: Melcher, Schmutz, Haidvogl & Moder 2007).

Data requirements for developing Europeanmethods

A pre-requisite for developing assessment methods isthe availability of adequate data. Existing fish datacollected for various purposes were used to develop themodels. Data were collated using a dedicated database(FIDES) that covered >8000 sites in 12 Europeancountries and 17 ecoregions (Beier, Degerman, Mel-cher, Rogers & Wirlof 2007). This included humanpressure data that were used to identity referenceconditions and calibrate assessment methods. Data onnatural drivers, i.e. physiographic characteristics of therivers, were used to predict reference conditions ofdisturbed sites (Fig. 1).

Fish sampling

To avoid potential variation in the quality of fish dataarising from differences in sampling methods usedacross Europe, fish data were confined to thosecollected during the first run of electric fishing surveyscarried out according to CEN standards on electricfishing (CEN 2003; Beier et al. 2007). The onlydifference was that CEN (2003) requires a river lengthof 10–20 times the channel width to be sampled. Thiswas not always achieved, and a threshold of 100 m2 offished area was defined as the selection criterion for theEFI, to exclude sites with very low sampling effort.Sampling method and effort were included in thepredictive models of EFI to account for differences insampling technique environments (Pont et al. 2006,2007). This sampling definition was a trade-off betweendata quality and availability, and defined the frame-work for the metrics that could usefully be tested andincluded in the models (see metrics selection).

Data representativeness and quality

Irrespective of the selection criteria, data representa-tiveness varied within and among regions and coun-tries. Although Western and Northern countries were

reasonably well covered, Mediterranean and easternregions were underrepresented. In addition, only a fewvery large rivers, such as the Rhine, were included (deLeeuw, Buijse, Haidvogl, Lapinska, Noble, Repecka,Virbickas, Wisniewolski & Wolter 2007).

Species guild classification

Using ecological guilds instead of single species is oneof the core concepts in IBIs and allows humanpressures to be correlated with functional aspects oforganisms to enable the application of standardmethods across fish assemblages with varying taxo-nomic composition. Although many fish species ofEurope have been classified into ecological guilds,classifications between countries have not been har-monised. Consequently, the 123 principal Europeanfish species in the data set were classified into consis-tent ecological guilds (Noble, Cowx, Goffaux &Kestemont 2007a). The assessment methods developedused traditional guild classification, such as trophicguild, reproductive behaviour and habitat preferences,but also included guilds such as migration behaviour,longevity and population age structure (0+ vs olderfish) to improve the characterisation of essentialecological requirements.

Reference conditions and classificationof human pressure

The concept of IBIs is based on the assumption thatwithin a given entity the variability of fish communitiesis low enough to distinguish between natural andhuman-induced variability. To accomplish this, IBIsare commonly calibrated by pre-classifying humanpressure using expert judgement (Lyons, Wang &Simonson 1996) or real pressure data (Hughes, Howlin& Kaufmann 2004). Consequently, pressure data areused for two purposes: to identify reference conditionsand to calibrate indices. In FAME, reference condi-tions were developed using field data from onlyminimally or slightly impacted sites. Historical data,although considered valuable information, could notbe used at the European level because only a fewcountries (i.e. Austria, Germany, the Netherlands)provided complete data sets.

Defining human pressure

Twenty-four potential pressures to fish communitieswere identified based on lists from the Impressgroup(2003) and AQEM Consortium (2002) supplementedthrough literature on fish-pressure studies and expert

SYNTHESIS OF FISH-BASED ASSESSMENT METHODS 371

� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd

judgement (Degerman, Beier, Breine, Melcher, Quata-ert, Rogers, Roset & Simoens 2007). As monitoringprogrammes differ considerably among countries interms of parameters used and how they are assessed,expert judgement was used whenever pressure datawere lacking or inadequate. Initially, it was intended tocollect pressure data covering the whole range ofpotential types of impacts on fish, but, because oflimited availability of pressure data, only five keyphysical/chemical pressures were considered: (1)hydrology; (2) morphology; (3) nutrients; (4) toxicsand acidification at the site scale and (5) connectivity atthe river segment scale. Connectivity was used only forSBM methods as there was no significant correlationwith fish at the continental scale (Pont et al. 2006). Amean of the individual pressures was used to calibratethe fish indices. The pressure index based on the foursite pressures was poorly correlated with biologicalpressures (e.g. introduction of fish) and connectivity,demonstrating that not all human pressures are cov-ered by this index. Also computing a pressure index asa mean of four or five pressure variables implies thatall pressures contribute in the same way to globalpressure conditions and that there are no interactionsbetween individual pressures. For example, a physicalmorphology pressure of class 5 and a chemical toxicitypressure of class 1 would result in a pressure index of 3indicating moderate conditions, but it is known thatfish assemblages would be severely affected by thebad physical habitat conditions. Consequently, theresponse to individual and combined effects ofpressures should be analysed in future studies.

Analyses of pressure-specific response found onlymetrics based on omnivorous species reacted(increased) to continuum disruptions in the SBMmethods (Schmutz et al. 2007). Among the ecologicalguilds considered for the biological assessment, diad-romous and potamodromous fish species are particu-larly sensitive to continuum disruptions, as aninterruption of migrations severely affects these species(Northcote 1998). Apart from an insufficient descrip-tion of the pressure (dimensions and passage possibil-ities of the migration barrier were not considered), thismight also have been caused by missing informationabout the reference situation of diadromous species.

Defining reference conditions

According to the WFD, reference conditions may bedefined either spatially based or by modelling tech-niques using a reference network containing enoughsites of high status to provide an acceptable level ofconfidence. Reference conditions equal the high status

(class 1) of the five-tiered WFD classification schemeand are defined as no or only very minor anthropogenicalterations of the physicochemical and hydromorpho-logical conditions and no or only very minor evidence ofdistortion of the biological conditions (Annex II,WFD). Hence, both abiotic and biotic conditions haveto meet reference criteria.

In FAME, human pressures were first quantifiedand then these conditions were compared with thebiological criteria (metrics) and tested for differencesbetween unimpacted and impacted conditions. Lessthan 10% of FIDES sites represented pure referencesites, i.e. pressure variables equivalent to class 1. Thesesites mainly occurred in northern Europe (Sweden) andheadwaters, and were thus not representative for allriver types. Consequently, slightly impacted sites(Class 2) were combined with Class 1 sites to calibratethe models. This approach may seem contradictory tothe WFD reference principle, but the calibration dataset was only used to identify and predict fish assem-blage types in the spatially based models (SBM) and tocalibrate the reference models for the EFI. The finalindices used the full range of impairment gradient anddiscriminated between class 1 and 2. However, as aresult, the EFI is not able to discriminate statisticallybetween 1 and 2 (Pont et al. 2006). Nevertheless, theboundary between 2 and 3 is more important as itrepresents the limit for restoration action.

Natural variability of European fish assemblages

Fish assemblage types

The �spatially based approach� is the underlyingmethodological principle of the WFD for monitoringand managing surface waters. The concept is thatrivers can be classified into homogenous water bodies(i.e. river segments) using abiotic criteria. However,because the lower the biotic heterogeneity withinidentified types, the higher the accuracy of the IBI(Smogor & Angermeier 2001), biotic instead of abiotictypologies were used. In principle, two spatial dimen-sions, the zoogeography across Europe and the longi-tudinal pattern within each river, structure fishassemblages at the large scale. In the WFD, Illies�(1967, 1978) ecoregions are supposed to provide aspatial framework for river classification; however,they have not been evaluated for their ability todiscriminate among fish assemblages at a continentalscale. In addition, longitudinal zonation proposedmore than a century ago (Fritsch 1872), has not beenevaluated at larger spatial scales, such as ecoregions orwhole continents.

S. SCHMUTZ ET AL.372

� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd

In the SBM approach, 60 fish types were identifiedranging from two to eight types per ecoregion (Sch-mutz et al. 2007). Grouping these 60 ecoregional fishassemblage types at the European level reduced thenumber to 15 types (Melcher et al. 2007). Althoughthis represents a reduction in the number of typescompared with the abiotic classification of the WFD,the variability within those types was sufficiently low todevelop robust methods; the reason being the fishassemblages across Europe have comparable guildcomposition and are mainly structured along thelongitudinal gradient. At both the ecoregional andEuropean scales, the classical trout to bream zonationwas observed. Most environmental parameters foundto structure fish assemblages describe the longitudinalgradient of streams and rivers. About five environ-mental descriptors were used in each ecoregion (Sch-mutz et al. 2007) and seven at the European scale(Melcher et al. 2007) to discriminate among fish types(Table 1). Melcher et al. (2007) demonstrated thatthere are only five European Fish Assemblage Types –EFT – that are distinct for particular ecoregions andthat the majority (10 types) are found in severalecoregions. This indicates that Illies� ecoregions areimpractical for explaining European fish assemblagepatterns.Correct assignment of sites to fish types by discri-

minant function analyses using environmental descrip-tors ranged from 18% to 100% within ecoregions(Schmutz et al. 2007) and improved to 70% at the

European level (Melcher et al. 2007). In particular, theSBM-EU approach showed that the environmentaldescriptors used were adequate to explain and predictthe spatial variability of fish assemblages in Europe.This approach has the advantage that no a prioriexpert classification of fish assemblage types is neces-sary and that the definition and assignment of sites totypes follows a standardised procedure.

Predicting site-specific fish assemblages

Oberdorff, Pont, Hugueny & Chessel (2001) usedstatistical procedures to elaborate models that predictmetrics in relation to factors describing the naturalenvironmental variability of French rivers at both localand regional scales. This technique was used in FAMEto develop the EFI. The EFI employs 11 environmen-tal descriptors (Table 1) to predict reference conditionsfor each metric of a given site using multiple regressionmodels (Pont et al. 2006). In concordance with theSBM approach, longitudinal descriptors were prevail-ing. Only one explicit regional descriptor, the mainriver region – a river basin classification based onnative fish fauna similarities, accounted for regionaleffects.

The level of variability explained in the 10 referencemodels for each of the EFI metrics was between 31%and 48% (Pont et al. 2006). Equivalent variability (20–68%) was found by Oberdorff et al.(2002) for an indexdeveloped in France that used similar environmentalpredictors and Porter, Rosenfeld & Parkinson (2000)explained 73–90% of fish species distributions usingmap-based macrohabitat variables. Oakes, Gido,Falke, Olden & Brock (2005) correctly classifiedspecies presence/absence at a level of 81%; Pont,Hugueny & Oberdorff (2005) achieved a level of 71–92%. The higher level of explanation in these studiescompared with the EFI models might be because theEFI does not predict species absence/presence butfunctional metrics composed of several species. Theconcept of ecological guilds assumes similar functionalroles of species within a guild category. Therefore, onemight expect analogous response of guild members toenvironmental drivers. However, the results of the EFImodels show that there is still a high variability withinthe guilds that probably could be attributed to thedifferent ecological requirements of individual species.

Metric selection

Biotic indices employ biological criteria (metrics) thatrespond to human alterations of ecosystems; theselection of metrics depends on their reaction to

Table 1. Environmental descriptors used in FAME methods

Environmental descriptors SBM-ecoregion SBM-EU EFI

Main river region · ·Longitude ·Latitude ·Mean air temperature · · ·Mean air temperature in July ·Mean air temperature in January ·Geology · ·Altitude · · ·Slope · · ·Distance from source · · ·Wetted width · · ·Size of catchment · ·Stream order ·Conductivity ·Average depth ·Huet�s zonation ·Sampling strategy ·Sampling method ·Sampled area ·

SBM: spatially based models, EFI: European Fish Index.

SYNTHESIS OF FISH-BASED ASSESSMENT METHODS 373

� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd

human pressures (Karr 1981). The WFD requires theassessment of species composition, abundance, sensi-tive species and population structure.

Metrics are often selected on the basis of expertjudgment, without checking for empirical relationsbetween metrics and environmental variables (Hughes& Oberdorff 1998; Angermeier, Smogor & Stauffer2000; McCormick, Peck & Larsen 2000). However,Angermeier et al. (2000) advised against building IBIsonly with metrics exhibiting strong empirical relations,because such indices may lack the ecological founda-tion needed for detection of a broad range of humanimpacts. Mebane, Maret & Hughes (2003) suggestedthat both an ecological foundation and empiricalrelationship should underlie the construction of IBIs.Redundant metrics should also be excluded (Mebaneet al. 2003; Hughes et al. 2004).

In FAME, a multiple-step selection procedure wasapplied to ensure that only metrics responsive tohuman pressures were selected for the assessmentmethods. In the SBM, correlation analyses were usedto pre-select metrics. Final metrics were chosen bydiscriminant function analyses for each fish assemblagetype using the five-tiered global pressure index (Mel-cher et al. 2007; Schmutz et al. 2007). For the EFI,only metrics for which reference values could bepredicted by environmental descriptors were pre-selected and only metrics discriminating betweendifferent levels of human pressures were selected (Pontet al. 2006).

The main difference in types of metrics selectedbetween the FAME methods was that the SBM-ERmainly employed sentinel metrics that are based onindividual species, whereby SBM-EU and EFI mainlyor solely used functional metrics (Fig. 2). The largerthe scale that the methods were designed for (spatial

gradient from SBM-EU to SBM-ER and EFI), themore general the characteristics of the fish assem-blages, i.e. functional metrics, that were employed toovercome taxonomic differences among regions. Sen-tinel species metrics are specific to individual fishassemblage types and are known to react to humanpressure in a predictable way but might not be presentor react in a different way in other types or ecoregions.

Species richness is a classical metric in IBIs (Nobleet al. 2007a, b) and, according to the biodiversityconcept one of the most important metrics to assessecosystem degradation (Magurran 2004). However,species richness plays a minor role in all FAMEmethods. Nevertheless, most FAME methods achieveda fairly high level of accuracy without species richnessmetrics.

Migratory species metrics are new metrics of impor-tance to reflect continuum disruptions. Althoughneither SBM-EU nor EFI used continuum disruptionin the global pressure index to test metrics reaction,both methods employed diadromous and potamodro-mous species guilds, proving the general sensitivity ofmigratory species to human disturbance.

Longevity is an important species trait for bioas-sessment (Plafkin, Barbour, Porter, Gross & Hughes1989; Doledec, Statzner & Bournard 1999) and wasalso tested in FAME by distinguishing between long-and short-living species (Noble et al. 2007a, b), but theresponses were not clear. Longevity metrics were notselected for the EFI and in the SBM-EU the responseof short- and long-living species was inconsistentamong fish assemblage types.

Different metric modalities (number of species,percentage of species, density, relative density, bio-mass, etc.) were tested (Noble et al. 2007a). Densityand number of species were the metric modalities most

25

Historical metrics

Longevity

Migration

Overall composition

ToleranceFeeding

HabitatReproduction

Sentinel

%0 5 10 15 20

Figure 2. Comparison of metric types selected for different FAME

methods: h SBM-ER, spatially based methods at ecoregional level;

SBM-EU, spatially based methods at European level; EFI, European

Fish Index.

60

0+

Biomass

N species

Density

%0 10 20 30 40 50

Figure 3. Comparison of metric variants selected for different FAME

methods: h SBM-ER, spatially based methods at ecoregional level;

SBM-EU, spatially based methods at European level; EFI, European

Fish Index.

S. SCHMUTZ ET AL.374

� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd

often employed (Fig. 3). An estimation of density, interms of catch-per-unit-effort (single pass) (Beier et al.2007), seems to be essential to achieve a sufficient levelof accuracy. Data for metrics describing the absence/presence or density of 0+ (young of the year) werestrongly limited. These metrics were only selected inSBM-ER, where sufficient data were available.

Index computation and scoring

Scoring of metrics and indices involves basic assump-tions affecting IBI behaviour and results. The classicaltri-section method proposed by Karr (1981) splits therange of metric values in three equal parts assuming alinear response of fish to human pressures. In FAME,scoring the indices was based on the pre-classificationof pressure presuming that the thresholds defined forpressure classes were the same as for ecological statusclasses. Both approaches lack profound biologicalfoundation. In future, more effort should concentrateon defining thresholds based on biological criteria suchas irreversibility, recovery time and extinction risk.Traditionally, indices are computed by summing up

individual metric scores assuming equal contributionof individual metrics to index score (Karr 1981;Bramblett, Johnson, Zale & Heggem 2005). The EFIwas computed in this way as the sum of 10 metrics(Pont et al. 2007). The SBM approach used discrimi-nant function analyses to model and predict statusclass membership of any given site (Melcher et al.2007; Schmutz et al. 2007). Discriminant functionanalyses have also been used in other studies to findpressure variables that best differentiate among IBIclasses (e.g. Joy & Death 2002). The advantage of thisapproach is that metric combinations are selected thathave the highest probability in explaining biologicalresponses to human pressures. Metrics are weighted inthe discriminant function according to their individualcontribution to the overall pressure–index relationship.The disadvantage of discriminant function analysis isthe contribution of individual metrics is �hidden� in thediscriminant functions of the model because of itsmulti-dimensional nature.

Index accuracy

Uncertainties arise at all levels of IBI development:field sampling, guild classification, pressure quantifi-cation, definition of reference conditions, metricsselection, index computation and scoring. Tacklingthe entire potential source of uncertainty would requireerror estimates for all types of uncertainty sources andideally computing a cumulative error estimate. To

date, error estimates were only performed for distinctsteps of IBI development, e.g. sampling intensity(Peck, Lazorchak & Klemm 2003) or cross-validationof index with human pressure (Hughes et al. 2004;Bramblett et al. 2005).

A precise IBI should be insensitive to natural spatialvariability. The natural spatial variability was explic-itly integrated into the development of the FAMEmethods. Being able to predict EFTs with environ-mental descriptors using the SBM-EU approach dem-onstrated that among variability of EFTs is higherthan within variability. Therefore, it can be assumedthat natural variation within EFTs has marginal effecton the index results (Melcher et al. 2007). Naturalvariability was integrated into the EFI by establishingstatistical models describing metric responses to natu-ral environmental variability in the absence of anysignificant human disturbance.

As only a limited number of sites with repeatsamples or time series were available, the effects oftemporal variability on index stability could not betested explicitly. However, temporal variability isinherently included in the data set because the samplescover about 20 yr in the various ecoregions.

Method accuracy was evaluated by cross-validatingecological status with global pressure status (Quataert,Breine & Simoens 2007). This method identifies bothtype I errors (unimpacted sites that were misclassifiedas impacted sites) and type II errors (impacted sitesthat were classified as unimpacted). In a preciseassessment method, both types of error should beequal and low. Accuracy of SBM-ER methods basedon cross-validation with pre-classification variedbetween 47% and 98% (mean 81%) when contrastingcalibration data set (classes 1 and 2) with degradedsites (classes 3, 4 and 5) (Schmutz et al. 2007). For theSBM-EU approach, on average, 85% of all sitescoincided with the pressure pre-classification (Melcheret al. 2007). The EFI correctly classified 80% ofreference and disturbed sites (Pont et al. 2007). AllFAME methods were about the same level of accuracy(Fig. 4) but type II errors were generally higher. Mostregional methods have similar accuracy to the FAMEmethods (Quataert et al. 2007).

Index validation with independent data

Cross-validation through comparison of the IBI withan independent estimate of the ecological status can beused to evaluate the error of assessment methods, butthe problem of circularity arises when the samepressure variables are used for calibration and valida-tion. Using pressure variables not employed for

SYNTHESIS OF FISH-BASED ASSESSMENT METHODS 375

� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd

calibration might be a solution to avoid circularity, butpressure variables are usually inter-related. A moreappropriate approach is to validate IBIs with anindependent data set not used for its development. InFAME, depending on the method developed, severalvalidation approaches were applied. In the SBM-ERapproach, splitting the database into 60 fish typeshampered method validation with independent databecause too few samples remained. For the SBM-EUapproach, the data set was randomly split into two andused for model development and validation respec-tively (Melcher et al. 2007). For the EFI, the referencedata set was divided randomly into three subsets: one

set for model calibration, one for validation of metric-specific models, and one for the final index validation(Pont et al. 2006).

End-user perspective

Any method developed should not only be scientifi-cally sound but also be cost efficient, transferable todifferent river basins and regional conditions and easyto use. Throughout the development of the methodol-ogies, specific collaboration was sought with agencies(Applied Partners) responsible for the implementationof the WFD, to collect data, seek feedback on theapplicability of the methods, use of software anddissemination of outputs (Backx 2004).

A major concern of end-users was the potential highcosts of sampling associated with the methods. Thetotal budget for monitoring fish in rivers variedconsiderably between countries because of differencesin the number of expected samples, the number of sitessampled per day and cost of equipment (fixed costs)and personnel (variable costs) (Table 2; Backx 2004).A large proportion of the variation in costs arisesbecause of the different sampling strategies used. Somecountries use multiple-run depletion sampling, whereasothers perform only single-run sampling. This is alsotrue for collecting data on fish length and weight.

Applied Partners recommended that the CEN stan-dard on electric fishing (CEN 2003) should be evalu-ated with respect to sampling costs and efficiencyrelated to the EFI metrics and eventually should berevised. More analyses on spatial and temporal vari-ability of fish data are necessary. Information ondiadromous species which is underrepresented inelectric fishing samples should also be considered forfish-based assessment. There are still problems sam-

0

10

20

30

40

50

60

70

80

90

Existingmethods

%

Type I errorCorrectly classifiedType II error

EFI SBM-EU SBM-ER

Figure 4. Comparison of method accuracy (cross-validation with

pressure pre-classification) of FAME methods with existing regional

methods. SBM-ER, spatially based methods at ecoregional level; SBM-

EU, spatially based methods at European level; EFI, European Fish

Index.

Table 2. Expected number of electric fishing sampling sites per country and per 1000 km2 area for the WFD monitoring programme,

the maximum number of sites sampled per day and the sampling costs (adapted from Backx 2004)

Country

Total number

of samples

Samples per

1000 km2

Samples

per day

Fixed

costs (€)

Variable

costs (€)

Costs per

sample (€)

Austria 400 4.8 2 4750 96 000 1679

Belgium Flanders 200 14.8 5 14 500 60 500 375

France 675 1.2 2 41 700 97 900 2945

Germany 1250

Greece 180 1.4 2 59 000 5000 1837

Lithuania 80 1.2 3 2030 5820 1000

Poland 540 1.7 3 25 200 55 000 149

Portugal 80 0.9 4600 61 425 943

Sweden 252 0.6 2 5975 251 700 512

The Netherlands 120 2.9 15 17 242 151 58 520

UK 1333 5.4 2 821

Mean (median) 226 1.5 2 14 500 60 500 943

S. SCHMUTZ ET AL.376

� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd

pling large rivers in an efficient and cost-effectivemanner (de Leeuw et al. 2007). Furthermore, guide-lines are needed on how to sample catchments andwater bodies representative of the WFD. Finally,although the project provided easy to handle, publicdomain software for the EFI (FAME Consortium2004) enabling all potential end-users to use themethod, interpretation requires expertise in fish-basedassessment methods.End-user feedback was used to select the final

method for supporting the WFD based on a numberof criteria (Table 3). No relevant differences in accu-racy were found between FAME and existing regionalmethods (ERM) where they exist, but many countrieshad not developed methods or assessment was basedmainly on expert judgement that lack scientific foun-dation and statistical robustness, and therefore sufferspoor objectivity and reliability. However, as ERMsmainly follow the classical IBI approach they are moretraceable than model-based FAME methods.Although different in their conceptual framework, allmethods seem to be equally simple in their practicalapplicability, but, as most ERMs were developedbefore the WFD came into force, they are not fullyin accordance with the WFD (reference approach,types of metrics). Nevertheless, the FAME methodsalso need to be further developed (e.g. integration ofpopulation age-structure) to be fully compliant withWFD requirements. Although there was no thoroughassessment of cost efficiency, the EFI and SBM-EU areno more expensive than the ERMs, as both requireonly single-pass electric fishing data, whereas for someERMs quantitative data are necessary. Furthermore,ERMs have limited spatial applicability and need

intercalibration to produce comparable results. Theonly method to use the same criteria and standardsacross Europe is the EFI; thus, given the similar levelsof accuracy between methods, the great objectivity ofstatistical methods and the advantage of standardisedmethods, the EFI was the preferred method amongstend-users.

Conclusions

The FAME project produced assessment methods atthe regional and European scales. Spatially based (fishassemblage type-specific) and site-specific methods(EFI) were developed and compared, with the EFIrecommended for the WFD assessment. The advanta-ges of the EFI include:1 applicability for a wide range of environmentalconditions across Europe;2 representative of a single index that does not requireintercalibration among countries;3 precision is comparable with spatially based andERMs;4 incorporation of natural variability in the index bypredicting reference metrics values for individual sites;5 standardised assessment of impacted sites based onstatistical evidence of deviation from unimpactedconditions;6 calibration against an independent assessment ofpressure;7 only requires low-cost single-pass electric fishingsampling;8 easy to use with simple public domain softwareimplemented in MS Excel�.

The EFI represents a major step towards a standar-dised fish-based assessment of European runningwaters. However, there are still limitations that needattention.

Extending the EFI to regions and environments notadequately covered in FAME

Three particular regions were notably weak in theFAME programme:• Mediterranean river systems, which are character-ised by species-poor but very diverse communities thatrequire better definition of the natural environmentalvariability of the rivers, specific species guilds andmetrics adapted to the particular type of pressure andconsideration of the presence of a large number ofalien species.• Large floodplain rivers, which have high diversity ofwater bodies with respect to morphology and fishfauna. Large mid-channel sections and temporarily

Table 3. Comparison of the European Fish Index (EFI) (Pont et al.

2007) with the spatially based method developed at the European

level (SBM-EU) (Melcher et al. 2007) and existing regional methods

from the end-users perspective

Criteria EFI SBM-EU

Existing

methods

Accuracy + + +

Scientific foundation + + ±

Objectiveness + + ±

Reliability + + ±

Traceability (black box) ± ± +

Simplicity of use + + +

WFD accordance ± ± )Cost efficiency + + ±

Standardised applicability

(ecoregions, river types, pressures)

+ ± )

European standard + ± )

+, strong; ±, medium; ), low significance.

SYNTHESIS OF FISH-BASED ASSESSMENT METHODS 377

� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd

connected floodplain water bodies form an integralpart of these water systems and the complex fishcommunities require several complementary samplingmethods and indices to make robust assessment.• Central/Eastern Rivers, e.g. the River Danube,which have high species diversity and endemism, thatneed to be considered when extending the EFI toeastern countries and predicting reference conditions.

Improving the capacity of the EFI to detectpressures

To improve the response of the EFI to hydromor-phological pressures, more precise pressure data areneeded. It is necessary to distinguish more accuratelybetween different types or pressures, e.g. to splitmorphological alteration into effects of impoundmentand channelisation as both alter the morphologicalconditions but result in different biological responses.Furthermore, most European rivers are impacted bymultiple pressure types, and biological assessmentshould help to identify which pressures affect fish themost to prioritise restoration measures. Thus, a moreprecise classification of pressure types (e.g. minimumflow), a better quantification of pressure variables (e.g.ratio of observed minimum flow to natural minimumflow) and links to drivers acting on these pressures(e.g. hydropower production) are needed.

Improving the ecological guild classification ofEuropean fish species

The ecological requirements of European fish species arereasonably well known but about half the speciescovered by the FAME project could not be classifiedbecause of missing information. These are mainly rareand/or endemic species with restricted distribution, butsometimes represent more sensitive species groups.Research on the ecological requirements of these speciesis necessary to fill existing gaps and to improve assess-ment methods. Additional guilds should also be testedfor their ability to reflect human disturbance. Forexample, temperature preferences of species could beused to detect thermal disturbances. A more preciseclassification of guilds such as a distinction betweenlithophilic and litho-obligate reproductive guilds mightreflect substrate alterations caused by channelisation orother pressures. Similarly, the effects of continuitydisruptions may be better elucidated by historic, pres-ence/absence information on diadromous species that isusually readily available.More detailed analyses are alsoneeded to evaluate the significance of longevity inbioassessment. For example, combinations of longevity

with other ecological requirements such as tolerance orhabitat requirements might be promising. Populationstructuremetrics are required by theWFD andmight bevery important where the response of the fish species ismore subtle. This is particularly the case in streams andrivers with very low species diversity (one or two species)where �functional community metrics� fail to indicatehuman pressure.

Testing new environmental descriptors for referencemodels

New environmental descriptors should be tested toimprove reference models, e.g. precipitation, streampower, neighbourhood relationship and landscapevariables. GIS-derived Europe-wide catchment datamight provide a useful and cost-effective source ofstandardised data in future approaches.

Testing new metrics

Further efforts are necessary to quantify continuumdisruptions at different spatial scales better and torelate these disturbances to migratory requirements offish species. As diadromous species are extinct in manyEuropean rivers, historical information could be usedto reconstruct former occurrences.

Field sampling and representative monitoring

Further tests are necessary for standardisation andoptimisation of field sampling to ensure cost-efficiencyand consistency at the European scale. Standardisedsampling protocols have to be developed for largefloodplain rivers. Analysing spatial and temporalstability of the EFI should give an indication ofmonitoring requirements to guarantee representativeassessments of catchments. Tools for up-scaling site-specific assessment results to entire water bodies andcatchments have to be developed.

Developing tools for prioritising restorationmeasures

The EFI and future versions will support harmonisedassessment and management of running waters inEurope. Providing a standardised assessment tool willenable comparative analyses of the ecological status ofEuropean rivers. Restoration and rehabilitation mea-sures can be evaluated between countries to assess ifgood ecological status has been achieved. Therefore,the EFI could become an important tool to define andprioritise restoration measures.

S. SCHMUTZ ET AL.378

� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd

Acknowledgements

We would like to thank the European Commission forfunding the FAME project (EVK1-CT-2001-00094).Special thanks are due to the many institutions inEurope providing data for the FAME project.

References

Angermeier P.L., Smogor R.A. & Stauffer J.R. (2000)

Regional frameworks and candidate metrics for assessing

biotic integrity in mid-Atlantic highland streams.

Transactions of the American Fisheries Society 129, 962–

981.

AQEM Consortium (2002) AQEM Software. Available at:

http://www.aqem.de.

Backx J. (2004) Transfer into water management. Report of

the Applied Partners WP11/D18 – R&D project FAME

EVK1-CT-2001-00094. Available at: http://fame.boku.

ac.at/downloads/D18_AppliedPartners_TransferWM.pdf.

Beier U., Degerman E., Melcher A., Rogers C. & Wirlof H.

(2007) Processes of collating a European fisheries database

to meet the objectives of the European Union Water

Framework Directive. Fisheries Management and Ecology

14, 407–416.

Bramblett R.G., Johnson T.R., Zale A.V. & Heggem D.G.

(2005) Development and evaluation of a fish assemblage

index of biotic integrity for northwestern Great Plains

streams. Transactions of the American Fisheries Society

134, 624–640.

CEN (2003) Water quality – sampling of fish with electricity.

European standard – EN 14011:2003. Brussels: European

Committee for Standardization, 18 pp.

Degerman E., Beier U., Breine J., Melcher A., Quataert P.,

Rogers C., Roset N. & Simoens I. (2007) Classification and

assessment of degradation in European running waters.

Fisheries Management and Ecology 14, 417–426.

Doledec S., Statzner B. & Bournard M. (1999) Species traits

for future biomonitoring across ecoregions: patterns

along a human-impact river. Freshwater Biology 41, 737–

758.

FAME Consortium (2004) Development, Evaluation and

Implementation of a standardised Fish-based Assessment

Method for the Ecological Status of European Rivers

(FAME). Available at: http://fame.boku.ac.at.

Fausch K.D., Lyons J., Karr J.R. & Angermeier P.L.

(1990) Fish communities as indicators of environmental

degradation. American Fisheries Society Symposium 8,

123–144.

Ferreira T., Caiola N., Casals F., Oliveira J.M. & de Sostoa

A. (2007) Assessing perturbation of river fish communities

in the Iberian Ecoregion. Fisheries Management and

Ecology 14, 519–530.

Fritsch A.J. (1872) Die Wirbeltiere Bohmens Ein Verzeichnis

aller bisher in Bohmen beobachteten Saugetiere, Vogel,

Amphibien und Fische. Archiv fur naturwissenschaftliche

Landesdurchforschung von Bohmen 2, 1–152.

Grenouillet G., Roset N., Goffaux D., Breine J., Simoens

I., de Leeuw J.J. & Kestemont P. (2007) Fish assem-

blages in European Western Highlands and Western

Plains: a type-specific approach to assess ecological

quality of running waters. Fisheries Management and

Ecology 14, 509–517.

Hughes R.M. & Oberdorff T. (1998) Application of IBI

concepts and metrics to waters outside the United States.

In: T.P. Simon (ed.) Assessment Approaches for Estimating

Biological Integrity using Fish Assemblages. Boca Raton,

FL: Lewis Press, pp. 79–83.

Hughes R.M., Howlin S. & Kaufmann P.R. (2004) A bio-

integrity index (IBI) for coldwater streams of Western

Oregon and Washington. Transactions of the American

Fisheries Society 133, 1497–1515.

Illies J. (1967) Limnofauna europaea. Eine Zusammenstellung

aller die europaischen Binnengewasser bewohnenden

mehrzelligen Tierarten mit Angaben uber ihre Verbreitung

und Okologie. Stuttgart: Gustav Fischer Verlag, 474 pp.

Illies J. (1978) Limnofauna Europaea. A Check-list of the

Animal Inhabiting European Inland Waters, with an

Account of their Distribution and Ecology. Stuttgart:

Gustav Fischer Verlag, 241 pp.

Impressgroup (2003) Analysis of Pressures and Impacts.

Guidance Document No. 3. Common Implementation

Strategy for the Water Framework Directive (2000/60/

EC). Brussels: Office for Official publications of the Euro-

pean Communities, 86 pp.

Joy M.K. & Death R.G. (2002) Predictive modelling of

freshwater fish as a biomonitoring tool in New Zealand.

Freshwater Biology 47, 2261–2275.

Karr J.R. (1981) Assessment of biotic integrity using fish

communities. Fisheries 6, 21–27.

de Leeuw J.J., Buijse A.D., Haidvogl G., Lapinska M.,

Noble R., Repecka R., Virbickas T., Wisniewolski W. &

Wolter C. (2007) Challenges in developing fish-based

ecological assessment methods for large floodplain rivers.

Fisheries Management and Ecology 14, 483–494.

Lyons J., Wang L. & Simonson T.D. (1996) Development

and validation of an index of biotic integrity for coldwater

streams in Wisconsin. North American Journal of Fisheries

Management 16, 241–256.

Magurran A.E. (2004) Measuring Biological Diversity.

Oxford: Blackwell Science, 256 pp.

Matthews W.J. (1998) Patterns in Freshwater Fish Ecology.

New York: Chapman & Hall, 756 pp.

McCormick F.H., Peck D.V. & Larsen D.P. (2000)

Comparison of geographic classification schemes for

SYNTHESIS OF FISH-BASED ASSESSMENT METHODS 379

� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd

Mid-Atlantic stream fish assemblages. Annuals of the

American Benthological Society 19, 385–404.

Mebane C.A., Maret T.R. & Hughes R.M. (2003) An index

of biological integrity (IBI) for Pacific Northwest rivers.

Transactions of theAmerican Fisheries Society 132, 239–261.

Melcher A., Schmutz S., Haidvogl G. & Moder K. (2007)

Spatially based methods to assess the ecological status of

European fish assemblage types. Fisheries Management and

Ecology 14, 453–463.

Noble R.A.A., Cowx I.G., Goffaux D. & Kestemont P.

(2007a) Assessing the health of European rivers using

functional ecological guilds of fish communities: standar-

dising species classification and approaches to metric

selection. Fisheries Management and Ecology 14, 381–392.

Noble R.A.A., Cowx I.G. & Starkie A.S. (2007b) Develop-

ment of fish-based methods for the assessment of ecolog-

ical status in English and Welsh rivers. Fisheries

Management and Ecology 14, 495–508.

Northcote T.G. (1998) Migratory behaviour of fish and its

significance to movement through riverine fish passage

facilities. In: M. Jungwirth, S. Schmutz & S. Weiss (eds)

Fish Migration and Fish Bypasses. Oxford: Fishing News

Books, Blackwell Science, pp. 3–18.

Oakes R.M., Gido K.B., Falke J.A., Olden J.D. & Brock

B.L. (2005) Modelling of stream fishes in the Great Plains,

USA. Ecology of Freshwater Fish 14, 361–374.

Oberdorff T., Pont D., Hugueny B. & Chessel D. (2001) A

probabilistic model characterizing fish assemblages of

French rivers: a framework for environmental assessment.

Freshwater Biology 46, 399–415.

Oberdorff T., Pont D., Hugueny B. & Porcher J.P. (2002)

Development and validation of a fish-based index for the

assessment of �river health� in France. Freshwater Biology

47, 1720–1734.

Peck D.V., Lazorchak J.M. & Klemm D.J. (2003) Environ-

mental Monitoring and Assessment Program-Surface

Waters: Western Pilot Study Field Operations Manual for

Wadeable Streams. Corvallis, OR: US Environmental

Protection Agency, 295 pp.

Plafkin J.L., Barbour M.T., Porter K.D., Gross S.K. &

Hughes R.M. (1989) Rapid Bioassessment Protocols for

Use in Stream and Rivers: Benthic Macroinvertebrates and

Fish. EPA/444/4-89-011. Washington, DC: US Environ-

mental Protection Agency, 150 pp.

Pont D., Hugueny B. & Oberdorff T. (2005) Modelling

habitat requirement of European fishes: do species have

similar responses to local and regional environmental

constraints? Canadian Journal of Fisheries and Aquatic

Sciences 62, 163–173.

Pont D., Hugueny B., Beier U., Goffaux D., Melcher A.,

Noble R., Rogers C., Roset N. & Schmutz S. (2006)

Assessing river biotic condition at a continental scale: a

European approach using functional metrics and fish

assemblages. Journal of Applied Ecology 43, 70–80.

Pont D., Hugueny B. & Rogers C. (2007) Development of a

fish-based index for the assessment of river health in

Europe: the European Fish Index. Fisheries Management

and Ecology 14, 427–439.

Porter M.S., Rosenfeld J. & Parkinson E.A. (2000) Predictive

models of fish species distribution in the Blackwater

drainage, British Columbia. North American Journal of

Fisheries Management 20, 349–359.

Quataert P., Breine J. & Simoens I. (2007) Evaluation of the

European Fish Index: false-positive and false-negative

error rate to detect disturbance and consistency with

alternative fish indices. Fisheries Management and Ecology

14, 465–472.

Roset N., Grenouillet G., Goffaux D., Pont D. & Kestemont

P. (2007) A review of existing fish assemblage indicators

and methodologies. Fisheries Management and Ecology 14,

393–405.

Schmutz S., Melcher A., Frangez C., Haidvogl G., Beier U.,

Bohmer J., Breine J., Simoens I., Caiola N., de Sostoa A.,

Ferreira M.T., Oliveira J., Grenouillet G., Goffaux D., de

Leuuw J.J., Noble R.A.A., Roset N. & Verbickas T. (2007)

Spatially based methods to assess the ecological status of

riverine fish assemblages in European ecoregions. Fisheries

Management and Ecology 14, 441–452.

Smogor R.A. & Angermeier P.L. (2001) Determining a

regional framework for assessing biotic integrity of

Virginia streams. Transactions of the American Fisheries

Society 130, 18–35.

Strange R.M. (1998) Historical biogeography, ecology, and

fish distributions: conceptual issues for establishing IBI

criteria. In: T. P. Simon (ed.) Assessing the Sustainability

and Biological Integrity of Water Resources Using

Fish Communities. Boca Raton, FL: CRC Press LLC,

pp. 65–78.

Virbickas T. & Kesminas V. (2007) Development of fish-

based assessment method for the ecological status of rivers

in the Baltic region. Fisheries Management and Ecology 14,

531–539.

S. SCHMUTZ ET AL.380

� 2007 The Authors. Journal compilation � 2007 Blackwell Publishing Ltd