National Water Quality Monitoring Council Methods and Data Comparability Board Report to Advisory Committee on Water Information September 10, 2003.

National Water Quality Monitoring Council

Methods and Data Comparability Board

Report to Advisory Committee on Water

Information

September 10, 2003

How we work…caught in the act!

Mission of the Board

Create a framework for collaboration and comparability among programs by...

identifying, examining, and recommending monitoring approaches that facilitate collaboration and yield comparable data and assessment results

Accomplishments “across the Board”

Provided an opportunity to bridge

the gap between water treatment

for human consumption and

natural water resources, as well as

the gap between human and

ecological health.

Leveraged the support of individual programs in the federal, state, and private sectors. For example NEMI provides the ability to compare methods which allows the selection of methods based on DQOs

Because of its interdisciplinary nature the Methods Board has…

Challenges “across the Board”

Members are volunteers – not always able to

participate or respond in a timely manner

Need for ground rules on how to operate under consensus

Limited

resources--

of all kinds

Developing and

implementing outreach and communication strategies

Topics to be Discussed

• Update on Methods Board Activities• Request that AWCI ask Federal Agencies and others to implement previously adopted recommendations

– Accreditation of federal laboratories– NEMI

• Water Quality Data Elements (Jerry Diamond)• On-line demonstration of NEMI (Larry Keith, Herb Brass)• Discuss expert systems including NEMI-CBR (Larry Keith)

Getting to comparability

– The 4 elements are the steps or building blocks moving us toward the goal of comparability.

– Each of the Board’s workgroups is focused on one or more of these elements or steps

– Effective and innovative outreach is an overarching need for each workgroup

DQOs & MQOs

Field Performanc

e

Lab Performanc

e

Data Reporting

NEMI * PBMS * Accreditation * WQDE * Biology * Nutrients*New

Technologies

Where are we going and how are we going to get there...

develop and deliver products in the short term

while thinking and planning strategically in the long

term...DQOs & MQOs

Field Performanc

e

Lab Performanc

e

Data Reporting

Accreditation Workgroup

Workgroup Co-ChairsBart Simmons, CA Department of Toxic Substance Control

Merle Shockey, USGS

Why Accreditation?

There has been the notion that “following the method” ensures accurate data. A method is simply one key component of generating reliable data. Consider an analogy…two chefs, in two kitchens, using the same recipe…

The same recipe in the hands of an inexperienced cook with less-than-terrific equipment is

a riskier proposition.

In the hands of a skilled,

experienced cook, using

fresh ingredients and with

all the right equipment, a

wonderful outcome will

result.

Both cooks, however, may be following the same

“method”.

Accreditation Workgroup accomplishments

– White Paper on the value of accreditation

– Issue Paper on the need for federal lab accreditation

– Independent coordination by Board members with INELA, NELAC, and ACWI

– Diverse representation on the workgroup was key to achieving consensus on recommendations

Previously ACWI Adopted

Recommendation Accreditation of Federal Laboratories

1All federal agencies (and commercial laboratories employed by federal agencies) performing analytical water testing, as part of compliance or ambient monitoring programs, be accredited under a recognized program, in order to better establish comparability of data and to meet the needs of specific federal agency programs. Each agencyshould evaluate the cost of implementing this recommendation as it applies to their individualsituation.

Previously ACWI Adopted

Recommendation Accreditation of Federal Laboratories

2The National Environmental Laboratory Accreditation Program (NELAP) is the Board’s recommended program, because NELAP adequately meets (or is taking measures that meet) the broad needs of the majority of federal laboratories performing water testing. Specifically, it is focused on uniform accreditation requirements across states (and therefore, potentially reduces accreditation costs for labs operating in several states), and allows Federal as well as state accrediting authorities.

Previously Adopted Recommendation Accreditation of Federal Laboratories

2For NELAP to serve as a satisfactory accrediting program for federal laboratories, NELAP needs to continue its efforts to:

• Obtain more state participation and reciprocity

• Address standards for ambient monitoring, field sample collection, and field measurements

• Promote the development of PBS implementation

Previously Adopted Recommendation Accreditation of Federal Laboratories

3The MDCB (and its parent organization, the NWQMC) will periodically re-evaluate NELAP’s suitability to serve as a national accreditation program in order: (1) to review the status of their progress in the aforementioned efforts, and (2) to encourage state, federal, and private participation in NELAP

Where are we going?

• Develop position paper on State laboratory accreditation - focus on issues/concerns• Develop position paper and approach to implementing field accreditation•Independent interaction by Board members with Institute for National Environmental Laboratory Accreditation (INELA) and NELAC – adopt approaches and recommendations brought forward by MDCB members• Presentations to INELA, NELAC, and ACWI at their upcoming meetings

National Environmental Methods Index (NEMI) Workgroup

Workgroup Co-ChairsDan Sullivan, USGSCo-Chair, Vacant

NEMI and NEMI-CBR

The National Environmental Methods Index (NEMI) is a database of methods applicable for monitoring water for chemical and microbiological pollutants.

– Online searches at www.nemi.gov– Endorsed by ACWI in 2001

NEMI-CBR is a database of methods applicable for anti-terrorism use with chemical, biological, and radiological (CBR) agents.

– Password protected secure database uses NEMI algorithms to save time and money.

Expert systems have been developed to help support the use of both databases.

ACWI Endorsement

The Advisory Committee on Water Information (ACWI) endorses the continued development and timely delivery of NEMI as a vital tool to enhance the generation of comparable data of known quality, across all entities that conduct water quality monitoring. Use of NEMI will assist in the design of water quality monitoring programs, so that data quality objectives and measurement quality objectives are more readily achieved.

Endorsement on May 16, 2001

NEMI accomplishments

• Public release October, 2002 – announced by joint USGS/USEPA letter

• Over 40,000 visits since public release• 600+ Methods Currently in NEMI

235 EPA methods 149 USGS methods 32 DOE radiochemical methods 75 ASTM methods 59 Standard Methods methods 8 AOAC methods 43 private sector methods

Where to Find NEMI

www.nemi.gov

Where are we going?

• Maintenance and upgrades to data base• Contributions of chemical, microbiological and radiochemical methods by external parties using new online forms (150 in parking lot)– focus on new and improved methods• Developing an approach to add field sampling methods and forming a Workgroup• Developed business rules and adding a number of biological methods

Where are we going?

• Adding water security methods – partially password protected• Adding remaining regulatory methods for drinking water and waste water (EMMI)• Support expert system development (EMMA) using CRADA and homeland security funds (WATER)• Methods for media in addition to water – resources to be provided by “non-water” programs – formation of external Steering Committee

NEMI Field Sampling Methods

• Form Workgroup to address identified methods types

– Routine water collection methods– Routine water quality measurements– Biological– New technologies– Microbiology– Sediment– Geomorphology/habitat

NEMI Field Sampling Methods

• Organizations to include in Workgroup– Federal Agencies (USGS, EPA, NOAA, FWS)– States (FL, NJ, others)– Volunteers (PA, others)– Tribes– ASTM– Standard Methods– Professional organizations (NABS, WEF,

AWWA, fisheries, others)– International (Environment Canada, British

Geological Survey, others)– Private sector– Academia– Others -- suggestions?

NEMI CBR

Create a central system for locating, evaluating, and retrieving analytical methods for chemical, biological, and radiochemical warfare agents in one federally managed location.• Allows EPA Water Protection Task Force to meet immediate homeland security needs• Costs minimized since NEMI database framework already developed• Password protected site

Water Protection Objectives

• Methods that provide highly selective identification (low false positives and

negatives) of target analytes as rapidly as possible• Accuracy and precision of methods less important than confident identification of presence or absence• Ability to confirm presence of target analytes

NEMI CBR Progress

• Refined NEMI fields – added rapidity of analyses, method selectivity, and class selectivity• Analytes already in NEMI have been revised to add these fields• New CBR methods are being added and linked to method summary• Expert system (similar to EMMA) being developed – WATER will help users prioritize considerations and serve as a planning and training tool. Will recommend approach when analyte does not have a suspected identity

Performance Based Systems (PBS) Workgroup

Workgroup ChairCliff Annis, Jr., Merck & Co., Inc.

What is PBS?

A performance based system permits the use of any scientifically appropriate method that demonstrates the ability to meet established performance criteria and complies with specified data quality needs or requirements

Well-defined MQOs & DQOs

Adequate supply of reference materials

for method validation

Validated or reference methods shown to meet

specific MQOs

Adequate training in development of

MQOs & validation of methods

For a performance based system to work, at least 5 darts have to hit the target...

Known performance characteristics

PBS Workgroup accomplishments

– Reached consensus on conception and definition of PBS (issue paper published as National Council report)

– Developed and conducting pilot studies addressing certain PBS implementation issues. Studies involve federal, state, municipal, and private labs:

• COD - Completed -- publish as a National Council report .

• Phosphorous – Lab study completed

• Macroinvertebrates – study begun in WI

Chemical Oxygen Demand Pilot

Lab competence with two COD methods Analyses necessary to demonstrate appropriate performance of new method for matrices of interest Analyses necessary to demonstrate that performance is maintained over time Study completed and report available -- request ACWI review prior to publishing as a National Council report -- request by October 10

Where are we going?

• Developing further pilots to address PBS implementation issue

-- Comparison of results using two total Nitrogen analytical methods – in development

• Provide usable DQO guidance and promote the role of DQOs in water monitoring programs

• Draft Fact Sheet available on the “Value of Comparability”

• Provide paper on “Defining and Assessing Comparability”

-- Compile and summarize results of previous comparability assessments

Biology Methods Workgroup

Workgroup Co-ChairsKatherine Alben, NYS Department of Health

Mike Miller, WI DNR

Why a “Biology Methods” Workgroup?

– Wide interest in a framework for comparing biological methods, particularly field population (community methods)

– Biological methods create unique challenges in terms of defining method performance, data quality, and method comparability

Biology Methods Workgroup accomplishments

– Attracted wide interest from monitoring community – Workgroup and Board increasingly viewed as coordination venue for a number of organizations

– PBS macro invertebrate pilot study and Biology WQDEs being developed in consultation with work group

– Bringing in new technologies as a priority to the Methods Board: e.g., DNA probes, immunoassays, new algal pigment methods. A number of these methods have been included in NEMI.

Where are we going?

• Continue to provide guidance to derive performance characteristics for field population/community and toxicity methods and develop a template/business rules for inclusion in NEMI

• Involvement in taxonomic certification effort

• Develop a data dictionary for biological WQDEs

• Coordinate/conduct PBS pilot studies to help define comparability of biological methods, particularly field biological assessment methods

• Coordinate, communicate, and promote new biological technologies that appear promising for water monitoring (ecological and human health)

New Technologies Workgroup

Workgroup Chair

Katherine Alben – NYS Department of Health

But wait…what about the impact of new technologies?

different methods used atdifferent times by

different programs withdifferent DQOs/MQOs…

Role of the Board with regardto new technologies

– Clearinghouse for analytical methods (NEMI)• to recognize contributors, facilitate technology

transfer, and support data comparability (without endorsement of specific methods)

– identify needs for new or improved monitoring techniques

– develop and promote guidelines to ensure methods and data comparability for the new methods

New Technologies Workgroup accomplishments

– Provide guidance to NEMI homeland security effort.

– Conducted sessions at the 3rd National Monitoring Conference. Developed list of researchers in the field.

– Organized a session at IAGLR (6/22/03) on achieving comparability in monitoring for algae using new technologies.

– provided methods to NEMI for new and advanced technologies.

Where are we going?

• Continue to provide methods to NEMI for new and advanced technologies.• Examine different technologies for monitoring algae – DQOs, MQOs, and comparability using case studies• Prepare technical paper on New Technologies for Early Warning Monitoring – focuses on probes and sensors.• Additional suggestions?

Outreach Workgroup

Workgroup ChairDennis McChesney - USEPA

Outreach products and plans

• Developed and distributed two newsletters (Across the Board). •Website redesign and update of information is nearing completion• Fact sheets developed and in review by National Council

• Coordinate Board participation in 2004 National Monitoring Conference• Ongoing update of speakers bureau – make products available for use by Board/Council members