Mod13-A

QA/QC FOR ENVIRONMENTAL MEASUREMENTUnit 4: Module 13, Lecture 1

Developed by: Zwiebel, Filbin Updated: June 14, 2005 U5-m13a-s*

QA/QC for environmental measurement What is quality assurance (QA)?

a broad plan for maintaining quality in all aspects of a program

- Keith et al. (1983)

establishes the need for quality control (QC)

Photos: 1.) taken from USGS web site http:/water.usgs.gov/ 2.) taken from NRRI Field Photos

Quality Assurance (QA) has been defined as a broad plan for maintaining quality in all aspects of a program. QA as it relates to environmental assessment also refers to the overall management system and includes: OrganizationPlanningData collectionDocumentationEvaluationand Reporting ActivitiesAll designed to address one purpose the success of the environmental assessment. QA is designed to assure that the data gathered and reported on in an environmental assessment meets the defined standards with a stated level of confidence. QA establishes the need for the implementation of quality control measures


QA/QC for environmental measurement Quality control refers to routine technical activities the purpose of which is to control error

QC can be considered the HOW of the QA process

applicable to field, lab and office procedures

Photos come from: 1.) http://oceanexplorer.noaa.gov/explorations/02hudson/logs/aug29/aug29.html - group data discussions2.) http://www.jmg.gov.my/Client_s_Charter/client_s_charter.html - accurate measurement at lab scaleQC information has been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

Quality Control refers to the routine technical activities the essential purpose of which is to control error. Errors are possible in the field, laboratory, and the office and so QC is an integral part of each of these functions. QC is generally part of the methods, standards, testing and certification infrastructure outlined by governmental agencies and professional organizations. QC should include both internal (controlled by those implementing the project) and external (controled by those outside of the project) measures (both are defined and discussed in later slides).


QA for environmental measurement Why do we need quality assurance for environmental measurement?understand data reliabilityquantify areas of analytical uncertaintystandardize measurement to allow for repeatable and comparable data across time and place

QA ensures that data will meet defined standards of quality with a standard level of confidence

Why do we need quality assurance in environmental measure?A) To understand just how reliable our data is and what amount of validity we can put in the measures they provide things can hardly ever be 100% accurate - (applies to sample collection, analytical analysis, and data evaluation)B) For the areas we dont know with 100 % accuracy, we need to give a value (quantify) how much accuracy were are able to assign to the information being evaluated - (includes all aspects from sampling to the evaluation of data results)C) To standardize measurement and allow for reliable measures that are precise, accurate, repeatable and comparable across time and place (all within an acceptable level of accuracy).

QA ensures that data will meet defined standards of quality with a standard level of confidence.

Data is used to make important decisions and requires the decision maker to have a level of confidence about the reliability of information (data) being used to make those decisions Historically, regulatory decisions were often made based upon taking someones word about the reliability of data - but as a society we are becoming more informed and more sophisticated (more skeptical?) in our decision processes. Would you make a decision on data if you werent sure it was valid?


QA for environmental assessment Need for consistency - monitoring a copper impacted stream

Is this problem getting worse or is being mitigated? How can you tell? Why is standardized measurement so important?

Photo from (Newell, NOAA, 1994) http://www.darcnw.noaa.gov/bbpix.htm - shows a creek contaminated with copper. The turquoise color is copper precipitate on rocks.

The goal is to make data reproducible over time and place. Data collected for pollution assessments and site mitigation projects is often gathered over time and used comparatively to determine whether a problem may be getting worse or is being mitigated. For this reason standardized measurements are needed to make any type of reasonable comparison. An example of the need for standardization and consistency - NOAA Case study is outlined below (http://www.darcnw.noaa.gov/Bbird.htm)BLACKBIRD MINE RESTORATION Date of incident: Mining activity began in late 1800s, peaked in late 1940s-1950s Location: An inactive mine site in the Panther creek watershed, a tributary of the Salmon River in east central Idaho Trustees: NOAA, State of Idaho and the U.S. Forest ServiceSource: Inactive mine siteRelease: Copper and other hazardous materials Primary Injury: Loss of chinook salmon and steelhead habitat Restoration: Reintroduction of chinook salmon and in-kind habitat restoration (estimated cost $5.2 million)SITE SUMMARY: Snake River spring/summer chinook salmon (listed as threatened under the Endangered Species Act) were eliminated from Panther Creek by the early 1960s due to hazardous substances contamination. Panther Creek is one of the larger tributaries of the Salmon River and historically supported substantial runs of chinook salmon (2,000 adults returning annually). The stream also supported significant numbers of steelhead. In 1993, the Environmental Protection Agency (EPA) proposed the site to its National Priorities List for further investigation and possible cleanup under Superfund authorities. In 1994, EPA initiated a Remedial Investigation/Feasibility Study for site cleanup. The Settlement: The Consent Decree settling the case requires the parties responsible for the release to restore the water quality in Panther Creek by 2002. The responsible parties were also required to fund a program to reintroduce chinook salmon to Panther Creek; implement a Biological Restoration and Compensation Plan (BRCP) to restore, enhance, and create anadromous and resident salmonid habitat in site-impacted and out-of-basin streams; fund trustee oversight of BRCP implementation; and reimburse trustees' past damage assessment costs.


Where is QA applicable?Who in the environmental arena uses quality assurance measures and when is it necessary?

nearly all REGULATORY decisions about: issuing permits, monitoring human health and environmental quality, pollution investigations and the progress of site remediation are based on one thing -- the data gathered from the site.

Quality Assurance (QA) Programs are used by: Regulatory agencies regulatory decisions about:environmental risk,remediation,issuing permits,monitoring surface and ground water quality, and site investigations

Regulatory decisions related to the above areas of compliance are based on data collected from the site, how it is collected, tested, analyzed, and summarized. The EPA and other regulatory agencies often evaluate how QA goals are met during a project by evaluating the projects Quality Assurance Project Plan (QAPP) we will discuss QAPPS furtherer later on in this presentation.


Where is QA applicable?What other environmental arenas utilize quality assurance measures? laboratories universities and research organization

PHOTO From NRRI

Quality Assurance/Quality Control Programs are also used by the following: Laboratories whether it is a analytical laboratory testing for individual chemical analytes in soil or groundwater, an engineering lab testing structural parameters of soil, or a lab technician counting the number of E. coli bacteria under a microscope for any of these analysis to mean something, they must all follow specific methods by which the quality of the analysis being conducted meets at least a minimum standard by which all similar type analysis can be evaluated. Universities and other Research Programs often use well know and established methods of data collection and analysis to assure the results of their research and study will be accepted as valid. Government established research programs such as the US Geological Survey (USGS) are often responsible for developing and standardizing such QA methods and programs.

Equate the need for QA in the environmental field with the manufacturing industry and needs for specific standards (electrical certifications, drug industry certifications, etc.)


Where is QA applicable?Non-research areas utilize environmental QA measures also business and industrylobbying groups and special interests legislatorslawyersothers

PHOTOS: 1.) http://www.lanl.gov/orgs/pa/News/060297.html 2.) http://gov.state.nv.us/SpecialSession072902.asp

Quality Assurance/Quality Control Programs are also used by the following: Businesses and industry required to meet certain limitations on use and output of a chemical compound or materials with the potential to cause environmental impacts, will use QA standards to assure agency acceptance of data, to provide meaningful results with which to assess compliance, to provide data that would be considered reliable in court.Environmental consulting firms often working for industry, a public interest groups, etc. use QA for the same reasons to assure that regulatory agencies will accept the data they present, again to assure that analyzed data meets the quality needs of the project, and to provide reliable data to limit potential liability.Others - Environmental and special interest organizations, lobbying groups, lawyers, etc. (list as many as your class is able related to the environmental industry note that QA also applies to other industries) - all have a need to assure the quality of data in environmental measurement related to compliance, integrity of presentation, and limiting liability.

Think of some specific examples where accurate data was important to make discussions affecting the public in your area. (e.g.: permitting of chemical out put from a major industry in your area, a recent major lawsuit concerning an environmental outcome - why was QA data important in this case? Recent laws, legislation, city counsel voting, etc. based upon the QA of data presented.)


Quality assurance vs. quality controlQuality Assurance (QA)broad program plan establishes the need for QC

Quality Controls (QC) individual checks and balancesthe HOW of QA

Use this slide as a opportunity to briefly review the meaning of QA and QC What are the key differences and similarities between the two?

Quality Assurance - a broad plan for maintaining quality in all aspects of a program, also refers to the overall management system and includes: Organization, Planning, Data collection, Documentation, Evaluation, and reporting activities. All designed to address the success of the environmental assessment.

Quality Control refers to the routine technical activities and established check and balances, the essential purpose of which is to control error. (applies to activities in the field, laboratory, and the office)


Where is QC applicable?Quality control is applicable in all aspects of a project including:

Field data collection and sampling

Laboratory analysis and processing

Data evaluation and assessment

Reporting and project documentation

QC provides steps that ensure data will meet defined standards of quality with a standard level of confidence

As we stated earlier QA ensures that data will meet defined standards of quality with a standard level of confidence. QC provides the steps by which the same goals are reached.

To be comfortable in the quality of data provided we need to be assured of the processes used in all areas of assessment - from when, where and how the sample was collected; to the processes used by the field technician or laboratory conducting the analysis; to how the data results are interpreted and by whom; to how the information is reported upon and level of accuracy documented. Without confidence in the data provided all arenas; from regulatory agencies, to research groups, to business officials, to legislators; would have to question many of the decisions they are required to make. Few agencies will use data unless methods of data collection, storage, and analysis can be documented. Clear and concise documentation of procedures also allows a project to continue over time and space using the same methods. This is particularly important to a project that extends over a long period of time and that intends to establish a baseline of quality information that can be compared over time.


QC in the fieldQC is particularly critical in filed data collectionoften the most costly aspect of a project data is never reproducible under the exact same condition or setting

sechi readingsfield filtrationlogging sea cores

Photos from left to right: 1.) provided by NRRI Collecting sechi disk reading 2.) http://climchange.cr.usgs.gov/info/lacs/watersampling.htm - USGS water sampling 3.) http://www.fe.doe.gov/techline/tl_hydrates_oregon.shtml - logging cores

This slide shows just a very few examples of diverse forms of field data being collected the class may want to discuss other examples, and why proper sample collection and data analysis is so important emphasize the reasons provided below. The first steps to a sound environmental assessment is the evaluation of competent data.and this evaluation begins in the filed. The quality and applicability of the samples and data collected here will guide all other aspects of the environmental evaluation. Data or sample collection from the designated time and place is never reproducible under the exact same conditions and setting. Because of this QA measures must be taken to assure the best possible (most reliable) set of data is obtained. Obtaining data and samples from the filed is also often the most costly aspect of any assessment program to recreate if data is deemed unreliable. These costs are a result of the need for specialized equipment, mobilization to the sample site, the need to redo all other aspects of evaluation, or missing one significant item could render other data useless.


QC in the fieldStandardized field method programsUS Army Corps of Engineers ASTM methodsUSGS classificationAPHAAIHANIST

273.psd

274.psd

275.psd

272.psd

276.psd

Logos copied from individual web sites as listed below:Governmental agencies (USGS, EPA, USDHHS, etc.) and professional organizations (ASTM, AIHA, etc. ) have worked together to develop a standards, testing and certification infrastructure to meet the needs of the environmental industry. This slide and the following discussion list a brief overview of many of the standards programs utilized in the industry today:ASTM American Society for Testing and Materials - a professional volunteer organization of chemists and engineers - responsible for the development of hundreds of standards related to both filed and laboratory data collection, testing, and analysis. Link to http://www.astm.org/USGS US Geologic Survey US Department of the Interior organization emphasis on geology, hydrogeology, engineering responsible for soil classification standards, filed gas measurement standards, in-situ water quality monitoring data collection, etc. Link to www.usgs.gov/ US Army Corps of Engineers has developed methods for the collection storage and manipulation of materials including sediments used in chemical and toxicological analysis. Link to www.usace.army.mil/ APHA American Public Health Association Professional association of individuals and organizations working to improve the public's health and to achieve equity in health status for all. responsible for most widely accepted standards related to waste water testing, treatment, and assessment. (American Public Health Association. 1995, 1998, 2001. Standard methods for the examination of water and wastewater. Washington, D.C., 17th- 19th Editions). Link to http://www.apha.org/.AIHA American Industrial Hygiene Association a professional organization of scientists concentrating on occupational and environmental health and safety. responsible for development of investigations standards in the areas of asbestos, mold, indoor air quality, etc. Link to www.aiha.org/ NIST - National Institute of Standards and Technologies - is an agency of the U.S. Commerce Department's Technology Administration . NIST Laboratories provide measurements and standards for U.S. industry. Link to http://www.nist.gov/


QC in the laboratoryLaboratory data analysis, measurement and acquisition:Chain of custody formsEquipment calibrationStorage practicesAnalytical methodsHolding timesMDLs

Once field data is collected, it must be evaluated with methods and standards that meet the quality requirements of your environmental investigation. Every aspect of analysis needs to be evaluated; form assuring that the samples being analyzed belong to your project (chain of custody), to preventing potential contamination from other sources (storage practices), to assuring the analytical result will be relevant (exp: the equipment is properly calibrated, and testing is conducted within holding times and using proper methods and levels of detection (minimum detection limits - MDLs) sufficient to quantify what it is you are assessing). Again, the goal is to maintain quality in all aspects of the analytical program.


QC in the laboratoryStandardized laboratory method programsEPA MethodsState Modified MethodsAPHA Methods ASTMAIHA

288.psd

291.psd

287.psd

Agency and organization logos were collected from the individual web sites as listed below:Examples of standards used in laboratory analysis include: EPA developed methodologies - most commonly used standards for analysis of pollutants and individual chemical analytes also responsible for the auditing and certification of laboratories. Link to http://www.epa.gov/safewater/certlab/labcint.htmlSome state agencies can also certify laboratories and/or have there own modified methods of federal program methods

- Exp: MN Department of Health Modified 465D water analysis list expands upon the EPAs standard list of analytes- Exp: WI Modified DRO and GRO (diesel- and gasoline- range organics) analysis on both soil and water samples modifies both field collection for samples, preservatives used, and laboratory analytical requirements. - List / explain some of the modifications your state or regional offices may use when compared to federal standardized requirements. APHA American Public Health Association Professional association of individuals and organizations working to improve the public's health and to achieve equity in health status for all. responsible for most widely accepted standards related to waste water testing, treatment, and assessment. (American Public Health Association. 1995, 1998, 2001. Standard methods for the examination of water and wastewater. Washington, D.C., 17th- 19th Editions). Link to http://www.apha.org/.ASTM - professional volunteer organization of chemists and engineers - establishes many methods and standards for assessing the characteristic properties of materials including chemical compounds and liquids (flash point tests, density analysis, etc.) soils (density, permeability, hydraulic conductivity, etc.) and other materials. Link to http://www.astm.org/AIHA American Industrial Hygiene Association a professional organization of scientists concentrating on occupational and environmental health and safety. responsible for development of laboratory methods and standardized requirements in the areas of asbestos, mold, indoor air quality, etc. Link to www.aiha.org/


QC in the officeIn office planning, data evaluation and summation of results:QAPP quality assurance

project plan outlines allproject managementassessment and oversightdata validation

www.cr.nps.gov/aad/collections/ mgt_pr.htm Photo from MN DNR

In actuality, although field data collection is very important, the assurance of quality (QA) in an environmental assessment implements QC measures that begin and end in the office (or in the processing stages); initiated with planning, and ending with evaluation of data and conclusions along with project presentation. If the first steps to a sound environmental assessment is the evaluation of competent datastarting with its proper collection in the fieldthe first steps to assuring that field collected data will be competent start in the office, or on the drawing board, with the creation of a Quality Assurance Project Plan (QAPP) (discussed in more detail in future slides and lesson 13B).A quality assurance project plan (QAPP) determines how QA data is collected and what QC measures will be used. The QAPP is a guide to conducting all aspects of an investigation and is originally outlined, developed, and compiled in the office. The QAPP is an invaluable planning and operating tool that should be developed in the early stages of an environmental assessment.In addition to development of the QAPP, other QC measures included in the office aspect of a project involve: Project management who will work on the project, what are their qualifications and training? Who will supervises and make sure things progress on schedule and on budget? Assessment and oversight who will evaluate the project work as it progresses? Who will take responsibility for the final outcomes?Data validation What statistical methods will be used to determine if the data is accurate and /or relevant?


QC for environmental measurement Why do we need quality control?To prevent errors from happening To identify and correct errors that have taken place

QC is used to PREVENT and CORRECT ERRORS

Why do we need quality control?The purpose of QC is to identify and correct errors that may have happened in sampling, analysis, or data evaluation and to prevent additional errors from happening in the future. We need QCs to establish a level of confidence in our QA.

Discuss with the students some of the possible errors that could arise while sampling in the fieldwhile testing in the laboratorywhile analyzing data results in the office.Briefly - how could some of these errors be discovered, corrected, and/or prevented?

EXAMPLES OF QUALITY CONTROL SYSTEMS PROVIDE:Constant checks on sensitivity and accuracy of instrumentation and analytic systemsInteractive response - quality control information is used to maintain calibration and instrument response.Real-time monitoring of instrument performance to ensure maintenance of calibrations and accuracy of results. Control Charts - used to monitor long-term performance of measurement and analytical systems and initiate corrective action when biases are detected


QC: Internal vs. external measuresInternal quality controlcontrollable by those responsible for undertaking the project or directly involved in the program

External quality controla set of measures established for and conducted by those people and organizations outside of the program

A good Quality Assurance Project Plan (QAPP) will provide for the validation of data and information through both internal and external QC measures.

Internal QC Internal controls refer to those aspect of an environmental investigation that are controllable by those responsible for undertaking the project or directly involved in the program - in a Quality Assurance Project Plan (QAPP) - internal controls would refer to the group or organization that developed the QAPP and the employees, volunteers, or participants designated specific tasks within the QAPP.

External QC - refers to a set of measures established for and conducted by those people and organizations outside of the program (example: an audit or replicate sample collection conducted by the state environmental regulatory agency, the regional EPA representative, or their designated auditing group).


Quality control (QC): Internal Internal Quality Control:Equipment calibrationProper training and certification of participantsProper sampling and containment techniques Proper data documentation

Photos are from the following sources: 1.) http://www.ars.usda.gov/is/np/fnrb/fnrb1001.htm asbestos containment chamber2.) http://il.water.usgs.gov/adcp/photos/class_apr99.html - recording data in the field QC information has been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

Some examples of internal quality control measures implemented by the group responsible for the project (including but not limited to the collection and evaluation of samples and data) would include:proper equipment calibration, training and certification of field technicians and other staff,proper documentation of all applicable information (see photo 2 field logging of collected data),prevention of cross contamination when collecting samples and data (see photo 1 asbestos chamber)

Can your group list examples of other internal measures of error control?


Quality control (QC): ExternalExternal quality control:Performance auditsSplit sample analysisReplicate (duplicate) sample analysis

Photos come from: 1.) http://www.ars.usda.gov/is/np/fnrb/fnrb1001.htm - auditing computer work on screen2.) http://www.tfhrc.gov/pubrds/novdec98/seven.htm - 3 person audit teamQC information has been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

External QC - set of measures established for those outside of the program

Examples of external QC controls include:performance audits conducted by outside personnel, usually agency representatives or hired consultantssplit sample analysis - where one sample is divided equally and sent to more than one laboratory (or annalist) for analysis, and collection of replicate samples (or duplicate samples when collecting only 2 samples) - two or more samples taken from the same location, at the same time, using the same method, but independently analyzed; generally by a person(s) from outside the organization conducting the investigation.


Quality Assurance Project Plans (QAPP) A quality assurance project plan (QAPP) is a project-specific QA document.

The QAPP outlines the QC measures used in implementing the project.

The QAPP is an invaluable planning and operating tool that should be developed in the early stages of the investigation, monitoring, or remediation project.Quality Assurance Project Plan (QAPPs) are used by agencies such as the EPA to ensure that the samples collected and analyzed, the data stored and manage, and reports that are written, are of high enough quality to meet project needs. When agencies are involved in data collection a QAPP is always used to oversee the agency selection of laboratories, data collection methods, selection of parameters to be measured, consistency of data analysis and confidence in data quality.Few agencies will use data unless methods of data collection, storage, and analysis can be documented. Clear and concise documentation of procedures also allows a project to continue over time and space using the same methods. This is particularly important to a project that extends over a long period of time and that intends to establish a baseline of quality information that can be compared over time.

QAPP agency uses have been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm


Quality Assurance Project Plans (QAPP) The QAPP governs work conducted in the field, laboratory, and the office. The QAPP: guides the selection of project parameters and proceduresguides data management and analysisprovides steps to determine the validity of specific sampling or analysis proceduresdesignates both internal and external QC measures

The QAPP governs work in all aspects of the field, laboratory and office. The QAPP guides all areas of an investigation (field, lab and office) and includes: the selection of parameters and procedures to be used in the investigation, including standards and methods to be used in both the field an laboratory (examples reviewed in the previous slides), data management and analysis - how will those viewing the data assess the results, apply statistical evaluation, and make conclusions - which analytical methods are best for the data that needs to be obtained?and, what steps are taken to determine the validity of specific sampling or analysis procedures both quality assessment and quality control (QA/QC) what conclusions are determined from the information?

Just as in the collection and evaluation of data, several agencies, programs and resources exist to describe how to develop and use a QAPP that is applicable to your area of study. We will talk about this in more detail in the following lesson when we discuss how to develop a QAPP and what key elements to include. However, for now we will concentrate on the WHY of QAPP development by discussing data objectives and key concepts necessary to implementing QA/QC The PARCC Parameters (following slides).


QA/QC: Data objective and key conceptsSuccessful data collection and analysis is dependant upon The PARCC Parameters: PrecisionAccuracyRepresentativenessCompletenessComparability

The key concepts of QA/QC are the PARCC Parameters the WHY of the QAPP

The PARCC Parameters include: PrecisionAccuracyRepresentativeness CompletenessComparability

The PARCC parameters are the WHY of the QAPP and QA/QC. The QAPP is developed in order that the key concepts of a well run and thorough environmental investigation may be met.


Key concepts of QA/QC: Precision -degree of agreement there is between repeated measurements of the same characteristiccan be biased meaning there is a consistent error in the resultsAccuracy - measures how close data results are to a true or expected value does not allow for bias

Photo taken from http://www.d.umn.edu/faculty/ UMD med school lab workerKey concept definitions have been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

Precision is the degree of agreement among repeated measurements of the same characteristic on the same sample or on separate samples collected as close as possible in time and place. Precision gives an idea of how constant and reproducible field or laboratory data are. However, this is only an indication of how consistent results are under similar conditions, and does NOT necessarily mean that results are a true value or accurate. (See target diagram on the following slide)

Precision may have a BIAS which is the degree of systematic error present in a process. Meaning, that when bias is present the result may be close together (precise) but will differ from the true value or expected result. (See target diagram on the following slide)

Accuracy is a measure of confidence in measurement. The smaller the difference between the measurement of a parameter and its true or expected value (the less bias), the more accurate the measurement and the more reproducible its result. (See target diagram on the following slide)


Key concepts of QA/QC: Accuracyaccuracy = (average value) (true value)precision represents repeatabilitybias represents amount of error

low bias and high precision = statistical accuracy

http://www.epa.gov/owow/monitoring/volunteer/qappexec.html

Information adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

Accuracy is the level of confidence in measurement. The smaller the difference between the measurement of a parameter and its true or expected value (the more accurate the measurement) the more reproducible its result. Accuracy is determined by comparing the result of analysis for a sample to the known value for that sample such as a standard or a spike. Accuracy needs to be considered when choosing a method of analysis What level of accuracy is needed? Grater accuracy allows for greater Repeatability - provides for a reliable measurement when you need to confirm the consistency of an environmental condition or look at change over time. ( e.g., for reliable interpretation of trends over time the nitrification of a lake or remediation of a pollutant). Repeatability is usually acquired by specifying and sticking to a particular sampling and analytical method (methods discussed in slides 12 and 14).

Accuracy is reflective of both bias and precision (see target diagram). Discuss with the students the need for accuracy to be represented by both high precision and low bias.


Key concept QA/QC: RepresentativenessRepresentativeness - extent to which measurements actually represent the true environmental condition or population at the time a sample was collected.Representative data should result in repeatable data

Does this represent this??

Photos: 1.) taken from WOW Model 8 (8-4-03)2.) taken from http://pubs.usgs.gov/fs/fs-0058-99/ - Minnesota LakeKey concept definitions have been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

Representativeness It stands to reason that the more accurate the data, the more representative it will be of the true or actual conditions at the site or area under investigation. As stated above, the more accurate the data, the more repeatable the results. Representativeness refers more to the sample location and selection. Are you really representing the conditions of the site when you choose specific sample or the specific sample location? (Simple Example: sampling for benthic organisms at the surface of a lake).


Key concepts of QA/QC: CompletenessCompleteness -comparison between the amount of data intended to be collected vs. actual amount of valid (usable) data collected.In the design of the QAPP will the goals of the plan meet assessment needs? Will sufficient data be collected?

Would this give usable data ??

Photo provided by NRRIKey concept definitions have been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

Completeness- is a data quality indicator generally expressed as a percentage. Completeness includes both the following (explanation and example for each provided below): 1.) an assessment of the amount of valid data obtained vs. amount planed for 2.) the design of a data collection plan to accurately assess the conditions

Deteriorated data gathering equipment will not gather usable data. (see photo) Deterioration of equipment need not be this extreme. It can include such simple problems as the biofouling of in-place data loggers or the sticking of the wiper on a self-wiping turbidity sensor. Or using equipment known to error under rigorous weather conditions. .

The Valid Data concept of completeness is important when you think of it as the amount of valid data obtained from a measurement system relative to the amount of valid data obtained under correct, normal conditions. In monitoring studies, it is not unusual to have some data that are invalid for a wide range of reasons (e.g., instruments that loose calibration, transcription errors, calculation/data reduction errors, lost samples, etc.). Generally, investigators try to minimize the amount of data lost because too much lost data can impact the statistical confidence of the analysis and cost extra money when data points need to be replaced, or whole datasets need to be discarded and re-sampled.

The Sample Design concept of completeness involves ensuring that the sample is designed to accurately assess conditions or trends across the entire population of interest. (e.g., using tools such as random sampling design or stratified sampling designs allow the investigator to fully characterize populations of interest; e.g., to fully characterize a fishery population you would want to sample at the lake AND its inflow and outflow streams, and you would want to look at all water layers when a lake is stratified - to avoid missing significant components in the population).


Key concepts of QA/QC: Comparability Comparability -the extent to which data can be compared between sample locations or periods of time within a project, or between projects

Will similar data from these sites be Comparable ??

Photos: 1.) taken from http://www.nrcs.usda.gov/programs/cta/ctasummary.html 2.) taken from http://www.nssl.noaa.gov/publicaffairs/backgrounders/backgrounder_ww.html - Winter data Key concept definitions have been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

Comparability - a method that works as well in fresh water may not work as well in saltwater (marine) environments! Different methods of analysis may have different limitations on their use because of the relative effects of different sample matrices. Another example - samples taken at different times of the day or at different seasons may not be comparable because of changes that occur with changing daylight or temperature. By specifying a consistent method you can compare two of more places or two places in time - why is this important?( Hint: e.g., to allow reliable comparisons in before and after measurements, or upstream and downstream measurements around an effluent at a point source)

Discuss with the students whether on not data collected under these varying climate conditions may or may not be comparable? What sort of sampling events may be affected by these varying conditions. Example: The collection of water samples for volatile organic compounds how would hot weather conditions effect the samples and their volatile nature. How about in turn cold weather? What if the samples froze? What if collected samples froze and broke the bottles in which they were stored? Discuss how this relates back to the concept of completeness. Discuss how it relates back to the need for QCs to prevent these problems from occurring.


Review: QA vs. QC Quality Assurance (QA)broad program plan establishes the need for QC

Quality Controls (QC) standardized tests and methodsthe HOW of QA



Quality Control refers to the routine technical activities and established check and balances, the essential purpose of which is to control error. QC employs standardized tests and methods used in the field, laboratory, and office to generate quality, consistent, and repeatable data.


Review: Quality Assurance Project PlansThe QAPP is a project-specific QA document.The QAPP outlines the QC measures to be taken for the project.

QAPP guides: the selection of parameters and proceduresdata management and analysissteps taken to determine the validity of specific sampling or analysis procedures

Review with students the need for a QAPP (introduced in slides 15 to 21)


Elements of a QAPPThe QAPP governs work conducted in the field, laboratory, and the office.

QAPP can be broken out into four areas of the projectProject management (office)Measurement and data acquisition (field and lab)Assessment and oversight (field, lab, and office)Data validation and usability (field, lab, and office)

Just as we discussed in the beginning of this presentation, QA/QC enters into all aspects of a project design, assessment, and evaluation. The QAPP governs actions to be taken in all aspects of the project including the field, laboratory, and the office. To assure a high quality investigation, consistent means of data acquisition and evaluation must be followed in all areas of the investigation.

A following lecture (13C) will concentrate on the development of a QAPP. How is one developed, by whom, and what to include and what considerations need to be addressed when compiling such a document and outlining such a program.


Review: QA/QC key conceptsThe PARCC Parameters are the WHY of QA/QC and assure successful data analysis: PrecisionAccuracyRepresentativenessCompletenessComparability

The key concepts and WHY of QA/QC are the PARCC Parameters

The QAPP is most specific on details surrounding data acquisition and evaluation, or work conducted in the field and laboratory. To assure that data gathered and analysis conducted is of high quality and consistency, the PARCC are employed.

Review again briefly with the students the PARCC Parameters and what sort of QC measures can be used to assure the obtainment of each including: PrecisionAccuracyRepresentative (Repeatability)CompletenessComparabilityHow can one be aware of the QC measures that are to be used during implementation of the project? How can one make sure they are using the QC methods properly? BY REVIEWING AND REFFERING BACK TO THE QAPP.

The following lecture (13B) will go into more detail on the Whys of implementing the QC portion of a project plan and some of the How behind the whys. We will introduce Data Quality Objectives (DQOs) and discuss how we evaluate quality of data. We will discuss QC sample(s) applicable to assess for each key parameter and touch upon statistical calculation of percussion along with determination of accuracy and bias


ReferencesEPA 1996, Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm


http://waterontheweb.org/index.html

4418This is the closing WOW slide that should appear at the end of all WOW presentations. This slide should not be altered in anyway. Do not add any additional elements to the slide, alter the web address, add additional copy, etc.

Photos: 1.) taken from USGS web site http:/water.usgs.gov/ 2.) taken from NRRI Field Photos

Quality Assurance (QA) has been defined as a broad plan for maintaining quality in all aspects of a program. QA as it relates to environmental assessment also refers to the overall management system and includes: OrganizationPlanningData collectionDocumentationEvaluationand Reporting ActivitiesAll designed to address one purpose the success of the environmental assessment. QA is designed to assure that the data gathered and reported on in an environmental assessment meets the defined standards with a stated level of confidence. QA establishes the need for the implementation of quality control measuresPhotos come from: 1.) http://oceanexplorer.noaa.gov/explorations/02hudson/logs/aug29/aug29.html - group data discussions2.) http://www.jmg.gov.my/Client_s_Charter/client_s_charter.html - accurate measurement at lab scaleQC information has been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

Quality Control refers to the routine technical activities the essential purpose of which is to control error. Errors are possible in the field, laboratory, and the office and so QC is an integral part of each of these functions. QC is generally part of the methods, standards, testing and certification infrastructure outlined by governmental agencies and professional organizations. QC should include both internal (controlled by those implementing the project) and external (controled by those outside of the project) measures (both are defined and discussed in later slides). Why do we need quality assurance in environmental measure?A) To understand just how reliable our data is and what amount of validity we can put in the measures they provide things can hardly ever be 100% accurate - (applies to sample collection, analytical analysis, and data evaluation)B) For the areas we dont know with 100 % accuracy, we need to give a value (quantify) how much accuracy were are able to assign to the information being evaluated - (includes all aspects from sampling to the evaluation of data results)C) To standardize measurement and allow for reliable measures that are precise, accurate, repeatable and comparable across time and place (all within an acceptable level of accuracy).

QA ensures that data will meet defined standards of quality with a standard level of confidence.

Data is used to make important decisions and requires the decision maker to have a level of confidence about the reliability of information (data) being used to make those decisions Historically, regulatory decisions were often made based upon taking someones word about the reliability of data - but as a society we are becoming more informed and more sophisticated (more skeptical?) in our decision processes. Would you make a decision on data if you werent sure it was valid?

Photo from (Newell, NOAA, 1994) http://www.darcnw.noaa.gov/bbpix.htm - shows a creek contaminated with copper. The turquoise color is copper precipitate on rocks.

The goal is to make data reproducible over time and place. Data collected for pollution assessments and site mitigation projects is often gathered over time and used comparatively to determine whether a problem may be getting worse or is being mitigated. For this reason standardized measurements are needed to make any type of reasonable comparison. An example of the need for standardization and consistency - NOAA Case study is outlined below (http://www.darcnw.noaa.gov/Bbird.htm)BLACKBIRD MINE RESTORATION Date of incident: Mining activity began in late 1800s, peaked in late 1940s-1950s Location: An inactive mine site in the Panther creek watershed, a tributary of the Salmon River in east central Idaho Trustees: NOAA, State of Idaho and the U.S. Forest ServiceSource: Inactive mine siteRelease: Copper and other hazardous materials Primary Injury: Loss of chinook salmon and steelhead habitat Restoration: Reintroduction of chinook salmon and in-kind habitat restoration (estimated cost $5.2 million)SITE SUMMARY: Snake River spring/summer chinook salmon (listed as threatened under the Endangered Species Act) were eliminated from Panther Creek by the early 1960s due to hazardous substances contamination. Panther Creek is one of the larger tributaries of the Salmon River and historically supported substantial runs of chinook salmon (2,000 adults returning annually). The stream also supported significant numbers of steelhead. In 1993, the Environmental Protection Agency (EPA) proposed the site to its National Priorities List for further investigation and possible cleanup under Superfund authorities. In 1994, EPA initiated a Remedial Investigation/Feasibility Study for site cleanup. The Settlement: The Consent Decree settling the case requires the parties responsible for the release to restore the water quality in Panther Creek by 2002. The responsible parties were also required to fund a program to reintroduce chinook salmon to Panther Creek; implement a Biological Restoration and Compensation Plan (BRCP) to restore, enhance, and create anadromous and resident salmonid habitat in site-impacted and out-of-basin streams; fund trustee oversight of BRCP implementation; and reimburse trustees' past damage assessment costs. Quality Assurance (QA) Programs are used by: Regulatory agencies regulatory decisions about:environmental risk,remediation,issuing permits,monitoring surface and ground water quality, and site investigations

Regulatory decisions related to the above areas of compliance are based on data collected from the site, how it is collected, tested, analyzed, and summarized. The EPA and other regulatory agencies often evaluate how QA goals are met during a project by evaluating the projects Quality Assurance Project Plan (QAPP) we will discuss QAPPS furtherer later on in this presentation. PHOTO From NRRI

Quality Assurance/Quality Control Programs are also used by the following: Laboratories whether it is a analytical laboratory testing for individual chemical analytes in soil or groundwater, an engineering lab testing structural parameters of soil, or a lab technician counting the number of E. coli bacteria under a microscope for any of these analysis to mean something, they must all follow specific methods by which the quality of the analysis being conducted meets at least a minimum standard by which all similar type analysis can be evaluated. Universities and other Research Programs often use well know and established methods of data collection and analysis to assure the results of their research and study will be accepted as valid. Government established research programs such as the US Geological Survey (USGS) are often responsible for developing and standardizing such QA methods and programs.

Equate the need for QA in the environmental field with the manufacturing industry and needs for specific standards (electrical certifications, drug industry certifications, etc.) PHOTOS: 1.) http://www.lanl.gov/orgs/pa/News/060297.html 2.) http://gov.state.nv.us/SpecialSession072902.asp

Quality Assurance/Quality Control Programs are also used by the following: Businesses and industry required to meet certain limitations on use and output of a chemical compound or materials with the potential to cause environmental impacts, will use QA standards to assure agency acceptance of data, to provide meaningful results with which to assess compliance, to provide data that would be considered reliable in court.Environmental consulting firms often working for industry, a public interest groups, etc. use QA for the same reasons to assure that regulatory agencies will accept the data they present, again to assure that analyzed data meets the quality needs of the project, and to provide reliable data to limit potential liability.Others - Environmental and special interest organizations, lobbying groups, lawyers, etc. (list as many as your class is able related to the environmental industry note that QA also applies to other industries) - all have a need to assure the quality of data in environmental measurement related to compliance, integrity of presentation, and limiting liability.

Think of some specific examples where accurate data was important to make discussions affecting the public in your area. (e.g.: permitting of chemical out put from a major industry in your area, a recent major lawsuit concerning an environmental outcome - why was QA data important in this case? Recent laws, legislation, city counsel voting, etc. based upon the QA of data presented.)



Quality Control refers to the routine technical activities and established check and balances, the essential purpose of which is to control error. (applies to activities in the field, laboratory, and the office) As we stated earlier QA ensures that data will meet defined standards of quality with a standard level of confidence. QC provides the steps by which the same goals are reached.

To be comfortable in the quality of data provided we need to be assured of the processes used in all areas of assessment - from when, where and how the sample was collected; to the processes used by the field technician or laboratory conducting the analysis; to how the data results are interpreted and by whom; to how the information is reported upon and level of accuracy documented. Without confidence in the data provided all arenas; from regulatory agencies, to research groups, to business officials, to legislators; would have to question many of the decisions they are required to make. Few agencies will use data unless methods of data collection, storage, and analysis can be documented. Clear and concise documentation of procedures also allows a project to continue over time and space using the same methods. This is particularly important to a project that extends over a long period of time and that intends to establish a baseline of quality information that can be compared over time.Photos from left to right: 1.) provided by NRRI Collecting sechi disk reading 2.) http://climchange.cr.usgs.gov/info/lacs/watersampling.htm - USGS water sampling 3.) http://www.fe.doe.gov/techline/tl_hydrates_oregon.shtml - logging cores

This slide shows just a very few examples of diverse forms of field data being collected the class may want to discuss other examples, and why proper sample collection and data analysis is so important emphasize the reasons provided below. The first steps to a sound environmental assessment is the evaluation of competent data.and this evaluation begins in the filed. The quality and applicability of the samples and data collected here will guide all other aspects of the environmental evaluation. Data or sample collection from the designated time and place is never reproducible under the exact same conditions and setting. Because of this QA measures must be taken to assure the best possible (most reliable) set of data is obtained. Obtaining data and samples from the filed is also often the most costly aspect of any assessment program to recreate if data is deemed unreliable. These costs are a result of the need for specialized equipment, mobilization to the sample site, the need to redo all other aspects of evaluation, or missing one significant item could render other data useless. Logos copied from individual web sites as listed below:Governmental agencies (USGS, EPA, USDHHS, etc.) and professional organizations (ASTM, AIHA, etc. ) have worked together to develop a standards, testing and certification infrastructure to meet the needs of the environmental industry. This slide and the following discussion list a brief overview of many of the standards programs utilized in the industry today:ASTM American Society for Testing and Materials - a professional volunteer organization of chemists and engineers - responsible for the development of hundreds of standards related to both filed and laboratory data collection, testing, and analysis. Link to http://www.astm.org/USGS US Geologic Survey US Department of the Interior organization emphasis on geology, hydrogeology, engineering responsible for soil classification standards, filed gas measurement standards, in-situ water quality monitoring data collection, etc. Link to www.usgs.gov/ US Army Corps of Engineers has developed methods for the collection storage and manipulation of materials including sediments used in chemical and toxicological analysis. Link to www.usace.army.mil/ APHA American Public Health Association Professional association of individuals and organizations working to improve the public's health and to achieve equity in health status for all. responsible for most widely accepted standards related to waste water testing, treatment, and assessment. (American Public Health Association. 1995, 1998, 2001. Standard methods for the examination of water and wastewater. Washington, D.C., 17th- 19th Editions). Link to http://www.apha.org/.AIHA American Industrial Hygiene Association a professional organization of scientists concentrating on occupational and environmental health and safety. responsible for development of investigations standards in the areas of asbestos, mold, indoor air quality, etc. Link to www.aiha.org/ NIST - National Institute of Standards and Technologies - is an agency of the U.S. Commerce Department's Technology Administration . NIST Laboratories provide measurements and standards for U.S. industry. Link to http://www.nist.gov/

Once field data is collected, it must be evaluated with methods and standards that meet the quality requirements of your environmental investigation. Every aspect of analysis needs to be evaluated; form assuring that the samples being analyzed belong to your project (chain of custody), to preventing potential contamination from other sources (storage practices), to assuring the analytical result will be relevant (exp: the equipment is properly calibrated, and testing is conducted within holding times and using proper methods and levels of detection (minimum detection limits - MDLs) sufficient to quantify what it is you are assessing). Again, the goal is to maintain quality in all aspects of the analytical program.Agency and organization logos were collected from the individual web sites as listed below:Examples of standards used in laboratory analysis include: EPA developed methodologies - most commonly used standards for analysis of pollutants and individual chemical analytes also responsible for the auditing and certification of laboratories. Link to http://www.epa.gov/safewater/certlab/labcint.htmlSome state agencies can also certify laboratories and/or have there own modified methods of federal program methods

- Exp: MN Department of Health Modified 465D water analysis list expands upon the EPAs standard list of analytes- Exp: WI Modified DRO and GRO (diesel- and gasoline- range organics) analysis on both soil and water samples modifies both field collection for samples, preservatives used, and laboratory analytical requirements. - List / explain some of the modifications your state or regional offices may use when compared to federal standardized requirements. APHA American Public Health Association Professional association of individuals and organizations working to improve the public's health and to achieve equity in health status for all. responsible for most widely accepted standards related to waste water testing, treatment, and assessment. (American Public Health Association. 1995, 1998, 2001. Standard methods for the examination of water and wastewater. Washington, D.C., 17th- 19th Editions). Link to http://www.apha.org/.ASTM - professional volunteer organization of chemists and engineers - establishes many methods and standards for assessing the characteristic properties of materials including chemical compounds and liquids (flash point tests, density analysis, etc.) soils (density, permeability, hydraulic conductivity, etc.) and other materials. Link to http://www.astm.org/AIHA American Industrial Hygiene Association a professional organization of scientists concentrating on occupational and environmental health and safety. responsible for development of laboratory methods and standardized requirements in the areas of asbestos, mold, indoor air quality, etc. Link to www.aiha.org/

In actuality, although field data collection is very important, the assurance of quality (QA) in an environmental assessment implements QC measures that begin and end in the office (or in the processing stages); initiated with planning, and ending with evaluation of data and conclusions along with project presentation. If the first steps to a sound environmental assessment is the evaluation of competent datastarting with its proper collection in the fieldthe first steps to assuring that field collected data will be competent start in the office, or on the drawing board, with the creation of a Quality Assurance Project Plan (QAPP) (discussed in more detail in future slides and lesson 13B).A quality assurance project plan (QAPP) determines how QA data is collected and what QC measures will be used. The QAPP is a guide to conducting all aspects of an investigation and is originally outlined, developed, and compiled in the office. The QAPP is an invaluable planning and operating tool that should be developed in the early stages of an environmental assessment.In addition to development of the QAPP, other QC measures included in the office aspect of a project involve: Project management who will work on the project, what are their qualifications and training? Who will supervises and make sure things progress on schedule and on budget? Assessment and oversight who will evaluate the project work as it progresses? Who will take responsibility for the final outcomes?Data validation What statistical methods will be used to determine if the data is accurate and /or relevant? Why do we need quality control?The purpose of QC is to identify and correct errors that may have happened in sampling, analysis, or data evaluation and to prevent additional errors from happening in the future. We need QCs to establish a level of confidence in our QA.

Discuss with the students some of the possible errors that could arise while sampling in the fieldwhile testing in the laboratorywhile analyzing data results in the office.Briefly - how could some of these errors be discovered, corrected, and/or prevented?

EXAMPLES OF QUALITY CONTROL SYSTEMS PROVIDE:Constant checks on sensitivity and accuracy of instrumentation and analytic systemsInteractive response - quality control information is used to maintain calibration and instrument response.Real-time monitoring of instrument performance to ensure maintenance of calibrations and accuracy of results. Control Charts - used to monitor long-term performance of measurement and analytical systems and initiate corrective action when biases are detected

A good Quality Assurance Project Plan (QAPP) will provide for the validation of data and information through both internal and external QC measures.

Internal QC Internal controls refer to those aspect of an environmental investigation that are controllable by those responsible for undertaking the project or directly involved in the program - in a Quality Assurance Project Plan (QAPP) - internal controls would refer to the group or organization that developed the QAPP and the employees, volunteers, or participants designated specific tasks within the QAPP.

External QC - refers to a set of measures established for and conducted by those people and organizations outside of the program (example: an audit or replicate sample collection conducted by the state environmental regulatory agency, the regional EPA representative, or their designated auditing group). Photos are from the following sources: 1.) http://www.ars.usda.gov/is/np/fnrb/fnrb1001.htm asbestos containment chamber2.) http://il.water.usgs.gov/adcp/photos/class_apr99.html - recording data in the field QC information has been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

Some examples of internal quality control measures implemented by the group responsible for the project (including but not limited to the collection and evaluation of samples and data) would include:proper equipment calibration, training and certification of field technicians and other staff,proper documentation of all applicable information (see photo 2 field logging of collected data),prevention of cross contamination when collecting samples and data (see photo 1 asbestos chamber)

Can your group list examples of other internal measures of error control?Photos come from: 1.) http://www.ars.usda.gov/is/np/fnrb/fnrb1001.htm - auditing computer work on screen2.) http://www.tfhrc.gov/pubrds/novdec98/seven.htm - 3 person audit teamQC information has been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

External QC - set of measures established for those outside of the program

Examples of external QC controls include:performance audits conducted by outside personnel, usually agency representatives or hired consultantssplit sample analysis - where one sample is divided equally and sent to more than one laboratory (or annalist) for analysis, and collection of replicate samples (or duplicate samples when collecting only 2 samples) - two or more samples taken from the same location, at the same time, using the same method, but independently analyzed; generally by a person(s) from outside the organization conducting the investigation.

The QAPP is an invaluable planning and operating tool that should be developed in the early stages of the investigation, monitoring, or remediation project.Quality Assurance Project Plan (QAPPs) are used by agencies such as the EPA to ensure that the samples collected and analyzed, the data stored and manage, and reports that are written, are of high enough quality to meet project needs. When agencies are involved in data collection a QAPP is always used to oversee the agency selection of laboratories, data collection methods, selection of parameters to be measured, consistency of data analysis and confidence in data quality.Few agencies will use data unless methods of data collection, storage, and analysis can be documented. Clear and concise documentation of procedures also allows a project to continue over time and space using the same methods. This is particularly important to a project that extends over a long period of time and that intends to establish a baseline of quality information that can be compared over time.

QAPP agency uses have been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htmThe QAPP governs work in all aspects of the field, laboratory and office. The QAPP guides all areas of an investigation (field, lab and office) and includes: the selection of parameters and procedures to be used in the investigation, including standards and methods to be used in both the field an laboratory (examples reviewed in the previous slides), data management and analysis - how will those viewing the data assess the results, apply statistical evaluation, and make conclusions - which analytical methods are best for the data that needs to be obtained?and, what steps are taken to determine the validity of specific sampling or analysis procedures both quality assessment and quality control (QA/QC) what conclusions are determined from the information?

Just as in the collection and evaluation of data, several agencies, programs and resources exist to describe how to develop and use a QAPP that is applicable to your area of study. We will talk about this in more detail in the following lesson when we discuss how to develop a QAPP and what key elements to include. However, for now we will concentrate on the WHY of QAPP development by discussing data objectives and key concepts necessary to implementing QA/QC The PARCC Parameters (following slides). The PARCC Parameters include: PrecisionAccuracyRepresentativeness CompletenessComparability

The PARCC parameters are the WHY of the QAPP and QA/QC. The QAPP is developed in order that the key concepts of a well run and thorough environmental investigation may be met. Photo taken from http://www.d.umn.edu/faculty/ UMD med school lab workerKey concept definitions have been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

Precision is the degree of agreement among repeated measurements of the same characteristic on the same sample or on separate samples collected as close as possible in time and place. Precision gives an idea of how constant and reproducible field or laboratory data are. However, this is only an indication of how consistent results are under similar conditions, and does NOT necessarily mean that results are a true value or accurate. (See target diagram on the following slide)

Precision may have a BIAS which is the degree of systematic error present in a process. Meaning, that when bias is present the result may be close together (precise) but will differ from the true value or expected result. (See target diagram on the following slide)

Accuracy is a measure of confidence in measurement. The smaller the difference between the measurement of a parameter and its true or expected value (the less bias), the more accurate the measurement and the more reproducible its result. (See target diagram on the following slide)

Information adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

Accuracy is the level of confidence in measurement. The smaller the difference between the measurement of a parameter and its true or expected value (the more accurate the measurement) the more reproducible its result. Accuracy is determined by comparing the result of analysis for a sample to the known value for that sample such as a standard or a spike. Accuracy needs to be considered when choosing a method of analysis What level of accuracy is needed? Grater accuracy allows for greater Repeatability - provides for a reliable measurement when you need to confirm the consistency of an environmental condition or look at change over time. ( e.g., for reliable interpretation of trends over time the nitrification of a lake or remediation of a pollutant). Repeatability is usually acquired by specifying and sticking to a particular sampling and analytical method (methods discussed in slides 12 and 14).

Accuracy is reflective of both bias and precision (see target diagram). Discuss with the students the need for accuracy to be represented by both high precision and low bias. Photos: 1.) taken from WOW Model 8 (8-4-03)2.) taken from http://pubs.usgs.gov/fs/fs-0058-99/ - Minnesota LakeKey concept definitions have been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

Representativeness It stands to reason that the more accurate the data, the more representative it will be of the true or actual conditions at the site or area under investigation. As stated above, the more accurate the data, the more repeatable the results. Representativeness refers more to the sample location and selection. Are you really representing the conditions of the site when you choose specific sample or the specific sample location? (Simple Example: sampling for benthic organisms at the surface of a lake).Photo provided by NRRIKey concept definitions have been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

Completeness- is a data quality indicator generally expressed as a percentage. Completeness includes both the following (explanation and example for each provided below): 1.) an assessment of the amount of valid data obtained vs. amount planed for 2.) the design of a data collection plan to accurately assess the conditions

Deteriorated data gathering equipment will not gather usable data. (see photo) Deterioration of equipment need not be this extreme. It can include such simple problems as the biofouling of in-place data loggers or the sticking of the wiper on a self-wiping turbidity sensor. Or using equipment known to error under rigorous weather conditions. .

The Valid Data concept of completeness is important when you think of it as the amount of valid data obtained from a measurement system relative to the amount of valid data obtained under correct, normal conditions. In monitoring studies, it is not unusual to have some data that are invalid for a wide range of reasons (e.g., instruments that loose calibration, transcription errors, calculation/data reduction errors, lost samples, etc.). Generally, investigators try to minimize the amount of data lost because too much lost data can impact the statistical confidence of the analysis and cost extra money when data points need to be replaced, or whole datasets need to be discarded and re-sampled.

The Sample Design concept of completeness involves ensuring that the sample is designed to accurately assess conditions or trends across the entire population of interest. (e.g., using tools such as random sampling design or stratified sampling designs allow the investigator to fully characterize populations of interest; e.g., to fully characterize a fishery population you would want to sample at the lake AND its inflow and outflow streams, and you would want to look at all water layers when a lake is stratified - to avoid missing significant components in the population).Photos: 1.) taken from http://www.nrcs.usda.gov/programs/cta/ctasummary.html 2.) taken from http://www.nssl.noaa.gov/publicaffairs/backgrounders/backgrounder_ww.html - Winter data Key concept definitions have been adopted from U.S. Environmental Protection Agency Volunteer Monitors Guide to: Quality Assurance Project Plans. 1996. EPA 841-B-96-003, Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C. 20460, USA http://www.epa.gov/owowwtr1/monitoring/volunteer/qappexec.htm

Comparability - a method that works as well in fresh water may not work as well in saltwater (marine) environments! Different methods of analysis may have different limitations on their use because of the relative effects of different sample matrices. Another example - samples taken at different times of the day or at different seasons may not be comparable because of changes that occur with changing daylight or temperature. By specifying a consistent method you can compare two of more places or two places in time - why is this important?( Hint: e.g., to allow reliable comparisons in before and after measurements, or upstream and downstream measurements around an effluent at a point source)

Discuss with the students whether on not data collected under these varying climate conditions may or may not be comparable? What sort of sampling events may be affected by these varying conditions. Example: The collection of water samples for volatile organic compounds how would hot weather conditions effect the samples and their volatile nature. How about in turn cold weather? What if the samples froze? What if collected samples froze and broke the bottles in which they were stored? Discuss how this relates back to the concept of completeness. Discuss how it relates back to the need for QCs to prevent these problems from occurring.Use this slide as a opportunity to briefly review the meaning of QA and QC What are the key differences and similarities between the two?


Quality Control refers to the routine technical activities and established check and balances, the essential purpose of which is to control error. QC employs standardized tests and methods used in the field, laboratory, and office to generate quality, consistent, and repeatable data. Review with students the need for a QAPP (introduced in slides 15 to 21)Just as we discussed in the beginning of this presentation, QA/QC enters into all aspects of a project design, assessment, and evaluation. The QAPP governs actions to be taken in all aspects of the project including the field, laboratory, and the office. To assure a high quality investigation, consistent means of data acquisition and evaluation must be followed in all areas of the investigation.

A following lecture (13C) will concentrate on the development of a QAPP. How is one developed, by whom, and what to include and what considerations need to be addressed when compiling such a document and outlining such a program. The QAPP is most specific on details surrounding data acquisition and evaluation, or work conducted in the field and laboratory. To assure that data gathered and analysis conducted is of high quality and consistency, the PARCC are employed.

Review again briefly with the students the PARCC Parameters and what sort of QC measures can be used to assure the obtainment of each including: PrecisionAccuracyRepresentative (Repeatability)CompletenessComparabilityHow can one be aware of the QC measures that are to be used during implementation of the project? How can one make sure they are using the QC methods properly? BY REVIEWING AND REFFERING BACK TO THE QAPP.

The following lecture (13B) will go into more detail on the Whys of implementing the QC portion of a project plan and some of the How behind the whys. We will introduce Data Quality Objectives (DQOs) and discuss how we evaluate quality of data. We will discuss QC sample(s) applicable to assess for each key parameter and touch upon statistical calculation of percussion along with determination of accuracy and bias

4418This is the closing WOW slide that should appear at the end of all WOW presentations. This slide should not be altered in anyway. Do not add any additional elements to the slide, alter the web address, add additional copy, etc.

Mod13-A

Documents

environmental assessment

defined standards of

quality control qcphotos

environmental measurementunit

accurate measurement

data evaluationb

comparable data

qa processapplicable