QA Strategy Workgroup Ambient Air Quality System Training Ambient Air Monitoring Quality Systems Requirements Mike Papp- OAQPS 24-H our,5-m inute Flow R ates 15 15.5 16 16.5 17 17.5 18 1 8 15 22 29 36 43 50 57 64 71 78 85 92 99 106 113 120 127 134 141 5-m in data F low R a te (L /m in
42
Embed
QA Strategy Workgroup Ambient Air Quality System Training Ambient Air Monitoring Quality Systems Requirements Mike Papp- OAQPS.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
QA Strategy Workgroup
Ambient Air Quality System Training
Ambient Air Monitoring Quality Systems Requirements
“(2) Establishment of a national network to monitor, collect, and compile data with quantification of uncertainty in the status and trends of air emissions, deposition, air quality, surface water quality, forest condition, and visibility impairment and to ensure the comparability of air quality data collected in different States and obtained from different nations.”
QA Strategy Workgroup
Ambient Air Quality System Training
3
Major 40 CFR Pt. 58 App. A QA Requirements
• Section 1– General Info– Applicability– Definitions– Differences SLAMs/PSD
QAPP - Project Specific•Identifies the reasons for collecting data and for collecting it in a specific way•Documents how the data are collected and how quality is maintained
All QAPPs Are Not Created Equal- the Graded Approach
The level of detail in each QA Project Plan will vary according to the nature of the work being performed and the intended use of the data.
High Road- Big important decisions with little uncertainty
Low Road – Small studies without major consequences or an allowance of greater uncertainty
OAQPS has developed a graded approach for ambient air monitoring
http://www.epa.gov/ttn/amtic/geninfo.html
QA Strategy Workgroup
Ambient Air Quality System Training
8
TURBO-QAPP
QA Strategy Workgroup
Ambient Air Quality System Training
9
QA Strategy Workgroup
Ambient Air Quality System Training
10
SOPs
• They can never be too detailed– Remember those coming after you
• They are easy to write!• Look at the reference methods in CFR for the
minimum requirements• Don’t reinvent the wheel –Share good SOPs-
Ask vendors for electronic versions• Use document control • Use quality bulletins for changes
Army Field Manual
QA Strategy Workgroup
Ambient Air Quality System Training
11
Independent Quality Management
• who functions independently of direct environmental data generation,
• who reports on quality issues to the senior manager having executive leadership authority for the organization; and
• who has sufficient technical and management expertise and authority to conduct independent oversight of and assure the implementation of the organization’s quality system in the environmental programs of the organization.
EPA QA Policy Requires: A quality assurance manager (QAM), or person/persons assigned to an equivalent position:
QA Strategy Workgroup
Ambient Air Quality System Training
12
Establishment of Primary Quality Assurance Organizations
Common factors of PQAOs• Operation by a common team of field operators
according to a common set of procedures;• Use of a common QAPP or standard operating
procedures;• Common calibration facilities and standards;• Oversight by a common quality assurance
organization; and• Support by a common management, laboratory or
headquarters.
QA Strategy Workgroup
Ambient Air Quality System Training
13
Possible PQAO Consolidations
• Consolidation of Tribes within a State
• Consolidation of Tribes across States within an EPA Region
• Consolidation of Tribes with State PQAO
• Consolidation of Tribes across EPA Regions
QA Strategy Workgroup
Ambient Air Quality System Training
14
Use of NIST Traceable Standards(CFR Part 58 App A Sec 2.6.1)
“Gaseous pollutant concentration standards must be traceable to either a National Institute of Standards and Technology (NIST) Traceable Reference Material (NTRM) or a NIST-certified Gas Manufacturer’s Internal Standard (GMIS), …
Vendors advertising certification…. must participate in the EPA Protocol Gas Verification Program or not use ‘‘EPA’’ in any form of advertising.”
QA Strategy Workgroup
Ambient Air Quality System Training
15
Certifications
Testifies to the accuracy of an instrument/standard/gas as
it compares to a NIST traceable standard.
•Make sure you are aware of how long a cylinder is certified for. It’s different for different gasses and concentrations
•Create a tracking mechanism
•Follow guidance for your MQOs in QAPP
QA Strategy Workgroup
Ambient Air Quality System Training
16
Use of Federal Reference Methods (FRM) anmd Federal Equivalent Methods (FEMs)
• All monitors used for comparison to NAAQS must be FRM/FEM (40 CFR Part 53)
• Non-NAAQS monitoring has more flexibility• EPA National Environmental Research
Laboratory (NERL) develops an FRM for each NAAQS
• NERL approves each FRM/FEM– Posted on AMTIC
• Each has acceptable operating range(s)– Pick the range that matches your need.
http://www.epa.gov/ttn/amtic/criteria.html
QA Strategy Workgroup
Ambient Air Quality System Training
17
PerformanceCharacteristics of Gaseous FRM/FEM Methods
• Acceptable levels of – detection– precision– bias– drift
• Relatively unaffected by interfering compounds
• Sensitive at typical ambient concentrations
• Commercially viable• Supports reliable and relatively
maintenance-free operation• Easily “QC-able”
QA Strategy Workgroup
Ambient Air Quality System Training
18
Quality Control and Evaluation
• Quality Control Checks
• Performance Evaluations
• Technical Systems Audits
QA Strategy Workgroup
Ambient Air Quality System Training
19
Understanding and Controlling Uncertainty in Order to Minimize Decision Errors
Bias +10% * +7%**Based on 3 years of data so any individual value can be higher than MQO gaseous P & B estimates at site level
QA Strategy Workgroup
Ambient Air Quality System Training
21
QC Checks
QC Checks have a variety of purposes• To estimate our errors and keep within limits
(internal program use, stay ahead of auditors)• To troubleshoot performance of equipment
(lamp intensity, pump efficiency)• To identify drift over time• To keep system stable (instrument checks,
shelter conditions)• To report to EPA for their use
A QC Check is essentially an audit on yourself...
QA Strategy Workgroup
Ambient Air Quality System Training
22
QC Checks• One Point QC Check
– Biweekly- Used to estimate precision and bias at the site level.
– Select QC Check concentration at around routine concentrations.
• Annual Performance – More thorough independent check– Independent standards (and possibly the person)– 3 consecutive ranges (some flexibility to concentration
levels in CFR
QA Strategy Workgroup
Ambient Air Quality System Training
23
NPAP-TTP PEP
NATTS PT &ORIA Round Robins
SRP PAMS Cylinders
Protocol Gas
Performance Evaluation-What and Why
Performance evaluations (PEs) are a type of audit in which the quantitative data generated in a measurement system are obtained independently and compared with routinely obtained data to evaluate the proficiency of an analyst, or a laboratory
QA Strategy Workgroup
Ambient Air Quality System Training
24
3 NPAP TTP Mobile Platforms
QA Strategy Workgroup
Ambient Air Quality System Training
25
QA Strategy Workgroup
Ambient Air Quality System Training
26
Technical Systems Audits
Two types of Audits:
Internal: Conducted by your own agency
QA group. This group is independent of the monitoring group
External: Conducted by your Regional EPA
staff or another agency. This group is independent of your agency completely
QA Strategy Workgroup
Ambient Air Quality System Training
27
Acceptance Criteria for Calibrations, QC Checks, and Instrument Audits
Look for these tables
All the ranges and parameters you will need are neatly laid
out here
QA Strategy Workgroup
Ambient Air Quality System Training
Statistical Data Assessments
100
audit
auditmeasdi
QA Strategy Workgroup
Ambient Air Quality System Training
29
Another Way of Looking at It…#
of O
bser
vatio
ns
% Difference
0 +10-10
Ideal: Precise w/o Bias
Precise w Negative Bias Precise w Positive Bias
Low Precision w/o Bias
Low Precision w Negative Bias Low Precision w Positive Bias
QA Strategy Workgroup
Ambient Air Quality System Training
30
Precision
Precision is more conservative when evaluated atthe 90% one-sided upper confidence level
Defines the “Width” of the
Bell Curve
Gaseous Estimate
21 1
22
1
)1(
)(_Pr
n
n
i
n
iii n
nn
ddnEstimateecision
truth
90th Percentile of a Chi-Squared Distribution
More Conservative
QA Strategy Workgroup
Ambient Air Quality System Training
31
truthBias
n
iidn
AB1
||1 Central
Tendency
95th Quartile of a Student-t Distribution
w/ n-1 df
Standard Error
)1(
|)|(||1 1
22
nn
ddnAS
n
i
n
iii
n
AStABABEstimateBias n 1,95.0||_
QA Strategy Workgroup
Ambient Air Quality System Training
32
DASC ToolHolds Up
to 500 Pairs of Values
Graphs & Stats
Automatically Updated
References Equations in Appendix A
QA Strategy Workgroup
Ambient Air Quality System Training
33
DASC Tool (Cont’d)
Easy to Use Navigation to Find
the Stats You Need
QA Strategy Workgroup
Ambient Air Quality System Training
34
AMP255 Features
Statistics Shown on a per Site
and Per PQAO Level as
Appropriate
QA Strategy Workgroup
Ambient Air Quality System Training
35
Box and Whisker Plots
*
*
*
+
Lower FenceQ1 – (1.5*IQR)
Upper FenceQ3 + (1.5*IQR)
InterquartileRange (IQR)
Outliers
Whiskermin
Whiskermax
Q1 (25th Percentile)
Q2 (50th Percentile)Q3 (75th Percentile)
Mean
QA Strategy Workgroup
Ambient Air Quality System Training
36
Box and Whisker Plots
Posted Box and Whisker Plots are Available at:www.epa.gov/ttn/amtic/parslist.html
Remember Me?
QA Strategy Workgroup
Ambient Air Quality System Training
Don’t ForgetThe
It contains a number of the QC Checks Also!!!
QA Strategy Workgroup
Ambient Air Quality System Training
38
What Data is in AQS?
• Collocated Data
• Flow Data
• Annual Performance Audits
• NPAP Data
• Technical Systems Audit Information???
QA Strategy Workgroup
Ambient Air Quality System Training
Guidance On All This!
QA Strategy Workgroup
Ambient Air Quality System Training
40
OR
GA
NIZ
AT
ION
/PR
OG
RA
M
PR
OJE
CT
PO
LIC
Y/R
EG
UL
AT
ION
S
Defensible Products and Decisions
EPA Program &Regional Policy
External PoliciesContracts - 48 CFR 46
Assistance Agreements - 40 CFR 30, 31, and 35
Internal EPA PoliciesEPA Order 5360.1EPA Manual 5360
Consensus StandardsANSI/ASQC E4ISO 9000 Series
Annual Review and Planning
(e.g., QAARWP)
SystemsAssessments(e.g., QSAs)
Quality SystemDocumentation
(e.g., QMP)
Training/Communication(e.g., Training Plan,
Conferences)
Supporting System Elements(e.g., Procurements,
Computer Hardware/Software)
TechnicalAssessments
IMPLEMENTATIONPLANNING ASSESSMENT
Conduct Study/Experiment
QA Project Plan
SystematicPlanning
(e.g., DQO Process)
StandardOperating
Procedures
Data QualityAssessment
Data Verification& Validation
Ambient AirCAA, CFR 50,53, 58
Regional Policy
OR
GA
NIZ
AT
ION
/PR
OG
RA
M
PR
OJE
CT
PO
LIC
Y/R
EG
UL
AT
ION
S
Defensible Products and Decisions
EPA Program &Regional Policy
External PoliciesContracts - 48 CFR 46
Assistance Agreements - 40 CFR 30, 31, and 35
Internal EPA PoliciesEPA Order 5360.1EPA Manual 5360
Consensus StandardsANSI/ASQC E4ISO 9000 Series
Annual Review and Planning
(e.g., QAARWP)
SystemsAssessments(e.g., QSAs)
Quality SystemDocumentation
(e.g., QMP)
Training/Communication(e.g., Training Plan,
Conferences)
Supporting System Elements(e.g., Procurements,
Computer Hardware/Software)
TechnicalAssessments
IMPLEMENTATIONPLANNING ASSESSMENT
Conduct Study/Experiment
QA Project Plan
SystematicPlanning
(e.g., DQO Process)
StandardOperating
Procedures
Data QualityAssessment
Data Verification& Validation
Ambient AirCAA, CFR 50,53, 58
Regional Policy
G-4
R5-G-5
EPA Guidance for planning, implementation, and assessment activities.