Quality Control/ Quality Assurance Annabelle Allison ITEP/TAMS Center
Quality Control/Quality Assurance
Annabelle Allison
ITEP/TAMS Center
2
Quality Assurance
PLAN
DO
CHECK
FIX
KEEP DOING IT!
3
Quality Assurance Definitions
Common sense written down
“Integrated system of management activities involving planning, implementation, assessment, reporting, and quality improvement to ensure that a process, item, or service is type and quality needed and expected by client”
4
Quality System
Structured, documented management system
Describes policies, principles, organizational authority for ensuring quality needed in product
Provides framework for planning, doing, documenting, & assessing work and for carrying out QA/QC activities
5
Quality Control (QC)
Technical activities Measures performance of the process Compares against defined standards“Customer” sets standards & requirements
QC is part of QA
6
Quality Control
Check (Measure… compare with limits… graph it… document and report)
Fix
7
Quality
System: management (common sense)
Assurance: policies (serve our community to produce needed result)
Control: assessments (we check up on our work)
8
American National Standard ANSI/ASQC E4-1994
Specifications and Guidelines for Quality Systems for
Environmental Data Collection and Environmental Technology Programs
National consensus standard authorized by American National Standards Institute (ANSI)Developed by American Society for Quality Control (ASQC)Consistent with ISO standardsEPA Guidance for Quality Assurance Project Plans: EPA QA/G-5 based on E4
9
Why is a QAPP Important?
Saves moneySaves timeHelps communicationNo later misunderstandingsEverything contained in or referenced in one documentPublic document in case of litigation
10
Using SOPs
Easier to write than QAPPs
Describe what is done
Once SOPs are written, QAPP is much easier
Reference and attach SOPs to QAPP
11
Using Existing Materials
Plagiarism when writing QAPPs and SOPs is okay!Guidance for the Preparation of Standard Operating Procedures (SOPs) for Quality-related Documents: EPA QA/G-6 Use template QAPPs for BAM and TEOM in TAMS diskette
12
6 Elements of Data Management
(1) Data processing and transmission (WHAT)
(2) Data end use and integrity protection (WHY)
(3) Data access (WHO)
13
6 Elements of Data Management (cont.)
(4) Data dissemination (WHERE)
(5) Data storage and retrieval (HOW)
(6) Data disposal (WHEN)
Document everything!
14
Error
…the difference between your answer and the “truth”
15
Continuous PM QC
Instrument stability checks (EPA calls this general QA in their 1995 memo; see Attachment A in the TAMS template QAPPs)
Precision
Bias
16
Instrument Stability
Internal diagnostics conducted with each measurement (no error codes)TEOM uses standard filter kit available from dealer BAM uses standard foil or internal sourceUse these stability test kits more frequently at beginning of program, thereafter as necessary and at least twice a year
17
Results of Stability Checks
Use dealer’s guidelinesWithin 2.5% for TEOM standard filterWithin 7% for BAM foilPlot results on control chart (see Excel table and linked chart as figure 14-1 in TAMS template QAPPs)
18
Precision Error
Unavoidable
Sometimes up, sometimes down–“random”
Estimated by measuring the same thing several times
Minimized by carefully following procedures
19
Precision
Collocated analyzers is too expensive, so use repeated measurements of the same thing to estimate precision
Flow rate is the most important parameter for PM
Flow rate changes with time—how much it varies is estimate of precision error
20
Two Ways To Estimate Precision of Flow Rate
At beginning of program, use external flow rate transfer standard in field and compare flow rate transfer standard’s result with that shown (or set) on analyzer (one-point check)
Conduct this check at least every 2 weeks
21
During the First Months
Ensure external flow rate transfer standard is checked against another flow rate transfer standard at least once (every six months if you want to use second, easier way to assess precision)Record every check of analyzer’s flow rate, verify it’s within 4% of external flow rate transfer standard’s flow rate for 3 consecutive biweekly checks
22
Easier Way To Estimate Precision after Several Months
Use “alternative procedure:” Record analyzer’s set point flow rate, assume same flow rate if measured by external flow rate transfer standardRecord analyzer’s indicated flow rateCalculate relative percent difference, use value as precision error of flow rate (use relative percent difference calculation; see cells B5 – G28 in the Tribal Data Analysis spreadsheet, pg “PM10 Automated”)
23
Bias Error
Minimized by calibrating equipment against a standard
Make sure standard is certified against NIST-traceable standard (keep certificate!)Compare transfer standard to field equipment
Measures any bias in your equipment
24
Definitions
EPA calls “accuracy” what others call “total error”
Accuracy, or total error, combines both precision and bias errors
25
Continuous PM Accuracy
Can estimate only accuracy of flow rate Most important parameter, so that’s okay
Use external flow rate transfer standard to compare indicated (or set) flow rate of analyzer with that shown by transfer standard
26
Accuracy (cont.)
Flow rate transfer standard must not be same used to calibrate analyzer
Audit may be conducted by tribal staff, but not routine site operator
Conduct flow rate audit at least once per year for each analyzer
27
Accuracy Calculations
Relative percent difference between audit flow rate transfer standard and your analyzer’s flow rate is estimated accuracy of your flow rate
No other (inexpensive) way to estimate accuracy, so this is your estimated accuracy, or total error