Bear Basin in the Gallatin Mountain Range. MONTANA: MEASUREMENT UNCERTAINTY Lab Accreditation Meeting San Diego, March 12 th.

Bear Basin in the Gallatin Mountain Range

MONTANA: MEASUREMENT UNCERTAINTY

Lab Accreditation Meeting San Diego, March 12th

MT Department of Agriculture Lab• Analytical Chemistry Lab

• 8.625 FTE

• Montana State University

• Bozeman, MT

• ~2,500 samples per year

MT Department of Agriculture Lab

• ~1/2 Feed /Fertilizer samples

• ~1/2 Pesticide samples

• ISO 17025 Accreditation November 2014

• 2 Feed, 1 Fertilizer Method

• 0.5 FTE QAO position

Outline

• What is Measurement Uncertainty (MU) ?

• ISO Requirements for MU

• MT Approach to MU-how we did it!

What is Measurement Uncertainty (MU)?

• Uncertainty associated with measurements

• Doubt about the result of any measurement

• Defines possible range/spread of results

• Tells something about the quality

What is Measurement Uncertainty ?

• Variation in results when they are repeated

• May be due to natural variations

• Take a number of readings

• Average to get to “true” value

What is Measurement Uncertainty ?

• How widely spread are the values

• Standard deviation-quantify spread

• Judge quality of measurement from the spread

• %RSD: Relative Standard Deviation

• %RSD=Divide STDDEV by AVER and X 100

What is Measurement Uncertainty ?

ISO MU Requirements

• Reporting does not give wrong impression

• Result incomplete without a statement of uncertainty

• Needs to be understandable and relevant to the user

• REQUIRED: Procedure to estimate uncertainty

ISO MU Requirements

• Need to look at MU mathematically

• Need to put a number to the uncertainty

• You do not have to hire a statistician

• Reasonable estimation based on method

MT Approach

• Sent Staff to MU training

• Become familiar with basic steps

• Become familiar with uncertainty language

• Goal: Reasonable Estimation

• Use a practical user friendly approach

MT Approach• Established rules for calculating uncertainty

• Follow conventional methods

• Express uncertainty as consistently as possible

• Express uncertainty values properly

• Others need to understand what we have done

MT Approach• 1: Identify Approach

• 2: Identify sources of uncertainty

• 3: Estimate size of uncertainty from each source=%RSD

• 4: Combine individual uncertainties =overall figure

• 5: Expand Uncertainty

• 6: Create Uncertainty Budget

Identify Approach• Two approaches to estimating uncertainty

• Type A: use statistics from repeated findings

• Type B: Non states, other means

• Use Type A—why?

• A2LA classifies test methods for determining MU

• Method based on published consensus methods

Type A

• Use Laboratory Control sample (LCS)

• Appropriate matrix and concentration

• LCS has been through all method steps

• Apply Statistics to estimate MU

Identify Sources of Uncertainty

• 1- Analytical method

• 2- Sample PREP

• Both use type A approach

MT Approach• Lasolacid Method MU

SAMPNO RESULT MATRIX COLDATEAB10717 26.93 Feed 21-Mar-11AB10403 28.11 Feed 22-Feb-11AB10165 27.50 Feed 20-Jan-11AB10047 27.75 Feed 06-Jan-11AB04245 30.43 Feed 10-Dec-10AB04026 27.16 Feed 18-Nov-10AB01036 31.54 Feed 08-Apr-10AB00554 29.30 Feed 24-Feb-10AB00159 29.59 Feed 21-Jan-10AB00090 28.54 Feed 13-Jan-10AB00036 26.44 Feed 06-Jan-10AA93899 31.65 Feed 03-Dec-09AA93890 30.90 Feed 02-Dec-09AA93652 30.42 Feed 09-Nov-09AA90612 27.81 Feed 06-Mar-09AA90468 28.43 Feed 19-Feb-09AA90437 27.91 Feed 13-Feb-09AA84116 28.01 Feed 17-Dec-08AA83899 28.97 Feed 26-Nov-08AA83773 29.68 Feed 18-Nov-08AA83741 30.33 Feed 14-Nov-08

x (g/T)= 28.92SD(g/T)= 1.527

MT Example• 21 data points from AAFCO

• Collected over 3 years

• (g/T) = 28.92

• SD (g/T) = 1.527

• RSD (%) = 5.280

• MU= 5.28%

Sample PREP MU EXAMPLE

• Design set of experiments

• 3 Different samples

• Each samples: mass reduction to 4 splits

• Each split: 3 instrument measurements

MT Approach• EXAMPLE

• Sample PREP MU

• Hypothetical data

• Repeat for 3 samples

• AVER SD splits 3 samples

• Calc: %RSD of AVER SD

• %RSD= MU Sample PREP

Sample 1 SD method

Split 1

25.00

22.00

23.00

AVER split 1 23.33 1.53

Split 2

19.00

19.00

16.00

AVER split 2 18.00 1.73

split 3

18.00

17.00

15.00

AVER split 3 16.67 1.53

AVER SD method 1.60

AVER splits 19.33

SD method/splits 3.53

AVER SD method 1.60

SD splits 1.93

%RSD 9.97

MU sample PREP EXAMPLE 9.97%

MT Approach

• Unfamiliar Concepts/Terms

•Combine Uncertainties

•Expand Uncertainty

•Uncertainty Budget

MT Example: Lasolacid• Add 2 uncertainties together

• MU Sample PREP: ~10 %• MU method: 5.28%

• Use root sum square method (RSS)

• UC =

• Uc = (~10.02 + 5.282)1/2 = 11.31

• Combined Uncertainties: 11.31%

MT Example

Coverage factor K

• Numerical factor used as a multiplier

• To expand uncertainty

• Include more results at greater confidence

• Typically in the range 2 to 3

Degree of Freedom

• Degree of freedom: number of measurements - 1

• For >20 values, k = 2.0, degree of freedom = ∞

• Tabulated values of k shown below:

n -1 1 2 3 4 5 6 7 8 9 10

k(corr) 12.71

4.30 3.18 2.78 2.57 2.45 2.37 2.31 2.26 2.23

n -1 11 12 13 14 15 16 17 18 19 20

k(corr) 2.20 2.18 2.16 2.15 2.13 2.12 2.11 2.10 2.09 2.09

Lasalocid Measurement Uncertainty• The data from LCS AAFCO Lasalocid 2007-30 was used to

estimate MU• U1: Sample prep 21 data points • U2: Testing process 21 data points

• Uc = (~10.02 + 5.282)1/2 = 11.31

• U = (2.0 x 11.31) = 22.62

Symbol Source of UncertaintyValue (%)

Distrb Div Uncert (1σ)Degree Freedm

U1 Sample Preparation ~10.0 N 1 ~10.0 ∞

U2 Testing process 5.28 N 1 5.28 ∞Uc Combined Uncertainty 11.31 %

U Expanded Uncertainty (k=2.0) 22.62%

Summary of Determination

• Combined Uncertainty : 11.31%

• Expand Uncertainty: 11.31 X 2.0 = 22.62%

• Average: 28.92 g/T

• 22.62% of 28.92g/T= ± 6.54 g/T

• 95% confident result will be 22.38-35.46 g/T

• YAHOO!!

Conclusions

• Deriving measurement uncertainty is so doable

• Standard protocol for determining uncertainty

• This is NOT a stumbling block for accreditation

• Practical approach—usable and manageable

References

• International Standard, ISO/IEC 17025, Section 5.4 Test and calibration methods and method validation. Second edition 2005-05-15, Reference number ISO/IEC 17025:2005(E).

• Montana Department of Agriculture Quality Management System for ISO accredited methods

• A2LA Introduction to Measurement Uncertainty-Training Course 2013

• A Beginners guide to Uncertainty in Measurement , Stephanie Bell, NPL, Issue 2

“A person who never made a mistake never tried anything new.”

Albert Einstein

MIDWEST AOAC meeting

• June 8-10th 2015

• Bozeman, MT

• SYMPOSIA:• Pesticides at work in the Agriculture LAB  • State AG FEED LABS going ISO-Say what?  • Veterinary Toxicology and Mycotoxins • NFTA technical presentations  • Food Safety:  Chemistry and Microbiology Applications • ICP workshop   • LIMS issues

• SPECIAL EVENTS:• Vendor expo and presentations• Tuesday evening social: Mountain Chalet surrounded by snowcapped mountains• Casper-Helfe Memorial Golf Outing