Developing a Total Uncertainty Program for the JAF Environmental Laboratory- Lessons Learned James Furfaro James Furfaro Entergy NNE Entergy NNE White.

Developing a Total Developing a Total Uncertainty Program for Uncertainty Program for the JAF Environmental the JAF Environmental

Laboratory-Laboratory-Lessons LearnedLessons Learned

James FurfaroJames Furfaro

Entergy NNEEntergy NNE

White Plains, White Plains, NYNY

The James A. FitzPatrick The James A. FitzPatrick (JAF) Environmental(JAF) Environmental

LaboratoryLaboratory

James A. FitzPatrick James A. FitzPatrick Environmental LaboratoryEnvironmental Laboratory

The JAF Environmental Laboratory does environmental analysis for:

-Indian Point Units 2 &3-James A. FitzPatrick Plant-Nine Mile Point Units 1 &2-Ginna

Self-Assessment of the JAF Self-Assessment of the JAF Environmental LaboratoryEnvironmental Laboratory

Compare the JAF E-lab to an industryCompare the JAF E-lab to an industry standard of excellence, ANSI N42.23.standard of excellence, ANSI N42.23.

ANSI N42.23 presents 11 elements for ANSI N42.23 presents 11 elements for a laboratory QA program.a laboratory QA program.

The JAF E-lab does not calculate total The JAF E-lab does not calculate total uncertainties for their analytical values.uncertainties for their analytical values.

Approach for Calculating Approach for Calculating Total Uncertainty (TU)Total Uncertainty (TU)

Use NIST Technical Note 1297 and the Use NIST Technical Note 1297 and the GUM.GUM.

Calculate TUs for each type of Calculate TUs for each type of analysis.analysis.

Look at the major error components Look at the major error components of each analysis.of each analysis.

Break the components down into Break the components down into counting errors and systematiccounting errors and systematic errors. errors.

““Guide To The Expression Guide To The Expression of Uncertainty in of Uncertainty in

Measurement” (GUM)Measurement” (GUM)

The NIST policy includes the The NIST policy includes the approach given in the “Guide To approach given in the “Guide To The Expression of Uncertainty in The Expression of Uncertainty in Measurement.” Measurement.”

The GUM was prepared by The GUM was prepared by individuals from many international individuals from many international organizations. (ISO 1995)organizations. (ISO 1995)

““Creating A Level of Creating A Level of Confidence for the Confidence for the

Measurement”Measurement”

y ± k c(y)where :

c(y) = the combined standard uncertainty of y. k = coverage factor: 1, 2 or 3. y = the estimate of the quantity you are trying to measure.

Root-Sum-Of-The-Squares Root-Sum-Of-The-Squares MethodologyMethodology

Let c(y) = y{[(x1)/x1]2 + [(x2)/x2]

2 +…[(xN)/xN]2}1/2

where: (xi) = the standard unc. of xi. c(y) = the combined standard unc. of the estimate y.

Estimate Error Terms -Estimate Error Terms -Sample ReceiptSample Receipt

Sample PreparationSample Preparation

Sample Counting and Sample Counting and AnalysisAnalysis

Data AnalysisData Analysis

Another Pair of HandsAnother Pair of Hands

In April 2001, hired a contractor to In April 2001, hired a contractor to help finish the estimate of the help finish the estimate of the error terms for each individual error terms for each individual analysis.analysis.

NIST and/or GUM methodology NIST and/or GUM methodology would be used.would be used.

Problems/ConfusionProblems/Confusion We were planning on issuing this We were planning on issuing this

work as a formal calculation, work as a formal calculation, December 2001.December 2001.

Our contractor presented us with Our contractor presented us with the calculation. He used a the calculation. He used a methodology based on Bruckner, methodology based on Bruckner, LANL. We were told that we LANL. We were told that we couldn’t use the root-sum-of-the couldn’t use the root-sum-of-the squares methodology.squares methodology.

Confusion-NIST uses the root-sum-Confusion-NIST uses the root-sum-of-the-squares methodology?of-the-squares methodology?

A Lesson LearnedA Lesson Learned

““When systematic errors are When systematic errors are small relative to random errors, small relative to random errors, the root-sum-of-the-squares the root-sum-of-the-squares methodology provides reasonable methodology provides reasonable estimates of the uncertainty.”estimates of the uncertainty.”

(Bruckner,1993Bruckner,1993)

A Lesson Learned A Lesson Learned (Continued)(Continued)

““When systematic errors are When systematic errors are large relative to random errors, large relative to random errors, the root-sum-of-the-squares the root-sum-of-the-squares methodology leads to uncertainty methodology leads to uncertainty estimates that are too small.”estimates that are too small.” (Bruckner,1993Bruckner,1993)

Bruckner’s ApproachBruckner’s Approach

Empirical formulaEmpirical formula It gives realistic estimates when It gives realistic estimates when

the systematic error is larger than the systematic error is larger than the random error.the random error.

Very difficult to use when you have Very difficult to use when you have several random and systematic several random and systematic error terms.error terms.

Four Important Four Important Parameters in the GUMParameters in the GUM

Sensitivity FactorsSensitivity Factors Avoid/minimize correlating effectsAvoid/minimize correlating effects Realistically Estimate Degrees of Realistically Estimate Degrees of

FreedomFreedom Estimate the type of distribution Estimate the type of distribution

for Type B evaluation.for Type B evaluation.

GUM Method for GUM Method for Estimating TUEstimating TU

Estimated standard deviations Estimated standard deviations are summed using the root-are summed using the root-sum-of-squares methodology sum-of-squares methodology to obtain the total standard to obtain the total standard deviation (uncertainty) for the deviation (uncertainty) for the measurement.measurement.

Law of Propagation of Law of Propagation of Uncertainty (GUM)Uncertainty (GUM)

[c(y)] = y{ [c1 x (x1)/x1]2 + [c2 x (x2)/x2]2 + … [cN x (xN)/xN]2}1/2

where:c(y) = the combined standard unc. of the estimate y. (xi) = the standard unc. of xi. ci = the sensitivity factor.

Confidence Interval (GUM)Confidence Interval (GUM)

y ± t/2,eff x c(y)

where :

t/2,eff = t statistic with a level of significance of . eff = the effective degrees of freedom.

2total = 1

2 + 22 + 3

2 + ….

Normal Distributions

y ± k total

Other Distributions

utotal2 = u1

2 + u22 + u3

2 + ….

u = estimated standard deviation of a component.

Effective Degrees of Effective Degrees of Freedom (GUM)Freedom (GUM)

N

i i

i

i

N

i i

i

eff

xxu

xxu

1

4

22

1

)(

)(

eff = the effective degrees of freedom.

Summary of Individual Summary of Individual Components-Gamma Components-Gamma

Spec.*Spec.*Assay Type

Media A1 A2 B1 B2 B3

Gamma Spec.

No Iodine Extract.

Net Peak Area Uncertainty (determined at count time)

0.8 % Calib. Stand.

1.2% Calib. Stand.

TBD Eff. Calib.

Gamma Spec.- I-131

Iodine Extract.- Water/Milk

Net Peak Area Uncertainty (determined at count time)

0.8% Calib. Stand.

1.2% Calib. Stand.

TBD Eff. Calib.

1% Iodine Ext. Yield

*Uncertainty components reported at 1 standard deviation.

Efficiency Versus EnergyEfficiency Versus Energylog-log plotlog-log plot

Summary of Individual Summary of Individual Components- Beta Components- Beta

AnalysisAnalysis

AssayType

Media A1 A2 B1 B2

GrossBeta

EvaporatedLiquid

Net CountRateUncertainty(determinedat counttime)

0.2%CalibrationStandard

1.7%CalibrationStandard

5%SelfAbsorptionFactor

GrossBeta

FilterPaper

Net CountRateUncertainty(determinedat counttime)

0.2%CalibrationStandard

1.7%CalibrationStandard

*Uncertainty components reported at 1 standard deviation.

Summary of Lessons Summary of Lessons LearnedLearned

Estimating error terms is a very Estimating error terms is a very time consuming process.time consuming process.

Try to identify the major random Try to identify the major random and systematic error terms for and systematic error terms for each analysis. If you overlook an each analysis. If you overlook an error term, you can always go back error term, you can always go back and “sharpen the pencil.”and “sharpen the pencil.”

Summary of Lessons Summary of Lessons Learned (Continued)Learned (Continued)

The GUM allows you to use the Root-The GUM allows you to use the Root-Sum-of-the-Squares Methodology.Sum-of-the-Squares Methodology.

When the systematic error component When the systematic error component is relatively large compared to the is relatively large compared to the random component, the GUM random component, the GUM methodology leads to reasonable methodology leads to reasonable estimates of the uncertainty. estimates of the uncertainty.

It does this through the use of four It does this through the use of four parameters.parameters.

Dr. Kenneth Inn says:“Always sharpen the pencil...,

…but Don’t Sharpen the Marshmallow.”

ReferencesReferences ANSI N42.23. “American National Standard ANSI N42.23. “American National Standard

Measurement and Associated Instrumentation Measurement and Associated Instrumentation Quality Assurance for Radioassay Laboratories, IEEE, Quality Assurance for Radioassay Laboratories, IEEE, NY 1997.NY 1997.

Bruckner, L.A. “Propagation of Variance Uncertainty Bruckner, L.A. “Propagation of Variance Uncertainty Calculation for an Autopsy Tissue Analysis” Health Calculation for an Autopsy Tissue Analysis” Health Physics, 67 (1):24-33, July 1994 (Appendix C).Physics, 67 (1):24-33, July 1994 (Appendix C).

Bruckner, L.A. “Including Random and Systematic Bruckner, L.A. “Including Random and Systematic Errors in Measurement Uncertainty” LA-UR-93-932, Errors in Measurement Uncertainty” LA-UR-93-932, Los Alamos National Laboratory, NM, 1993Los Alamos National Laboratory, NM, 1993

““Guidelines for Evaluating and Expressing the Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results” Uncertainty of NIST Measurement Results” NiST Technical Note 1297, 1994.NiST Technical Note 1297, 1994.

““Guide to the Expression of Uncertainty in Guide to the Expression of Uncertainty in Measurement” ISO, 1995.Measurement” ISO, 1995.