Sampling:oenthesourceofmost uncertaintyinanalycalmeasurements · 2020. 7. 24. · Samplingaspartofthe* Measurement*Process* Sampling Physical sample preparation Analysis Sampling

Sampling:  o,en  the  source  of  most  uncertainty  in  analy6cal  measurements  

Prof  Michael  H.  Ramsey  School  of  Life  Sciences,  University  of  Sussex,  Brighton,  UK  [email protected]    L.S.  Theobald  Lecture  for  2012,  Analy9cal  Research  Forum,                        GSK/University  of  HerEordshire,              8th  July  2013  

Overview  –  ques9ons  addressed  1.  Is  uncertainty  of  measurement  (U)  –  inevitable?  evaluaHon  beneficial?  2.  Where  does  the  measurement  process  start?  3.  What  are  the  sources  of  U?  4.  What  are  methods  for  esHmaHng  U  from  sampling  &  analysis    

–  with  case  studies  in  different  media  (e.g.  soil,  food)  5.  How  large  is  U,  and  what  proporHon  arises  from  sampling?  

–  for  food,  soil,  gas,  water  -­‐  with  ex  situ  and  in  situ  measurements  on  range  of  analytes    

6.  How  much  U  is  acceptable?  –  are  measurements  fitness-­‐for-­‐purpose?  

7.  How  can  U  be  changed  to  achieve  fitness-­‐for-­‐purpose?    8.  ImplicaHons  for  scope  and  role  of  analyHcal  chemistry?  9.  Conclusions  

Uncertainty:  inevitable?  -­‐  and  evalua6on  beneficial?  

•  It  is  well  established  that  any  analyHcal  measurement  result  is  only  an  esHmate  of  the  true  value  of  the  analyte  concentraHon.  

•  Measurement  results,  therefore,  inevitably  contain  uncertainty.    

•  Knowing  the  REALISTIC  value  of  this  uncertainty  gives  vital  benefits  to  users  who  interpret  the  measurements  e.g.  –  to  test  a  scienHfic  hypotheses  (e.g.  H1:  contaminaHon  has  declined)  –  to  make  a  management  decisions  (probabilisHcally)  

•  e.g.  ac9on  required    

Uncertainty  of  measurement  (U)  

‘An  esHmate  a]ached  to  a  test  result  which  characterises  the  range  of  values    within  which  the  true  value  is  asserted  to  lie’  (ISO  3534-­‐1:  3.25,  1993*)    •  e.g.  U’  =  20%  on  measurement  of  390  mg/kg  Pb  in  soil  

–  Range  of  U’  =  390+20%  =468  mg/kg                                  to                  390-­‐20%  =312  mg/kg  

•  We  can  never  know  the  true  value  (could  be  460  mg/kg)  •  Effect  on  comparison  with  threshold  value  of  450  mg/kg  

i.e.    Measured  value  390  mg/kg  is  below  the  threshold  value      True  value  460  mg/kg  is  above  the  threshold  value  

             Measured  value  gives  ‘false  nega9ve’  classifica9on  •  All  we  need  to  know  is  how  far  from  the  truth  we  might  be    

–  e.g.  with  19/20  chance  if  being  right  =  95%  confidence  •  How  can  we  esHmate  uncertainty  of  measurements  –  including  that  from  

sampling?  –described  below  

•  Recent  defini9on  of  U  more  cryp9c:    ‘Parameter,  associated  with  the  result  of  a  measurement,  that  characterizes  the  dispersion  of  the  values  that  could  reasonably  be  a]ributed  to  the  measurand’  (GUM_JCGM_100,  2008)  

�   390  

468  

312  

x  

Aims  for  U  in  Analy6cal  Chemistry    

•  AnalyHcal  chemists  generally  aim  to:-­‐  –  reduce  U  of  measurements,  by    –  improving  the  precision,  and  reducing  bias  in  analyHcal  methods  

•  This  aim  has  two  problems:  it  assumes:-­‐  –  no  other  significant  sources  of  U    –  a  minimum  level  of  uncertainty  is  always  desirable  

-­‐  even  when  cost  of  pursuit  may  be  disproporHonate  to  benefits  produced    -­‐  review  these  assump9ons  

Where  does  the  measurement  process  start?  

•  The  soluHon  requires  a  fresh  look  at  the  ‘measurement  process’  •   Measurement  process  aims  to:  

–  esHmate  true  value  of  analyte  concentraHon  (i.e.  the  measurand)  in    –  some  mass  of  material  (i.e.  the  sampling  target).    

•  Measurement  process  therefore  starts  when:  –  porHon  of  material  (i.e.  a  sample)  is  selected    –  to  represent  the  sampling  target    

•  e.g.  an  area  of  land,  or  a  batch  of  product  –  unless  the  sampling  target  is  defined  as  the  lab  sample  (in  some  regula9ons)  

•  Therefore  U  not  just  from  analyHcal  process  –    –  must  include  contribuHons  from  primary  sampling  and    –  all  of  the  physical  preparaHon  of  the  sample    

•  e.g.  prior  to  receipt  by  the  lab  •  More  evident  for  in  situ  measurements    

–  e.g.  placing  PXRF  implicitly  involves  taking  a  sample  •  analysts  is  also  acHng  as  the  ‘sampler’  •  has  implicaHons  for  the  scope  of  analyHcal  science  

Sampling  as  part  of  the  Measurement  Process  

Sampling

Physical sample preparation

Analysis

Sampling Target Collection of a single sample, or several increments combined into composite sample

Primary Sample Comminution and/or splitting

Sub-sample Further comminution and/or splitting

Laboratory sample

Physical preparation, e.g. drying, sieving, milling, splitting, homogenisation

Test sample Selection of test portion for chemical treatment preceding chemical analysis

Test portion Chemical treatment leading to analytical determination

Test solution Analytical determination of analyte concentration

Process step Form of material

Description of process step

More  careful  use  of  the  word  ‘sample’  

TradiHonal  view  of  measurement  process  (clear  boxes)  

Who  is  responsible  for  primary  sampling?  -­‐  how  is  quality  of  sampling  ensured?  

•  AnalyHcal  chemists  tradiHonally  considered    measurement  process  to  begin  when  sample  received  by  the  lab    

•  So  theory  and  pracHce  of  sampling  lek  to  other  people.    •  Different  philosophy  and  approach  to  ensuring  ‘quality’  has  

therefore  developed    for  sampling  •  e.g.    ‘correct’  sampling  giving  rise  to  a  ‘representaHve’  sample    (Gy  1992*)  

–  not  compaHble  with  approach  used  for  chemical  analysis.  –  e.g.  ‘correct’  analyHcal  methods  are  not  assumed  to  give  true  values  of  analyte  concentraHon.  

•  Needs  new  integrated  approach  for  whole  measurement  process**  

*Gy,  P.M.  (1992)    Sampling  of  heterogeneous  and  dynamic  material  systems;  theories  of  heterogeneity,  sampling  and  homogenizing.  Elsevier,  Amsterdam    pp  653.  (page  567)  **  Ramsey  M.H.  and  Boon  K.A.  (2010).  Geostandards  and  GeoanalyHcal  Research,  34,  293-­‐304.                                                            doi  10.1111/j.1751-­‐908X.2010.00104.x    

Sources  of  uncertainty:  heterogeneity  and  ambiguity      

•  All  sampling  targets  are  heterogeneous  to  some  extent,    •  All  measurement  procedures  have  some  level  of  ambiguity    

–  e.g.  exactly  where  to  take  a  sample    

•  Heterogeneity  and  ambiguity  combine  and  interact  to  contribute  U  in  final  measurement  result.    –  e.g.  analogy  with  green  grains,  then  with  le]uce,  soil,  bu]er  

•  Visual  analogy  of  the  effects  of  in  situ  heterogeneity    –  white  grains  contaminated  with  green  grains    –  what  is  the  concentraHon  of  green  grains?    

•  expressed  as  %  of  all  grains  •   what  is  the  uncertainty  of  single  measurements?  

Pb heterogeneity by X-ray Micro-Probe = 51%

Analogy  

Es9mate  the  concentra9on  of  ‘contaminated’  green  grains  in  a  batch  

Sample  area  =  10%  of  popula9on  area  

Expect  4  green  grains  

=  Batch  =  Popula6on  

40  green  grains  

out  of  400  

True  Conc.  =  10%  

Analogy  to  show  U  caused  by  heterogeneity  in  the  sampling  target  

Sample  #1  

6  grains  

out  of  40  

ConcentraHon  

=  15%  of  grains  

Single  measurement  

-­‐  Posi9on  of  sample  ambiguous  

-­‐   no  idea  of  whether  conc  es9mate  is  reliable    

-­‐   no  idea  of  uncertainty  

Sample  #2  

3  grains  

out  of  40  

Conc  =  7.5%  of  grains  

Two  measurements  

-­‐ Can  judge  that  conc  es9mate  is  not  very  reliable  

-­‐ see  uncertainty  (U)  is  appreciable  

-­‐ What  is  U?  

Sample  #3  

2  grains  

out  of  40  

Conc  =  5%  

Sample  #4  

5  grains  

out  of  40  

Conc  =  12.5%  

Es6ma6ng  uncertainty(random  component)      Standard  uncertainty  (u  =  smeas)  =  4.56%  of  grains  

Expanded  uncertainty  (U,  2s  for  95%  conf)  =  9.1%  of  grains  

RelaHve  expanded                                                                =  91%  of  measurement    U  first  measurement  =  15  +/-­‐  9.1  %  of  grains            U  =  range  within  which  the  true  value  (10)  lies  =  5.9  to  24.1%  -­‐  Can  be  es9mated  from  just  2  observa9ons  (from  each  of  8  targets)      Only  includes  random  component    –  not  yet  systema6c  component,  e.g.  from  sampling  bias    –  e.g.  grains  not  randomly  distributed,  so  sample  under-­‐represents  green  grains  (e.g.  lower  down  in  container)    

c  –  U    =  5.9%  

•    c  =  15%  of  grains  

 c  +  U    =  24.1%  

Ð    True  value  =  10%  

U ' =100× 2smeasx

Obs Conc % of grains

1 15 2 7.5 3 5 4 12.5

10=x

Methods  to  es6mate  uncertainty  from  sampling  (&  analysis)    

•  Various  methods  developed  to  esHmate  measurement  uncertainty  arising  from  sampling  

•  Some  empirical,  some  based  on  modeling,  details  in:-­‐  •  Ramsey  M.H.,  and  Ellison  S.  L.  R.,(eds.)  (2007)  Eurachem/

EUROLAB/  CITAC/Nordtest/  AMC  Guide:  Measurement  uncertainty  arising  from  sampling:  a  guide  to  methods  and  approaches                                                                                Eurachem  ISBN  978  0  948926  26  6.  h]p://www.eurachem.org/index.php/publicaHons/guides/musamp  

 •  Empirical  approach  requires  inclusion  of  4  components:  

–  AnalyHcal  precision  &  bias  –  Sampling  precision  &  bias  

Method#  

Method description

Samplers (People)

Protocols  

Component estimated  

Sampling  Precision  

Sampling Bias  

Anal.  Precision  

Anal.  Bias  

1  

Duplicates  

single  

single  

Yes  

No  

Yes  

No1  

2  

Multiple protocols  

single  

multiple  

between protocols  

Yes  

No 1  

3  

CTS  

multiple  

single  

between samplers  

Yes  

Yes 2  

4  

SPT  

multiple  

multiple  

between protocols +between samplers  

Yes  

Yes 2  

Four  empirical  methods  for  es6ma6ng  uncertainty  including  that  from  sampling    

CTS = Collaborative Trial in Sampling ,

and SPT = Sampling Proficiency Test (e.g. Analyst, 2011, 136, 7, 1313-1321)

1 = Estimate analytical bias using CRM, 2 = Analytical bias partially or completely included where multiple labs involved

Simplest  Empirical  method  is  ‘Duplicate  Method’ (#1)  

1 m

The  Duplicate  Method  • Define  your  sampling  target  and  sampling  protocol  

• Use  a  balanced  design    

•   Take  a  sample  at  the  nominal  sampling  target.  •   Take  a  second  sample  displaced  from  original  (in  space  or  Hme)  to  reflect  ambiguity  in  sampling  protocol  (&  reflect  analyte’s  in  situ  heterogeneity)  •   Carry  out  duplicate  analyses  on  both  the  sample  duplicates    • Take  duplicate  samples  from  10%  (≤  8)  of  targets    •   EsHmate  uncertainty  components  using  ANOVA  (Analysis  of  Variance)  –  usually  Robust  

Sampling Target

Analysis 1 Analysis 2 Analysis 1 Analysis 2

Sample 1 Sample 2

Sampling Target

Analysis 1 Analysis 2 Analysis 1 Analysis 2

Sample 1 Sample 2

Sampling Target

Analysis 1 Analysis 2 Analysis 1 Analysis 2

Sample 1 Sample 2

S1A1                S1A2                  S2A1              S2A2  

Case  Study  using  duplicate  method  Nitrate  Concentra6on  in  LeUuce  

•  Nitrate  a  potenHal  risk  to  human  health  •  EU  threshold  4500  mg/kg  for  batch  concentraHon  •  Current  sampling  protocol  specifies  taking  10  heads  to  make  a  single  composite  sample  from  each  batch  (in  ‘W’  or  ‘star’  design)  European  Direc9ve  79/700/EEC.  OJ  L  207,  15.8.1979,  p26.    

•  Usual  ambiguity  in  the  protocol    –  e.g.  where  to  start  and  orienta9on  

•  What  is  the  uncertainty  in  measurements?  •  Is  sampling  protocol  valid  (suitable  for  rouHne  use)?  •  Full  details  in  Eurachem  Guide  (Appendix  A1)  &  Lyn  et  al.,  (2007)  ACQUAL,  12,    67-­‐74    

EsHmaHng  U  with  Duplicate  Method  using  Balanced  Design  

Target

Sample 1

Analysis 1 Analysis 2

Sample 2

Analysis 1 Analysis 2

At 10% of Sampling targets in whole survey n ≥ 8

- aim to represents these targets in general

Nitrate  conc.  in  Duplicate  Samples  

Most  analyHcal  duplicates  agree  well  <  x0.1  (approx)  

Sampling  duplicates  agree  only  <  x0.2  (approx)  

Range  of  conc.  between  batches  x1.6  (approx)  

Is  level  of  Uncertainty  OK?  

<4500?

>4500?

Reliable  decisions  whether  batch  is  >  4500  mg/kg?  

S1A1    

S1A2    

S2A1    

S2A2    

3898    

4139    

4466    

4693    

3910    

3993    

4201  

4126

5708    

5903  

4061  

3782  

5028    

4754    

5450    

5416    

4640    

4401    

4248    

4191    

5182    

5023    

4662    

4839    

3028    

3224    

3023    

2901    

3966    

4283    

4131    

3788    

mg/kg  

ANOVA  output                                                                    for  Uncertainty  es6ma6on  

CLASSICAL ANOVA RESULTS

Mean = 4345.5625

Standard Deviation (Total) = 774.5296

Sums of Squares = 12577113 4471511 351320

Between-target Sampling Analysis

Standard Deviation 556.2804 518.16089 148.18063

Percentage Variance 51.583582 44.756204 3.6602174

ROBUST ANOVA RESULTS:

Mean = 4408.3237

Standard Deviation (Total) = 670.57617

Between-target Sampling Analysis Measurement

Standard Deviation 565.39868 319.04834 167.94308 360.5506

Percentage Variance 71.090791 22.636889 6.2723172 28.909209 Relative Uncertainty – 14.474814 7.6193626 16.357719

(% at 95% confidence)

u  in  mg/kg  

U’  in  %  (k=2)  

From  ROBAN  sokware,  free  download  from:  h]p://www.rsc.org/Membership/Networking/InterestGroups/AnalyHcal/AMC/Sokware/ROBAN.asp  

Uncertainty  es6mate  for  LeUuce  •  Using  Analysis  of  Variance  (ANOVA)    

•  on  duplicate  measurements  •  Robust  staHsHcs  to  accommodate  outlying  values  

•  Gives  repeatability  precision  smeas  =  u  =  361  mg/kg    •  Expanded  U’  =  16%    •  U  from  analyHcal  bias  (from  CRM/  or  spike)    

•  can  be  added  –  not  staHsHcally  significant  in  this  case  

•  Does  not  include  U  from  any  sampling  bias    –  Can  be  included  using  values  from  Sampling  Proficiency  Test  (SPT)  –  with  >8  organisaHons  

U ' =100× 2smeasx

U ' =100× 2*3614408

U  es6mates  for  range  of  foods/analytes  

0

10

20

30

40

50

60

70

80

Infan

t milk

(Pb)

Milk (a

dded

wate

r)

Spread

able

fats (

fats)

Butter

(mois

ture)

Sausa

ges (

meat)

Glassh

ouse

lettu

ce (N

itrate)

Apples

(Prop

argite

)

Infan

t wet

meals

(Cd)

Strawbe

rries (

(Myc

lobuta

nil)

Pistac

hio nu

ts (A

flatox

in)

U'(%

)

Sources  of  U  –  sampling  or  analysis?  

0

10

20

30

40

50

60

Infan

t milk

(Pb)

Milk (ad

ded w

ater)

Spread

able

fats (

fats)

Butter

(mois

ture)

Sausa

ges (

meat)

Glassh

ouse

lettu

ce (N

itrate)

Apples

(Prop

argite

)

Infan

t wet

meals

(Cd)

Strawbe

rries (

(Myc

lobuta

nil)

Pistac

hio nu

ts (A

flatox

in)

U %

(95%

con

f)

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Ratio

of v

sam

p an

d vm

eas

Usamp%Uanal%Ratio

Sampling  generally  dominated,  but  someHmes  chemical  analysis  (e.g.  if  conc  near  DL)  Recent  meta-­‐analysis  of  128  analyte/food  combinaHons  has  show  that:  Median  U’  =  16%,  with  70%  of  the  cases  having  sampling  as  main  cause  (FSA  E01088)  

Alterna6ve  methods  to  es6mate  U    

             Unbalanced  design  -­‐  33%  less  cost*      Simplified  design                                                                                              -­‐  use  external  esHmates  of  analyHcal  U    •  Include  sampling  bias  in  U  using  mulHple  samplers/protocols  (Method  4)….  

* Rostron P. and Ramsey M.H. (2012) Accreditation and Quality Assurance: J. for Quality, Comparability and Reliability in Chemical Measurement 17, 1, 7-14. http://link.springer.com/article/10.1007%2Fs00769-011-0846-2

•       Modify  duplicate  method  to  reduce  the  cost  –  using  either:-­‐    

Including  Sampling  Bias  in  Es6mate  of  Uncertainty    using  Sampling  Proficiency  TesHng  

Sampling  Proficiency  Test.  Ramsey  et  al.,  Analyst,  2011,  136,  1313-­‐1321  

U%  on  concentra6on  from  SPT  on  moisture  in  20t  fresh  buUer:-­‐    

≈  Duplicate  Method  U%  =  0.39  %    

SPT  Includes  bias  between-­‐samplers  (n=9)    U%  =  0.87%  -­‐  larger  x  2.2  

Two  samplers  had  poten6ally  non-­‐proficient  z-­‐scores  (z  >  3)  

Samplers;  I  (z  =  -­‐3.2)  &  G  (z  =  +3.5)  

If  only  proficient  samplers,  U%  =  0.69%  

-­‐  U  s6ll  larger  x  1.8  

one laboratory under repeatability conditions.15 The former

approach could be called more precisely a ‘measurement profi-

ciency test’ (MPT), as each participant undertakes the whole of

the measurement process. The latter approach eliminates the

effect of any analytical bias between the participants, helping the

performance score to reflect only the participant’s sampling

performance. However, the centralised analysis in the latter

approach will also tend to reduce the estimate of the total

measurement uncertainty to an unrealistic level, unless it is

dominated by the sampling component, or the analytical

component is added independently.

Specific experimental design of SPT on moisture in butter

The determination of moisture in butter is an important

parameter considered by the UK Rural Payments Agency for the

acceptability of butter, either for the intervention scheme26 or as

‘butter for manufacture’.27 A study of previously frozen butter28

showed that the measurement uncertainty from sampling was

low for moisture (Usamp ! 2.5% relative or 0.39% m/m), due

primarily to low levels of heterogeneity in the spatial distribution

of the moisture in the butter. However, the closeness of the

measured concentration (15.76% m/m in this case) to the legal

limit (16% m/m) makes the sampling an important source of

uncertainty. This is because it contributes 96% of the measure-

ment uncertainty, and can therefore lead to misclassification of

the butter against the legal limit.

The sampling target selected for this SPT was broadly defined

as a batch of 20.1 t of fresh butter made in a creamery in SW

England. The experimental design used is essentially the general

one described in Fig. 1, with 9 participant samplers (identified by

letters A–I), and full details are reported elsewhere.25 The butter

is packaged in the normal way, in around 804 cardboard boxes

each containing a 25 kg block wrapped in a blue polythene sheet,

and stored in a cold room chilled to 0–5 "C, on separate pallets to

allow access to all boxes. This SPT design generally meets all of

the selection criteria (Table 1), but with two partial exceptions.

The 100 g samples were taken with a metal ‘trier’, that was a half-

round tube around 30 cm long, with a 2 cm internal diameter and

a handle at one end. The trier was inserted into the selected 25 kg

block of butter at an angle of around 45" (variable between

samplers 0–90" observed), twisted and then removed. The top

and bottom portions of the resultant core (#20 cm, but variable)

were rejected and the rest of the core transferred into a plastic pot

(volume around 500 mL). Two cores were taken from each 25 kg

block, to give a sample mass of around 100 g in each pot. Each

sampler sampled six blocks, randomly selected from the whole

batch, that generated a sample mass of around 600 g. This

composite sample represented the 20.1 t of product, giving

a sample ratio of 0.003% m/m (600 g/20 100 000 g). Criterion 1

was therefore satisfied. Each sampler took around 75 min to

execute the sampling protocol, and the whole SPT took 5 hours

to complete (criterion 8), during which time there was no

detectable change in the concentration of the analyte (criterion

3). The variability in the position and angle at which the trier was

inserted, and in the length of the discarded portion of the core,

left room for improving the consistency of the sampler’s

performance (criterion 7).

The two criteria that were not fully met included the sufficient

heterogeneity of the material (criterion 4). The heterogeneity at

around 0.08% RSD was low as expected, but the precision of the

gravimetric method (0.17% RSD) was not low enough to quan-

tify this heterogeneity accurately, and should ideally have been

less than 10% of that value (i.e. 0.008% RSD). The analytical

precision was low enough to quantify the between-sampler effect

(0.39% RSD, 0.78% at 95% confidence). This made the presence

of non-proficient samplers or bias between samplers detectable,

but potentially hard to quantify accurately (discussed below).

The independence of the samplers (criterion 10) was the second

criterion that was not fully met. Although the samplers were

mainly from different local authorities, they had received some

degree of common training, causing a possible lack of indepen-

dence. This possibility was supported by the observation that

many of the samplers adopted a very similar protocol that was

implemented in a very similar way, despite the ambiguity in the

written version of the protocol.29 The importance of the selection

of participant samplers with ‘required skill’ (criterion 6) was

investigated by including one non-professional and relatively

untrained sampler (identified by letter I). This sampler was

included to see if the SPT could detect this nominal lack of

sampling proficiency.

The routine sampling protocol normally employed in the

factory was slightly modified for all of the samplers, to enable the

more accurate evaluation of the uncertainty and the scoring of

the participants. Each sampler took two sets of six increments

from the whole batch, the second set constituting the duplicate

sample. This was different from the routine protocol in which

one set of six increments was taken, and then divided into two

sets of three increments that were combined physically to

represent the two halves of the whole 20 t target. Routinely, the

position of the split between the two halves of the batch was

made at a random point based upon the location of the third and

fourth increments. This latter routine practice caused some

ambiguity in the definition of the sampling target, which was

resolved in the data processing, as explained below. This illus-

trates that although methods for the evaluation of uncertainty

can generally be applied without changing the essence of the

sampling protocol, there are some situations where the routine

Fig. 1 Experimental design typically applied in sampling proficiency

tests (SPTs), adapted from35.

1316 | Analyst, 2011, 136, 1313–1321 This journal is ª The Royal Society of Chemistry 2011

Benefits  of  Knowing  Uncertainty    

Benefit  #1:-­‐Improving  reliability  of  decisions  –  e.g.  for  poten9ally  contaminated  lekuce  –  Risk  assessment:-­‐  

•  Hazard  >  threshold?,    •  Exposure  >Tolerable  Daily  Intake?  

–  Saves  money  on  consequences  of  :-­‐  •  unnecessary  destrucHon  of  batch  =  false  posi9ve  •  undetected  contaminaHon  (e.g.  liHgaHon)  =  false  neg.  

–  Test  scienHfic  hypotheses:-­‐    –  e.g.  to  compare  different  invesHgaHons  -­‐  in  space  or  Hme  

(e.g.  H1:  contaminaHon  level  has  decreased)  

Know  the  U  →  make  more  reliable  decisions  C

onta

min

ant c

once

ntra

tion

• •

•

•

Threshold

(e.g. 4500 mg/kg)

‘false positive’

‘false negative’

Ð Ð

True  value  

• UnderesHmate  of  U  -­‐  can  cause  unreliable  decisions  

Ð

   Effect  of  U  on  interpreta6on  

Threshold (T)

C

C-U

C+U

Uncontaminated PossiblyContaminated

ProbablyContaminated

Contaminated

Concentration (C)

- Probabilistic Classification

How  does  this  effect  le]uce  data  from  Case  Study  ?  

Uncontaminated      Possibly                            Probably                    Contaminated            Contaminated        Contaminated            (C  –  U  >  T)  

   Effect  of  U  on  interpreta6on  

Nitrate  concentra6ons  (mg  kg-­‐1)  for  S1A1  (rouHne  sample)  with  the  associated  measurement  uncertainty  (calculated  from  U  =  16.4%).    

e.g.  Target  F  value  of  the  measurand  (or  true  value)  between  4332  mg  kg-­‐1  and  6032  mg  kg-­‐1,  =  ‘Probably  Contaminated’,  compared  with  threshold  (T)  4500  mg  kg  -­‐1  

ProbabilisHc  classificaHon  has  only  one  batch  definitely  uncontaminated  (G),  whereas  determinisHc  classificaHon  has  4  batches  uncontaminated  (A,  B,  G  &  H)    

Only  one  batch  (C)  is  Definitely  Contaminated  –  posiHon  taken  by  some  regulators  (e.g.  EC)  

SAMPLEC - U C + U Probabilistic

TARGETClassification

A 3898 639.3 3259 4537 Poss ContB 3910 641.2 3269 4551 Poss ContC 5708 936.1 4772 6644 ContD 5028 824.6 4203 5853 Prob ContE 4640 761 3879 5401 Prob ContF 5182 849.8 4332 6032 Prob ContG 3028 496.6 2531 3525 Uncont.H 3966 650.4 3316 4616 Poss Cont

S1A1 Detn. Class

Uncertainty

C  -­‐  U  >  T  

Benefit  #2  Judging  fitness-­‐for-­‐purpose  in  valida6on  

•  Equivalent  as  asking  ‘How  much  U  is  acceptable?’  •  One  opHon  -­‐use  the  opHmised  uncertainty  (OU)  method*  •  Balance  the  cost  of  measurement    

-­‐  against  the  cost  of  making  incorrect  decisions  -­‐  Knowing  sampling  and  analyHcal  components  -­‐  judge  whether  either  is  not  FFP    -­‐  therefore  where  improvements/  increased  expenditure  required  

*  Lyn,  J.A.,  Ramsey,  M.H.,  and  Wood,    R.  (2002)  Analyst,  127,  1252  –  1260      

Acceptable  level  of  Uncertainty?  

0

200

400

600

800

1000

1200

1400

1600

0 200 400 600 800

Uncertainty mg/kg(smeas)

Cos

t (£)

(E

xpec

tatio

n of

Los

s)

Actual U

Optimal U

Cost  of  falsely  discarded  batches  

Cost  of  lowering  U    on  measurement  

Actual  U  ~2x  higher  than  OpHmal  

Judging  FFP  for  range  of  foods  •  Applied  Duplicate  Method  to  sampling  10  different  foods/analytes,  range  of  costs,  

heterogeneity,  situaHons  

Sampling   Analysis   False non-comp   False comp  Infant milk   Pb   15.51   45   400,000   100,000  Milk   Added water   8.33   10   500   5 000  Spreadable  fats   Fat   2.44   60   3000   5 000  Butter   Moisture   4.5   7.5   4590   18680  Sausages   Meat   14.35   90   100   2 000  Glasshouse  lettuce   Nitrate   40   40   2400   2640  Apples   Propargite   195   83   5000   1000000  Infant wet  meals   Cd   8.71   45   2,000   100,000  Strawberries   Myclobutanil   10   80   5000   100000  

Pistachio nuts   Aflatoxin   6.29   60   5,000   50,000  

Product   Analyte   Costs (£)  

0500

1000150020002500300035004000

0.00 0.05 0.10 0.15 0.20 0.25

MU (% m/m)

Loss

(£)

% moisture in butter

0

50

100

150

200

250

300

0.00 0.01 0.02 0.03 0.04 0.05

MU (mg kg-1)

Loss

(£)

Hg in fresh Tuna

050

100150200250300

0 2 4 6 8 10 12 14

MU (% m/m)

Loss (

£)

% meat in sausages

0500

1000150020002500

0 0.2 0.4 0.6 0.8 1MU (µg kg-1)

Lo

ss (

£)

Aflatoxin in nuts

Op6mising  U  (&  sampling)  for  range  of  foods  

U  higher  than  opHmal  

U  higher  than  opHmal  U  lower  than  opHmal  

U  opHmal  ~  FFP  

Benefit  #3  of  Knowing  Uncertainty    

Ra6onal  basis  for  alloca6on  of  finance,    to:-­‐  1.  Measurement  as  a  whole,  and  2.  ApporHonment  between  sampling  and  analysis  Allows  achievement  of  opHmal  uncertainty    -­‐  and  fitness  for  purpose  of  whole  measurement  

method  –  e.g.  lekuce  

How  can  U  be  changed  to  achieve  FFP?    

•  Graph  for  le]uce  shows  that  U  is  too  high  –  need  to  reduce  it  •  Need  to  know  source  of  U    

–  from  sampling  or  from  chemical  analysis?    –  Duplicate  Method  +  ANOVA    -­‐  tells  us  sampling  78%  of  U  

•  We  need  to  reduce  the  U  by  a  factor  of  2  (360→180)  

•  Sampling  theory  predicts  (e.g.  Gy’s)  increasing  sample  mass  by  factor  of  4  (=  22)  

•  ReducHon  in  U  was  achieved  in  pracHse  →  FFP  –  By  taking  composite  sample  with  40  heads  instead  of  10  –  Make  whole  method  valid  (i.e.  suitable  for  rouHne  use)  –  Full  details  in  Lyn  et  al.,  (2007)  ACQUAL,  12,    67-­‐74  

Benefits  #4  of  Knowing  Uncertainty    

Provides  tool  for  monitoring  Quality  of  Sampling  -­‐  Be]er  than  assuming:-­‐  

-­‐  ‘correct’  sampling  protocol  was  ‘correctly’  implemented  -­‐  resulHng  in  a  ‘representaHve’  sample  

-­‐  Gives  quanHtaHve  esHmate  of  sampling  quality  -­‐  Bring  sampling  within  similar  QC/QA  to  analysis  -­‐  Other  tools  to  improving  quality:-­‐    

-­‐  Validate  sampling  protocol  (with  CTS)  -­‐  Train  and  cerHfy  samplers  (with  SPT)  

How  best  to  express  large  Uncertainty  

•  When  U’  is  very  large  (>30%?),  we  need  be]er  ways  of  expressing  it  –  If  U  range  goes  <  0,  can’t  be  the  ‘true  value’  

•  New  ways  of  expressing  in  situ  heterogeneity1  use  heterogeneity  factors  (HF):  –  e.g.  HF  =  2,  gives  UCL  =  c*2,  LCL  =  c/2    

•  More  accurately  match  the  real  frequency  distribuHon  –  oken  ~  lognormal  for  very  heterogeneous  targets  

•  Can  be  modelled  as  a  funcHon  of  spaHal  scale1  

•  –  also  express  uncertainty  as  a  factor  (UF),  not  as  U’%?    

1  Ramsey  M.H.,  Grace  Solomon-­‐Wisdom  G.O.  and  Argyraki  A.  (in  press)  EvaluaHon  of  in  situ  heterogeneity  of  elements  in  solids:  implicaHons  for  analyHcal  geochemistry.  Geostandards  and  GeoanalyHcal  Research  (GGR-­‐0236.R1)    

U  in  6  Ex  situ  cont.  land  inves6ga6ons  Site# Source of

Pollution End use Prime

contaminant Sampling method

U’% %Prop Samp

PropAnal

1 Mine Sn/Cu Housing Arsenic Trial Pit 65 94 6

2 Gasworks waste

Public access

Lead Trial Pit

51 93 7

3 Infill after WWII Bombing

Private gardens

Lead Window 25 99.9 0.1

4 Gasworks Commercial Dev.

Total PAH Trial Pit

186 <1 >99

5 Railway sidings

Public access

Copper Trial Pit

158 Simplified design - not separated

6 Ex-firing range

Housing Lead Hand auger 75 72 28

Boon  K.A.,  Ramsey  M.H.  and  Taylor  P.D.  (2007.  Geostandards  and  GeoanalyHcal  Research,    31,  237-­‐249  (doi:  10.1111/j.1751-­‐908X.2007.00854.x  )    

Median  U’  is  70%.                                          Sampling  typically  contributes  >  90%  of  U  

U  of  On-­‐Site  vs  Ex-­‐situ  Measurements  on  Contaminated  Land  

Site Location

Key Contaminant

Anal Method

On site? In or Ex

situ?

Umeas (%) random

Usamp (%) Prop of U from

sampling

NW  England   Arsenic   PRXF   On  site   179   178   99  

NW  England   Arsenic   HG-­‐AAS   Ex  situ   110   110   100  

S.  Coast   TPH   Site-­‐LAB   On-­‐site   42   31   54  

S.  Coast   TPH   GC-­‐FID   Ex  situ   82   62   57  

NW  England   Arsenic   PXRF   In  situ   154   149   93  NW  England   Arsenic   HG-­‐AAS   Ex  situ   110   110   100  

S.Wales   Pb   PXRF   On-­‐site   62   57   85  

S.Wales   Pb   ICP-­‐AES   Ex  situ   54   39   53  

S.Wales   Zn   PXRF   On-­‐site   49   45   87  

S.Wales   Zn   ICP-­‐AES   Ex  situ   30   29   91  

S.Wales   TPH   Site-­‐LAB   On-­‐site   46   34   53  

S.Wales   TPH   GC-­‐FID   Ex  situ   97   91   89  

Median  On-­‐site   49   45   85  Median  Ex-­‐situ   89   77   90  

Median  Both   62   57   87  

Duplicate  method  equally  applicable  to  on-­‐site*  and  in  situ**  measurements  U  of  on-­‐site  measurements  are  similarly  dominated  (~85%)  by  sampling  -­‐  Even  more  so  for  in  situ  measurements  (93%)  *Boon  K.A.  and  Ramsey  M.H.  (2012)  Science  of  the  Total  Environment  h]p://dx.doi.org/10.1016/j.scitotenv.2011.12.001  **Ramsey  M.H.  and  Boon  K.A.  (2011)  Applied  Geochemistry  h]p://dx.doi.org/10.1016/j.apgeochem.2011.05.022  

U  from  sampling  other  media    

•  Duplicate  method  applicable  to  many  different  media  &  analytes    •  Levels  of  U’  vary  greatly  from  <4%  to  >72%  •  ContribuHon  from  sampling  oken  dominates  (median  92%),  but  not  

always  in  situ  γ-­‐ray  spec  of  Cs-­‐137  at  nuclear  site    –  integrates  heterogeneity  of  5  ton  sample  

•  Landfill  Gas1  -­‐  temporal  heterogeneity  increases  Usamp  

•  Ground-­‐Water  –  heterogeneity  sHll  dominant  •  U  esHmates  ignoring  sampling  will  be  too  low,  and  not  improved  by  

lab  acHon  1.  Squire,  S  and  Ramsey,  M.H.  (2001).  J.  Environmental  Monitoring,  3,  3,  288  -­‐  294  2.  Ramsey  M.H.,  and  Ellison  S.  L.  R.,(eds.)  (2007)  Eurachem/EUROLAB/  CITAC/Nordtest/  AMC  Guide:  Measurement  uncertainty  arising  

from  sampling:  a  guide  to  methods  and  approaches.    Eurachem,  p59  3.  Rostron,  P.,  Heathcote,  J.A.,  Ramsey,  M.H.  Journal  of  Environmental  RadioacHvity  (submi]ed).    

Sampling  Target   Analyte   Mode   U'   Prop  Samp   Ref    Landfill  Gas  

   Methane   in  situ   3.6   92.0   1  Oxygen   in  situ   4.8   96.0      

Groundwater   Fe-­‐dissolved   ex  situ   10.1   97.0   2  Nuclear  Decom.  Site  

Background  Cs-­‐137      

in  situ   42.6   0.0   3  ex  situ   47.4   84.6      

Moderate  Cs-­‐137      

in  situ   12.6   65.5      ex  situ   72.6   99.7      

FFP  for  the  6  Cont.  Land  Inves6ga6ons  

•  Can  judge  FFP  by  closeness  to  unity  of  raHo  of  actual/opHmal  Umeas  •  Measurements  are  ~FFP  in  one  case  (Site  2),  and  be]er  than  FFP  at  Site  3  •  Most  measurements  not  FFP  (have  more  U  than  is  opHmal)  -­‐  by  factor  of  2.4  •  Because  sampling  dominates  U,  improvement  factor  for  sampling  similar  2.8  •  Usamp  typically  62%,  but  opHmal  (by  OU/OCLI  method)  =  24%  (range  16-­‐54%)  

Site number

Key Contaminant

Mean Actual Optimal Ratio actual/ optimal

smeas

Actual Usamp(%)

Optimal Usamp Ratio

actual/ optimal

ssamp

smeas smeas (%)  (µg g-1) (µg g-1)  (µg g-1)    

1 Arsenic 411 131 49 2.7 62 22 2.8  

2 Indeno(123-cd)pyrene 13.6 3.5 2.1 1.6 49 27 1.8  

3 Lead 1590 200 533 0.4 25 54 0.5  4 Total PAH 75.8 34 11 3 71 24 3.0  5 Arsenic 20.9 16 8 2.1 - - -­‐  6 Lead 749 311 60 5.2 83 16 5.2  

Median       243   83   30   2.4   62   24   2.8  

Scope  and  role  of  analy6cal  chemistry  

•  SituaHon  presents  a  golden  opportunity  for  measurement  scienHst  to  broaden  their  role.    

•  Sampling  personnel  are  usually  poorly  trained  to  tackle  these  measurement  issues,  but    

•  AnalyHcal  chemists  could  usefully  extent  their  exisHng  experHse  to  consider  the  whole  measurement  process.    

•  Also  applies  to  in  situ  and  on-­‐site  measurements    •  Broaden  analyHcal  science  to  include  design  of  primary  sampling  

Conclusions  1.  Sampling  is  part  of  the  measurement  process    2.  Uncertainty  of  measurement  (U)  needs  to  include  the  contribuHon  from  

primary  sampling  and  sample  prep  –  both  random  &  systemaHc  components  3.  We  now  have  methods  to  esHmate  U  (from  all  sources  including  sampling)    

–  Applicable  to  any  measurement  (&  sampling)  protocol,  in  any  medium  for  any  analyte  4.  EsHmated  values  of  U  are:    

–  Inevitable,  and  usually  larger  than  Uanal    expressed  by  the  lab  –  Usually  dominated  by  the  contribuHon  from  sampling  (e.g.  ~  80%)  –  Beneficial  to  the  users  of  measurements  for  more  reliable  interpretaHon    

5.  The  fitness-­‐for-­‐purpose  of  measurements  can  be:  –  Judged  by  comparison  with  the  opHmal  level  of  U  (=  minimal  overall  cost)  –  Only  judged  realisHcally  when  the  contribuHon  for  sampling  (&  sample  prep)  included  –  Oken  achieved  most  cost-­‐effecHvely  by  improving  the  sampling  (rather  than  analysis)    

•  e.g.  by  increasing  the  number  of  increments  in  a  composite  sample  –  A  be]er  way  of  judging  data  quality  requirements  than  arbitrary  AQC  targets  

6.  The  scope  of  analyHcal  chemistry  (and  role  of  measurement  scienHsts)  should  be  widened  to  include  the  process  of  primary  sampling….  

7.  Enables  unified  approach  to  ensuring  measurement  quality  (based  on  U)  

Acknowledgements  Co-­‐workers  Ariadne  Argyraki  Jenny  Lyn  Paul  Taylor  Katy  Boon  Peter  Rostron  Sharon  Squires  Jong  Chun  Lee  Jacqui  Thomas  Grace  Solomon-­‐Wisdom  Ilaria  Palestra    

Collaborators  Mike  Thompson  Steve  Ellison    Roger  Wood  Andrew  Damant  Phil  Po]s  Mike  Gardner  Anna  Whi]aker  John  Heathcote  Members  of  Eurachem  &  RSC/AMC  Working  Groups  on  U  from  Sampling  

Funding    EPSRC  FSA  &  MAFF  TSB,  DTI  &  VAM  Dounreay  Site  RestoraHon  Ltd  NDA  CL:AIRE  EA  Corus  NaHonal  Grid  Commonwealth  Scholarships