Understanding the uses, and limitations, of attributes ...foodsafetyasiapacific.net/wp-content/uploads/2013/...and-Sampling.pdf · Understanding the uses, and limitations, of attributes

Understanding the uses, and

limitations, of attributes sampling

plan

Tom Ross

Food Safety Centre, University of Tasmania and

International Commission on Microbiological Specifications for Foods

FAO Regional Training Course, Hanoi, 16 November 2013


Overview

• attributes sampling plans

• statistics of sampling plans

• limitations of testing

• other complications

• sources of advice on sampling and sampling plan

design


Sampling plans

• “attributes” plans

– “2 class” or “3 class”

– test against a (probability of a) specific level of the hazard

– used by buyers, regulators

• “variables” plans

– incorporate all measured levels of the hazard to assess compliance

– must know mean and standard deviation of hazard levels, i.e. a history of quantitative results


“attributes” sampling plans:

• intended to tell what proportion of the units in

the lot meet some criterion (i.e., the

“attribute” we are interested in)

• how many are “acceptable”; how many “pass”,

or “fail”


attributes sampling plans

defined by three (or four) quantities:

• ’m’ – the microbiological limit (e.g., <1 per 25g, 100 cfu/g) desired (= the “attribute”)

– may implicitly dictate the sample size

• ‘n’= the number of samples from the lot that must be tested

• ‘c’ = the number of samples that may exceed ‘m’

• ‘M’ – applied in some cases to recognise occasional deviations, a level higher than ‘m’ but one which must never be exceeded


2 and 3-class attributes plans

2 class:

the criterion is absolute, sample units either “pass” or “fail”

described only by m, c, n

3 class:

the criterion includes a third category, i.e., “marginally acceptable (“M”)

described by M, m, c, n


log count / g

Rela

tive p

rop

ort

ion

of

sam

ple

un

its in

lo

t

log count / g

Rela

tive p

rop

ort

ion

of

sam

ple

un

its in

lo

t

Two- and three-class attributes plan m m

2-class plan 3-class plan M

acceptable

unacceptable

acceptable

unacceptable

marginally acceptable


“If the test is negative, the batch is free from pathogens”

Wrong!

In fact, we can never demonstrate ‘zero tolerance’, only a

certain level of confidence that the contamination is

below a certain level

Common Misconception about

Presence/Absence Testing


Sampling and the Probability of Detection

<50% defective?

<10% defective?

<1 % defective?


Sampling and the Probability of

Detection

• probability of detection depends on

– actual proportion of samples that are unacceptable

– number of samples examined

– it is easily described mathematically (a binomial sampling process)

- (in fact, the ‘hypergeometric’ distribution is more correct, but for low contamination values, the results are nearly equivalent)

Pacceptance = (1 - Pdefective)number of samples examined


Binomial Distribution

• Pacceptance often = 0.05 (i.e., 95% confidence that a ‘bad’

batch will be detected and rejected

• Pdefective = real proportion defective or the upper limit of

our tolerance of units that don’t meet our criterion

Pacceptance = (1 - Pdefective)number of samples examined


“attributes” sampling plans

confidence in the result of testing is based on:

– number of samples taken

– true (or required) frequency of compliance

plan performance (confidence) is summarised as

‘operating characteristic curve’


Probability of accepting a lot (Operating Characteristic Curves)


0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Proportion defective sample units in a lot

Pro

ba

bil

ity o

f a

cce

pti

ng

th

e l

ot

n=5

Probability of acceptance with 5 negative

samples depends in true ‘defect’ rate


Pacceptance = 0.05 with 5 samples (all negative); depends on true ‘defect’ rate


n = 5, c = 0

• means that we can be 95% confident only that, on

average, less than half the units in the lot exceed

the ‘attribute’, m.

0.05 = (1 - Pdefective) 5

=> Pdefective = 0.46


what’s acceptable (“appropriate level of protection”)

• % 50 defective (=contaminated)?

• 1 % contaminated?

• 0.1% contaminated?

• 0.01% contaminated (1 in 10,000 chance)?


samples required for 95% confidence of rejection of a non-compliant lot


• no sampling plan can ever guarantee absolute safety of the food

• the sampling plan can only provide a certain level of confidence.

• it is virtually* impossible to use sampling and testing to determine whether a lot of product is acceptable when the tolerable level of failure is very low, or zero

• better to rely on preventive approaches


Limitations of Testing

• methods may not recover injured cells

• microbiological condition of the food changes over time

• microoorganisms aren’t uniformly distributed throughout the

lot

• even if they were completely evenly distributed the samples

would not contain identical numbers of cells





• microoorganisms aren’t uniformly distributed throughout the

lot

• even if they were completely evenly distributed the samples

would not contain identical numbers of cells


Variability and acceptable levels


Variability and acceptable levels





• microoorganisms aren’t uniformly distributed throughout the lot

• even if they were completely evenly distributed the samples would

not contain identical numbers of cells


Poisson processes

• random sampling from a population when assessing

a (binomial) attribute is a Poisson process

• the chances of making a particular number of

observations, in a given time, or space, based on

some true average density, or frequency, is defined

by the Poisson distribution

• for example….


• if the concentration of a pathogen in a food:

– were exactly 1 cell per cm3, and

– the cells were perfectly evenly distributed, and

– our sample size was exactly one cm3

• our samples of 1 cm3 would not always contain 1

cell….

Poisson processes


Poisson sampling “error”

i.e., mean concentration is one cell per sample unit


i.e. but a sample could, by chance, contain four

cells …




or could contain two cells …


Poisson distribution

• The probability that a certain number of ‘events’

(e.g., number of positive samples) will be observed,

when the true average of those ‘events’ is known, is

given as:

P(observing n positives) =

(true_averagen ) ´ e-( true _ average )

n!


ICMSF Sampling Plans – ‘risk’ basis

incre

asin

g s

everity

of c

on

seq

uen

ce

increasing probability of unacceptable hazard


Examples of Sampling Plans

ICMSF: Microorganisms in Foods, 2. Sampling for Microbiological Analysis. Principles and Specific Applications,

2nd Edition, Blackwell Scientific Publications, 1986 (ISBN-0632-015-675).


Advice on Sampling Plan Design and Interpretation (http://www.icmsf.org)


Attributes Sampling Plans

• are like the results of MPN tests

• if we can quantify, or estimate, the variability of

contamination levels in a lot, we can calculate the

sensitivity and confidence of the sampling plan

using the equations and understanding just

presented

• its complicated!


Food Control, 20 (2009): 967 - 979


ICMSF Sampling Plan Spreadsheet (www. icmsf.org)


ICSMF Sampling Tool

can be used to:

• interpret the sensitivity of a specified sampling plan

• design a sampling plan to ensure that the batch satisfies a

microbiological limit

• relies on specification of the variability (as SD)

automates the calculations

utilises macros (security settings issues)


ICMSF Sampling Plan Spreadsheet (http://www.icmsf.org/main/software_downloads.html;“Download

Standard Program”)


ICICMSF Sampling Plan Spreadsheet

interpretation

design


ICMSF Sampling Plan Spreadsheet


ICMSF Sampling Plan Spreadsheet (http://www.icmsf.org/main/software_downloads.html;“Download

Standard Program”)


Other Sampling Plan Tools: Baseline

http://www.baselineeurope.eu/index.aspx


Other Sampling Plan Tools: FAO

http://www.fstools.org/sampling/


Other Sampling Plan Tools: FAO Histamine Sampling Tools

http://www.fstools.org/histamine/


Conclusions

• in the absence of non-destructive testing, we can never

guarantee that a lot contains no unacceptable units

• transparent, science-based criteria and sampling plans

can be developed and their performance compared based

on knowledge of the SD of contamination within a lot

• correct design, and interpretation, of sampling plans is

complex

• user-friendly tools are available, for free, to assist


Useful Reading

van Schothorst, M., Zwietering, M.H., Ross, T., Buchanan, R.L and Cole, M.B. International

Commission on Microbiological Specifications for Foods. (2009). Relating

microbiological criteria to food safety objectives and performance objectives. Food

Control, 20: 967-979.ICMSF

R.C. Whiting, A. Rainosek, R.L. Buchanan, M. Milioti, D. LaBarre, W. Long, A. Ruple and S.

Schaub. (2006). Determining the microbiological criteria for lot rejection from the

performance objective or food safety objective. International Journal of Food

Microbiology, 110: 263–267.

ICMSF (International Commission on Microbiological Specifications for Foods), 2002.

Microorganisms in Foods, Microbiological Testing in Food Safety Management, Vol. 7.

Kluwer Academic/Plenum Pub, NY. 362 pp.

Understanding the uses, and limitations, of attributes ...foodsafetyasiapacific.net/wp-content/uploads/2013/...and-Sampling.pdf · Understanding the uses, and limitations, of attributes

Documents