NF VALIDATION AFNOR CERTIFICATION VALIDATION OF THE METHOD COLILERT‐18 with QUANTI‐TRAY or QUANTI‐TRAY 2000 For the enumeration of Escherichia coli Protocol for bathing waters SUMMARY REPORT – JANUARY 2017 – V1 Expert laboratory : Manufacturer : ISHA IDEXX Laboratories, Inc. 25 avenue de la République IDEXX Drive, Westbrook 91300 MASSY Maine 04 092 FRANCE USA This report of analysis concerns only objects subjected to analysis. Its reproduction is authorized only in the form of complete photographic facsimile. It contains 54 pages. Only some assays reported in this document are covered by the accreditation of the Section Laboratory of COFRAC. They are identified by the symbol (*). Assays realized at ISHA : 25 avenue de la République, 91300 Massy, France.
54
Embed
2017 01 IDEXX Colilert-18 Quanti-Tray BW SR v1 · Colilert‐18 / Quanti‐Tray or Quanti‐Tray 2000 developed by IDEXX for the enumeration of Escherichia ... Read results according
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
NF VALIDATION
AFNOR CERTIFICATION VALIDATION OF THE METHOD
COLILERT‐18 with QUANTI‐TRAY or QUANTI‐TRAY 2000
For the enumeration of Escherichia coli
Protocol for bathing waters
SUMMARY REPORT – JANUARY 2017 – V1
Expert laboratory : Manufacturer : ISHA IDEXX Laboratories, Inc. 25 avenue de la République IDEXX Drive, Westbrook 91300 MASSY Maine 04 092 FRANCE USA
This report of analysis concerns only objects subjected to analysis. Its reproduction is authorized only in the form of complete photographic facsimile. It contains 54 pages. Only some assays reported in this document are covered by the accreditation of the Section Laboratory of COFRAC. They are identified by the symbol (*). Assays realized at ISHA : 25 avenue de la République, 91300 Massy, France.
Table of contents 1. Introduction ................................................................................................................................................. 4
1.1. Validation repository and validation history ....................................................................................... 4
1.2. Alternative method ............................................................................................................................. 4
3. Interlaboratory study ................................................................................................................................ 13
3.1. Study organisation ............................................................................................................................. 13
4. Extension study ......................................................................................................................................... 20
4.1. Results and interpretation ................................................................................................................. 20
4.1.1. Results from Enterolert‐E / Quanti‐Tray 2000 comparative study ............................................ 20
4.1.2. Results from Colilert‐18 / Quanti‐Tray study ............................................................................ 22
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 3/54
1. Introduction This summary report presents the results of the validation study, under the brand NF Validation, of the method Colilert‐18 / Quanti‐Tray or Quanti‐Tray 2000 developed by IDEXX for the enumeration of Escherichia coli in bathing waters.
1.1. Validation repository and validation history The aim of this validation study is to evaluate the performance of the alternative method against the reference method NF EN ISO 9308‐3: 1999. The different steps of the validation are the following:
Step: Date: Version of the validation protocol for an alternative commercial method as compared to a reference method:
‐ Initial validation ‐ June 2012 ‐ Version 1, May 2010 ‐ Extension ‐ November 2014 ‐ Version 2, May 2013 ‐ First renewal ‐ June 2016 ‐ Version 2, may 2013
1.2. Alternative method Colilert‐18 detects E. coli in bathing waters. It is based on IDEXX’s patented Defined Substrate Technology (DST):
‐ when total or fecal coliforms metabolize Colilert‐18’s nutrient‐indicator, ONPG, the sample turns yellow, ‐ when E. coli metabolize Colilert‐18’s nutrient‐indicator, MUG, the sample also fluoresces.
Colilert‐18 can simultaneously detect these bacteria at 1 CFU/100 mL within 18 hours even in the presence of as many as 2 million heterotrophic bacteria per 100 mL. The protocol of the alternative method is presented in figure 1. Figure 1 : protocol of the alternative method
1. Add contents of one pack to a 100 mL room temperature water sample in a sterile vessel. When Colilert‐18 is used for E. coli detection in marine water, samples must be diluted at least tenfold. Multiply the MPN value by the dilution factor to obtain the correct quantitative result. 2. Cap vessel and shake until dissolved. 3. Pour sample/reagent mixture into a Quanti‐Tray or a Quanti‐Tray 2000 and seal in an IDEXX Quanti‐Tray Sealer. 4. Place the sealed tray in a 36±2°C incubator for 18 hours to 22 hours (pre‐warming to 36°C is not required). For incubation in a water bath, submerge the Tray below the water level using a weighted ring. 5. Read results according to the Result Interpretation table. Count the number of positive wells and refer to the MPN table provided with the trays to obtain a Most Probable Number.
1.3. Application scope The application scope of the alternative method concerns the category bathing waters, which groups two types of waters:
‐ fresh waters ‐ sea waters.
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 4/54
1.4. Reference method The standard NF EN ISO 9308‐3 (1999): Detection and enumeration of E. coli and coliforms – part 3: miniaturized method (MPN) for detection and enumeration of E. coli in surface and waste water, was used as the reference method. The protocol of the reference method is presented in figure 2. Figure 2 : protocol of the reference method
Dilutions preparation
‐ Dilute 9 mL of sample in 9 mL of special diluent (1/2) ‐ Transfer 1 mL of in 9 mL of special diluent (1/20)
Inoculation
‐ Inoculate 200 µL of the 1/2 dilution in each of the first 64 wells of the microplate ‐ Inoculate 200 µL of the 1/20 dilution in each of the 32 wells of the microplate
Incubation
‐ Cover the microplate with sterile adhesive ‐ Incubate the microplate at 44 ± 1° C for 36 h to 72 h
Reading and interpretation
Read results according to the Result Interpretation table. Count the number of positive wells using Wood lamp and refer to the MPN table provided with the trays to obtain a Most Probable Number. Express the
result in MPN E. coli / 100 mL
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 5/54
2. Methods comparative study The following characteristics were studied during the comparative study of the methods: the relative accuracy, the linearity of the alternative method, the selectivity of the alternative method, the limit of detection and the limit of quantification of the alternative method, the practicability of the alternative method.
2.1. Relative accuracy The relative accuracy is defined as the closeness of agreement between test result and the accepted reference value.
2.1.1. Number and nature of samples Two types of water were tested (duplicate) with reference method and alternative method: freshwater and seawater. Different types of analyzed samples are summarized in table 1. Table 2 : Number and nature of samples analyzed
Water type Number of samples analyzed Number of samples used
Sea waters 53 20
Fresh waters 41 22
Total 94 42
Globally, 94 samples were analyzed and 42 results were used. 16 naturally contaminated samples were analyzed. Others samples were artificially contaminated (cf. appendix 1). The contamination levels used cover the entire measurement range of the alternative method.
2.1.2. Results Figure 3 presents the two‐dimensional graphs for the two matrices. The y‐axis is reserved for the alternative method and the x‐axis for the reference method. . The representation of a line of equation “y = x“ figures dashed on the graphs. Raw results are in appendix 2. Figure 3 : two‐dimensional graphs for relative accuracy in log CFU and log MPN / test portion (black line: y=x)
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 6/54
2.1.3. Statistical analysis The relationship of relative accuracy between the reference method and the alternative method is evaluated with the linear model: 'y = a + bx '. This formula corresponds to the equation of the linear regression drawn from raw results obtained by experimentation, y representing the alternative method and x the reference method. There is a perfect accuracy (or there is no systematic bias) between the two methods if this equation is equal to the theoretical 'y = x' equation, which applies in the ideal model where the two methods behave similarly. The intercept is theoretically zero in this ideal model (hypothesis [a = 0]). The estimated intercept obtained with the two methods is checked using p {a = 0}. If the alternative method is a systematic bias against the reference method, the probability p {a = 0} is less than α = 0.05. The 'b' slope is theoretically equal to 1 in the ideal model (hypothesis [b = 1]). The estimated slope obtained with the two methods should pass by p {b = 1}. Statistically, if the alternative method does not give the same values as the reference method, the probability p {b = 1} is less than α = 0.05. The linear regression method is chosen over the value of the robustness of the ratio R of overall repeatability standard deviation: ‐ If Rob.R > 2, linear regression by least‐squares (OLS 1) with the x‐axis for the reference method, ‐ if Rob.R < 0.5, a linear regression by least‐squares (OLS 2) with the x‐axis for the alternative method, ‐ If 0.5 < Rob.R < 2, orthogonal regression (GMFR) with the x‐axis to the reference method. Table 2 : statistical data (log MPN / test portion) for the enumeration of E. coli in bathing waters
Matrix Rob.R Regression used T a t(a) b t(b) Probabilities (%)
Sea waters The hypothesis [b = 1] is accepted but the hypothesis [a = 0] isn’t accepted. However, the correlation coefficients and equation are satisfactory as shown below: ‐ r = 0.984, ‐ log Alt. = 1.053 log Ref. – 0.232
Fresh waters The two hypothesis [b = 1 and a = 0] aren’t accepted. However, the correlation coefficients and equation are satisfactory as shown below: ‐ r = 0.979, ‐ log Alt. = 1.157 log Ref. – 0.494
Bathing waters (seawaters + freshwaters) The two hypothesis [b = 1 and a = 0] aren’t accepted. However, the correlation coefficients and equation are satisfactory as shown below: ‐ r = 0.988, ‐ log Alt. = 1.087 log Ref. – 0.337
Remark: The limits of detection of the two protocols of the alternative method and of the reference method are different, based on different dilution factors and MPN tables: ‐ 1 MPN/100 mL for the alternative method in fresh waters, ‐ 10 MPN/100 mL for the alternative method in sea waters, ‐ 15 MPN/100 mL for the reference method. That’s why, for fresh waters and bathing waters, if the data of the alternative method inferior to the limit of detection of the reference method are not taken into account (2 samples involved), the following values are obtained (data and calculations in appendix 3): Fresh waters: Bathing waters:
With these values, the statistical exploitation shows that the two hypothesis [b = 1 and a = 0] are accepted with α = 5%.
2.1.4. Conclusion The relative accuracy of the alternative method is satisfactory.
2.2. Linearity The linearity is the ability of the method when used with a given matrix to give results that are in proportion to the amount of analyte present in the sample, that is an increase in analyte corresponds to a linear or proportional increase in results.
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 8/54
2.2.1. Contamination levels The couples matrix / strain are presented in Table 4. For each couple, four contamination levels were tested in duplicate by the reference method and the alternative method. Table 4 : couples matrix – strain analyzed
2.2.2. Results Figure 4 presents the two‐dimensional graphs for the two couples matrix‐strain. The y‐axis is reserved for the alternative method and the x‐axis for the reference method. The representation of a line of equation “y = x“ figures dashed on the graphs. Raw results are in appendix 4. Figure 4 : two‐dimensional graphs for linearity in log CFU and log MPN / test portion (black line: y=x)
2.2.3. Statistical analysis Statistical interpretations are made according to requirements of standard NF ISO 16140 (see table 5). The choice of the linear regression method is compared to the value of the robustness of the ratio R of the standard deviations of repeatability overall:
‐ if Rob.R> 2, a linear regression least squares (OLS 1) is used with the x‐axis for the reference method, ‐ if Rob.R <0.5, a linear regression least squares (OLS 2) is used with the x‐axis for the alternative method, ‐ if 0.5 <Rob.R <2, an orthogonal regression (GMFR) is used with the x‐axis to the reference method.
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 9/54
The relationship between the 2 methods is not linear: ‐ if Rob.F > critical F or, ‐ if P (Rob.F) < α (= 0.05).
2.2.4. Conclusion The relationship between the two methods is linear for the two couples (E. coli / sea water and E. coli / fresh water). The correlation coefficients are satisfactory. So, the linearity of the alternative method is satisfactory.
2.3. Limit of detection and limit of quantification The detection and quantification limits are checked in accordance with the standard EN ISO 16140. Three parameters are determined. Here are their ISO 16140 definitions:
‐ the critical level (LC) is the smallest amount which can be detected (not null), but not quantified as an exact value. Below this value, it cannot be sure that the true value is not null. At this level, the false negatives probability β is 50 % (β is the second type of statistical error). ‐ the detection limit (LOD) is higher than the critical level, because it involves a power, the probability 1 ‐ β, which has to be well over 50 %, for example 95 %. ‐ the quantification limit (LOQ) is the smallest amount of analyte, (that is the lowest actual number of organisms), which can be measured and quantified with defined precision and accuracy under the experimental conditions by the method under validation.
2.3.1. Test protocols The limits of detection and quantification were determined by analysing a pure culture of E. coli by the alternative method. Five levels of contamination (including level 0), with six replications for each level, were studied in sterilized water.
2.3.2. Results Results are shown in the following tables and in appendix 5. Table 6 : data (s0 and x0) of E. coli enumeration (underlined: the reference level)
Level (CFU/100mL) Number of positive samples Standard deviation (s0) Bias (x0)
0 0 0.000 0
0.2 1 0.408 0
0.4 2 0.516 0
1.5 3 0.548 0.5
3 6 1.627 1.5
Table 7 : LC, LOD and LOQ values of the alternative method
Parameter Formula Values obtained
Critical level (LC) 1.65 s0 + x0 1.40
Limit of detection (LOD) 3.3 s0 + x0 2.31
Limit of quantification (LOQ) 10 s0 + x0 5.98
2.3.3. Conclusion The detection limit and quantification limit of the alternative method are satisfactory.
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 10/54
2.4. Selectivity The selectivity of the alternative method is evaluated by its inclusivity and its exclusivity. Inclusivity is the ability of the alternative method to detect the target analyte from a wide range of strains. Exclusivity is the lack of interference by a relevant range of non‐target strains with the alternative method.
2.4.1. Test protocols Twenty E. coli strains and thirty non‐target strains (from the national, international and ISHA internal collections) were analyzed. The assays were performed by the alternative method protocol.
2.4.2. Results Raw results are in appendix 6. All target strains tested are detected by the alternative method except for one strain (which is not detected by the reference method either). For the thirty non‐target strains tested, no positive result was observed. See tables below.
2.4.3. Conclusion The selectivity of the alternative method can be considered as satisfactory.
2.5. Practicability The practicability was evaluated according to the 13 criteria defined by AFNOR Technical Committee. 1‐ Mode of packaging of test components The Colilert‐18 reagent is conditioned on single capsules. The Quanti‐Tray devices are conditioned par ten in aseptic bag. 2‐ Volume of reagents Unknown. 3‐ Storage conditions of components and shelf‐life of unopened products The Colilert‐18 reagent should be conserved at 2 – 8°C. The Quanti‐Tray devices should be conserved at 4 – 30°C. 4‐ Modalities after first use Each Colilert‐18 test serves a unique analysis and should not be reused. 5‐ Equipment and specific local requirements Quanti‐Tray® Sealer model 2X. Wood lamp. 6‐ Reagents ready to use or for reconstitution None. 7‐ Training period for operator with no experience with the method The duration of training is estimated to be 1 hour. 8‐ Handling time and flexibility of the method in relation to the number of samples The duration of analysis according the reference method is more important than the duration of use of alternative method.
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 11/54
9‐ Time required for results The time to obtain results for the alternative method is 18 hours for negative samples and positive samples. Concerning the reference method, the delay for negative samples is between 24 and 48 hours and for positive samples, the delay is between 48 and 72 days. 10‐ Operator qualification Identical as necessary for the reference method 11‐ Steps common with the reference method None. 12‐ Traceability of analysis results None. 13‐ Maintenance by laboratory None.
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 12/54
3. Interlaboratory study The main object of the interlaboratory study is to determine the variability of the results obtained by different laboratories analysing identical samples and to compare these results within the framework of the comparative study of the methods.
3.1. Study organisation
3.1.1. Participating laboratories The interlaboratory study was realized by the expert laboratory and fifteen participating laboratories.
3.1.2. E. coli absence in the matrix Before spiking, the absence of E. coli was verified in the batch of seawater used according to the reference method.
3.1.3. Strain stability in the matrix The strain stability in seawater matrix was evaluated for 3 days at (5±3)°C. The strain used was E. coli (ISHA code: ESC.1.119). The samples were analysed at D0, D+1 and D+2 by the reference method. The results are summarized in table 10. Table 8 : results (E. coli / 100 mL) of the stability study of the strain ESC.1.119 in seawater matrix
Day Level 1 Level 2 Level 3
D0 60 534 1049
D1 75 563 882
D2 30 504 861
The results show that the E. coli strain used is stable for 2 days at (5±3)°C in SHW matrix.
3.1.4. Samples preparation and spiking The matrix was inoculated with the target strain suspension to obtain 4 contamination levels: ‐level 0 : 0 CFU/100 mL,
‐level 1 : from 50 to 100 CFU/100 mL, ‐level 2 : from 250 to 500 CFU/100 mL, ‐level 3 : from 1000 to 1500 FCU/100 mL.
The matrix was distributed at 50 mL in sterile bottles. Every bottle was individually spiked and homogenized. Eight samples per laboratory were prepared (2 samples per contamination level). Each laboratory received 8 samples to analyse, 1 sample to quantify the endogenous microflora and 1 water sample containing a temperature probe. The results of the enumerations of the heterophilic flora, the target levels and the real levels of contamination are presented in table 9. Table 9 : target level, real level and TVC of the matrix
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 13/54
3.1.5. Samples labelling The labelling of the bags was realized as follows: a code to identify the laboratory: from A to O (cf. table 10) and a code to identify each sample, only known by the expert laboratory. The samples and the temperature control vials (water sample with a temperature probe) were stored at 4°C before shipping. Table 10 : sample code by contamination level
Contamination level (MNP E. coli / 100 mL) Sample code
0 4 / 8
50 to 100 6 / 7
250 to 500 1 / 3
1 000 to 1 500 2 / 5
3.1.6. Samples shipping The samples were shipped in a coolbox April 16th , 2012.
3.1.7. Samples reception and analysis The coolboxes were received April 17th, 2012 by all the participating laboratories. The control temperature was recorded upon receipt of the package and the temperature probe sent to the expert laboratory. The samples were analysed on April 17th, 2012. The expert laboratory concurrently analysed a set of samples under the same conditions with both methods.
3.2. Results
3.2.1. Temperature and state of the samples The temperature readings at reception, the state of the samples and the data from the thermal probe are shown in table 11. Table 11 : temperature and state of the samples upon reception and data of the temperature probes for the transportation time of samples (/: data not available)
Laboratory Temperature (°C) State of the samples
Temperature recorded by the probe
Mean SD
A 4.1°C Ok 2.9 1.0
B 5.2°C Ok 3.4 0.5
C 6.7°C Ok 3.7 0.3
D 6.8°C Ok 2.5 0.4
E 6.4°C Ok 2.4 1.0
F 3.8°C Ok / /
G 2.0°C Ok 2.4 0.5
H 3.0°C Ok 2.9 0.3
I 5.2°C Ok 2.5 0.3
J 6.0°C Ok / /
K 2.1°C Ok 2.2 0.5
L 4.8°C Ok 2.6 0.4
M 1.6°C Ok 2.4 0.6
N 6.1°C Ok 2.3 0.7
O 4.8°C Ok 1.6 0.8
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 14/54
The analysis of thermal profiles of probes showed for all participants that the average of temperature during the shipment is comprise between 1.6 and 3.7°C.
3.2.2. Total viable counts Raw results are in appendix 7. For the whole laboratories, the total viable counts at 22°C vary between <1 and 240 CFU/mL. Concerning the total viable counts at 36°C, the results were varying between <1 and 7 CFU/mL.
3.2.3. Expert laboratory and collaborating laboratories results The overall results are presented in Table 12 and in appendix 8. The results of the reference method are presented for a reading of the microplates after 36 at 72 hours of incubation at 44 ± 1°C. For alternative method, reading of Quanti‐Tray devices was performed between 18 and 22 hours. The results of all laboratories are presented in the following tables. Table 12 : E. coli MPN enumeration results per 100 mL seawater samples (MR: reference method, MA: alternative method, R1: repetition 1 and R2: repetition 2)
Laboratory
Level 0
MR MA
R1 R2 R1 R2
MPN/ 100 mL
Low limit
High limit
MPN/ 100 mL
Low limit
High limit
MPN/100 mL
MPN/100 mL
A <15 / / <15 / / <10 <10
B <15 / / <15 / / <10 <10
C <15 / / <15 / / <10 <10
D <15 / / <15 / / <10 <10
E <15 / / <15 / / <10 <10
F <15 / / <15 / / <10 <10
G <15 / / <15 / / <10 <10
H <15 / / <15 / / <10 <10
I <15 / / <15 / / <10 <10
J <15 / / <15 / / <10 <10
K <15 / / <15 / / <10 <10
L <15 / / <15 / / <10 <10
M <15 / / <15 / / <10 <10
N <15 / / <15 / / <10 <10
O <15 / / <15 / / <10 <10
Expert <15 / / <15 / / <10 <10
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 15/54
Laboratory
Level 1
MR MA
R1 R2 R1 R2
MPN/100 mL
Low limit
High limit
MPN/100 mL
Low limit
High limit
MPN/100 mL
MPN/100 mL
A 93 41 206 93 42 207 86 108
B 127 63 253 109 52 230 10 41
C 94 42 208 94 42 208 75 63
D 127 63 253 <15 / / 41 63
E 110 52 231 15 2 106 52 63
F 46 15 142 61 23 163 63 41
G 77 32 186 160 86 298 98 135
H 15 2 106 46 15 142 51 52
I 125 62 251 61 23 163 97 30
J 61 23 163 61 23 163 52 52
K 94 42 208 93 42 207 40 41
L 94 42 208 144 75 276 52 122
M 197 63 253 46 15 142 95 109
N 94 42 208 46 15 142 62 74
O 127 63 253 126 63 252 119 109
Expert 126 63 252 30 8 121 40 84
Laboratory
Level 2
MR MA
R1 R2 R1 R2
MPN/ 100 mL
Low limit High limit MPN/ 100 mL
Low limit High limit MPN/ 100 mL
MPN/ 100 mL
A 697 486 981 332 212 521 496 487
B 529 363 769 434 290 650 331 404
C 332 212 521 438 293 655 389 457
D 177 98 321 465 314 689 408 374
E 234 138 394 434 290 650 425 369
F 195 111 344 393 258 598 259 238
G 415 275 626 393 258 598 292 482
H 585 408 840 465 314 689 387 331
I 654 462 927 500 341 733 393 269
J 412 272 622 375 244 575 393 309
K 344 221 537 504 344 738 754 530
L 606 424 866 640 451 909 350 529
M 476 322 703 580 403 833 231 512
N 559 387 808 640 451 909 305 231
O 585 408 840 668 473 944 496 437
Expert 697 479 953 559 387 808 616 459
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 16/54
Laboratory
Level 3
MR MA
R1 R2 R1 R2
MPN/ 100 mL
Low limit High limit MPN/ 100 mL
Low limit High limit MPN/ 100 mL
MPN/ 100 mL
A 1049 773 1423 882 642 1213 591 712
B 858 622 1182 489 333 720 809 771
C 773 555 1075 851 617 1174 733 512
D 647 456 917 838 606 1157 581 738
E 514 352 751 1007 740 1371 581 847
F 690 490 972 805 580 1116 556 594
G 580 403 833 943 690 1290 754 573
H 759 544 1058 759 544 1058 733 581
I 1305 973 1751 742 531 1037 727 663
J 918 670 1258 543 375 783 906 884
K 1136 841 1535 838 606 1157 909 1017
L 1007 740 1371 968 709 1321 776 933
M 882 642 1213 872 633 1200 988 1334
N 882 642 1213 968 709 1321 733 622
O 1567 1174 2092 893 650 1227 836 794
Expert 633 445 901 1034 761 1405 1010 833
3.3. Interpretation The data presented in the following paragraphs were calculated from the results in log10 MPN/100 mL in the same way that the presentation of the results of the preliminary study.
3.3.1. Bias calculation Table 13 shows the target value, the mean, standard deviation of fidelity, the relative bias and the bias of each level of contamination for the alternative method. Table 13 : Calculation of the alternative method bias
Values log (MPN /mL)
Contamination level Low Medium High
Target value 1.971 2.667 2.937
Average 1.795 2.580 2.870
Relative bias ‐8.93% ‐3.26% ‐2.26%
Bias ‐0.176 ‐0.087 ‐0.067
3.3.2. Accuracy profile Tables 14 and 15 show the values of tolerance and the tolerance limits of the alternative method for a probability value of tolerance of 80% (table 14) and of 90% (table 15).
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 17/54
Table 14 : Values and tolerance limits of the alternative method with ß = 80 %
Probability of tolerance
Niveaux Log CFU/L log (MPN /mL)
Low Medium High
80%
Low tolerance value 1.482 2.415 2.742
High tolerance value 2.107 2.746 2.999
Low tolerance limit ‐0.489 ‐0.253 ‐0.067
High tolerance limit 0.137 0.079 0.062
Table 15 : Values and tolerance limits of the alternative method with ß = 90 %
Probability of tolerance
Niveaux Log CFU/L
Low Medium High
90%
Low tolerance value 1.389 2.366 2.704
High tolerance value 2.201 2.795 3.037
Low tolerance limit ‐0.582 ‐0.302 ‐0.233
High tolerance limit 0.230 0.128 0.100
Figures 5 and 6 show the accuracy profiles using respectively ß = 80% and ß = 90%. Figure 5 : Accuracy profile of the alternative method with tolerance probability of 80 % and acceptability limits at 0,5 log
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 18/54
Figure 6 : Accuracy profile of the alternative method with tolerance probability of 90 % and acceptability limits at 0,6 log
Comments The accuracy profile obtained from the results of the reference method and the alternative method shows that the bias of Colilert method for the enumeration of E. coli in bathing waters is acceptable. The tolerance limits of the alternative method for a probability of 90% tolerance are included within the limits of acceptability of 0,6 log.
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 19/54
4. Extension study The aim of the extension study was to compare the results obtained with Colilert‐18 or Enterolert‐E with the use of a Quanti‐Tray 2000 or the use of a Quanti‐Tray, in order to allow the use of both devices in the framework of a certification NF Validation concerning each IDEXX kit using a Quanti‐Tray or a Quanti‐Tray 2000.
4.1. Results and interpretation Two sets of results are available:
‐ ISHA data from the comparative study for the NF Validation certification of the method Enterolert‐E with Quanti‐Tray 2000, ‐ IDEXX data from an analysis of a tap water using Colilert‐18 associated with Quanti‐Tray 2000 and with Quanti‐Tray.
4.1.1. Results from Enterolert‐E / Quanti‐Tray 2000 comparative study
Raw results Results have been collected from samples used in the comparative study for the validation of the method Enterolert‐E in the common enumeration range of the two devices, namely from 10 to 2000 MPN/100 mL. A minimum of 10 results was asked by the Technical Board: it’s a total of 18 samples that have been taken into account. Results are available in the summary report of the AFNOR Certification validation of the Enterolert‐E method. A two‐dimensional graph is shown in figure 7, presenting the results obtained with the Quanti‐Tray 2000 (the “validated” Quanti‐Tray for the Enterolert‐E method) as the reference method. Figure 7 : Comparison of results obtained with Quanti‐Tray 2000 and with Quanti‐Tray for the validation of the Enterolert‐E method
0,0
0,5
1,0
1,5
2,0
2,5
3,0
3,5
4,0
0,0 0,5 1,0 1,5 2,0 2,5 3,0 3,5 4,0
Quan
ti‐Tray (log MPN/100 m
L)
Quanti‐Tray 2000 (log MPN/100 mL)
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 20/54
Statistical interpretation
Validation protocol for an alternative commercial method as compared with a reference method:
A statistical interpretation has been performed according to the requirements of the Validation protocol for an alternative commercial method as compared with a reference method, considering the Quanti‐Tray 2000 as the reference device and using the tests for the relative accuracy. Results are available in the summary report of the AFNOR Certification validation of the Enterolert‐E method. According to this protocol, the relationship of relative accuracy between QT‐2000 and QT is evaluated with the linear model: 'y = a + bx'. This formula corresponds to the equation of the linear regression drawn from raw results obtained by experimentation, y representing the QT‐2000 devices and x the QT‐devices. There is a perfect accuracy (or there is no systematic bias) between the two methods if this equation is equal to the theoretical 'y = x' equation, which applies in the ideal model where the two methods behave similarly. The intercept is theoretically zero in this ideal model (hypothesis [a = 0]). The estimated intercept obtained with the two methods is checked using p {a = 0}. If the alternative method is a systematic bias against the reference method, the probability p {a = 0} is less than α = 0.05. The 'b' slope is theoretically equal to 1 in the ideal model (hypothesis [b = 1]). The estimated slope obtained with the two methods should pass by p {b = 1}. Statistically, if the alternative method does not give the same values as the reference method, the probability p {b = 1} is less than α = 0.05. The results of the statistical tests are shown in the table below.
Rob.R Regression used T critical a t(a) b t(b) Probabilities (%)
The equation for the regression line is as follows: log Alt = 1.040 log Ref – 0.097. Hypothesis [a = 0 and b = 1] is accepted for the comparison of the enumeration of enterococci with the Enterolert‐E method using a Quanti‐Tray versus a Quanti‐Tray 2000.
Student‐Fisher test A Student‐Fisher test has been also performed from the data obtained during the validation of the Enterolert‐E method. The results of the test are shown in the table below:
t-Test: Paired Two Sample for Means Parameter Quanti-Tray Quanti-Tray 2000
Mean 1.998 2.015 Variance 0.280 0.259
Observations 36 36 Pearson Correlation 0.883
Hypothesized Mean Difference 0 df 35
t Stat -0.398 P(T<=t) one-tail 0.346 t Critical one-tail 1.690 P(T<=t) two-tail 0.693 t Critical two-tail 2.030
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 21/54
Both one‐tailed and two‐tailed tests conclude that there is no statistically significant difference between the enumeration of enterococci with Quanti‐Tray or with Quanti‐Tray 2000 at α=0.05.
4.1.2. Results from Colilert‐18 / Quanti‐Tray study
Raw results Results were obtained from IDEXX Company. An Escherichia coli suspension was spiked in a neutralized tap water from 30 to 180 CFU/100 mL and then analyzed with Colilert‐18 associated with Quanti‐Tray and with Quanti‐Tray 2000. Results are available in the summary report of the AFNOR Certification validation of the Enterolert‐E method. Two two‐dimensional graphs are shown in figure 8, presenting the results obtained with the Quanti‐Tray (the “validated” Quanti‐Tray for the Colilert‐18 method in drinking waters) as the reference method. Figure 8 : Comparison of results obtained with Quanti‐Tray 2000 and with Quanti‐Tray for the enumeration of Escherichia coli in tap water
Statistical interpretation A Student‐Fisher test has been performed from the data obtained. The results are shown in the table below.
t-Test: Paired Two Sample for Means
Parameter Quanti-Tray Quanti-Tray 2000 Mean 104.8 109.1
Variance 2119.6 3043.9 Observations 19 19
Pearson Correlation 0.892 Hypothesized Mean Difference 0
df 18 t Stat -0.745
P(T<=t) one-tail 0.233 t Critical one-tail 1.734 P(T<=t) two-tail 0.466 t Critical two-tail 2.101
0
50
100
150
200
250
0 50 100 150 200 250
Quanti‐Tray 2000 (MPN/100 mL)
Quanti‐Tray (MPN/100 mL)
0,0
0,5
1,0
1,5
2,0
2,5
0,0 0,5 1,0 1,5 2,0 2,5
Quanti‐Tray 2000 (log MPN/100 mL)
Quanti‐Tray (log MPN/100 mL)
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 22/54
Both one‐tailed and two‐tailed tests conclude that there is no statistically significant difference between the enumeration of Escherichia coli with Quanti‐Tray or with Quanti‐Tray 2000 at α=0.05.
4.2. Conclusion The assays realized showed that the enumerations with the NF Validation certified IDEXX methods can be performed either with a Quanti‐Tray device or with a Quanti‐Tray 2000 device according to the expected concentration of the target analyte in the sample without introducing any bias in the measurement.
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 23/54
5. Conclusion
Comparative study The linearity and relative accuracy of the Colilert‐18 / Quanti‐Tray or Quanti‐Tray2000 method for the enumeration of E. coli in bathing waters are satisfactory. The bias between the two methods is acceptable. The limits of detection and quantification of the method are satisfactory. Colilert‐18 / Quanti‐Tray or Quanti‐Tray2000 method for the enumeration of E. coli is specific and selective. Extension study showed that the enumerations with the NF Validation certified IDEXX methods can be performed either with a Quanti‐Tray device or with a Quanti‐Tray 2000 device according to the expected concentration of the target analyte in the sample without introducing any bias in the measurement.
Interlaboratory study The bias of the alternative method is relatively stable from the low level of contamination to the high level of contamination. For all levels of contamination, the tolerance limits are between the limits of acceptability, meaning that at least 90% of the results will be between the limits of acceptability as defined at 0,6 log.
Massy, January 13th, 2017 François Le Nestour
Head of the Unit innovation Biology
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 24/54
Code Souche Origine Stress appliquéIntensité du stress
Numéro Eau
ESC.1.116 Escherichia coli Eau de puits 4 j à 4°C + 10 min à -80°C + 5 min à 51°C 1,88 52 Plage de la RoquilleESC.1.116 Escherichia coli Eau de puits 4 j à 4°C + 10 min à -80°C + 5 min à 51°C 1,88 90 La sommeESC.1.116 Escherichia coli Eau de puits 4 j à 4°C + 10 min à -80°C + 5 min à 51°C 1,88 94 TroyesESC.1.117 Escherichia coli Eau de puits 4 j à 4°C + 10 min à -20°C + 5 min à 51°C 1,3 53 Plage de CarnonESC.1.117 Escherichia coli Eau de puits 4 j à 4°C + 10 min à -20°C + 5 min à 51°C 1,3 91 Saint Quentin en YvelinesESC.1.117 Escherichia coli Eau de puits 4 j à 4°C + 10 min à -20°C + 5 min à 51°C 1,3 95 EtampesESC.1.119 Escherichia coli Eau de distribution 4 j à 4°C + (5 min à -80°C + 5 min à 36°C) x2 0,9 54 Plage du CouchantESC.1.119 Escherichia coli Eau de distribution 4 j à 4°C + (5 min à -80°C + 5 min à 36°C) x2 0,9 92 Villennes sur SeineESC.1.123 Escherichia coli Eau 4 j à 4°C + (5 min à -20°C + 5 min à 36°C) x2 1,1 55 Plage du Point ZeroESC.1.123 Escherichia coli Eau 4 j à 4°C + (5 min à -20°C + 5 min à 36°C) x2 1,1 93 Saint Leger en YvelinesESC.1.111 Escherichia coli Eau de fontaine 4 j à 4°C + 30 min à -80°C + 10 min à 36°C + 10 min à 51°C 1,1 47 Saint RochESC.1.111 Escherichia coli Eau de fontaine 4 j à 4°C + 30 min à -80°C + 10 min à 36°C + 10 min à 51°C 1,1 20 FécampESC.1.111 Escherichia coli Eau de fontaine 4 j à 4°C + 30 min à -80°C + 10 min à 36°C + 10 min à 51°C 1,1 21 Mesnil Val plageESC.1.111 Escherichia coli Eau de fontaine 4 j à 4°C + 30 min à -80°C + 10 min à 36°C + 10 min à 51°C 1,1 22 Dieppe
ESC.1.113 Escherichia coli Eau de puits4 j à 4°C + 30 min à -20°C + 10 min à 36°C + 10 min à 51°C 1,6 48
Plage Saint Maurice (Palavas les flots)
ESC.1.114 Escherichia coli Eau de puits 4j à 4°C + 10 min à -80°C + 60 min à 36°C 1,1 50 Plage des dunes (Carnon-Plage)ESC.1.114 Escherichia coli Eau de puits 4j à 4°C + 10 min à -80°C + 60 min à 36°C 1,1 51 Plage du grand traversESC.1.120 Escherichia coli Eau 30 min à 56°C 1,7 23 QuendESC.1.120 Escherichia coli Eau 30 min à 56°C 1,7 24 Saint MargueriteESC.1.122 Escherichia coli Eau 10 min à -20°C + 7 min à 51°C 1,7 25 Saint Pierre en PortESC.1.122 Escherichia coli Eau 10 min à -20°C + 7 min à 51°C 1,7 26 Veulette sur merESC.1.123 Escherichia coli Eau 10 min à -20°C + 5 min à 51°C 0,5 27 CharronESC.1.123 Escherichia coli Eau 10 min à -20°C + 5 min à 51°C 0,5 28 La RochelleESC.1.112 Escherichia coli Effluent secondaire (30 min à -80°C + 15 min à 55°C) x2 0,9 1 Saint BrevinESC.1.112 Escherichia coli Effluent secondaire (30 min à -80°C + 15 min à 55°C) x2 0,9 2 BerckESC.1.124 Escherichia coli Eau de rivière 7 min à 51°C 0,6 15 Cayeux sur mer
Appendix 1 - Bacterial stress
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 25/54
Appendix 2
Relative accuracy results
IDEXX - Colilert-18 / Quanti-Tray or Quanti-Tray 2000 - BW Summary report - v1 January 2017
Institut Scientifique d'Hygiène et d'Analyse 26/54