Learning From Issues in Assay Cross-Validation / Method Transfers Peter van Amsterdam Abbott Healthcare Products, Weesp, Netherlands Presented at: The 5th Bioanalysis Workshop April 13-14, 2011, Montreal, Canada
May 10, 2015
Learning From Issues in Assay Cross-Validation / Method Transfers
Peter van AmsterdamAbbott Healthcare Products, Weesp, Netherlands
Presented at: The 5th Bioanalysis WorkshopApril 13-14, 2011, Montreal, Canada
Assay Cross-Validation/Method Transfers Date: 14 April 2011
2© 2011 Abbott
Contents
1. The Landscape
2. The Regulations
3. A few case studies
4. Method Transfers
5. Cross-Validation
6. Global Trials: blank matrices
7. Global Trials: OTC/Co-med interference
8. Questions / Discussion
Assay Cross-Validation/Method Transfers Date: 14 April 2011
3© 2011 Abbott
The Landscape
ToMethod Transfers
CRO PharmaCRO Y (site to site) N (rare)FromPharma Y (contracting) Y (site to site)
Cross-ValidationsComparatorCRO Pharma
Reference CROAlways, Never or Depends?
Pharma
Assay Cross-Validation/Method Transfers Date: 14 April 2011
4© 2011 Abbott
The Landscape
Contracting Site to SiteMethod Consistency
Pharma CRO CRO Pharma
Only 1 site active (move methods) Free, Similar, Identical or Depends?
Multiple sites active (expand business, capacity, regions) Free, Similar, Identical or Depends?
Role of CROContracting OutIncidental Regular
Pharma Innovative Increasing Importantanceof the Role of the CROGeneric
Assay Cross-Validation/Method Transfers Date: 14 April 2011
5© 2011 Abbott
Quiz (1)
• Who is (ultimately) responsible?
– Sponsor– CRO
• Who will be inspected?
– Sponsor– CRO
• Who pays the largest price for incompliance?
– Sponsor– CRO
FDA: Guidance for IndustryBioanalytical Method Validation
B. Partial Validation
Partial validations are modifications of already validated bioanalytical methods. Partial validation can range from as little as one intra-assay accuracy and precision determination to a nearly full validation. Typical bioanalytical method changes that fall into this category include, but are not limited to:
· Bioanalytical method transfers between laboratories or analysts
Assay Cross-Validation/Method Transfers Date: 14 April 2011
6© 2011 Abbott
EMA: Guideline On Validation Of Bioanalytical Methods
4.2 Partial validation
In situation where minor changes are made to an analytical method that has already been validated, a full validation may not be necessary, depending onto the nature of the applied changes. Changes for which a partial validation may be needed include transfer of the bioanalytical method to another laboratory, change in equipment, calibration concentration range, storage conditions etc. All modifications should be reported and the scope of revalidation or partial validation justified.
In most cases, provision of additional accuracy and precision data or relevant additional stability data on the modified issue may be sufficient.
Assay Cross-Validation/Method Transfers Date: 14 April 2011
7© 2011 Abbott
Guidance for IndustryBioanalytical Method Validation
C. Cross-Validation
Cross-validation is a comparison of validation parameters when two or more bioanalytical methods are used to generate data within the same study or across different studies. An example of cross-validation would be a situation where an original validated bioanalytical method serves as the reference and the revised bioanalytical method is the comparator. The comparisons should be done both ways.
When sample analyses within a single study are conducted at more than one site or more than one laboratory, cross-validation with spiked matrix standards and subject samples should be conducted at each site or laboratory to establish interlaboratory reliability. Cross-validation should also be considered when data generated using different analytical techniques (e.g., LC-MS-MS vs. ELISA) in different studies are included in a regulatory submission.
Assay Cross-Validation/Method Transfers Date: 14 April 2011
8© 2011 Abbott
EMA: Guideline On Validation Of Bioanalytical Methods
4.3 Cross validation
Where data are obtained from different study sites, comparison of those data is needed, and a cross validation of the applied analytical methods should be carried out. Differences in sample preparation or the use of another analytical method may result in different outcomes between the study sites. Cross validation should be performed in advance of study samples being analysedif possible. For the cross validation, the same set of QC samples should be analysed by both analytical methods. The outcome of the cross validation is critical in determining whether the obtained data are reliable and whether they can be compared and used. The difference between the two measurements should not exceed 15%.
Assay Cross-Validation/Method Transfers Date: 14 April 2011
9© 2011 Abbott
Regulatory: Conclusions
• Method TransferPartial Validation
– Accuracy & Precision– Other: depends
• (Essentially) different methods during drug development or muliplelabs in one study
Cross-Validation
– QCs & Samples (FDA)– QCs only (EMA)
Assay Cross-Validation/Method Transfers Date: 14 April 2011
10© 2011 Abbott
Quiz (2)
• Is Cross-Validation required for Pop-PK, PK/PD, S&M (pool data frommultiple studies)?
– Yes– No– Depends
Assay Cross-Validation/Method Transfers Date: 14 April 2011
11© 2011 Abbott
Assay Cross-Validation/Method Transfers Date: 14 April 2011
12© 2011 Abbott
Case: GC-ECD 3-keto-delta-4,6 Steroid (1990)
• Requirements:
– LLOQs: Parent 0.1 ng/ml, active metabolite 0.5 ng/ml– Range: At least 2 decades– No interference from endogenous steroids
• In house GC-MS instrument capacity to limited GC-ECD method
• Large volume and high time pressure clinical program launched
• Contracting out to 3 ~mid sized CROs
• Complicated and laborious assay.
– L/L and HPLC clean-up with fraction collection.– Required freshly distilled solvents– 2 analytes, 2 IS, 2 different GC set-ups
Assay Cross-Validation/Method Transfers Date: 14 April 2011
13© 2011 Abbott
Case: GC-ECD 3-keto-delta-4,6 Steroid (1990)
• Multiple studies running at multiple labs in parallel– Sponsor wanted to be in full control – Daily reporting of CAL & QC data + faxing selected chromatograms
• Issue: method only ran well on sponsor’s HP 5890.
– Less well on CRO’s HPs (2 CROs)• Troubleshooting and training of CRO staff by sponsor• Set-up solvent distillations
– Hardly at all on Varians (1 CRO)• Ship 2 sponsor’s HP 5890 to CRO• Training of CRO staff by sponsor• Set-up solvent distillations
Nightmare Method Transfer
Assay Cross-Validation/Method Transfers Date: 14 April 2011
14© 2011 Abbott
Case HPLC-EC Aryl-piperazine (1995)
• Requirements:
– LLOQ: Parent 0.1 ng/ml– Highly selective (multiple metabolites)– Range: 3 decades
• Early 90s
– HPLC-FLU failed, GC-NFID failed, GC-ECD failed. – Extremely complicated LC-GC method involving a combined hydrolysation /
derivatisation step worked, however only 1 column manufacturer and judged not-transferable: discontinued.
• New method HPLC-EC, but with extensive L/L and SPE sample clean-up and column swithing
• Large volume pre-clin & clinical program. Contracting out to 2 CROs
• Demand: exact copy of sponsor method
Assay Cross-Validation/Method Transfers Date: 14 April 2011
15© 2011 Abbott
Case HPLC-EC Aryl-piperazine (1995)
• CRO1: FAIL, CRO2: FAIL (and fail again)– Troubleshooting at CRO staff by sponsor
– Restart: CRO1: FAIL, CRO2: FAIL
– Extensive in-house training CRO technicians
– ReRestart: CRO1: OKish (large % batch failure), CRO2: OKish (background / positive blanks)
• CRO3: First time right and hardly ever failed a batch
• CRO4: OKish. – Strong desire for variation on method. Chrom & QC data looked very good.
– BUT: Indications of issues with separation from metabolites.
Nightmare Method Transfer
Case LC-MS/MS Benzoxazol-piperazine (2000)
• Requirements:
– LLOQ: Parent 0.02 ng/ml– Range: At least 3 decades
• Not very complicated or laborious assay, but challenging LLOQ
• After FIH, clinical studies and long term Tox to be contracted out.
– Demand: similar to sponsor method
• Problem free study conduct:
– in-house, CRO1 (until change of personnel), CRO2 (entire program), US-site (NA studies only), CRO3, co-development partner
• Except…
– BA lab of Tox CRO never, really never, got the method performing well for more then 2 batches.
Assay Cross-Validation/Method Transfers Date: 14 April 2011
16© 2011 Abbott
Dream Method Transfer
Case CTD of old drug (2005)
• Lypophilic aliphatic t-amino compound. No UV absorbance. No derivatization possible.
• Long development history
• Multiple methods and labs
– GC-FID, GC-NFID– Direct MS, GC-MS with some random IS, GC-MS with analogue IS– LC-MS with analogue, LC-MS/MS with deuterated IS
• Local validations only
– Oldest work: very limited
• No cross-validations done
Assay Cross-Validation/Method Transfers Date: 14 April 2011
17© 2011 Abbott
Challenging 2.7.1
Assay Cross-Validation/Method Transfers Date: 14 April 2011
18© 2011 Abbott
Method Transfers: Lessons Learned
• ‘Green Fingers’ methods are hard to transfer
– Match equipment, chemicals, consumables, glassware as much a reasonably possible
– Thorough training of CRO staff by Sponsor at CRO and/or at Sponsor site.– Exact copy of a method is no guarantee for success
• ‘Standard’ methods are relatively easy to transfer
– Match types of equipment, chemicals, consumables & glassware– Allow local adaptations to the method as long as they stay within the basic
principles of the reference method.– Be ready for failure and have a back-up scenario.
• Strive for consistency in methods over time
– Do Cross-Validations when methods change in their principles
Method Transfers: Validation
• Within lab (method upgrades, same sample work-up or measurement principles)
– Partial validation (accuracy & precision batches)
• From lab to lab (similar and identical methods)
– Full validation except long term stability & specificity/selectivity
• From method to method (different sample work-up or measurement principles)
– Full validation including long term stability & specificity/selectivity
Assay Cross-Validation/Method Transfers Date: 14 April 2011
19© 2011 Abbott
Cross-Validations (Interlab)
• Originator lab / method is the reference lab / method
– Regardless whether it is CRO or Sponsor
• Reference lab prepares test sample sets
– 20 Spiked Blanks (QCs)• Spanning the entire range. May be Routine QCs & Freshly prepared
– 20 Pooled Study Samples• Spanning the entire range
– Samples are blinded– Spare sets for possible future cases
• Labs analyze 2 sets. Usual run acceptance criteria. Create analysis report
• Reference lab writes interlab report
• Criteria:
– CV duplicates < 15%– Slope between 0.85 and 1.15
Assay Cross-Validation/Method Transfers Date: 14 April 2011
20© 2011 Abbott
Quiz (3)
• A Cross-Validation shows substantial differences between two techniques or labs, but perfectly correlated. Would you accept?
– Yes– No– Depends
Assay Cross-Validation/Method Transfers Date: 14 April 2011
21© 2011 Abbott
Global Studies: Blank Matrices
• Should we match the blank matrix with the regional origin of thesample?
• Considering:– Normal ranges on many many blood parameters span on average about
one decade– Blank matrices are different from person to person– There are no practical differences between populations in common lab
values (clinical chemistry)– Differences between males and females are larger then between any
other groups– Cultural differences, especially eating habits and types of food are of
larger impact on the matrix then genetic differences.
• My Answer: No
Assay Cross-Validation/Method Transfers Date: 14 April 2011
22© 2011 Abbott
Reference Ranges
Source: http://en.wikipedia.org/wiki/Reference_ranges_for_blood_tests
Assay Cross-Validation/Method Transfers Date: 14 April 2011
24© 2011 Abbott
Global Studies: OTC interference evaluation
• Testing on OTC interference is commonly done by a number of labs– Acetaminophen, Ibuprofen, Levonorgestrel, Nicotine, Caffeine, ....– Standard list varies from lab to lab
• Should we keep doing this? Extent? Or stop?
• Considering:– Country to country differences in OTC drugs – Difficult and quite costly to include OTC drug metabolites– Herbal products containing of 100s of compounds in various compositions– Influence of food on plasma and urine composition– Very selective tools: L/L or SPE + LC-MS/MS
• My Answer: No
Targeted OTC and Co-medication interference testing
During drug development:
• Day-by-day: check on used co-medication or OTC when for a given subject:
– Analyte levels are substantially different from the general picture– IS response substantially differs from other samples– Broad, odd or extra peaks are detected
• Early phase III: obtain listings on used OTC drugs and any othermedication used during clinical trials.
– Evaluate list of compounds with respect to the likeliness to interfere• Molecular weight, Structure, Concentrations, Chemicophysical properties
– Test selected potentially suspect compounds at relevant concentrations in the assay
Assay Cross-Validation/Method Transfers Date: 14 April 2011
26© 2011 Abbott
Questions !
• CRO <-> Sponsor– Who is responsible: sponsor or CRO?– Who will be inspected: sponsor or CRO?– Who pays the largest price for incompliance: sponsor or CRO? – Sponsor SOPs or CRO SOPs: yes, no or depends?
• Cross-Validation– Is Cross-Validation required for Pop-PK studies: yes, no, depends?– What if when the comparator is not matching the reference lab data?– Transferred assays must be 1-to-1 copies of the original: yes, no, depends– Transferred methods must cross-validated: always, never, depends
• Multi-site / Global– Should we match the blank matrix with the regional origin of the sample?– Should we take the international OTC market into account when doing OTC
interference tests?
Assay Cross-Validation/Method Transfers Date: 14 April 2011
© 2011 Abbott 27