Regulatory 101 Putting AIQ to Use The difference between: - Operational Qualification (OQ) and - Performance Qualification (PQ) Paul Smith September 17, 2015 Confidentiality Label 1
Regulatory 101
Putting AIQ to Use
The difference between: - Operational Qualification (OQ) and - Performance Qualification (PQ)
Paul Smith
September 17, 2015
Confidentiality Label
1
September Seminar
July Seminar Focused On These areas
Previously: 10 Steps to a “New Instrument”….
DQ URS User Requirement Specification
• What you need the instrument to do • Purchasing decision
Design Qualification
• Why suitable • Range of use
Analytical Need
1.
Approval To Buy
2.
Instrument “Selection”
3.
Order Instrument
4. Delivery
5.
Install 6.
Review & Approval
8.
Release For Use
9. Maintenance 10.
PQ
Installation Qualification
• Installed correctly • Lab. is suitable
Operational Qualification
• Functional tests - works as expected – in the Lab. • Test range of use
Performance Qualification
• Application test – works with your applications
IQ
OQ
Qualification 7.
September 17, 2015
Confidentiality Label
2
WebEx Structure Contents
A. The Instrument Life Cycle Changes in Requirements Related Regulatory Information
B. Regulations & Historical Interpretation
Operational Qualification (OQ) Performance Qualification (PQ) C. Examples of OQ & PQ
Questions ?
Divided into 3 sections. Within the time frame, can’t answer detailed questions about specific instrumentation……. However, if you have a specific Qualification compliance question - you would like an answer to - use the on-line system to ask, and we will provide reference information. A key part of this WebEx is to highlight current regulatory information that can help laboratories make qualification and compliance decisions…..
September 17, 2015
Confidentiality Label
3
The Instrument Life Cycle
Historical Perspective
Management of Evolution of Instrument Life Cycle
USP <1058> OQ/PQ
September 17, 2015
Confidentiality Label
4
Purchase Qualify Train Maintain Use
Historical Instrument Compliance Requirements
September 17, 2015
Confidentiality Label
5
Is the instrument suitable for its intended use ?
Might have been considered as a sequential series of activities….
Independent Roles………
What is Laboratory Compliance ?
September 17, 2015
Confidentiality Label
6
Laboratory Compliance framework is used for:
• Justify - the Scientific validity of the results
• Defend - the Results & how they were generated
Complexities of Current Compliance Requirements
September 17, 2015
Confidentiality Label
7
Is the instrument suitable for its intended use ?
1. Is the data you are generating Valid ? (and the decisions made on that data)
2. If the instrument developed a fault – would you detect it ?
3. If someone “cheated” in your lab. – would you detect it ?
4. What are the risks around what you do – how are you managing them ?
Historically, a rational, technical response has been used……
• Instrument Working
• Method Validated
• System Suitability • Ref. Samples
“Sensitivity” of your ability to detect faults ?
Data Integrity ……
Risk Management ……
Historical Instrument Compliance Requirements
September 17, 2015
Confidentiality Label
8
Maintain Use
Maintain
Qualify
Use
Repair
Re-Qualify
All of these ensure instrument compliance status:
Qualify Qualify Maintain Use
“User” Maintenance
Planned Maintenance
GAMP 5 and USP <1058>
September 17, 2015
Confidentiality Label
9
Historically, GAMP and USP <1058> have been perceived as being very different, but, complement each other and there are more similarities than differences……..
1. Infrastructure
GAMP Category (software focus)
3. Non-Configured Products
4. Configured Products
5. Custom Applications
<1058> Category (usage focus)
A. Apparatus – Document • No Calibration Requirement • No Measurement Capability
[Appendix M4, GAMP 5]
B. Instruments – Calibrate • Follow SOP • Calibrate
C. Systems - Qualify
2. No Longer Used
Same principles – increasing complexity = more work to test / document “suitable” Diverse interpretation - in industry The framework followed and actions taken – are more important
GAMP GPG USP Chapter <1058>
[21 CFR, FDA]
[USP, e.g. <1058>]
[Method, Process, Software]
[GAMP 5, ASTM E2500]….
Calibration 1 – Terms Used in Regulated Industries
September 17, 2015
Confidentiality Label
10
1. Calibration
2. Qualification
3. Validation
4. Verification
May have a
“specific” meaning or
may be “label”
There is a need to “position” calibration - in the context of OQ / PQ [Qualification]
Calibration can mean something which is very specific to an action performed on a particular type of instrument, or, it can be a descriptive “label” to describe a kind of activity……. [e.g. “Calibration” = Qualification].
Calibration 2
September 17, 2015
Confidentiality Label
11
There are several references to Calibration in the CFR:
On Line eCFR:
21 CFR SECTION Calibration CFR Reference
58 (BIMO) 21 CFR 58.63(a)
110 (Food) 21 CFR 123.8(a)(2)(ii)
211 (Drugs) 21 CFR 211.68(a) 21 CFR 211.160(b)(4)
606 (Biologics) 21 CFR 606.60(a)
820 (Medical Devices) 21 CFR 820.72(a)
[1]
[57]
[54] [13]
[11]
21 CFR 820.72(b) [42] [3]
21 CFR 606.60(b) [6]
• These are the CFR references provided in the annual FDA 483 spreadsheet
• [57] = Number of 483 citations in 2014 [for this CFR]
• Search FDA FDA Warning Letters – [cut and paste the CFR from excel file into the Warning Letters search !]
FDA Warning Letter FDA 483 Inspectional Observations
Requirement General Requirements
Instrument Life Cycle
12
Design Qualification
(DQ)
Operational Qualification
(OQ)
Installation Qualification
(IQ)
Performance Qualification
(PQ)
Ongoing System
Suitability
Major instrument upgrade or New Analytical Method – DQ !
Regular or move or
major repair
OQ Verifies
DQ
Qualified State
Evolution of Compliance
September 17, 2015
Confidentiality Label
15
Qualification / Validation Time Line…..
PICSVF - 1994
GAMP 3 - 1998
GAMP 4 - 2001
GAMP 5 - 2008
PICSVF Formed - 1991
Pharmaceutical Industry Computer Systems Validation
Forum
Version 1 Validation
Management Paper
1994
AAPS
2008 AIQ <1058>
American Association of Pharmaceutical
Science
2003-4
Furman Paper Holistic Vs
Modular HPLC
2015
Draft <1058>
January 2002
May 1987 June1997
FDA Process Validation Guide
Lines General Principles
of Software Validation
(Draft)
General Principles
of Software Validation
(Final)
Impact of Diversification & History
September 17, 2015
Confidentiality Label
16
• Manufacturers / Laboratories - Different Interpretation of Requirements • Laboratories Just Managed any Differences…… • Cost – Reduced Instrument Availability • Potential Risk – Justifying / Managing Different Approaches
• No Strong Compliance Driver for Harmonisation……….
May 1987
FDA Process
Validation Guide Lines
2008 AIQ <1058>
GAMP 5 - 2008
These were after diversification of Qualification…..
USP <1058> Analytical Instrument Qualification (Draft)
September 17, 2015
Confidentiality Label
17
OQ Operational Qualification (OQ) is the documented collection of activities necessary to demonstrate that an instrument will function according to its operational specification testing in the selected environment. OQ demonstrates fitness of purpose for the user's ways of working, and should reflect the contents of the DQ document. Testing activities in the OQ phase may consist of these test parameters.
Operational Qualification
• Test the instrument in the environment • Operational Specification
• Demonstrate Fitness For Purpose
• Should Reflect the DQ Document
Key Messages
USP <1058> Analytical Instrument Qualification (Draft)
September 17, 2015
Confidentiality Label
18
PQ Performance Qualification (PQ) is the documented collection of activities necessary to demonstrate that an instrument consistently performs according to the specifications defined by the user, and is appropriate for the intended use. The PQ verifies the fitness for purpose of the instrument under actual conditions of use. After IQ and OQ have been performed, the instrument's continued suitability for its intended use is demonstrated through continued performance qualification.
Performance Qualification
• Specifications Defined by User
• Conditions of Use
• Implies System Suitability
Key Messages
USP <1058> Analytical Instrument Qualification (Draft)
September 17, 2015
Confidentiality Label
19
PQ PQ tests are usually based on the instrument's typical on-site applications and may consist of analyzing known components or standards. The tests should be based on good science and reflect the general intended use of the instrument
Performance Qualification
• Based on on-site application
• Reflect the intended use of the instrument Key Messages
USP <1058> Analytical Instrument Qualification (Draft)
September 17, 2015
Confidentiality Label
20
What does this mean ? Relates to historical, diverse interpretation of what an OQ and PQ contain, but not necessarily helpful…
Performing the activity is far more important than phase under which the activity is performed.
Historical OQ/ PQ Interpretation 1.
September 17, 2015
Confidentiality Label
21
Interpretation of process validation guidance - 3 Validation batches…… (for process).
For an instrument such as a pH Meter, calibration is the is the key test (matching USP <1058> Group B).
May 1987
FDA Process
Validation Guide Lines
OQ Perform Test Once (e.g. pH Meter Calibration)
PQ Perform Test 3 Times (e.g. pH Meter Calibration)
Historical OQ/ PQ Interpretation 2.
September 17, 2015
Confidentiality Label
22
So, the results of testing reference samples are evaluated against acceptance criteria…….
Diagnostic Tests Are Built into
The Instrument
OQ “Turn the instrument on, if no error messages are reported, the instrument has passed the OQ”
PQ “Proceed to test the PQ of the instrument with reference samples.”
Historical OQ/ PQ Interpretation 3.
September 17, 2015
Confidentiality Label
23
Implies that the OQ is a periodic evaluation (documenting the instrument is working correctly at the time of testing) and the PQ is an ongoing evaluation………..
OQ “Tests performed to document & evaluate that the instrument works as expected in the laboratory,…”
PQ “Tests performed to document & evaluate that the instrument will continue to work as expected in the Laboratory….”
FDA Method Validation Guidance….
Trend Analysis on Method Performance (system suitability ?)…..
Historical OQ/ PQ Interpretation 4.
September 17, 2015
Confidentiality Label
24
This interpretation has it’s “roots” in the 1994 Furman paper, based on HPLC instruments that were available 21 years ago. In principle, an instrument could pass a series of “module tests”, but fail an “overall test”. Therefore, an injector precision test, could be a module or holistic test (some companies define an injection test of a Chromatography system as a PQ).
HPLC OQ “During an OQ, the performance of each module or component is evaluated”
PQ “During the PQ, the “Holistic” or overall performance of the instrument is evaluated”
Where Does Calibration Fit In ?
September 17, 2015
Confidentiality Label
25
Although calibration can have a very specific meaning (which is not within the boundary of Qualification), some of the “common ways” CALIBRATION might be used, include:
1. Adjust – the response of an instrument based on external measurement or reference source
3. Check – the response of the instrument is within performance limits
2. Use – the response of a reference material to calculate values
For 1, before any adjustment is made, a test of the measurement device should be made, to highlight the performance “as found”. Some auditors have requested as found measurements for systems as complex as HPLC, FT-IR, LC-MS.. Etc. Impact assessment required…….
For 2, this represents “standardisation” or System Suitability type tests
For 3, this is a control test before using an instrument. A balance check weight is used to document acceptable performance before use.
What is the Role of Metrology Tests ?
September 17, 2015
Confidentiality Label
26
Where an instrument performs what could be classifies as a “Metrology Function” (where the functional performance can be evaluated independently, such as flow, rotation, temperature… Etc.), it is good practice to test the functionality in an equivalent (Metrology) way. Results of holistic tests do not provide 100 % assurance of the performance of the instrument (IMO) !
HPLC instruments for example, should include a mixture of metrology and holistic tests……..
Laboratory Instrument: Compliance Framework
September 17, 2015
Confidentiality Label
29
Who can Perform Installation, Qualification, Maintenance and Repair ?
There are no regulatory barriers about who “Should” perform qualification:
FDA# Friendly Compliance
# [and other Regulators]
• Pharmacopeia's • GMP (CFR, EU or other) • FDA Warning Letters
• How to “Justify” the Qualification ? [Are documents designed to support this ?]
• Are the people Trained ? [How do you know the quality of the training ?]
• How is Data Integrity Assured ? [Electronic data traceability ?] [Is repeat work “tracked” and visible ?]
• Is it Scientifically Valid ? [Can you explain / “defend” it ?]
• Does it Meet Your Requirements ? [Configured to your analytical range of use ?]
But….. you need to consider….
The regulations do not specify how often qualification should be performed !
How Different is Your Use of the instrument ?
[from the OQ]
How Complex is the instrument ?
Is a PQ Required ?
September 17, 2015
Confidentiality Label
30
pH Meter
HPLC / GC
Balance
Titrator
UV-Visible
Hyphenated
OQ Metrology Test [External Agent]
PQ System Suitability
[Equivalent]
OQ Functional
Use Test
PQ Application Based Test
Manage Auditors Expectations…
What About HPLC / GC
September 17, 2015
Confidentiality Label
31
In some companies: The Qualification Policy is PQ = System Suitability
Is this a Risk ?
“The auditor requested our PQ for the instrument. We advised that we include system suitability test and that is how we evaluate the ongoing performance of the instrument”
One of the risks associated with any audit are in part, how well the lab. Can “defend” their policy……
The auditor stated: “Routine tests do not constitute Qualification”…..
HPLC Example – Modify System Suitability….
September 17, 2015
Confidentiality Label
32
• Detected in OQ ?
• Detected in SST ?
• Detected in Enhanced SST
Failure
Risk – Some Instrument Failures Not Detected !
Implications for USP <1058> OQ
September 17, 2015
Confidentiality Label
33
Associate the DQ documents to the analytical testing………. Review Analytical Methods To List:
Maximum and Minimum Range of Use (in the Methods).
Configure the OQ testing to COVER the RANGE of USE. Feedback Loop: Update DQ & OQ – NEW METHODS Ensures Range of Use Tested
OQ Verifies
DQ
If the OQ is “Fixed”, the range of use Cannot be tested !
September 17, 2015
Confidentiality Label
34
HPLC OQ and PQ PQ OQ
Does the Instrument Work as Expected In the Laboratory :
• Tested Under Standard Conditions • OQ Service • Linked to RANGE of USE
Does the Instrument Work as Expected With the Analytical Methods:
• Tested with Method Conditions • Representative Method
OQ Parameter Min. Max.
Column Temp. 25 55
Wavelength 205 nm 273 nm Pump Flow 1 mL/Min. 2.5 mL/Min.
Autosampler Temp. 4 oC 8 oC
Gradient Step. 10 % 85 %
% RSD (Inj. Precision) 1.00 % (UV-Vis)
% Carry Over 0.20 % (UV-Vis)
PQ Parameter Value
Column Temp. 45
Wavelength 264 nm
Pump Flow 1.5 mL/Min.
Autosampler Temp. 4 oC
Gradient Prog. 10 to 65 %
% RSD (Inj. Precision) 0.5 %
% Carry Over 0.1 %
OQ Verifies: Functionality Range of Use
Technical Justification Vs Explanation
September 17, 2015
Confidentiality Label
35
Where justification is provided for not including / performing a functional test of an instrument.
The auditor may have a different Interpretation ……. RISK
For example, the principle of operation of an FT-IR Instrument: - Internal He-Ne Laser Therefore, in principle, because of the internal Laser, no wavelength calibration is required….
In Practice: it is a much lower risk to test the Wavenumber accuracy !
FDA Warning Letter Search – 10 Years
September 17, 2015
Confidentiality Label
36
21 CFR 211.68 21 CFR 211.160(b) 50 103
Some of the most common qualification problems that come up in FDA warning letters include:
• Calibration not performed or parts of the system missed (Justification) • No electronic raw data for the work • The tools included in the calibration were not calibrated • The people performing the work were not suitably trained
Relationship Between 483 Observations….
September 17, 2015
Confidentiality Label
37
Correlation Between: Complete test data included in records
Calibration/Inspection/Checking not done
21 CFR 211.194(a)
21 CFR 211.68(a)
Over 2006 to 2014
This graph shows that there is a
correlation between “Complete Data” and
instrument calibration !
Drive for Harmonisation ?
September 17, 2015
Confidentiality Label
38
Increased Regulatory Focus on Data Integrity: Is expected to motivate companies towards harmonised compliance!