Jeff Gardner Principal Consultant

Mitigating Risk of Out-of-Specification Results During Stability Testing of

Biopharmaceutical Products

Jeff GardnerPrincipal Consultant

36th AnnualMidwest Biopharmaceutical Statistics Workshop

May 21, 2013

What Is an OOS Result?Business Consequences of OOS Results

- Regulatory requirements- Marketing/business impact

Risks of OOS Associated with Statistical Approaches to Estimating Shelf Life

- ICH Q1E Guidance- Mixed model (random slopes) estimation

Recommendations for Managing OOS Risk

Outline

From 21 CFR 314.81(b)(1)(ii):“Any failure of a distributed batch to meet [i.e. conform to] any of the specifications established in an application.”

From ICH Q6A/Q6B:“’Conformance to specifications’ means that the drug substance and/or drug product, when tested according to the listed analytical procedures, will meet the listed acceptance criteria.”

Expectation is that any test result, not just the average of results or majority of results, meets the specified acceptance criterion.

What Is an OOS Result?

Regulatory Requirements• Field alert required within 72 hours of identification

Commercial Impact• Potential product recall• Possible reduction of shelf-life

Business Impact of OOS Results

Illustration

363024181260

105

104

103

102

101

100

99

98

97

96

95

94

Months

Pote

ncy

95

105

S 0.885278R-Sq 57.1%R-Sq(adj) 49.9%

Regression90% CI90% PI

As seen here, a portion of the population of results extends into the OOS region, meaning that some risk exists of seeing OOS results.

“An appropriate approach to… shelf life estimation is to analyze a quantitative attribute… by determining the earliest time at which the 95 percent confidence limit for the mean intersects the proposed acceptance criterion.”

The above applies in either of two cases (contingent on assumption of common slope using “poolability” test):• Linear regression analysis is performed on data from a single batch

• In this case, mean = batch mean

• Linear regression analysis is performed on data from multiple batches• In this case, mean = mean of all batches in analysis• This mean is then taken to be an estimate of the product mean

ICH Q1E Guidance

Numerous critiques exist in literature which address “flaws” or limitations of the statistical approach outlined in ICH Q1E.

Methodology treats batch as a fixed effect, thereby limiting inference to only the sample of batches being evaluated.

Calculation of a (1- α)% prediction interval about the mean is only useful in defining the region of likely results from the batch(es) being evaluated.

This is despite the stated aim of the guidance which is to provide a “high degree of confidence” that all future batches will remain within acceptance criteria.

ICH Q1E Limitations

A statistical methodology is needed that more closely aligns with stated objectives of the ICH guidance documents.

Consider the model

A = Population intercept (mfg. process mean)B = Mean product degradation rate

ai = Initial value for batch i

bi = Degradation rate for batch i

xj = Stability test interval j

eij = Random error when measuring batch i at test interval j

, ,

Mixed Effects Model

Inclusion of random effects ai and bi in the model allows for calculation of prediction limits which more closely align with stated objective of ICH guidance.

When the model parameters are known, (1- α)% prediction intervals can be constructed at each xj such that

and

Constructing the intervals in this way assumes that , and are all independent.

Prediction Intervals Using Mixed Model

The first equation above is “philosophically” closest to ICH Q1E in that it governs the distribution of individual batch means at each stability testing interval.

Reflects test result(i.e. analytical method)

variability

When the model parameters are unknown (which is pretty much always the case),

and

Prediction Intervals Using Mixed Model

For sake of discussion, we will use the first equation as the basis for determining shelf life since it aligns closest with ICH Q1E.

Using α=0.05, shelf life = maximum xj such that Upper Acceptance Limit (UAL)

or

Lower Acceptance Limit (LAL)

Once shelf life has been determined, calculate

Pj(OOS) = P( LAL | xj = Shelf Life)+

P(OOS) = P( UAL | xj = Shelf Life)

Determination of Shelf Life

Shelf life and Pj(OOS) were determined as described above for n = 25 = 32 experimental settings in order to examine the relationship between the two.

A: 0, -0.4B: -0.015, -0.04 0.15, 0.33 0.008, 0.024 0.05, 0.15

Experimental Settings

All numbers are expressed as multiples of unit scale where

1 = (two-sided)

or1 = UAL (one-sided)

𝜑batch and 𝜑indiv were calculated for each combination of settings for A, B, , , and as specified below.

Example 1: P(OOS) at 24-Month Expiry

batch and indiv were calculated in this manner at 0 through 24 months for each of the 32 experimental settings.

A BMonth

(xj)Mean

(A + Bxj)P(OOS)

0 0.33 -0.015 0.008 0.15 0 0.00 -0.543 0.00290 0.33 -0.015 0.008 0.15 3 -0.05 -0.589 0.00430 0.33 -0.015 0.008 0.15 6 -0.09 -0.639 0.00640 0.33 -0.015 0.008 0.15 9 -0.14 -0.691 0.00960 0.33 -0.015 0.008 0.15 12 -0.18 -0.745 0.01440 0.33 -0.015 0.008 0.15 18 -0.27 -0.862 0.03060 0.33 -0.015 0.008 0.15 24 -0.36 -0.988 0.0594

Since parameters are pre-defined (known), z statistics were used in lieu of t for these calculations.

Probability of OOS at Expiry

0 3 6 9 12 15 18 21 240

0.010.020.030.040.050.060.070.080.09

Relationship Between Estimated Shelf Life and Probability of OOS at Expiryfor Varying Settings of Random Coefficients Model Parameters

Supported Expiry

Prob

abili

ty o

f OO

S Re

sult

The cumulative P(OOS) at xj is given by

Determining Cumulative P(OOS)

Determining cumulative probability of OOS provides a direct measure of business risk of field alert/product recall.

Note: this probability significantly increases if testing is performed more frequently later in shelf life

Therefore the cumulative probability is most useful when calculated only for those intervals that are specified when establishing the stability testing protocol.

Example 2: Cumulative P(OOS)A B

Month(xj)

Mean(A + Bxj)

P(OOS) Cum.P(OOS)

0 0.33 -0.015 0.008 0.15 0 0.00 0.0029 0.00290 0.33 -0.015 0.008 0.15 3 -0.05 0.0043 0.00720 0.33 -0.015 0.008 0.15 6 -0.09 0.0064 0.01350 0.33 -0.015 0.008 0.15 9 -0.14 0.0096 0.02300 0.33 -0.015 0.008 0.15 12 -0.18 0.0144 0.03710 0.33 -0.015 0.008 0.15 18 -0.27 0.0306 0.06660 0.33 -0.015 0.008 0.15 24 -0.36 0.0594 0.1220

Note: same model parameters used here as in Example 1.

Cumulative Probability of OOS

This chart illustrates that determining shelf life based on mean alone (x-axis) does not address the risk of OOS results.

0 3 6 9 12 15 18 21 240

0.02

0.04

0.06

0.08

0.1

0.12

0.14

Relationship Between Estimated Shelf Life and Cumulative P(OOS)for Varying Levels of Random Coefficients Model Parameters

Supported Expiry

Cum

ulati

ve P

roba

bilit

y of

OO

S

Application

Consider the following:

Example 3: 12 Month Assay Data

1815129630

105

103

101

99

97

95

Month

Assa

y

Spec Limit

Spec Limit

A1341A1342A1343

Lot

Plot of Assay (% Label Claim) vs Month

Example 3 cont’d: Q1E Approach

• Per ICH Q1E, p-value for the month*lot effect (<0.25) requires determination of individual confidence limit for each lot.

• Shelf life is based on the earliest time at which any lot’s confidence limit intersects the lower acceptance limit (shown in previous graph).

• In this case, the data supports a maximum shelf life of 15 months which per ICH Q1E would have to be verified with real-time data.

Example 3 cont’d: Mixed Model Approach

The point estimates for A, B, , , and can be used to determine Pj(OOS).

(Note: Output is from SAS®PROC MIXED)

AB

σ 𝑖𝑛𝑡2

σ 𝑠𝑙𝑜𝑝𝑒2

σ 𝑒𝑟𝑟2

Example 3 cont’dUsing = ,

Based on a nominal α=0.05, the cumulative P(OOS) suggests a shelf life of 12 months.

Summary

Estimation of shelf-life is based on the earliest point at which a batch’s true mean is likely to intersect a specification limit. • Substantial risk exists for observing individual OOS analytical

results during the expiry period• OOS risk is greatest toward the end of product shelf-life

Summary of ICH Guidance

A prediction interval based on a mixed effects model can be constructed with either of two aims:• philosophical alignment with ICH guidance (i.e. including

only variance components for batch intercept & slope variability)

• minimizing risk of OOS results (by including residual variance component as well as batch variance components)

Either of the above methods are an improvement upon the statistical approaches outlined in ICH guidance.

Estimating Expiry Using Mixed Model

It is recommended that both prediction interval calculations are employed for preliminary shelf life estimation.• Difference in shelf life estimates = impact of analytical

method variability on shelf-life• Additional analysis may be considered for evaluating how

this impact can be lessened via increased replication• Such understanding may provide guidance toward optimal

method validation

Impact of Analytical Method Variability

It’s recommended that a cumulative probability of OOS occurrence accompanies each preliminary shelf life estimate

1. calculate P(OOS) at each scheduled stability testing interval

2. calculate cumulative P(OOS) across all stability testing intervals

It may be worth considering whether the final proposed shelf life should be based on first determining a maximum cumulative P(OOS) rather than by focusing on lower bound on the population of batch slopes.

Formally Assessing Risk of OOS

Using experimental settings defined earlier, examine the sampling distributions of model parameter estimates via Monte Carlo simulations:• How does sampling error contribute to over-estimation or

under-estimation of P(OOS)?• What impact does analytical method variability have on shelf

life estimates if P(OOS) is used instead of the distribution of batch means at a given testing interval?

Future Research

End of Presentation