Investigating Out of Trend Results (OOT’s) Out of Trend Results (OOT’s) What is OOS and OOT OOS (Out Of Specification) is a comparison of one result verses a predetermined specification

Investigating Out of Trend Results (OOT’s)

Dr. Pradeep Nagalkar

Head

Quality Control (Bio. & Chem.) Haffkine Bio-Pharma. Corp. Ltd., Mumbai


- We will be discussing

- 1. What is OOT & OOS.

- 2. Examples of OOT & OOS.

- 3. Regulatory issues

- 4. Method for detection of OOT

- 5. Types of OOT

- 6. Documentation / SOP Consideration.

- 7. Investigation

- 8. Depth of investigation

- 9. Conclusion

Dr.Pradeep Nagalkar / March 2015


What is OOS and OOT

OOS (Out Of Specification) is a comparison of one result

verses a predetermined specification criteria while OOT

(Out Of Trend) is a comparison of many historical data

values verses time.

OOS investigations focus on determining the truth about

that one value while OOT investigation focus on

understanding non-random changes.



What is OOS and OOT

Example :

The specification limit for assay is : 95.0 – 105.0% w/w of label claim

1. For a particular batch , the result obtained 94.2%w/w This result is

out of specifications and is called OOS. An investigation is required.

Root Cause Analysis (RCA) is required for OOS investigation. Once a

root cause is identified, corrective and preventive measures need to

be taken.

2. The result obtained 95.8 w/w. Although the results are well within

the specifications, we should compare the result with the previous

batches trend. If found the average value of the trend 99.05 w/w then

this batch result (95.8 w/w) is called Out Of Trend (OOT). A through

understanding and control of the process is required.



OOS regulatory issues have been well understood and

documented in the literature. However, OOT issues, for

product stability, raw materials, in-process and finished

product, data identification and investigation are less

understood, but rapidly gaining regulatory interest.

Though FDA’s draft guidance indicates that much of the

guidance presented for OOS can be used to examine OOT

results, there is no clearly established legal or regulatory

basis for requiring the consideration of data that is within

specification but does not follow expected trend.



Methodology [3 sigma approach]

A minimum of 25 – 30 batches data shall be compiled for fixing the

Trend range.

Results that shall be obtained from the 25 batches tabulated, average

value, minimum and maximum values are noted.

Standard deviation will be calculated for these 25 batches. Excel

spread sheet shall be used for Standard deviation calculation.

Standard deviation will be multiplied by 3 to get the 3 sigma (3 ))

value.

Maximum limit will be arrived by adding the 3 1 value to the Average

value of 25 batches.

Minimum limit will be arrived by subtracting the 3 1 value from the

Average value of 25 batches. Minimum value may come in negative

also at times. Dr.Pradeep Nagalkar / March 2015


The maximum and minimum limits in above steps shall be

taken as the Trend range for upper and lower limits.

Any value that shall be out of this range will be considered

as Out of Trend (OOT) value or Outlier value.

Wherever specification has only Not more than, then only

Maximum limit for trend can be considered. Minimum limit

should be excluded.

Wherever specification has only Not less than, then only

Minimum limit for trend can be considered. Maximum limit

should be excluded.

Wherever specification has range then both the Maximum

and Minimum limits for trend should be considered.



Methods of Identifying OOT Results

1. 3 sigma approach :

Advantages: One advantage of this approach is that as long as

the assumptions are met, the rate of false positives can be set

when one calculates the limits.

Disadvantage: The products with limited data, the appropriate

limits may be difficult to determine. This can lead to wrongly

centered, too narrow or too wide OOT limits.

2. Regression-Control-Chart Method:

Advantage: Suitable for identification of the present OOT

results in case where is no historical stability data.

Disadvantage: The main disadvantage of this method is the

necessity of having results for each time point due to the fact

that the construction of the regression line was based on

gradually adding the values in each subsequent time point.



Methods of Identifying OOT Results

3. The by-time-point Method:

Advantages: The main advantage is that the absence of having

a result in any time point did not affect the analysis of the

previous or next point result.

Disadvantage: A large history of data is preferred for proper

use of this method.

4. Slope-Control-Chart Method:

Advantage: Any small change in the value of the tested

attribute from point to point is precisely recorded in the slope

value of each time point.

Disadvantage: If the test of the attribute were omitted in any

time point for various reasons, the limits of that time point may

not be appropriate.

Dr.Pradeep Nagalkar /

March 2015


There are mainly three types of OOT alerts:

– 1. An Analytical Alerts is observed when a single

result is aberrant but with in specification limit ( i.e.

outside normal analytical or sampling variation and

normal change over time).

– 2. A Process Control Alerts occurs when a succession

of data points shows an atypical pattern that was

possibly caused by changes to the laboratory or

manufacturing process. These data points might

originate from the same stability study or from multiple

studies assayed within a reasonably close time.

Despite the deviating trends, no pending potential OOS

situation occurs. Dr.Pradeep Nagalkar / March 2015


There are mainly three types of OOT alerts :

– 3. A Compliance Alert defines a case in which an OOT

result suggests the potential or likelihood for OOS

results to occur before the expiration date within the

same stability study (or for other studies) on the same

product.



SOP for OOT : Considerations

An investigation's depth may depend on the nature of the

data, the product history and characteristics being

measured, the potential risk to product quality, and the

potentially related safety or efficacy implications. A

review of the OOT alert procedures' performance might

coincide with the annual product review.

When preparing an SOP for handling OOT test results, it

is prudent to consider risks to product quality when

determining the company's assessment of OOT test

results. The depth of an investigation and the corrective

measures taken may depend on the potential or implied

risk to product quality. Dr.Pradeep Nagalkar / March 2015



How to set alert limits :SOPs may identify scientifically

sound approaches to determine procedures with

associated limits for the identification of each type of

OOT alert for stability test attributes such as assay and

degradation products. Once an approach is identified,

some potential topics to be included in the SOP are data

requirements, personnel responsible for setting the limits,

and methods used to detect each type of OOT alert.




How to set alert limits:

For new products, available stability data will be limited.

Established products often will have sufficient historical

data. The method for detecting OOT results may vary

depending on the type of OOT alert. Ideally, a method for

each OOT should be specified.

How to use limits : The SOP may contain a section about

how OOT alert limits are applied and which personnel will

be responsible for comparing new stability results with

the OOT alert criteria and timeliness of the review.




Investigation :

Investigation requirements could include personnel

responsibilities with defined timelines, documentation

requirements, and proper internal notification

requirements. The investigation's depth required for each

OOT alert will depend on potential risks to product

quality. The SOP should not be overly prescriptive

because the steps taken to investigate OOT results will

depend on the nature of the initial finding and the

investigation findings.




Investigation :

Nonetheless, SOPs should provide basic criteria for

conducting the investigations. Moreover, the steps taken

to investigate OOT results should be scientifically

defensible, and the rationale for the investigation's depth

should be documented. Although an OOT result usually

will be investigated as soon as possible after it is

identified, it also can be addressed more broadly during

the annual product review.

The notification requirements and the department

responsible for performing the annual product reviews

also could be included. Dr.Pradeep Nagalkar / March 2015


SOP for OOT : Considerations Depth of Investigation :

– If an analytical alert is observed, a laboratory

investigation may be performed. If the laboratory

investigation is inconclusive, then the supervisor may

take no further action at that time and monitor

subsequent time points closely. A review of results for

related or correlated test attributes also can facilitate

the investigation. Depending on the product, history,

and nature of the analytical alert data, the company

may decide to investigate whether a manufacturing

error occurred in cases in which a laboratory error is

not conclusive.




Investigation :

More data (i.e., the next stability test point) may be

needed to indicate whether the result is part of a long-

term trend (process control alert) or just an isolated

result.

If subsequent results within the study and across other

studies are not out of trend, then the initial analytical

alert result may be an isolated incident that requires no

further investigation.



SOP for OOT : Considerations Investigation :

– A process control alert may indicate unexpected

changes in product or assay performance. A stability

study with an unusual trend may reveal that a

characteristic's stability profile changed, whereas

multiple analytical alerts may suggest that the

measurement process is no longer in control.




Investigation :

When a process control alert is evident, the investigation will often begin with an assessment of the effect of potential changes to the laboratory process (e.g., changes in instrument, column, integration technique, or standard). It also may extend to the manufacturing process (e.g., looking for changes in materials, personnel, equipment, and procedures).




Investigation :

A compliance alert identifies a situation in which a particular study (or several related studies) may not meet specification limits during the expiration period, one may need to conduct a thorough investigation. In general, this investigation begins with the laboratory process and expands to the manufacturing process if no conclusive laboratory root cause is identified. The manufacturing investigation might include other batches (from the same or related products) that may be associated with this predicted failure to identify whether the discrepancy is isolated or systematic.




Investigation :

The investigation also could assess whether more analytical testing, an investigation into the manufacturing process, a product recall, or shorter dating is required. An investigation of a compliance alert aims for the early detection of potentially failing product and the identification of possible causes. If a root cause is determined, then appropriate action should be taken such as the identification of potential preventative actions.




Periodic review of limits :

SOPs may specify a periodic review process for OOT alert

limits. The frequency of the review and the personnel

responsible for completing it may be addressed in the

SOPs. This assessment can involve identifying a previous

result that was once within trend but is now an OOT

result. Or, it may find that a previous OOT result is now

within trend. It is natural that earlier conclusions be

revised in light of new information because developing

OOT alert criteria is an active and continuing process.



Conclusion :

Identifying OOT results is a growing interest for FDA and

vaccine Industry point of view. Despite this, very little has

been discussed in the scientific literature or in regulatory

guidance on this topic.

The vaccine industry still lacks having a proper guideline

for the identification of OOT results among results.

As a result, industry is not harmonized in the way they

conduct OOT analysis.

The method to determine an OOT should not be too

complex. However, some-thing too simplistic may not be

sensitive enough to identify a true OOT or may give a high

rate of false signals. Dr.Pradeep Nagalkar / March 2015


Conclusion :

Challenges:

What statistical approach should be used for OOT assessment,

What minimum data to be used,

What data to be used for updating the limits,

How is an OOT result is confirmed,

What action to be taken if an OOT result is confirmed,

How are OOT investigation incorporated in APR


Thank you

Investigating Out of Trend Results (OOT’s) Out of Trend Results (OOT’s) What is OOS and OOT OOS (Out Of Specification) is a comparison of one result verses a predetermined specification

Documents