Biostatistics in development of Medical Devices By T. Mudde - Clinquest (Qserve Conference 2013)

Fulfilling the Promise of Medicine Together

Clinquest

CLINQUEST SERVICES

Biostatistics in

Development of Medical Devices

2 Clinquest Services Qserve Conference

OUTLINE

3 Clinquest Services Qserve Conference

• Guidelines

• Clinical Investigations

(Statistical Section) Protocol

• Precision and Bias

• Adaptive Designs

EUROPEAN GUIDELINES

Clinquest Services Qserve Conference 4

“Clinical data must be provided for ALL medical devices. This may be

literature review or clinical investigation depending on device class and

use.”

When must/should a clinical investigation be undertaken?

To ensure a high level of safety and performance, demonstration of

compliance with the general safety and performance requirements

should be based on clinical data that, for class III medical devices and

implantable medical devices should, as a general rule, be sourced

from clinical investigations to be carried out under the responsibility of

a sponsor who can be the manufacturer or another legal or natural

person taking responsibility for the clinical investigation.

Depending on clinical claims, risk management outcome and on the

results of the clinical evaluation, clinical investigations may also have

to be performed for non-implantable medical devices of classes I, IIa

and IIb.

FDA GUIDELINES


The collection and evaluation of sound clinical data are the basis of the approval process for many medical devices.

Goals for device studies (including IVD studies) are:

• Producing valid scientific evidence, demonstrating

reasonable assurance of the safety and effectiveness of the

product

• Protecting the rights and welfare of study subjects

FDA GUIDELINES


Valid scientific evidence is defined as:

“Evidence from well-controlled investigations, partially controlled

studies, studies and objective trials without matched controls,

well-documented case histories conducted by qualified experts,

and reports of significant human experience with a marketed

device, from which it can fairly and responsibly be concluded by

qualified experts that there is a reasonable assurance of the

safety and effectiveness of a device under its conditions of use.”

GUIDELINES


Clinical Investigation Plan

The clinical investigation plan (CIP) shall define:

• Rationale of the clinical investigation

• Study objectives

• Study design

• In/exclusion criteria

• Endpoint choice

• Proposed analyses: statistical considerations

• Monitoring

• Conduct and record-keeping: data management

GUIDELINES


Statistical Section Clinical Investigation Plan

The statistical section should at least include:

• Analysis populations used

• Missing data strategy

• Definition of endpoints

• Hypotheses to be tested (efficacy/safety endpoints)

• Planned statistical methodology

• Sample size justification

• Sensitivity analyses planned

• Planned interim analyses

• Detailed Statistical Analysis Plan


INTERACTION SPONSOR- BIOSTATISTICIAN

Study Design Endpoint Choice

Sample Size Hypotheses

Study Objective

Bias

Precision

PRECISION / BIAS


Precision small

Bias large Precision large

Bias large

Precision small

Bias small

Precision large

Bias small

X x X x x x

x x x x x x

X x X x x x

x x x x x x

PRECISION / BIAS


• Precision: random fluctuations

Maximizing precision by:

- work with large samples

- work with homogeneous samples

- use paired designs (patient/sample is its own control)

- use extra information, e.g. baseline measurements

• Bias: systematic deviate from true result

Minimizing bias by:

- randomization (decreases selection bias and impact of confounding)

- blinding (decreases observation bias)

- use extra information, e.g. baseline measurements (decreases impact of confounding)

- avoid missing data

N

ses ..

EXAMPLE STENT STUDY


Comparison of Stenting versus Balloon Angioplasty (PTA) for the

treatment of below the knee artery disease (Class III).

Efficacy Study Objective: - Does our Stent perform better than balloon angioplasty?

Ethical, control standard care

Design: - Parallel Arm, Multiple Centers

- Superiority Trial

- Randomized? Yes

- Blinded? No

- Duration FU? Long enough for acceptable evaluation of

performance and safety (1 year)

- Interim Analyses? No

Primary efficacy endpoint: - Binary in-segment stenosis at 12 months by angiography

STENT STUDY


Possible bias:

- Objective primary endpoint

- Observation bias, blinding not possible

- Use ITT analysis population in case of

randomized study

- Confounding possible?

- Missing data

ANALYSIS DATASETS


• ITT Analysis set: All randomized patients; analyzed as

randomized.

• Per Protocol set: Set of patients with minimal violations against

the protocol as: errors in treatment assignment,

use of excluded medication, poor compliance,

loss to follow-up and missing data; analyzed as

treated.

To be set before database lock.

• Full Analysis set: The analysis set that is as complete as

possible and as close as possible to the

intention to treat ideal of including all

randomized subjects (ICH).

ANALYSIS DATA SETS


• ITT 50 patients on stent 50 patients on angioplasty

• PP 55 patients on stent 45 patients on angioplasty

PP analysis generally used as sensitivity analysis

R

Angioplasty N1=50

5 patients too severe

Stent N2=50

5 patients go over on

stenting

Hospital

A

B

Total

Died 63 16 79

Survived 2037 784 2821

Total

Mortality rate

2100

3.0%

800

2.0%

2900

BIAS BY CONFOUNDING VARIABLES


Mortality rate comparison

Relative risk hosp. B vs. hosp. A = 0.66,

34% lower risk of dying in hosp. B than in

hosp. A

Good

A

Health

B

Total

Poor

A

Health

B

Total

Died 6 8 14 57 8 65

Survived 594 592 1186 1443 192 1635

Total

Mortality rate

600

1.0%

600

1.3%

1200 1500

3.8%

200

4.0%

1700


33% higher risk of dying in hosp. B than

in hosp. A


5% higher risk of dying in hosp. B than

in hosp. A



Good

A

Health

B

Total

Poor

A

Health

B

Total

Died 6 8 14 57 8 65

Survived 594 592 1186 1443 192 1635

Total

Mortality rate

600

1.0%

600

1.3%

1200 1500

3.8%

200

4.0%

1700

Mortality rate confounded by health status because :

- Health status has impact on outcome

- Health status not equally distributed over the two hospitals

Statistical analysis: Adjusted relative risk hospital B vs. hospital A = 1.14,

14% higher risk of dying in hospital B than in hospital A, taking health status

into account.


Clinquest Services Qserve Conference

18

• Identical distribution of prognostic factors over treatment groups

(stratified randomization)

• Subgroup analyses (problem of multiple testing and low power)

• Retrospective control for prognostic/confounding factors during

statistical analysis

HOW TO COPE WITH PROGNOSTIC/CONFOUNDING FACTORS ?

FACTORS?

Confounding variable is a background factor which:

– is not equally distributed over the risk groups

– influences the outcome variable(s)

Primary efficacy endpoint:

Try to reject H0

-If p-value < 0.05 reject H0: there is a statistically significant therapy effect

-If p-value ≥ 0.05 accept H0: there is no statistically significant therapy effect.

H0 : π A = π B no stent therapy effect π A π B 0

HA : π A π B stent therapy effect π A π B 0

STENT STUDY, HYPOTHESES TO BE TESTED


STENT STUDY, STATISTICAL INFERENCE


ITT Population

-40.0%

-30.0%

-20.0%

-10.0%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%P

erc

en

tag

e w

ith

S

ten

osis

Stent Treatment

PTA Treatment

Difference

STENT STUDY, STATISTICAL INFERENCE


Diabetes

-60.0%

-50.0%

-40.0%

-30.0%

-20.0%

-10.0%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

StentTreatment

PTATreatment

-60.0%

-50.0%

-40.0%

-30.0%

-20.0%

-10.0%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

StentTreatment

PTATreatment

Non-diabetes

p-value<0.001

p-value=0.313

Stent PTA

Diabetes 64.6% 64.4%

Diabetes: prognostic factor, not confounding

STENT STUDY, MISSING DATA


Therapy

Stent (N=113)

PTA (N=115)

Total

Restenosis 15 (22.4%) 31 (41.9%) 46

No Restenosis 52 43 95

Total

Missing

67

46 (40.7%)

74

41 (35.7%)

141

87 (38.2%)

worst case

best case

Restenosis:

23 Stent,

21 PTA

STENT STUDY, SAMPLE SIZE CALCULATION


Stent study:

With 61 lesions in each therapy arm, a clinically meaningful difference of 25% in 12 months in-segment restenosis rate can be detected with a power of 80% at a two sided significance level of 5%, assuming a 12 months restenosis rate of 45% in the group of subjects who only receive balloon angioplasty. The total sample size was increased to almost 110 lesions per therapy arm to account for a 45% drop out rate.

Therapy

Stent (N=113)

PTA (N=115)

Total

Restenosis 15 (22.4%) 31 (41.9%) 46

No Restenosis 52 43 95

Total

Missing

67

46

74

41

141

87

Drop out rate: 38.2%

Power to detect 25% difference = 85.8%



• Performed on the basis of assumptions concerning:

- How accurate can the primary endpoint be measured?

- What is the clinically significant difference you like to see

(HA)?

- What do you expect as outcome for the control group?

• Performed on the basis of agreements about:

- significance level: (e.g. 5%)

- power: 1- (e.g. 90%)



Hypothesis Testing Type I error / Type II error

(Type I error)= Significance level P(reject H0 | H0 is true) = P(false positive)

of the test

(1 - ) = Power of the test: P(reject H0 | HA is true)

(Type II error): P(accept H0 | HA is true) = P(false negative)

Power: Probability to accept correctly, by a given , the alternative

hypothesis.

H0 HA

ADAPTIVE DESIGNS


Adaptive Design Clinical Study: a study that includes a

prospectively planned opportunity for modification of one or more

specified aspects of the study design and hypotheses based on

analysis of data (usually interim data) from subjects in the study

(FDA).

Goal: to make the study more efficient: shorter duration, fewer

patients, more information

Possible problems:

Operational bias: revisions not previously planned and made or proposed after an un-blinded interim analysis raise major concerns about study integrity.

Multiple testing: control of type I error rate

Regulatory concerns: FDA communication/review needed

ADAPTIVE DESIGNS


Planned Adaptive Design Fixed Sample Design

• Allocation Rule

- Can be fixed but can change based - Randomization remains

on accruing data fixed throughout the study

• Sampling Rule

- How many subjects sampled at next - Only one stage

stage (cohort)

- Sample size recalculation based on - Fixed sample size

interim results

• Stopping Rule

- When to stop a trial: efficacy, futility - No early stopping

• Decision Rule

- E.g. dropping study arm - No changes

EXAMPLE CIN-EVENT STUDY


Comparison experimental device system with standard hydration

protocol for preventing the incidence of CIN (Contrast Induced

Nephropathy), after the administration of contrast media. (Class

IIb).

Efficacy Study Objective: - Evaluation of the comparability between experimental device and

standard care in CIN events within 3 days post contrast administration.

Design: - Parallel Arm, multiple centers

- Superiority Trial

- Randomized? Yes, stratified by Y/N NSTEMI

- Blinded? No

- Duration FU? 3 days for primary efficacy endpoint, 90 days for safety

- Interim Analysis? Yes

Primary efficacy endpoint: - CIN event within 3 days; missing information for CIN: failure

CIN-EVENT STUDY, SAMPLE SIZE CALCULATION


Assumptions:

3-day CIN event rate control arm: 15%

Assuming 80% power and with a 1:1 randomization allocation ratio, 155 subjects in each group are required to demonstrate the expected difference of 10% in the incidence of CIN between groups at a significance level of 0.05 based on a 2-sided test.

The study will randomize a total of 326 patients to account for a 5% lost to follow-up but based on the interim analysis, could randomize up to a maximum of twice the initial sample size, or 652 patients.

CIN-EVENT STUDY, SAMPLE SIZE RECALCULATION


1

1ˆ Measured difference at interim analysis

Expected difference at final analysis

CIN-EVENT STUDY, SAMPLE SIZE RECALCULATION


Sample size recalculation will be performed based on the method of modification of sample size in group sequential clinical trials employing conditional power and maintaining type I error rate. The sample size will be adjusted such that the calculated conditional power obtained is 80%.

The trial can be stopped for reasons of overwhelming efficacy (p-value<0.003) or futility (conditional power under the current trend <20%).

QUESTIONS


?

Biostatistics in development of Medical Devices By T. Mudde - Clinquest (Qserve Conference 2013)

Business

primary efficacy

stent therapy

clinical investigation

implantable

relative risk

precision

clinical investigation

mortality