How to design a study

How to design a study

Nikolaos P. Polyzos M.D. PhD

What kind of study should I perform?

It depends on what you are seeking for..

General types of studies

Observational studies Checking association or relationship

Interventional studies (Clinical trials)

Checking a specific treatment protocol

Type of studies and level of evidence

RCTs

Metaanalyses

of RCTs

Cohorts, case control studies, cross-

sectional surveys

Case series, case reports, guidelines, expert opinions

LEVEL I evidence

LEVEL II evidence

LEVEL III evidence

Clinical trials

Randomized and non randomized

Treatment arm --with or without a control arm

Testing the safety and effectiveness of a drug or intervention

Clinical Trials-Phases

Phase I Safety testing- small groups of patients 10-15 patients

Phase II Pilot studies to confirm the effectiveness of the drug less than 100

patients

Phase IIILarge groups of patients for statistical confirmation of effect and

incidence of side-effects >100 patients

Phase IVPost marketing studies. Fine tuning and new rare findings from a very

large population

Randomized clinical trials

LEVEL I evidence

Randomized controlled studies(1)

Description:After assessment of eligibility and recruitment, but before the intervention

to be studied begins, are randomly allocated to receive one or other of the alternative treatments under study

Advantage:RCTs are considered by most to be the most reliable form of scientific

evidence in the hierarchy of evidence that influences healthcare policy and practice

Disadvantage: Costs Time Rare events

Designing a clinical and especially a randomized trial …….

Is it that easy?It c

an be..

THE STUDY PROTOCOL

…..possibly the most important part of your trial

The proper study protocol

1. Research question-rationale

2. Exact study design

3. Inclusion-exclusion criteria

4. Randomization procedure, allocation concealment, blinding

5. Timing of blood sampling and monitoring

6. Clearly defined interventions

7. The primary outcomes

8. Appropriate statistical analysis

9. Feasibility of the study

10. Data management

11. Ethical considerations

1. The research question & hypothesis

Simple –one question and one answer

In accordance with the available evidence

Ask yourself

1. Why is such a study valuable?

2. Can it change clinical practice?

2. Study design

Parallel-group – each participant is randomly assigned to a group, and all the participants in the group receive (or do not receive) an intervention.

Crossover – over time, each participant receives (or does not receive) an intervention in a random sequence.

Cluster – pre-existing groups of participants (e.g., villages, schools) are randomly selected to receive (or not receive) an intervention.

Factorial – each participant is randomly assigned to a group that receives a particular combination of interventions or non-interventions

(e.g., group 1 receives vitamin X and vitamin Y, group 2 receives vitamin X and placebo Y, group 3 receives placebo X and vitamin Y, and group 4 receives placebo X and placebo Y).

3. The population to include..

Clearly defined

Easy to recruit

Easy to follow

Unclear population…not replicable results

47 randomized trials using 41 definitions for poor ovarian responders

No more than 3 trials used the same definition Even trials from the same research group used

different definition

Who are the poor ovarian

responders

4.Randomization procedure

Simple randomization Block randomization Computer generated list

4. Concealment of patients allocation

The procedure for protecting the randomisation process so that the treatment to be allocated is not known before the patient is entered into the study

Methods to ensure1. sequentially numbered, opaque, sealed envelopes

(SNOSE);

2. sequentially numbered containers;

3. pharmacy controlled randomization;

4. central randomization

4. Blinding

Procedures that prevent study participants, caregivers, or outcome assessors from knowing which intervention was received

Types

1. Single-blind

2. Double-blind

3. Open Label

5. Patients’ monitoring

6. Clearly defined interventions

7. The primary outcomes

Do not select surrogate outcomes

e.g. pregnancy ---not oocytes, nor embryos

Exact timing when the primary outcome is measured

8. Statistical analysis

Who is going to be the

“person behind your numbers”

Crucial part of any clinical trial Wrong analysis=wrong results Improper test= not valid study A statistical mistake can jeopardise the work

of years

8. Sample size calculation

Power-sample calculation

1. 80% power, level of significance 0.05

2. Estimate your sample based on previous evidence

3. Do not make your sample size based on unrealistic assumptions

During the protocol formulation describe the statistics you will use

9. The feasibility of conducting the study

Resources available ( funding, personnel)

Available number of patients

Easy to follow your patients

10. Data management

Uniform templates for extracting data

11. Ethical considerations

IRB approval Inform consents Trial registration Sponsorship

Institutional Review Board (IRB) approval

All institutions should have (by law) an IRB to evaluate whether it is ethical to conduct the study

Submit your protocol and wait for acceptance prior conducting a trial

Explain in detail the rationale, the population, the interventions and the goals of the study

Inform your patients properly…. Get their written consent…..

Very detailed information for the treatment (drugs, duration, procedures)

What is the current gold standard

Why do you expect the new treatment to show difference

Do not use scientific terms…let them understand

Inform about potential side-effects and who is taking any liability in such a case

Obtain their signature

Register your trial !

After approval of IRB register your trial in a trial registry (e.g. clinicaltrials.gov)

Most journals require trial registration prior the conduction of the study

Declare any indirect or direct funding from the industry

ICJME suggests reporting of any potential conflict of interest related or not to the study

Industry funding should be reported

Why is so crucial to design and perform a clinical trial correct?

A research finding is less likely to be true when

1. the studies conducted in a field are smaller

2. effect sizes are smaller;

3. where there is greater flexibility in designs, definitions, outcomes, and analytical modes;

4. when there is greater financial and other interest and prejudice;

Top cited articles are they always telling the truth NO

Initial findings…might prove wrong in the future

Highly cited studies (>1000 citations) with efficacy claims

16% were contradicted by subsequent research

16% were found to have initially stronger effects.

44% were replicated also with a larger sample size in subsequent research compared with the original highly cited study)

24% had remained largely unchallenged.Ioannidis JP, JAMA 2005

ALL RANDOMIZED TRIALS REPRESENT TOP LEVEL OF EVIDENCENO

Poor study design….wrong conclusions

RANDOMIZED TRIALS ARE EASILY PUBLISHED

NOShow a benefit and get published!!

Negative RCTs may left unpublished….

INDUSTRY SPONSORHIP DOES NOT AFFECT THE OUTCOMES OF RCTS

Get the money …..and the results can benefit the drug

that sponsors you!

Industry and positive results…

titel41 19-04-2023

BMJ. 2003 May 31;326(7400):1167-70.

SPONSORSHIP ---- 4 times more like to have a positive result in your trial

New drugs are cost-effective even if they cost thousands of Euros

…..only If you get some sponsorship of course

Industry and positive results in cost-effectiveness analysis (CEAs) of novel drugs

82% of Sponsored cost-effectiveness analyses show that drugs are cost effective

Do not follow ethical guidelines and you can publish everything

Not always….

Data fabrication….even in the top journals

Sudbø  case (Lancet)

Among 908 in the study 250 had the same birth date

Copy-paste…..

Conclusions

Build your protocol

Select carefully your population

Decide which study design you will choose based on your resources

Always consider your study design when you interpret your results

No trial is without a limitation and always results can be due to chance…

The higher the level of evidence …the most likely to be true

Conclusions

Select your people for conducting your trial

1. Investigators

2. Study nurses

3. Co-ordinator

4. Statisticians

Follow rigorously the ethical guidelinesDo not attempt to publish at any cost!

Otherwise………….. You will simply get into…

Thank you