8/10/2019 400863 Lecture 12 Understanding Results 3 2014
1/40
Foundations of Research and Evidence- Based Practice
Lecture 12
Understanding Results 3Nerida L. Klupp
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
2/40
Where are we up to.
Step 1. Is the result from data that is
Continuous
Categorical
Step 2. Is the result
Descriptive Inferential
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
3/40
Where are we up to
Step 3. If inferential, is the result about
Statistical Significance Clinical Significance
We will continue on with
clinical significance..
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
4/40
Minimum Important Difference
Minimum Important Difference (MID) is the
smallest worthwhile difference
(improvement) expected by a patient toproceed with a treatment
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
5/40
MID Example
Pain scale 0-10 (O is no pain, 10 is worst painimaginable)
You have heel pain and have marked this as 8
on pain scale Treatment options are
A) $10 comfy cushioned heel pad
B) $300 custom orthoses, daily strengthening andstretching
C) $5000 surgery, 6 weeks functional interruption,post-operative discomfort and some risk of harm
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
6/40
MID Example
What is the smallest improvement (points on apain scale) you would expect to make itworthwhile to have each treatment?
Treatment MID
A
B
C
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
7/40
What do you do with it?
You compare the treatment effect size with theminimal important difference. E.g. The amount ofimprovement in pain from having surgery for
spinal pathology
MID for having surgery= 80% improvement
95% Confidence Interval = Between 50% and 60%This means the best evidence suggests average improvement
in pain following this procedure is between 50% and 60%. The
patient does not proceed because their minimum
improvement wanted to undergo surgery is at least 80%.
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
8/40
Minimal important difference
At this stage you need to know!
What MID means
How it is used on a tree plot to determineclinical significance
To set an MID for a patient or populationevidence question requires further clinicalreasoning skills. You are not expected tosuggest MID values until third year.
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
9/40
So what is a tree plot?
The box represents the point estimate andthe line represents the confidence interval
Confidence Interval = 6-10kg
Mean = 8kg
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
10/40
treatment is not
worthwhile
Smallest clinicallyworthwhile effect or MID
treatment is
worthwhile
no effect
2 4 6 8 10 12 14 16-10 -8 -6 -4 -2 0
Tree Plot
Line of no effect
(no difference between groups)
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
11/40
treatment is not
worthwhile
Smallest clinically worthwhile
effect or MID: 5 kg weight loss
treatment is
worthwhile
no effect
Treatment effect = 8 kg weight loss following resistancetraining (95% CI 6 to 10) in favour of treatment group
2 4 6 8 10 12 14 16-10 -8 -6 -4 -2 0
Tree Plot
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
12/40
Clinical Significance
Statistical significance is that p-value < 0.05
Clinical significance is when the treatmenteffect (confidence interval) is equal or
more than the MID.
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
13/40
treatment is not
worthwhile
Smallest clinically worthwhile
effect or MID: 5 kg weight loss
treatment is
worthwhile
no effect
2 4 6 8 10 12 14 16-10 -8 -6 -4 -2 0
Tree Plot
Is this result clinically significant?
Why?
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
14/40
If you put it all together
Difference in means =
The result that matters is the difference in weightloss between the groups (not within a group)
8kg (95%CI 6kg-10kg) p
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
15/40
But this is for continuousdata .
what do we do if the data is
categorical?
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
16/40
Revision
Continuous data involves an actualmeasurement e.g. average student mark onassignment = 62 marks
Categorical data involves a proportion of eventse.g. how many fails compared to passes = 5 failgrades compared to 10 pass grades
Both examples could provide information aboutstudent performance in an assignment
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
17/40
Categorical results
Categorical data are presented asproportions experiencing an event oroutcome of interest.
Risk ratio is a comparison of risks
Also called relative risk
Risk does not mean a harmful thing
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
18/40
TITANIC
SURVIVAL
ALIVE DEAD
FEMALE 308 154
MALE 142 709
Risk of death on Titanic
TOTAL
462
Females
851Males
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
19/40
Risk of death on Titanic
The risk ratio can compare the probability ofdeath according to sex
The probability of a woman dying was 154
(women who died) out of 462 (total women)(154/462=0.33) or 33%.
The probability of a man dying was 709 (menwho died) out of 851 (total men)(709/851=0.8331) or 83%.
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
20/40
Risk of death on Titanic
The risk ratio can compare the probability ofdeath according to sex
83% (male death proportion) compared to 33%
(female death proportion)= (0.8331/0.3333).
= There is a 2.5 greater probability of death
for males than for females.
Risk ratio = 2.5
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
21/40
Categorical results
Minimal important differences (MIDs) are alsoset in exactly the same way. How much morelikely do you want an event to occur to make anintervention worthwhile?
Confidence intervals are also the same e.g. RR =3 (95%CI 2.2- 3.8)
In the actual research the outcome was three timesmore likely, but in the population we can estimate with95% certainty that the outcome will be between 2.2and 3.8 times more likely
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
22/40
Example
The box represents the point estimate andthe line represents the confidence interval
Confidence Interval = 2.2 3.8
Risk Ratio = 3
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
23/40
No effect
One thing is different
Continuous: If there is no difference between
2 actual measures, the difference = 0 E.g. If both groups lose 5kg then the difference
between their improvements = 0
Categorical: If there is no difference betweenrisks, the difference value is 1
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
24/40
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
25/40
treatment is not
worthwhile
Smallest clinically worthwhileeffect or MID: 2.5 times greater risk
treatment is
worthwhile
no effect
Risk Ratio = 3 (95% CI 2.2 to 3.8) in favour of
intervention group
Minimal Important Difference = 2.5
1 1.5 2 2.5 3 3.5 4 4.5 50.001 0.01 0.1 0.5
Tree plot for categorical data
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
26/40
Clinical significance
A result is clinically significant if
The point estimate is larger than the MID
Most of the confidence interval larger than MID
A narrow confidence interval. If it is narrow, it is
called precise.
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
27/40
treatment is not
worthwhile
Smallest clinically worthwhile
effect or MID: 5 kg weight loss
treatment is
worthwhile
no effect
Mean treatment effect= 2 kg in weight loss followingresistance training(95% CI -4 to 0) in favour of control group
group
2 4 6 8 10 12 14 16-10 -8 -6 -4 -2 0
Study A
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
28/40
treatment is not
worthwhile
Smallest clinically worthwhile
effect or MID: 5 kg weight loss
treatment is
worthwhile
no effect
Mean treatment effect= 8 kg in weight loss followingresistance training(95% CI -2 to 18) in favour of treatment group
2 4 6 8 10 12 14 16-10 -8 -6 -4 -2 0
Study B
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
29/40
Activity
From the Study A and B on the previous two slides
1) Which study found the treatment effect wasclinically significant ? (the intervention works)
2) Which study found the treatment effect was not
clinically significant? (the control group did better-the intervention did not work)
3) Which study has the most precise results?
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
30/40
treatment is not
worthwhile
smallest clinically
worthwhile effect or
MIDvery harmful
treatment
very effective
treatment
treatment is
worthwhile
no effect
Study A
Study B
Study C
Tree plot of effect size
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
31/40
Systematic reviews
The results of systematic reviews arecalled forest plots
They plot multiple tree plots.
One tree plot = one clinical trial
Results will be either mean differences orrisk/odds ratios depending on type of data
A meta-analysis combines or pools thedata of the trials
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
32/40
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
33/40
Music for pain relief: Categorical
Categorical Forest Plot: Risk Ratio
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
34/40
Music for pain relief: Continuous
Continuous Forest Plot: Mean Difference
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
35/40
Tips on Reading Results
Dont skip the results section!!
Decide on what is your primary research question
Decide on your minimal important difference Find the point estimate and confidence intervals
for that question
Make sure they are for group comparisons
And ignore all the less relevant numbers!
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
36/40
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
37/40
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
38/40
Other studies
These lectures on results have onlydiscussed evidence on interventionswhatabout studies about diagnosis, prognosis,
and aetiology? They might use risk ratios
You will learn some other common results
for these types of studies next year.
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
39/40
Still to come
Final lecture
Lecture 14 Revision session on challenging topics
Exam advice
Bring any questions before your exam
8/10/2019 400863 Lecture 12 Understanding Results 3 2014
40/40
Thank you!