Top Banner
Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community dwelling older people with disabling foot pain: randomised controlled trial. BMJ 2011;342:d3411.
26

Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

Mar 26, 2015

Download

Documents

Paige Parsons
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

Appraisal of an RCT using a critical appraisal checklist

Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to

prevent falls in community dwelling older people with disabling foot pain: randomised

controlled trial. BMJ 2011;342:d3411.

Page 2: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

Is the included population explicitly described?Is the setting explicitly described?• The RCT should clearly explain how it recruited people, and the

inclusion/exclusion criteria it used, so that you can decide if this is representative of the target population and generalisable to your practice

• Examples of fair recruitment are: random recruitment/recruitment of consecutive patients in a treatment centre. However, an intervention study will need to have informed consent from participants, and so it may use volunteer recruitment/recruitment from advertising to ensure that it recruits enough people

• Also remember: inclusion and exclusion criteria should be relevant for the study method and question — i.e., a study shouldn’t exclude people with certain weight or height, if this is not relevant to the study outcomes or question posed

Page 3: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

In this study: Is the included population explicitly described?

• Look in Methods section for details of recruited participants. (The results section should detail the actual people randomised from this recruited population group.) Here, participants were recruited from the treatment centre database and by advertisements. Inclusion and exclusion criteria are clearly defined and sensible for study method and question

Page 4: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

In this study: Is the setting explicitly described?

• Look in Methods section in this RCT for details of the setting. Sometimes this information may be in the results section

Page 5: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

Is the intervention and control explicitly described?

• The study should describe the intervention and control in sufficient detail, such that these can be replicated

• Watch out for statements such as “usual care” — the RCT should define what this involves

Page 6: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

In this study:Is the intervention and control explicitly described?

• Look in Methods section for details of the intervention and control

etc…

Page 7: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

Are the outcomes explicitly described?

• The study may report on several different outcomes • A primary outcome is the prespecified outcome that is considered

to be the most important outcome of the study. It is best to have only one primary outcome in a study

• Secondary outcomes are additional outcomes of interest. However, the study is often not powered to detect differences between groups in these outcomes

• All outcomes should be clearly described in sufficient detail, including the timing of measurement, so that they could be replicated

Page 8: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

In this study:Are the outcomes explicitly described?

• Yes, look in Methods section. This RCT also refers to another protocol paper with further information, which you should also read

Page 9: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

Are the outcome measurements valid? (objective, using standard/validated measures)

• Outcomes can be continuous (vary on a continuum or scale, e.g., weight, pain score), or dichotomous (yes/no outcomes e.g., death, heart attack)

• Clinically relevant outcomes include outcomes that are important to the patient, e.g., survival, symptom improvement. Ideally an RCT should report on clinically relevant outcomes

• Proxy measures are sometimes used because these may be easier/cheaper to measure, or can be assessed in a shorter time frame, e.g., intraocular pressure in glaucoma trials, or HbA1c in diabetes trials. However, proxy measures do not always correlate with clinical outcomes

• Also watch out for:– Reporting significant outcomes only — an RCT should report on all its prespecified

outcomes whether they are significant or not– Reporting continuous outcomes on unvalidated scales. It can be difficult to

compare results of these with other studies

Page 10: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

In this study: Are the outcome measurements valid?

• Look again at the outcomes section — particularly the primary outcomes

• There may be further information in the introduction or discussion sections if you are not familiar with the topic

Page 11: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

Is the randomisation method described?Is the study adequately randomised?

• If a study is successfully randomised, then it should be impossible to predict in advance which group each person will be assigned to. The ideal is that everyone has a known and equal chance of being in either group

• The following allocation methods are not true randomisation methods, as it may be possible to “guess” which group each person will be in in advance, so watch out for these! – Date of birth, day of the week, medical record number, month of the

year, or the order in which they are included in the study (alternate)

Page 12: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

In this study:Is the randomisation method described?Is the study adequately randomised?

• Look in Methods for details of randomisation used

Page 13: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

Is the level of blinding reported? Could the level of blinding affect the interpretation of the results? (observer bias, bias in analysis, use of objective or subjective outcome measures, etc)

• Double-blinding — i.e., both patients and investigators are unaware of treatment allocation is ideal, and minimises measurement bias. However, sometimes this may not be possible, for example, where the intervention involves a physical treatment or surgery or where adverse effects of a drug clearly compromise blinding. Single-blinding (where either one of the patients or the assessors are unaware of treatment allocation) may, however, be possible and acceptable in these circumstances, depending on the study design

• Also remember — if the outcome is highly objective (e.g., death) then the requirement for blinding may be less critical, because measurement bias should not really affect the result. However, if the outcome is subjective (e.g., self-rated symptoms or function) then blinding of the assessor may be crucial to avoid the influence of bias on the result

Page 14: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

• Generally check the Methods section for information on blinding applied; however, sometimes further clarification may be found in other parts of the review — e.g., this RCT gives further clarification in the discussion section

• This RCT was single-blinded (assessor blinding). It was not possible to blind the patients because of the physical nature of the intervention

In this study:Is the level of blinding reported?

… …

Page 15: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

In this study:Could the level of blinding affect the interpretation of the results? • To assess whether the level of blinding affects the interpretation of the

results, look again at the outcomes section, and whether you decided if the outcomes were objective or subjective

• In this study, the primary outcome of proportion of fallers, multiple fallers, and falling rate are relatively objective measures and so single blinding applied should not affect the interpretation of these results. Secondary outcome measures are more subjective, however, and so level of blinding applied, particularly no patient blinding, may suffer from bias

Page 16: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

Are both groups treated in exactly the same way apart from the intervention?

• Bear in mind that this includes all analyses, tests, and contact with the study personnel, as well as any other allowed treatments in the study

• This may not always be possible/ethical depending on the nature of the interventions; however, you should decide whether any differences in treatment during the study could affect the results

Page 17: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

In this study:Are both groups treated in exactly the same way apart from the intervention?

• Look again at the Methods (description of interventions) section. You should also look at the results section to check that both groups had similar assessments and follow-up

Page 18: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

Are treatment and control groups similar at baseline?Are any baseline differences explained/accounted for at analysis?

• If the randomisation process worked, then the groups should be similar at baseline (remember that the goal of randomisation is that everyone should have a similar and equal chance of being in either group)

• The study should specify whether any differences between groups are statistically significant, because any significant differences between groups at the beginning of the study could potentially affect the results

Page 19: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

In this study: Are treatment and control groups similar at baseline?Are any baseline differences explained/accounted for at analysis?

• Yes. Look at the results section. RCTs often contain a table describing baseline characteristics, as in this case

Page 20: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

Are all randomised people accounted for at the end of the RCT? Does the RCT present an intention-to-treat analysis?Are final results based on more than 80% of people randomised?

• Most trials will suffer from some loss to follow-up for various reasons (including patient withdrawal due to safety/efficacy/consent, lost contact, etc). However, to avoid bias and maintain the validity of the initial randomisation, loss to follow-up should be minimal — generally less than 20% of the randomised population. However, it should be noted that if few patients have the outcome of interest, then even small losses can bias the results

• An intention-to-treat analysis is where the groups are analysed based on the initial treatment to which they were randomised (not by treatment eventually administered or whether they finished the trial!). This type of analysis is generally considered optimal for most RCTs (with the possible exception of non-inferiority or equivalence trials)

Page 21: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

In this study: Are all randomised people accounted for at the end of the RCT? Does the RCT present an intention-to-treat analysis?Are final results based on more than 80% of people randomised?

• Yes. Look at the Methods and Results sections (to see what actually happened). RCTs often contain a figure which shows the flow of people through the study, as in this case

Page 22: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

Are all clinically relevant outcomes reported?

• This requires your judgement — look at what outcomes were reported, and how these answer the question posed by the study

Page 23: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

Does the RCT report a direct statistical analysis between groups?Does the RCT report appropriate test statistics?• The RCT should compare similar results between groups directly. Indirect comparisons, for

example, comparison with baseline within a group, are difficult to interpret• The appropriate type of test statistic presented will depend on what type of outcome was

measured. For dichotomous outcomes, the results can be expressed in many ways, including: Relative Risk (RR) = risk of the outcome in the treatment group / risk of the outcome in the

control group Absolute Risk Reduction (ARR) = risk of the outcome in the control group – risk of the

outcome in the treatment groupRelative Risk Reduction (RRR) = absolute risk reduction / risk of the outcome in the control group.Number Needed to Treat (NNT) = inverse of the ARR and is calculated as 1 / ARR.

• The study should present a confidence interval or P value so that you can judge the statistical significance of the result. However, bear in mind that “statistically significant” and “clinically significant” are different!

Page 24: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

In this study: Does the RCT report a direct statistical analysis between groups?Does the RCT report appropriate test statistics?

• Yes, generally look first at the Methods section and then at the Results section

Page 25: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

In this study: Does the RCT report on the clinical relevance/importance of the results?

• Yes, generally find this information in the discussion section of the RCT

Page 26: Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community.

In this study: Are the setting and intervention generalisable to routine clinical practice/care?

• Again, this requires your judgement — the study may make suggestions but you will know best what is generalisable to your area of practice