Top Banner

Click here to load reader

Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community

Mar 26, 2015

ReportDownload

Documents

  • Slide 1

Appraisal of an RCT using a critical appraisal checklist Spink MJ, et al. Effectiveness of a multifaceted podiatry intervention to prevent falls in community dwelling older people with disabling foot pain: randomised controlled trial. BMJ 2011;342:d3411. Slide 2 Is the included population explicitly described? Is the setting explicitly described? The RCT should clearly explain how it recruited people, and the inclusion/exclusion criteria it used, so that you can decide if this is representative of the target population and generalisable to your practice Examples of fair recruitment are: random recruitment/recruitment of consecutive patients in a treatment centre. However, an intervention study will need to have informed consent from participants, and so it may use volunteer recruitment/recruitment from advertising to ensure that it recruits enough people Also remember: inclusion and exclusion criteria should be relevant for the study method and question i.e., a study shouldnt exclude people with certain weight or height, if this is not relevant to the study outcomes or question posed Slide 3 In this study: Is the included population explicitly described? Look in Methods section for details of recruited participants. (The results section should detail the actual people randomised from this recruited population group.) Here, participants were recruited from the treatment centre database and by advertisements. Inclusion and exclusion criteria are clearly defined and sensible for study method and question Slide 4 In this study: Is the setting explicitly described? Look in Methods section in this RCT for details of the setting. Sometimes this information may be in the results section Slide 5 Is the intervention and control explicitly described? The study should describe the intervention and control in sufficient detail, such that these can be replicated Watch out for statements such as usual care the RCT should define what this involves Slide 6 In this study: Is the intervention and control explicitly described? Look in Methods section for details of the intervention and control etc Slide 7 Are the outcomes explicitly described? The study may report on several different outcomes A primary outcome is the prespecified outcome that is considered to be the most important outcome of the study. It is best to have only one primary outcome in a study Secondary outcomes are additional outcomes of interest. However, the study is often not powered to detect differences between groups in these outcomes All outcomes should be clearly described in sufficient detail, including the timing of measurement, so that they could be replicated Slide 8 In this study: Are the outcomes explicitly described? Yes, look in Methods section. This RCT also refers to another protocol paper with further information, which you should also read Slide 9 Are the outcome measurements valid? (objective, using standard/validated measures) Outcomes can be continuous (vary on a continuum or scale, e.g., weight, pain score), or dichotomous (yes/no outcomes e.g., death, heart attack) Clinically relevant outcomes include outcomes that are important to the patient, e.g., survival, symptom improvement. Ideally an RCT should report on clinically relevant outcomes Proxy measures are sometimes used because these may be easier/cheaper to measure, or can be assessed in a shorter time frame, e.g., intraocular pressure in glaucoma trials, or HbA1c in diabetes trials. However, proxy measures do not always correlate with clinical outcomes Also watch out for: Reporting significant outcomes only an RCT should report on all its prespecified outcomes whether they are significant or not Reporting continuous outcomes on unvalidated scales. It can be difficult to compare results of these with other studies Slide 10 In this study: Are the outcome measurements valid? Look again at the outcomes section particularly the primary outcomes There may be further information in the introduction or discussion sections if you are not familiar with the topic Slide 11 Is the randomisation method described? Is the study adequately randomised? If a study is successfully randomised, then it should be impossible to predict in advance which group each person will be assigned to. The ideal is that everyone has a known and equal chance of being in either group The following allocation methods are not true randomisation methods, as it may be possible to guess which group each person will be in in advance, so watch out for these! Date of birth, day of the week, medical record number, month of the year, or the order in which they are included in the study (alternate) Slide 12 In this study: Is the randomisation method described? Is the study adequately randomised? Look in Methods for details of randomisation used Slide 13 Is the level of blinding reported? Could the level of blinding affect the interpretation of the results? (observer bias, bias in analysis, use of objective or subjective outcome measures, etc) Double-blinding i.e., both patients and investigators are unaware of treatment allocation is ideal, and minimises measurement bias. However, sometimes this may not be possible, for example, where the intervention involves a physical treatment or surgery or where adverse effects of a drug clearly compromise blinding. Single-blinding (where either one of the patients or the assessors are unaware of treatment allocation) may, however, be possible and acceptable in these circumstances, depending on the study design Also remember if the outcome is highly objective (e.g., death) then the requirement for blinding may be less critical, because measurement bias should not really affect the result. However, if the outcome is subjective (e.g., self-rated symptoms or function) then blinding of the assessor may be crucial to avoid the influence of bias on the result Slide 14 Generally check the Methods section for information on blinding applied; however, sometimes further clarification may be found in other parts of the review e.g., this RCT gives further clarification in the discussion section This RCT was single-blinded (assessor blinding). It was not possible to blind the patients because of the physical nature of the intervention In this study: Is the level of blinding reported? Slide 15 In this study: Could the level of blinding affect the interpretation of the results? To assess whether the level of blinding affects the interpretation of the results, look again at the outcomes section, and whether you decided if the outcomes were objective or subjective In this study, the primary outcome of proportion of fallers, multiple fallers, and falling rate are relatively objective measures and so single blinding applied should not affect the interpretation of these results. Secondary outcome measures are more subjective, however, and so level of blinding applied, particularly no patient blinding, may suffer from bias Slide 16 Are both groups treated in exactly the same way apart from the intervention? Bear in mind that this includes all analyses, tests, and contact with the study personnel, as well as any other allowed treatments in the study This may not always be possible/ethical depending on the nature of the interventions; however, you should decide whether any differences in treatment during the study could affect the results Slide 17 In this study: Are both groups treated in exactly the same way apart from the intervention? Look again at the Methods (description of interventions) section. You should also look at the results section to check that both groups had similar assessments and follow-up Slide 18 Are treatment and control groups similar at baseline? Are any baseline differences explained/accounted for at analysis? If the randomisation process worked, then the groups should be similar at baseline (remember that the goal of randomisation is that everyone should have a similar and equal chance of being in either group) The study should specify whether any differences between groups are statistically significant, because any significant differences between groups at the beginning of the study could potentially affect the results Slide 19 In this study: Are treatment and control groups similar at baseline? Are any baseline differences explained/accounted for at analysis? Yes. Look at the results section. RCTs often contain a table describing baseline characteristics, as in this case Slide 20 Are all randomised people accounted for at the end of the RCT? Does the RCT present an intention-to-treat analysis? Are final results based on more than 80% of people randomised? Most trials will suffer from some loss to follow-up for various reasons (including patient withdrawal due to safety/efficacy/consent, lost contact, etc). However, to avoid bias and maintain the validity of the initial randomisation, loss to follow-up should be minimal generally less than 20% of the randomised population. However, it should be noted that if few patients have the outcome of interest, then even small losses can bias the results An intention-to-treat analysis is where the groups are analysed based on the initial treatment to which they were randomised (not by treatment eventually administered or whether they finished the trial!). This type of analysis is generally considered optimal for most RCTs (with the possible exception of non-inferiority or equivalence trials) Slide 21 In this study: Are all randomised people accounted for at the end of the RCT? Does the RCT present an intention-to-treat analysis? Are final results based on more than 80% of people randomised? Yes. Look at the Methods and Results sections (to see what actually happened). RCTs often contain a figure which shows the flow of people through the study, as in this case Slide 22 Are all clinically relevant outcomes reported? This requires your judgement look at what outcomes were reported, and how the