The Evidence Base of Neuropsychological Rehabilitation in Acquired Brain Impairment (ABI): How Good is the Research

119

Address for correspondence: Michael Perdices, PhD, Department of Neurology, Royal North Shore Hospital, St Leonards NSW 2065, Australia. E-mail: [email protected]

The Evidence Base of NeuropsychologicalRehabilitation in Acquired BrainImpairment (ABI): How Good is the Research?

Michael PerdicesDepartment of Neurology, Royal North Shore Hospital, Sydney, Australia

Regina Schultz and Robyn TateRehabilitation Studies Unit, Northern Clinical School, Faculty of Medicine, University of Sydneyand Royal Rehabilitation Centre Sydney, Australia

Skye McDonaldSchool of Psychology, University of New South Wales, Australia

Leanne TogherSchool of Communication Sciences and Disorders, University of Sydney, Australia

Sharon Savage, Kiri Winders and Kate SmithRehabilitation Studies Unit, Northern Clinical School, Faculty of Medicine, University of Sydneyand Royal Rehabilitation Centre Sydney, Australia

In the context of evidence-based clinical practice (EBCP), the reliability of empiricaldata is largely determined by the methodological quality of research design.

PsycBITE™ (Psychological Database of Brain Impairment Treatment Efficacy) is aweb-based database listing all published, empirical reports on the effectiveness ofnonpharmacological interventions for the psychological consequences of acquiredbrain impairment (ABI). The aim of this study was to survey the listings of PsycBITE™and examine the methodological quality of the reports it contains. Reports listed inPsycBITE™ include systematic reviews (SRs), randomised controlled trials (RCTs),non-RCTs, case series (CSs) and single-subject designs (SSDs). They are indexedaccording to research design, neurological group, patient age group, target area andintervention type. The PEDro Scale is used to rate the methodological quality of RCTs,nonRCTs and CSs, with maximum obtainable methodological quality rating (MQR) of10/10, 8/10 and 2/10 respectively. A search identified 1298 reports indexed inPsycBITE™. The largest proportion was SSDs (39%), followed by CSs (22%), RCTs(21%), non-RCTs (11%) and SRs (7%). The majority of reports was concerned withstroke (41%), traumatic brain injury (29%) and Alzheimer’s and related dementias(22%). The most frequently investigated deficits were communication/language/speech disorders (24%); independent/self-care activities (19%); behaviour problems(17%); memory impairments (17%); anxiety, depression, stress, adjustment (15%).Approximately half of the RCTs, non-RCTs and CSs were rated for methodologicalquality. Mean MQR scores for RCTs, non-RCTs and CSs were 4.49, 2.85 and 1.15respectively. While some PEDro criteria were met by a high proportion of RCTs andnon-RCTs (≥ 70%), other criteria were only met by a small proportion of reports (aslow as 1.6%). There was no significant difference in MQR scores between RCTs

BRAIN IMPAIRMENT VOLUME 7 NUMBER 2 SEPTEMBER 2006 pp. 119–132

Guyatt, Rennie and Hayward (2000) propose thatthere are four general domains of clinical ques-tions in medical practice, namely those relating totherapy, harm, diagnosis and prognosis. Arguably,all four types of question are pertinent to someextent in the context of rehabilitation practice forthe treatment of the neuropsychological conse-quences of acquired brain impairment (ABI).Questions relating to therapy are, however, partic-ularly relevant. In terms of therapy issues, theclinician’s primary concern is to determine theefficacy of a given intervention in ameliorating apatient’s cognitive, functional, behavioural or psy-chological deficits. There are three main sourcesthat clinicians can draw upon to guide and informtheir decision about which therapy should be usedto treat a given deficit in a specific patient withABI. First, they can draw upon their own personalexperience about what has successfully worked inthe past to treat the deficit in question in the typeof patient under consideration. Second, they canrely on anecdotal evidence, that is, evidencereflecting other colleagues’ opinions and experi-ence and/or what might be considered ‘acceptedknowledge’ about those therapies that are effec-tive. Third, they can examine the evidence pro-

vided by relevant empirical studies published inthe literature.

Determining the efficacy of a therapeuticintervention is tantamount to evaluating the credi-bility of the data (or evidence) that demonstratesthe putative effect of that therapy. If conclusionsdrawn about the efficacy of a given therapy are tobe reliable, they must be founded on unbiaseddata. It is extremely difficult to assess objectivelythe reliability of conclusions about treatment effi-cacy that are drawn either from personal experi-ence or from anecdotal evidence. Evidence fromthese two sources is almost invariably based onunsystematic observations, tends to be derivedfrom small sample sizes and, more importantly, islikely to be influenced by sources of bias whichmight, at times, be well nigh impossible to eitheridentify or evaluate.

By contrast, information derived from pub-lished empirical studies about treatment efficacyis more amenable to objective evaluation. Thereliability (i.e., credibility) of evidence supportingthe efficacy of a therapeutic intervention islargely, though not exclusively, determined by themethodological quality of the empirical study. Putsimply, well-designed, well-conducted studieswill produce reliable data; poorly designed and

120

MICHAEL PERDICES ET AL.

TABLE 1Hierarchy of Clinical Evidence†

Level of evidence Study design

I Evidence obtained from a systematic review of all relevant randomised controlled trialsII Evidence obtained from at least one properly designed randomised controlled trial.III Evidence obtained from well-designed pseudo randomised controlled trials (alternate allocation

or some other method).Evidence obtained from nonrandomised controlled trials: such as comparative studies (includingsystematic reviews of such studies) with concurrent controls and allocation not randomised,cohort studies, case-control studies, or interrupted time series with a control group.Evidence obtained from comparative studies with historical control, two or more single armstudies, or interrupted time series without a parallel control group.

IV Evidence obtained from case series, either posttest or pretest/posttest

Note: † adapted from National Health & Medical Research Council (1999).

focusing on different neurological groups or target areas. Furthermore, there was nodiscernible improvement in MQR score for RCTs published over the last threedecades. The methodological quality of studies investigating the efficacy of rehabili-tation interventions in ABI has been consistently modest over several decades. Thisis largely attributable to poor adherence to fundamental tenets of research design,and requires urgent remediation. RCTs (and to a lesser extent, non-RCTs) areresearch methodologies which can potentially yield a high level of evidence, but onlyif they are adequately designed. PsycBITE™ has the facility to raise awareness ofthese issues and be instrumental in promoting EBCP in the field of ABI.

121

THE EVIDENCE BASE OF NEUROPSYCHOLOGICAL REHABILITATION IN ABI

conducted studies will produce unreliable data.Inferences about treatment efficacy made from theformer will be credible, inferences made from thelatter will not. In effect, there is a an inherent hier-archy in the quality of empirical data pertaining totreatment efficacy, ranging from the relativelypoor, unreliable evidence drawn from sourcessuch as anecdotal evidence, to that provided bywell-conducted, methodologically robust studies.This hierarchy of evidence has been explicitly for-malized (Table 1) and is an integral aspect of evi-dence-based clinical practice (EBCP).

In essence, EBCP is about solving clinicalproblems, and has been defined as ‘the integra-tion of best research with clinical expertise andpatient values … which optimizes clinical out-comes and quality of life’ (Sackett, Strauss,Richardson, Rosenberg, & Haynes, 2000).Central to this concept is the identification andcritical appraisal of evidence from the literature.The extant hierarchy of empirical evidence man-dates that clinicians should seek the highestavailable evidence from the hierarchy to guidetheir choice of therapeutic intervention.

There are four basic steps that the clinicianmust follow in order to implement an evidence-based approach (Haynes, Sackett, Gray, Cook, &Guyatt, 1996). First, the clinical problem has to beprecisely defined. In the context of providing ther-apy for neuropsychological deficits in patients withABI, this would essentially entail a precise andobjective delineation (i.e., assessment) of the deficitin question, as well as its functional impact, inorder to operationalise rehabilitation goals. Second,the information required to resolve the problemmust be ascertained, which means that the availableevidence pertaining to treatment of the specifieddeficit must be examined. Third, this evidence mustbe appraised for its reliability (i.e., credibility) andrelevance. Fourth, clinicians must determine how(and whether) the effects of a given treatmentreported in the literature can be applied to an indi-vidual patient. A central consideration in thisrespect is whether the clinical and prognostic char-acteristics of the patient in question are similar tothose of participants in the study reporting theeffects of therapy. Determining how the advantages(and any possible disadvantages) of the therapymay impinge on the patient’s values, and involvingthe patient in decisions about management options,are also important considerations in this final eval-uation (Guyatt, Rennie, & Hayward, 2000).

EBCP has increasingly gained currency in thelast two and a half decades and is now the univer-sally accepted standard. Nonetheless, time limita-tions have been identified as, and remain, the

biggest challenge to its implementation (Guyatt,Rennie, & Hayward, 2000). Steps 1 (problem def-inition) and 4 (applicability of therapy) rely on theexperience and expertise of the clinician andrequire both time and infrastructure resources.Step 2 (gathering information) is very costly interms of time resources, a generally scarce com-modity among clinicians. What is essentiallyrequired is a comprehensive, and ideally exhaus-tive, search of the relevant literature. This can beextremely time-consuming given the immensechanges that have occurred in rehabilitation prac-tice in ABI over the past three decades. In the mid-1970s there were virtually no resources to guideclinicians in the selection and implementation oftreatment strategies for ABI. Now clinicians areoverwhelmed by the sheer volume of informationavailable. Not only has the number of publishedstudies, including randomised controlled trials(RCTs) and non-RCTs (see Appendix A for defi-nition), examining the efficacy of rehabilitationtherapies in ABI proliferated dramatically inrecent years, but numerous systematic reviews(SRs) have also begun to appear in the literaturefor neuropsychological interventions since 1992(see Results).

Of even more concern is Step 3 (appraising theevidence). Many healthcare practitioners have diffi-culty understanding statistics, do not know how toadequately interpret the quality of research data orhow to evaluate conclusions (Young, Glasziou, &Ward, 2002; Metcalfe et al., 2001). This representsa further bottleneck in the implementation ofEBCP. Until recently, the only option available toclinicians was to search for relevant informationthrough a number of generic databases such asPsycINFO and Medline and evaluate the reliabilityof evidence as best they could. The fact that nosingle repository of the relevant literature was avail-able constituted another significant impediment forthe implementation of EBCP (Metcalfe et al.,2001). There is a need, therefore, for tools that notonly make research findings more accessible toclinicians working with ABI, but also improve theirability to evaluate and apply those findings.

PsycBITE� (Psychological Database of BrainImpairment Treatment Efficacy; Tate, Perdices, etal., 2004) is a recent, web-based resource(www.psycbite.com) launched in June 2004 anddeveloped to meet these needs. PsycBITE™ is adatabase of all published, empirical reports on theeffectiveness of non-pharmacological interventionsfor the psychological consequences of ABI. Theseinterventions encompass a range of disciplines andresearch designs. The aim of this article is to presenta survey of reports indexed on PsycBITE� .

122

MICHAEL PERDICES ET AL.

MethodThe procedure used to compile PsycBITE� hasbeen described in detail elsewhere and is sum-marised in Figure 1 (Tate et al., 2004). Briefly,research reports are extracted from seven existinggeneric databases, using 85 search terms. Thesearch terms are deliberately tailored to providemaximum sensitivity. This is done in order toensure that all pertinent reports are extracted fromthe literature, at the expense of generating a largenumber of false positives (low specificity). Reportslisted in PsycBITE� include SRs, RCTs, non-RCTs, case series (CSs) and single-subject designs(SSDs; see Appendix A for definitions).References generated by the initial search are thenscreened for eligibility. For inclusion in thedatabase, the report must meet the following fivecriteria: (1) it must be published as a full-lengthpaper in a peer-reviewed scientific journal; (2) thepopulation treated is human, with brain impairmentof acquired aetiology that involves structural/mor-phological changes; (3) the population is older than5 years of age; (4) treatment comprises at least oneintervention that is psychologically-based and/ortargets at least one psychological consequence ofABI; and (5) the report provides empirical dataregarding treatment efficacy.

Reports meeting eligibility criteria are thencategorised on five main indices according toresearch design, neurological group, target area,

intervention type and age group using 73 termschosen to enable end-users of PsycBITE� toperform customised searches (Table 2). For themain indices of neurological group, target area,and intervention type, reports can be indexed on amaximum of four categories, if necessary. Forexample, a report that is mainly concerned withremediation of memory problems, but alsofocuses on living skills and various other cogni-tive problems would be categorised under‘memory impairments’ as the primary index fortarget area, but would also be categorised under‘interpersonal and social skills’ and ‘cognitiveproblems’. That is, it is possible for a report tocontribute to the frequency count on more thanone category on the neurological group, targetarea, and intervention type main indices.

Once indexed, reports are listed on thePsycBITE™ website and individually examinedand rated for methodological quality by trainedclinicians/clinical researchers. Methodologicalquality ratings (MQRs) are allocated using thePEDro scale (Moseley, Maher, Herbert, &Sherrington, 1999; Maher, Sherrington, Herbert,Moseley, & Elkins, 2003). The PEDro scale con-sists of 11 items, and incorporates the 9-itemDelphi list (Verhagen et al., 1998), the 3-itemJadad scale (Jadad et al., 1996), and also covers anumber of the items from the CONSORT state-ment (Moher, Schulz, & Altman, 2001). The firstitem on the PEDro scale relates to the externalvalidity, while the following 10 items assessinternal validity and whether the trial containssufficient statistical information to make it inter-pretable (see Appendix B).

At present, only RCTs, non-RCTs and CSs,but not SRs listed on PsycBITE� are rated formethodological quality, and a specific scale forrating SSDs is in the process of development(Tate, McDonald, et al., 2004). RCTs attract amaximum MQR of 10/10 on the PEDro scale, butbecause of their inherent methodological limita-tions, non-RCTs only attract a maximum MQR of8/10 (no points can be obtained for either Criteria2 or 3 — see Appendix B) and CSs a maximumMQR of 2/10 (points can only be obtained forCriteria 8 and 11 — see Appendix B).

MQRs are listed on the website providing aquantitative measure of quality for each report.End-users are thus able to quickly identify trialsthat are likely to contain valid, clinically usefulinformation. Searches on PsycBITE� are per-formed using simple interactive menus that allowend users to closely tailor the search to their spe-cific needs (see McDonald et al., (in press) for a

3) Indexing of reports for target area,intervention type, neurological group, age

group, and study design

1) Identification of all potentially relevantreports from seven generic databases:

AMED, CINAHL, Cochrane Library,EMBASE, ERIC, Medline, PsycINFO

2) Screening for eligibility

4) Rating for methodological quality

5) Entry of citation details, abstract, indexingterms and ratings on PsycBITE

TM database

FIGURE 1Compilation procedure for PsycBITETM database.

TABLE 2Indexing Terms Used in PsycBITE�

Research design

Systematic reviewRandomisedcontrolled trialNonrandomisedcontrolled trialCase seriesSingle-subject design

Neurological group

Alzheimer’s and relateddementiasAnorexia nervosa/bulimia nervosa (withABI)Brain infectionsBrain tumours/neoplasmsDegenerative disorders/diseasesDrug and alcohol-related brain damage/disordersEpilepsy/seizures/convulsionsHypoxia/anoxiaMultiple sclerosis/MSNeurotoxicityStroke/CVA (cerebro-vascular accidents)Traumatic brain injury(TBI)/head injuryNonspecified braininjuryOther

Target area

Anxiety, depression,stress and adjustmentAttention problemsBehaviour problemsCognitive deficitsComa/altered statesof consciousnessCommunity re-entry/instrumental ADLsCommunication/language/speech disordersDelusions/delirium/psychotic disordersEmotional processingdisorderExecutive functioningdeficitsFatigue and low worktoleranceIndependent/self-careADLInsight and awarenessInterpersonal psycho-social and social skillsMemory impairmentsMovement and motorproblemsMultiple problemsPainReading, writing andarithmeticSensory/perceptual/visuospatialimpairmentsQuality of lifeOther

Intervention

Anger/anxiety/stress management therapyAssistive devicesAttention remediationBehaviour therapy/behaviourmodificationCarer training/educationCognitive/neuropsychologicalrehabilitationCognitive–behaviour therapy/cognitive therapyCommunication/language/speech skills remediationComplementary/alternativetherapiesCommunity re-entryComputer/technological interventionsCounselling/psychotherapy/family supportEducation/psychoeducationExecutive skills remediationFamily, narrative, solution-focused therapyFatigue managementGoal settingIndependent/self-careskills/ADL techniquesMemory remediationMovement/motor skills remediationMusic therapyMultidisciplinary rehabilitationPsychosocialrehabilitation/social skillsReading writing and arithmetic remediationRecreational/leisure/fitnessrehabilitationSensory/perceptual/visuospatial skillsSexuality counsellingVocational rehabilitationOther

Patient age group

ChildrenAdolescentsAdults

123

THE EVIDENCE BASE OF NEUROPSYCHOLOGICAL REHABILITATION IN ABI

detailed demonstration on how searches onPsycBITE� are performed).

This article presents a detailed survey of reportsindexed on PsycBITE� based on records as atMarch 1, 2006. Information was extracted to ascer-tain: (a) the number and research design of reportscurrently listed in the database, (b) the neurological

group studied, (c) the target area addressed in eachreport, and (d) the methodological quality ofreports as reflected in MQR scores. More specif-ically, MQR scores for RCTs were also examinedfor selected neurological groups and problemareas. For the purpose of this article, the fre-quency of a given category in either of these two

124

MICHAEL PERDICES ET AL.

main indices was the combined frequency withwhich that category occurred on the four possibleentries allowed for each main index.

ResultsResearch DesignBy March 1, 2006, searches of generic databaseshad yielded 187,232 reports. Of these, 168,638had been culled for further processing, with18,594 yet to be processed. Of the culled reports,2152 (1.3%) were eligible and met the five selec-tion criteria described in the Method section. Atotal of 1298 (60%) reports have been indexed andlisted in PsycBITE� . The majority of thesereports were, by far, SSDs (39%; n = 509). Theproportion of RCTs (21%; n = 276) and CSs(22%; n = 280) was comparable, while SRs (7%;n = 87) and non-RCTs (11%; n = 146) were rela-tively under-represented. Ninety-five per cent ofall reports in each research design category con-cerned adults. Approximately 46% of RCTs, 47%of non-RCTs and 42% of CSs had also been ratedfor methodological quality.

There was a steady increase in the publicationrate of reports over the last 50 years (Figure 2). In1953, Gottschalk published the earliest empiricaltreatment study of ABI we have identified in the lit-erature, a SSD investigating the effects of psy-chotherapy on children with epilepsy. Published

reports of SSDs increased progressively over thefollowing decades, with 176 studies publishedbetween 2001 and 2006. Similar patterns are evi-dent for other research designs (Figure 2). Forinstance, the earliest RCT identified in the literaturesearches was published in 1977 by Weinberg et al.,and a further three RCTs were published by the endof that decade (David, Enderby, & Bainton, 1979;Gasparinni & Satz, 1979; Di Carlo, 1980). By con-trast, 136 RCTs were published within in the last 5years. SRs also began to appear in the literature in1992 (de Pedro-Cuesta, Widen-Holmqvist, &Bach-y-Rita, 1992; University of York, 1992), andhave become increasingly common since then.Similarly, the number of target areas targeted by theresearch has increased dramatically over time.Gottschalk (1953) treated memory problems inchildren with epilepsy, and this was the only targetarea targeted by the research in the 1951–1960decade. Research in the following decade targetedtwo target areas, communication/language/speechdisorders (Goodkin, 1966; Byers-Brown & Ives,1969) and behaviour problems (Brookshire, 1970).In the 1991–2000 decade, research targeted 21 dif-ferent target areas.

Neurological GroupA single neurological group was the focus of themajority of reports (88%), with only a small pro-portion of studies reporting on two (7%), three

0

100

200

300

400

500

600

1951-1960 1961-1970 1971-1980 1981-1990 1991-2000 2001-2006Decade

Fre

qu

ency

SR RCT non-RCT CS SSD

FIGURE 2Cummulative frequency of publications by research design over time.

TABLE 3Neurological Group by Research Design of all Studies Indexed in PsycBITE�

Neurological group Research design

All reports N (%)* SR N (%) RCT N ( %) Non-RCT N (%) CS N (%) SSD N (%)

Stroke/CVA 534 (41) 39 (45) 113 (41) 55 (38) 110 (39) 217 (43)Traumatic braininjury (TBI)/head injury 338 (29) 11 (13) 48 (17) 36 (25) 88 (31) 155 (30)Alzheimer’s andrelated dementias 282 (22) 22 (25) 71 (26) 47 (32) 70 (25) 72 (14)Multiple sclerosis 66 (5) 30 (34) 24 (9) 11 (8) 20 (7) 8 (2)Degenerative disorders 58 (4) 11 (13) 14 (5) 5 (3) 14 (5) 14 (3)Brain infections 55 (4) 1 (1) 3 (1) 6 (4) 11 (4) 34 (7)Nonspecified brain injury 49 (4) 4 (5) 11 (4) 5 (3) 11 (4) 18 (4)Epilepsy/seizures/convulsions 42 (3) 5 (6) 10 (4) 2 (1) 8 (3) 17 (3)Hypoxia/anoxia 35 (3) 0 (0) 2 (1) 5 (3) 10 (4) 18 (4)Drug/alcohol-relatedbrain damage 22 (2) 0 (0) 3 (1) 5 (3) 6 (2) 8 (2)Other 18 (1) 1 (1) 1 (< 1) 1 (1) 9 (3) 7 (1)Neurotoxicity 17 (1) 0 (0) 1 (< 1) 1 (1) 4 (1) 2 (< 1)Neoplasms 16 (1) 3 (0) 1 (< 1) 1 (1) 8 (3) 3 (1)

Note: * Column percentages do not add to 100 because some reports contribute to the frequency count for more than onecategory for neurological group.

125

THE EVIDENCE BASE OF NEUROPSYCHOLOGICAL REHABILITATION IN ABI

(4%) or four (1%) distinct neurological groups.An overwhelming majority of reports (92%) wasconcerned with one of three neurological groups,namely stroke/CVA (41%), traumatic brain injury(TBI; 29%) or Alzheimer’s/related dementia(22%). To date, other types of ABI have receivedrelatively scant attention, with 5%, or less, of allreports concerned with any other single neurolog-ical group (Table 3).

When reports were examined according toresearch design, a similar pattern was evident.Stroke/CVA was the most common neurologicalgroup and occurred in comparable proportion ineach class of research design (SRs = 45%; RCTs =41%; non-RCTs = 38%; CSs = 39%; SSDs = 43%).Alzheimer’s/related dementia was the second mostfrequent neurological group targeted by both RCTs(26%) and non-RCTs (32%), while TBI was thethird most common group of interest among boththese research designs (RCT = 25%; non-RCT =17%). An inverse pattern was evident for CSs andSSDs. TBI was the second most common neuro-logical group among CSs (31%) and SSDs (30%),while Alzheimer’s/related dementia was the thirdmost common in both types of study (CSs = 25%;SSDs = 14%). Greater variability occurred amongSRs. Multiple sclerosis (34%) was the second mostcommon neurological group in this research designand Alzheimer’s/related dementia (25%) the third

most common. Only 13% of SRs focused on TBI,the lowest proportion among the various studytypes. The same proportion of SRs (viz. 13%) wasalso concerned with degenerative disorders, agroup investigated among 5% or less of any of theother research designs. No other neurologicalgroup indexed on PsycBITE™ was the focus ofstudy in more than 5% of reports in any researchdesign category.

Target AreaOver half of all reports (58%) focused on a singletarget area, over one quarter (28%) on two, andonly 12% and 3% tackled three or four target areasrespectively. Reports were predominantly con-cerned with one of seven target areas for interven-tion: communication/language/speech disorders(24%); independent/self-care ADL (19%);behaviour problems (17%); memory impairments(17%); anxiety, depression, stress, adjustment(15%); multiple problems (13%); and cognitivedeficits (10%). Most other target areas wereaddressed by less than 5% of all studies (Table 4).The largest, and comparable proportions of SRstargeted other (42% — i.e., problems not definedin other categories) and multiple problems (38%).Approximately half that proportion of SRs tar-geted independent/self-care ADL (19%) andbehaviour problems (17%). Only three other

126

MICHAEL PERDICES ET AL.

target areas — anxiety, depression, stress andadjustment (13%), cognitive deficits (11%) andcommunication/language/speech disorders (10%)— were the focus of at least 10% of SRs.

RCTs and non-RCTs predominantly investi-gated similar target areas. A comparable proportionof RCTs targeted anxiety, depression, stress andadjustment (27%) and independent/self-care ADL(26%) and a smaller but also comparable propor-tion of RCTs targeted behaviour problems (19%)and multiple problems (17%). The majority, and

comparable proportions, of non-RCTs also tar-geted independent/self-care ADL (24%) and anx-iety, depression, stress and adjustment (21%).Smaller and, again, comparable proportions ofnon-RCTS targeted multiple problems (18%),memory impairments (17%), and behaviour prob-lems (16%). Only 14% of RCTs and 12% of non-RCTs targeted communication/ language/speechdisorders, while most other target areas were thefocus of approximately 10% or less of reports ofeither research design. The majority of CSs also

TABLE 4Target Area by Research Design of All Studies Indexed in PsycBITE�

Target Area Research DesignAll reports N (%)* SR N (%) RCT N ( %) Non-RCT N (%) CS N (%) SSD N (%)

Communication/language/speechdisorders 311 (24) 9 (10) 37 (14) 18 (12) 48 (17) 199 (39)Independent/self-careADL 236 (19) 17 (19) 72 (26) 36 (24) 66 (23) 45 (9)Behaviour problems 221 (17) 15 (17) 52 (19) 24 (16) 35 (13) 95 (19)Memory impairments 212 (16) 7 (8) 31 (12) 26 (17) 41 (14) 107 (22)Anxiety, depression,stress and adjustment 193 (15) 11 (13) 75 (27) 30 (21) 48 (18) 29 (6)Multiple problems 162 (13) 33 (38) 46 (17) 27 (18) 50 (18) 8 (2)Cognitive deficits 131 (10) 10 (11) 29 (11) 22 (15) 39 (14) 30 (6)Movement and motorproblems 88 (7) 6 (7) 28 (10) 8 (5) 20 (7) 26 (5)Sensory/perceptual/visuospatial impairments 78 (6) 5 (6) 17 (6) 8 (5) 16 (6) 32 (6)Quality of life 75 (6) 5 (6) 41 (15) 12 (8) 14 (5) 3 (1)Interpersonal psychosocialand social skills 71 (6) 4 (5) 16 (6) 12 (8) 26 (9) 13 (3)Attention problems 53 (4) 3 (3) 14 (5) 5 (3) 12 (4) 19 (4)Reading, writing andarithmetic 53 (5) 0 (0) 4 (1) 0 (0) 5 (2) 46 (10)Executive functioningdeficits 45 (4) 0 (0) 7 (3) 6 (4) 9 (3) 23 (5)Community re-entry/instrumental ADLs 44 (3) 2 (2) 14 (5) 7 (4) 15 (5) 10 (2)Other 22 (2) 37 (42) 16 (9) 8 (5) 13 (5) 14 (3)Coma/altered statesof consciousness 14 (1) 2 (2) 1 (< 1) 1 (1) 5 (2) 5 (1)Fatigue and low worktolerance 14 (1) 2 (2) 0 (0) 1 (1) 3 (1) 3 (1)Insight and awareness 8 (< 1) 0 (0) 2 (1) 0 (0) 2 (1) 4 (1)Delusions/delirium/psychotic disorders 6 (< 1) 1 (1) 1 (< 1) 1 (1) 0 (0) 3 (1)Pain 4 (< 1) 0 (0) 1 (< 1) 1 (1) 0 (0) 1 (< 1)Emotional processingdisorder 3 (< 1) 0 (0) 0 (0) 1 (1) 1 (< 1) 2 (< 1)

Note: * Column percentages do not add to 100 because some reports contribute to the frequency count on more than onecategory for target area

0

5

10

15

20

25

30

35

40

Fre

qu

en

cy

. 0 1 2 3 4 5 6 7 8 9 10 .

MQR Score

Rated RCTs (N=128)

0

5

10

15

20

25

30

35

40

Fre

qu

en

cy

. 0 1 2 3 4 5 6 7 8 9 10 .

MQR Score

Rated non-RCTs (N=68)

0

10

20

30

40

50

60

Fre

qu

en

cy

. 0 1 2 3 4 5 6 7 8 9 10 .

MQR Score

Rated CS (N=122)

FIGURE 3Distribution of MQR scores by research design.

127

THE EVIDENCE BASE OF NEUROPSYCHOLOGICAL REHABILITATION IN ABI

targeted independent/self-care ADL (23%), anxi-ety, depression, stress and adjustment (18%), aswell as memory impairments (18%) and commu-nication/language/ speech disorders (17%). Incontrast to other research designs, communica-tion/language/speech disorders was the most fre-quently investigated target area among SSDs

(39%). Approximately half that proportion of SSDstargeted memory impairments (22%) andbehaviour problems (19%) and only 10% or less ofSSDs focused on any other single target area.

Methodological QualityThe frequency distribution of MQR scores foreach research design is shown in Figure 3. Scoresfor RCTs had an approximately normal distribu-tion, with a mean MQR just below the median ofthe range at 4.49. Although a small number ofRCTs received relatively low scores none of thetrials received a score of zero. Conversely, noRCT achieved a perfect 10/10 score. The distribu-tion of scores for non-RCTs was slightly posi-tively skewed, with a mean MRQ of 2.85. Again,none of the trials received a score of zero, or themaximum score of 8/10. CSs had a mean MQR of1.15, with 51% of studies obtaining a score of 1/2.

The percentage of RCTs and non-RCTs thatsatisfied specific criteria on the PEDro scale isshown in Figure 4. Criteria 2 and 3 were only appli-cable to RCTs, while criteria 1 and 4 to 11 wereapplicable to both types of research design. A sig-nificantly higher proportion of RCTs (42%) thannon-RCTs (10%) met criterion 7 (viz. ‘All asses-sors who measured at least one key outcome areblinded’ — χ2(1) = 21.1, p < .001), while a signifi-cantly higher proportion of non-RCTs (68%) thanRCTs (54%) met criterion 8 (viz. ‘Measures of atleast one key outcome are obtained from more than85% of the subjects initially allocated to groups’ —χ2(1) = 3.46.1, p < .043). There were no significantdifferences between the two research designs in theproportion of reports that met other criteria. At least70% of both types of research design met criteria 1,10 and 11, while the proportion of reports meetingcriteria 5, 6 and 9 was relatively low for both typesof research design.

Specific neurological groups and target areaswere selected if there were at least five rated RCTsin a given category. Trials concerned withstroke/CVA had the highest mean MQR (4.77), butthere was no significant difference in mean MQRamong the six neurological groups examined, F(5,116) = 0.55, p = .74. The highest mean MQR (5.86)was obtained by trials targeting movement andmotor problems but, once again, there was no sig-nificant difference in mean MQR among the 10target areas examined, F(9, 107) = 1.12, p = .36).The mean MQR of 126 RCTs published during1981 to 1990, 1991 to 2000 and 2000 to 2005 was4.64, 4.32 and 4.57 respectively, showing no clearindication of improvement in methodologicalquality over time.

128

MICHAEL PERDICES ET AL.

DiscussionThere is, to date, a considerable and steadilygrowing body of published research reporting onthe efficacy of nonpharmacological rehabilitationfor the neuropsychological consequences of ABI.The number of reports in the published literaturehas increased dramatically since the 1950s.Moreover, the sequelae of ABI targeted by theresearch has also expanded to encompass a widerange of problems in the cognitive, social, inter-personal, and emotional spheres of function,extending well beyond the bounds of deficits inspecific cognitive abilities.

Unfortunately, despite the growing volumeand broadening scope of investigations, themethodological quality of research remainsmodest. This clearly represents a setback for theimplementation of EBCP. In the framework ofEBCP, SRs provide the highest level of evidenceto inform clinical practice. Findings reported in aSR can be considered a reliable indicator of theefficacy (or lack of efficacy) of a given interven-tion. A ‘limitation’ of SRs is that, in the context ofthe ABI literature, they constitute the smallestproportion of the research output (7%) and cur-rently few, if any, are available on some neurolog-ical groups (e.g., hypoxia/anoxia) and target areas(e.g., insight/awareness).

RCTs represent the next highest level of clin-ical evidence in the EBCP hierarchy and, in

theory, provide the clinician with reliable guide-lines to evaluate the appropriateness of particulartreatments. RCTs (21%) are also more numerousin PsychBITE� than SRs (7%), and provideinformation on some neurological groups andtarget areas not covered by SRs. Yet, resultsreported in a particular RCT cannot be automati-cally accepted as strong and reliable (i.e., credi-ble) evidence just because the study is an RCT; itneeds to be interpreted in the context of the trial’smethodological quality. Moseley, Sherrington,Herbert, and Maher (2000) suggest that a score of5 or above on the PEDro Scale (i.e., MQR ≥ 5) isindicative of a well-designed and well-conductedRCT. The mean MQR for RCTs listed inPsycBITE� was 4.49, just below this cut-off. Ourfindings suggest that sources of bias are not suffi-ciently well controlled in RCTs investigating theefficacy of treatments for the neuropsychologicalconsequences of ABI. As a consequence, the reli-ability of findings reported in these studies is sig-nificantly compromised.

Why is the methodological quality of RCTs somodest? Several factors are worth considering. Onthe one hand, it is understandable why there wasrelatively high attainment of some criteria on thePEDro Scale. For example, criterion 2 (randomallocation of subjects) was satisfied by all RCTs.This is not surprising given that this criterionencapsulates the pivotal definition of an RCT.Criteria 10 (between-intervention group compar-

0

10

20

30

40

50

60

70

80

90

100

Per

cen

t

1 2 3 4 5 6 7 8 9 10 11

Criterion RCTS non-RCTs

FIGURE 4Proportion of RCTs and non-RCTs meeting specific criteria on the PEDro Scale.

129

THE EVIDENCE BASE OF NEUROPSYCHOLOGICAL REHABILITATION IN ABI

isons reported) and 11 (point measures and mea-sures of variability provided) pertain to such fun-damental aspects of data analysis and reportingthat it is of grave concern that between 10% and20% of RCTs failed to meet one of these criteria.On the other hand, it is also understandable whyattainment of other criteria was relatively low. Forinstance, the concept of ‘intention to treat’ (crite-rion 9) is relatively new and has yet to gain widecurrency in the psychological literature. It is there-fore not surprising that only 13% of RCTs met thiscriterion. Also, no matter how well an RCT isdesigned, some criteria are difficult to meet, giventhe intrinsic limitations imposed by the nature ofthese investigations. In RCTs investigating the ther-apeutic effects of an experimental drug, blinding ofsubjects (criterion 5) and therapist (criterion 6) isreadily achievable because subjects can be giventwo identical looking tablets, one containing theactive agent and the other containing a placebo.Blinding of subjects and therapists in neuropsycho-logical trials is remarkably difficult to achieve. Inthe context of an RCT investigating the efficacy ofpsychological interventions, subjects would beblinded if two participants in the trial, one from the‘treatment’ group and one from the ‘control’ group,were unable to determine which particular groupthey were in, after having ‘compared notes’ aboutwhat they did during the ‘treatment’ sessions. Forsimilar reasons, blinding of therapist is also diffi-cult to achieve in these types of investigation.Indeed, only two of the 128 RCTs (and one of the68 non-RCTs) presently rated and listed inPsycBITE� satisfied criteria 5 and 6.

It is, however, unsatisfactory that adherence toother, and arguably fundamental criteria (e.g., cri-terion 4: treatment and control groups must besimilar at baseline on the most important outcomeand prognostic indicators is relatively poor (41%).Some of the blame may be attributed to ‘real-world’ practical constraints of clinical research.For instance, adherence to criterion 4 might bedifficult to achieve because random allocation ofsmall samples may produce groups that are notcomparable in terms of the key outcome variables.Attempts to rectify the situation might be unsuc-cessful either because it is difficult to recruit newparticipants (e.g., low base-rate of the conditionbeing investigated, limited financial resources), orstatistical correction might be inappropriate. Thepoint is, that although there might be practical andunderstandable reasons why compliance withsome criteria is poor, this state of affairs nonethe-less reflects an unacceptably and regrettably lowstandard of methodological rigor, which urgentlyrequires significant improvement. One bright note

is the fact that despite their relatively poor meanMRQ, 49% of RCTs obtained a rating score abovethe cut-off suggested by Moseley et al. (2000),while 11% obtained an MQR ≥ 7, indicating rela-tively high methodological rigor. Moreover, therewas no single neurological group or target areatargeted in RCTs where research was significantlyinferior in terms of methodological quality.

The methodological quality of non-RCTs wasdisappointingly low. While this type of study isclearly inferior to SRs and RCTs in terms of thelevel of evidence it can furnish, it cannot be com-pletely discarded as a source of empirical data.Indeed, it can be argued that a well-conducted,methodologically robust non-RCT may providemore reliable evidence on the efficacy of an inter-vention than a poorly designed RCT. Non-RCTssurveyed in this study showed a pattern of com-pliance on specific criteria of the PEDro Scalecomparable to that of RCTs. Consequently, theissues discussed above are equally applicable tonon-RCTs. To a great extent, the methodologicalquality of non-RCTs is poor, because fundamentalaspects of research design are not being met. Inbrief, despite the fact that some high qualityresearch is appearing in the literature, researchesin the field of ABI need to significantly improvetheir standards. We hope and envisage thatPsycBITE� will contribute to this by providing areadily accessible, standardised framework forevaluating clinical research and thus raise aware-ness of these issues. We also envisage that, as aconsequence, PsycBITE� will be instrumental inpromoting EBCP in the field of ABI. PsycBITE�was launched in July 2004. Over the next fewyears we anticipate that the relatively low andunchanging level of methodological qualityobserved in RCTs over the last 25 years will showdiscernible improvement.

The findings of this study confirm the pointthat the empirical value of CSs is severely limited.These research designs can only obtain a maxi-mum MQR of 2/10, which makes the reliability oftheir findings highly questionable. Given the lim-itations inherent in CSs, combined with the factthat the time and resources required to mount thistype of study may be comparable to those neededfor an RCT or non-RCT, it is necessary to ask whythis type of research design continues to be used.

SSDs constitute the most numerous method-ological design in the literature. This type ofresearch design offers a very important advantageover (even well-designed) RCTs. The efficacy oftreatment demonstrated by an RCT may not applyto a specific patient if that patient’s clinical charac-teristics do not closely match those of the group

130

MICHAEL PERDICES ET AL.

studied in the RCT. Furthermore, even if a treat-ment is shown to be successful, it is unlikely that allpatients in the trial will benefit from treatment, par-ticularly if the treatment effect is small in statisticalterms (Guyatt, Rennie, & Hayward, 2000). By con-trast, the applicability to an individual patient oftreatment effects reported in a SSD is easier todetermine. SSDs offer the additional advantage thatin terms of time and resource requirements, theyprovide an accessible research vehicle which canbe readily integrated with clinical practice. We arein the final stages of developing an instrument,analogous to the PEDro scale, to rate the method-ological quality of SSDs (Tate et al., 2004).Preliminary trialing of the scale indicates that anumber of SSDs show an admirable level ofmethodological rigor. In essence, SSDs are a valu-able, and to date a relatively underrated, comple-ment to SRs and RCTs. An SSD may, for instance,be used to ‘pilot’ the efficacy of an interventionprior to undertaking a more complex andtime/resource consuming RCT. Well-designedSSDs reach well beyond a simplistic methodologyconsisting of single pre- to posttreatment measures(Barlow & Hersen, 1984). Sophisticated SSDsincorporating, among other things, multiple base-lines across subjects and behaviours, as well as ran-domisation of the order of active treatment phasescan, potentially, provide a level of evidence withinthe hierarchy of EBCP comparable to that of RCTs(Guyatt, Rennie, & Hayward, 2000).

The majority of the research studies surveyedfor this report tended to focus on relatively fewtypes of ABI and target a relatively small ‘pool’ ofdeficits. The limited focus on neurological groupmay, to some extent, reflect the gross prevalenceof ABI in the general clinical setting. Even so, itis puzzling to find that so little research attentionhas been given to relatively common types ofABI, such as cerebral neoplasms. It is also per-plexing that deficits which are relatively frequentsequelae of ABI (e.g., executive deficits, attentiondeficits, impaired insight) are infrequentlyaddressed in any of the research designs. Reasonsfor this may be partly historical and partly a func-tion of the complexity of some areas of investiga-tion. Interventions for executive functions, forexample, have been notoriously difficult to mountand have only recently started to emerge in thepublished literature with any degree of frequency.Clearly, more attention needs be given to theseaspects of ABI and future research efforts mustaddress these imbalances.

Despite the aforementioned imbalances inresearch focus, the literature now offers a muchwider scope of information than it did 20 years ago

about interventions for the neuropsychological con-sequences of ABI. Such proliferation of researchcreates a daunting task for the clinician attemptingto embrace and implement the principles of EBCP.Given the dearth of studies available in some targetareas, it is a point of concern that interventions forthese aspects of dysfunction may be implementedin the rehabilitation setting without substantive evi-dence for their efficacy. The challenge is to pro-mote research within these relatively neglectedareas and build a sufficiently large corpus ofresearch to justify the selection and implementationof interventions in clinical practice. As the task ofsearching for and identifying relevant informationbecomes increasingly more complex and time-con-suming, PsycBITE� offers a number of advan-tages that have been closely detailed elsewhere(Tate et al., 2004; Tate et al., 2006).

ReferencesBarlow, D.H., & Hersen M. (1984) Single case experimen-

tal designs: Strategies for studying behaviour change(2nd ed.). Needham Heights, MA: Allyn & Bacon.

Brookshire, R.H. (1970), Control of ‘involuntary’ cryingbehavior emitted by a multiple sclerosis patient.Journal of Communication Disorders 3(3), 171–176.

Byers-Brown B., & Ives, L. (1969) The re-education ofa dysphasic adult: An experiment in co-operationbetween a speech therapist and an educational psy-chologist. British Journal of Disorders ofCommunication, 4, 176–196.

David, R.M., Enderby, P., & Bainton, D. (1979).Progress report on an evaluation of speech therapyfor aphasia. British Journal of Disorders ofCommunication 14(2), 85–88.

de Pedro-Cuesta, J., Widen-Holmqvist, L., & Bach-y-Rita, P. (1992). Evaluation of stroke rehabilitationby randomized controlled studies: A review. ActaNeurologica Scandinavica, 86, 433–439.

Di Carlo, L.M. (1980). Language recovery in aphasia:Effect of systematic filmed programmed instruc-tion. Archives of Physical Medicine andRehabilitation 61(1):41–44.

Gasparinni, B., & Satz, P. (1979). A treatment ofmemory problems in left hemisphere CVA patients.Journal of Clinical Neuropsychology 2, 137–150.

Goodkin, R. (1966). Case studies in behaviouralresearch in rehabilitation. Perceptual and MotorSkills, 23, 171–182.

Gottschalk, L.A. (1953). Effects of intensive psy-chotherapy on epileptic children: Report on threechildren with idiopathic epilepsy. Archives ofNeurology and Psychiatry, 70, 361–384.

Guyatt, G., Rennie, D., & Hayward, R. (2002). User’sguides to the medical literature: A manual for evi-dence based practice. Chicago: AMA Press.

Haynes, R.B., Sackett, D.L., Gray, J.M., Cook, D.J., &Guyatt, G.H. (1996). Transferring evidence from

131

THE EVIDENCE BASE OF NEUROPSYCHOLOGICAL REHABILITATION IN ABI

research into practice: 1. The role of clinical careresearch evidence in clinical decisions. ACP J Club,125, A14–6.

Jadad, A.R., Moore, R.A., Carroll, D., Jenkinson, C.,Reynolds, D.J., Gavaghan, D.J., & McQuay H.J.(1996) Assessing the quality of reports of random-ized clinical trials: Is blinding necessary?Controlled Clinical Trials. 17(1), 1–12.

Maher, C.G., Sherrington, C., Herbert, R.D., Moseley,A.M., & Elkins, M. (2003). Reliability of the PEDroscale for rating quality of RCTs. Physical Therapy.83, 713–721.

McDonald, S., Tate, R., Togher, L., Perdices, M., Mosely,A., Winders, K., Schultz, R., & Smith, K. (2006)Improving evidence-based practice in rehabilitation:Introducing PsycBITETM. Aphasiology (in press).

Metcalfe, C., Lewin, R., Wisher, S., Perry, S., Bannigan,K., & Moffett, J.K. (2001). Barriers to implement-ing the evidence base in four NHS therapies.Dieticians, occupational therapists, physiothera-pists, speech and language therapists.Physiotherapy. 87, 433–441.

Moher, D., Schulz, K.F., & Altman, D.G. (2001). TheCONSORT statement: revised recommendations forimproving the quality of reports of parallel-grouprandomised trials. Lancet, 357(9263), 1191–1194.

Moseley, A.M., Maher C, Herbert R.D., & Sherrington C.(1999). Reliability of a scale for measuring themethodological quality of clinical trials. Proceedingsof the VIIth Cochrane Colloquium, Rome, p. 39.

Moseley, A., Sherrington, C., Herbert. R., & Maher, C.(2000). The extent and quality of evidence in neu-rological physiotherapy: An analysis of thePhysiotherapy Evidence Database (PEDro). BrainImpairment, 1(2), 130–140.

National Health and Medical Research Council. (1999). Aguide to the development, implementation and evalu-ation of clinical practice guidelines. Canberra:NHMRC.

Sackett, D.L., Strauss, S.E., Richardson, W.S.,Rosenberg, W., & Haynes, R.B. (2000). Evidence-based medicine: How to practice and teach EBM(2nd ed.) Edinburgh: Churchill Livingstone.

Tate, R.L., McDonald, S., Togher, L., Perdices, M.,Moseley, A., & Winders, K. (2004). (Abstract).Rating the methodological quality of single-caseexperimental designs: The PsycBITETM Scale.Brain Impairment. (Suppl. Vol. 5), 165.

Tate, R.L., Moseley, A.M., Perdices, M., McDonald, S.,Togher, L., Schultz, R., Savage, S., & Winders, K.(2006). (Letter). Update on Cicerone’s systematicreview of cognitive rehabilitation: The PsycBITEperspective. Archives of Physical Medicine andRehabilitation, 87, 446.

Tate R.L., Perdices, M., McDonald, S., Togher, L.,Moseley, A., Winders, K., Kangas, M., Schultz, R., &Smith, K. (2004). Development of a database of reha-bilitation therapies for the psychological conse-quences of acquired brain impairment.Neuropsychological Rehabilitation, 14(5), 517–534.

University of York. (1992). Stroke rehabilitation.Effective Health Care 1(2), 12.

Verhagen, A.P., de Vet, H.C., de Bie, R.A., Kessels, A.G.,Boers, M., Boutler, L.M., & Knipschild, P.G. (1998).The Delphi list: A criteria list for quality assessmentof randomised clinical trials for conducting system-atic reviews developed by Delphi consensus. Journalof Clinical Epidemiology, 51, 1235–1241.

Weinberg, J., Diller, L., Gordon, W.A., Gerstman, L.J.,Lieberman, A., Lakin, P., Hodges, G., & Ezrachi, O.(1977). Visual scanning training effect on reading-related tasks in acquired right brain damage.Archives of Physical Medicine & Rehabilitation,58(11), 479–486.

Young, J.M., Glasziou, P, & Ward, J.E. (2002). Generalpractitioners’ self-ratings of skills in evidence basedmedicine: A validation study. BMJ, 324, 950–951.

APPENDIX ATypes of Research Design Indexed on PsycBITE�

Systematic ReviewA systematic review is an overview of primary studies that contains an explicit statement of objectives,materials, and methods and has been conducted according to explicit and reproducible methodology. Theaim of such a review is to address a specific clinical question by assembling, synthesising and apprais-ing the results of all relevant studies in the area, using methods which limit bias. A meta analysis is oneform of systematic review. It is a mathematical synthesis of the results of two or more primary studiesthat addressed the same hypothesis in the same way. A meta analysis must be a component of a system-atic review or adhere to systematic review methodology to be included in PsycBITE� .

Randomised Controlled Trial (RCT)A RCT compares at least two treatments/interventions (one of which can be a no-treatment control or a wait-list control condition), which involves random allocation (participants are randomly allocated to groups for

132

MICHAEL PERDICES ET AL.

either the intervention/treatment being studied or control/placebo using a random mechanism, such as cointoss, random number table, or computer-generated random numbers) and the outcomes are compared.

Non-Randomised Controlled Trial (non-RCT)This type of trial is similar to an RCT in that it compares at least two treatments/interventions (one of whichcan be a no-treatment control or a wait-list control condition) with the exception that participants have notbeen randomly allocated to groups. Non-RCTs however may be observational in nature, such that theresearcher does not have control over the intervention being received by one group (see cohort studybelow). Outcomes of the treatment group and control group may be examined prospectively or retrospec-tively. In other non-RCTs, participants with an already established outcome or disease are selected andinformation is obtained about the previous exposure to the treatment/intervention or other factor being stud-ied. Pseudo or intended-to-be-randomised trials are also included in this category where participants areallocated to groups for intervention/treatment or control/placebo using a non-random method (such as alter-nate allocation or by odd or even hospital numbers). Note, however, that pseudo randomised trials do notget a point for criterion 2 on the PEDro scale.

Cohort StudyA cohort study is a type of observational study where the outcomes for a group of exposed participants(e.g., who have received an intervention) are compared with the outcomes for another group from a simi-lar population or ‘cohort’ (which may either be historical or contemporaneous) that were not exposed (e.g.,did not receive the intervention). Cohort studies can be prospective or retrospective. Where these cohortsare followed forward for the outcome of interest they are also known as longitudinal studies. This form ofcohort study differs from a controlled trial as the exposure is outside the control of the investigators.

Case Series (CS)A group/s or series of participants are exposed to one treatment/intervention where outcomes are mea-sured in participants before and after exposure to the treatment/intervention. No control group isincluded. Detailed information regarding individual participants is often presented.

Single-Subject Design (SSD)This type of trial compares the effectiveness of interventions using the same (single) participant ashis/her own control by assessing treatment effects over several different points in time as treatments aresystematically applied and sometimes withdrawn. The single-case trial may take the form of variationsof an ABAB (cross-over) or multiple baseline design.

APPENDIX BAdapted Summary of Criteria Comprising the PEDro Scale

1 Eligibility criteria of subjects were specified (not included in MQR Score)2 Subjects were randomly allocated to interventions (in a crossover study, subjects were randomly

allocated an order in which treatments were received)3 Allocation was concealed4 The intervention groups were similar at baseline regarding the most important outcome measures

and prognostic indicators5 There was blinding of ≥ 90% of subjects6 There was blinding of ≥ 90% of therapists who administered the therapy7 There was blinding of ≥ 90% of assessors who measured at least one key outcome8 Measures of at least one key outcome were obtained from ≥ 85% of the subjects initially allocated

to groups9 All subjects for whom outcome measures were available received the treatment or control condition

as allocated or, where this was not the case, data for at least one key outcome was analysed by‘intention to treat’

10 The results of between-intervention group statistical comparisons are reported for at least one keyoutcome

11 The study provides both point measures and measures of variability for at least one key outcome.