Spanish, French, and British Cross-cultural Validation of the European Brain Injury Questionnaire

LWW/JHTR HTR200133 December 11, 2010 0:17 Char Count= 0

Copyright c© Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

J Head Trauma RehabilCopyright c© 2010 Wolters Kluwer Health | Lippincott Williams & Wilkins

Spanish, French, and BritishCross-cultural Validation of theEuropean Brain Injury Questionnaire

Alfonso Caracuel, PhD; Andrew Bateman, PhD; Thomas W. Teasdale, PhD;Antonio Verdejo-Garcıa, PhD; Miguel Perez-Garcıa, PhD

Objective: To explore the factor structure of the European Brain Injury Questionnaire and to assess the cross-culturaland construct validity of this questionnaire by using Rasch analysis. Participants: A total of 366 individuals with trau-matic brain injury or stroke were recruited from 3 different countries: Spain (116 participants), the United Kingdom(110 participants), and France (140 participants). Analyses: We first performed a factor analysis and then appliedRasch analysis to the resulting factors to examine construct and cross-cultural validity. Results: Three subscaleslabeled Depressive Mood, Cognitive Dysfunction, and Poor Social and Emotional Self-regulation were extractedusing the factor analysis. In the Rasch analyses, 8 items were removed because of misfit and 7 items showed differ-ential item functioning by country. Conclusion: Rasch analyses showed good fit to the model, unidimensionality,construct validity, and good reliability of the 3 European Brain Injury Questionnaire subscales. However, only theDepressive and Cognitive subscales showed cross-cultural validity. Keywords: adults, acquired brain injury, behavior,cognition, construct validity, cross-cultural validity, EBIQ, emotion, factor analysis, outcome assessment, questionnaire, Raschanalysis, stroke, subjective assessment, traumatic brain injury

THE EUROPEAN BRAIN INJURY QUESTION-NAIRE (EBIQ) is a tool developed by an inter-

national group as a cross-culturally valid measure ofthe consequences of acquired brain injury (ABI). TheEBIQ, originally designed from a holistic clinical per-spective, measures a wide range of subjective cognitive,emotional, and social difficulties stemming from ABI,along with estimates of basic activities of daily livingfunctions.1 The items were selected from clinical experi-ence and some were adapted from the Symptom Check-list (SCL90R) and the Katz Adjustment Scales.2 Thequestions cover the broad areas of personal, familial andeconomic activities, social relationships, cognitive func-tions, somatic factors, and depression.3 There was agree-ment in the development consortium that the questionsalso needed to be ecologically relevant4 and specifically

Author Affiliations: Personalidad, Evaluacion y Tratamiento PsicologicoDepartment and Neurosciences Institute F. Oloriz, University of Granada,Spain (Drs Caracuel, Verdejo-Garcıa, and Perez-Garcia); The OliverZangwill Centre for Neuropsychological Rehabilitation, Ely,Cambridgeshire, United Kingdom (Dr Bateman); and Department ofPsychology, University of Copenhagen, Denmark (Dr Teasdale).

The original development of the EBIQ was done under the leadership anddirection of the late Professor Gerard Deloche, who also provided the Frenchdata incorporated here.

Corresponding Author: Alfonso Caracuel, PhD, Personalidad, Evaluacion yTratamiento Psicologico Department and Neurosciences Institute F. Oloriz,University of Granada, Facultad de Psicologia, 18710, Granada, Spain([email protected]).

designed for ABI populations, using clear, brief languagethat was not culture specific.1 Parallel forms for individ-uals, close relatives, and clinicians are available. Symp-toms occurring in the preceding month are to be ratedon a 3-point scale (“not at all,” “a little,” and “a lot”).

The EBIQ assesses subjective symptoms in the areasof life that are important to individuals and relatives. Inrehabilitation settings there has been a growing emphasison subjective person-centered evaluation of people withbrain injury.5,6 The EBIQ is currently used by profes-sionals for several purposes including establishing base-line measurements,7–10 measuring differences betweengroups (eg, to discriminate between consequences ofABIs of different etiologies, discrepancies between re-ports of individuals and caregivers), or exploring possi-ble symptom changes across time (eg, onset, discharge,follow-up).11,12 The instrument is also widely used forthe assessment of rehabilitation outcomes.13–20

Several methodological approaches have been appliedto explore the psychometric properties and structure ofthe EBIQ. First, to assess the relations among items ofthe EBIQ, the original consortium used nonmetric Mul-tidimensional Scaling for grouping items according toa “radex-hypothesis” derived from Facet theory.1 Thisanalysis identified 9 subscales: 1 global “Core” scale and8 domain-specific subscales (Cognitive, Impulsivity, So-matic, Depression, Physical, Communication, Motiva-tion, and Isolation). Internal reliability by the Cron-bach alpha was reported for the 9 scales as ranging

1

LWW/JHTR HTR200133 December 11, 2010 0:17 Char Count= 0

Copyright c© Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

2 JOURNAL OF HEAD TRAUMA REHABILITATION

between 0.47 and 0.90 (median 0.63) for self-reportsand between 0.54 and 0.92 (median 0.66) for relatives’reports. Construct validity was considered satisfactorysince the EBIQ scales discriminated between individu-als with ABI and controls, individuals with and withoutaphasia, groups varying in time since injury and differentetiologies. For example, individuals with traumatic braininjury (TBI) scored significantly higher on the cognitive,impulsivity and isolation subscales, whereas individualswith stroke scored higher on the physical and communi-cation domains.1 Test-retest reliability for the 9 subscalesover 4 weeks ranged between 0.55 and 0.90 with a meanvalue 0.76.5

Second, factor analysis has also been used in severalstudies that have consistently detected the existence of3 factors labeled Depression, Cognitive difficulties, and Ir-ritability/Impulsivity or Social interaction difficulties acrossdifferent samples.3,21–23 The most recent method appliedfor the assessment of the EBIQ is Rasch analysis.7,11 Thisanalysis complements the former approaches and it hasbeen increasingly used for examining item bias, assess-ing psychometric properties of each factor as a poten-tial subscale and improving ordinal measures. The mainproblem for ordinal scores is that the interval between0 and 1 is not necessarily equal to the interval between1 and 2 etc. The Rasch model is the only method thatuses interval units when measuring human performance,attitudes, and perceptions.24 When a questionnaire sat-isfies all Rasch model requirements, its item scores canbe validly summed and used in subsequent paramet-ric statistical analyses.25 To meet this criterion, an as-sessed instrument must show unidimensionality26; thatis, there is a single latent trait along its construct. The la-tent trait of the EBIQ could be called ABI disorder, a labelfor the interrelated consequences across somatic, cogni-tive, emotional, and behavioral domains after an ABI.Nonetheless, although the complex constructs createdby the interaction of several domains could potentiallyshow unidimensionality, this does not seem to be thecase of the EBIQ. Thus, previous Rasch analysis with aBritish sample found that the EBIQ was a multidimen-sional instrument with several subscales. Domain-basedsubscales related to impulsivity, depression, communica-tion, cognition, fatigue, and somatic difficulties, and fac-tor analysis-based subscales relating to depression, cog-nition, and social difficulties had shown appropriate fitsto the Rasch model.7

Rasch analysis is also used for testing cross-culturalvalidity of instruments. This is a key issue within therehabilitation field because it may allow researchers tomake adequate comparisons when data from severalcountries are pooled27 to achieve larger sample sizes.Rasch analysis uses differential item functioning (DIF)to check the equivalence or stability of items acrossgroups of respondents.28 The presence of DIF implies

variance in latent trait manifestation across the factorsinvolved including cultures, diagnostics, gender, or timeof administration.29,30 To date there are no studies usingRasch analysis for assessing the cross-cultural propertiesof the EBIQ at the items level. At the domain-basedsubscales level, results of the original international vali-dation project showed some indications of different cul-tural performance across the 8 participant countries.1 Forexample, in a set of 1-way analysis of variance (ANOVA),5 of the pairwise comparisons between countries weresignificant for the patients with ABI and 6 for the rela-tives. These results were regarded as a broadly negativefinding at the subscale level;1 however, Rasch analysismight be used to identify those individual items thatshow DIF by country.29,30 In addition, more symptomsin almost all subscales were found in a Brazilian controlgroup than a French one. The presence of some differ-ences connected to countries is not surprising becauseit is well-known that linguistic equivalence does notguarantee metric equivalence.30 Furthermore, culture in-fluences responses to questionnaires in patients31 andprofessionals.29 Some factors connected to this culturalinfluence are specific customs, attitudes, and attributionstoward illness and symptoms. These factors could affectindividuals, relatives, and professionals.32 Other influ-ences stem from conventions, relevance of time in cer-tain activities, attitudes toward testing in general andtoward some specific questions related to self-control orsexuality, patterns of abilities such as problem solving.33

This study had 2 aims. The first was to explore the fac-tor structure and the overall psychometric properties ofthe items on the EBIQ using factor analysis and Raschanalysis in a sample from 3 different cultures. Our sec-ond aim was to perform a cross-cultural validity assess-ment by checking DIF due to nationality factor.

METHODS

Participants and settings

Data from 366 individuals with ABI were retrospec-tively collected from outpatient departments in reha-bilitation facilities in 3 European countries. Selectioncriteria were as follows: (a) documented moderate or se-vere TBI or stroke (eg, initial Glasgow Coma Scale lessthan 13; PTA greater than 24 hours or a period of uncon-sciousness longer than 6 hours); (b) time since injury over30 days; (c) minimum age 15 years; (d) the absence ofsevere language comprehension problems; and (e) livingat home and substantially self-reliant in daily life activ-ities. British and Spanish samples represented consecu-tive referrals who met the selection criteria. Participantswere recruited from the Oliver Zangwill Centre betweenNovember 1996 and November 2005 and from the Vir-gen de las Nieves Hospital between January 2002 andApril 2010 for the UK and Spanish samples, respectively.

LWW/JHTR HTR200133 December 11, 2010 0:17 Char Count= 0

Copyright c© Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Cross-cultural Validation of the EBIQ 3

British participants were selected from a larger sample of226 subjects included in a recent study7 if they met thecriteria for the current study. The French sample wasalso extracted from the larger pool of 520 participantsincluded in the original EBIQ validation study1 by se-lecting those participants who met the specific criteriafor the current study. All were public facilities and re-ferral centers within their respective areas. Participantscompleted the questionnaire at the first appointment aspart of the regular assessment protocol in the presenceof, and if necessary with the assistance of, a clinician.

A sample size of 366 participants guarantees at the99% confidence level that no item calibration is morethan 1 unit away from its stable value in the logit scaleafter Rasch analysis.34 Roughly two-thirds of the partici-pants were males; mean age was 37.3 ranging from 15 to91 years; the mean number of months between the ABIoccurrence and completing the EBIQ was 21.16 (SD =19.45). In all, 66.4% of participants had a diagnosis ofTBI and the remainder were individuals with stroke.

Materials

The self-rating version of the EBIQ was used. Theversions for the 3 languages were those used in the orig-inal international project. The English version is avail-able at http://teasdale.psy.ku.dk/EBIQ.pdf. The EBIQcomprises 63 questions about problems and difficultiesoccurring within the preceding month. The 3 possibleresponses (“not at all,” “a little,” and “a lot”) were codedas 0, 1, and 2 points, respectively, reflecting an increasingdegree of symptoms.

Analysis

Factor analysis was used to determine the structure ofthe EBIQ. Considering that there is no robust theoreti-cal support underlying some groups of related symptoms(emotional, behavioral, and social)35 in the EBIQ, an ex-ploratory factor analysis, rather than a confirmatory one,was used. Separate Rasch analyses of the 3 subscales wererun afterwards. Rasch analysis was conducted to deter-mine unidimensionality and overall fit of the subscalesto the Rasch model, individual item fit, targeting of thesubscales to the severity of participants, functioning ofresponse categories, and the presence of DIF by age,gender, etiology, time since injury, and country.

Rasch analysis is increasingly applied to rehabilita-tion research but is relatively unknown to many.30

Therefore, we have incorporated here some backgroundinformation about the Rasch model to facilitate anunderstanding of its general principles and applica-tions. Brief and clear explanations can be found inHagquist et al36 and Tennant and Conaghan.37 A glos-sary of Rasch measurement terminology is available athttp://www.rasch.org/rmt/rmt152e.htm.

The Rasch model24 is a probabilistic model of mea-surement within Item Response Theory. Originally de-veloped in the context of cognitive tests, the Raschmodel states that the probability that a person will affirma given item is a logistic function of the difference be-tween the person’s ability and the difficulty of the item.27

In the EBIQ context, item “difficulty” refers to the like-lihood of the symptom being endorsed (ie, symptomseverity), and a person’s “ability” refers to the numberof symptoms endorsed (ie, overall disorder severity).7

People with low disorder severity should endorse itemcategories connected to low symptom severity. Raschanalysis tests the extent to which the observed patternof responses fits the pattern expected by the probabilis-tic model. Items and persons are calibrated and placedon a common scale: items according to their “difficulty”of endorsement and persons according to their disorderseverity. The unit is called logit30 and allows for the mea-surement of the distance between person location andevery item location in an interval scale. Construct va-lidity is determined by examining the hierarchy of theitems and by evaluating the “fit” of individual items tothe latent construct.38

Factor analysis and Rasch analysis were performedusing SPSS (IBM SPSS Statistics, Somers, New York)for Windows Version 17 and RUMM2020 software(RUMM Laboratory Pty Ltd, Perth, Australia),39 respec-tively. Data from the EBIQ were evaluated against Raschmodel expectations according to protocols described byTennant et al.29

RESULTS

Missing data

Inspection of missing data revealed that only 4 itemshad missing data frequencies greater than 5% of itemsample. These were items 36, 39, 52, and 56 with 5.2%,11.9%, 6.3%, and 8.5% of unmarked responses, re-spectively. Remaining missing data were evenly spreadthroughout the questionnaire.

Factor analysis

A factor analysis was conducted to investigate the di-mensional structure of the EBIQ. A 3-factor solutionshowed the best data fit. According to the decreasingcurve of the eigenvalues, the first 3 obtained compo-nents explained 37.98% of the common variance amongthe 63 items. Following Varimax rotation, the first 3 fac-tors explained 14.7%, 12.1%, and 11.1% of the commonvariance, respectively. Data fit was confirmed by a Kaiser-Meyer-Oklin value of 0.91 and a significant Bartlett’sTest of Sphericity (P < .05). Factors were interpreted onthe basis of item loadings. A salient factor loading foritem-level data was defined as 0.40 or more.40 All but 8

www.headtraumarehab.com

LWW/JHTR HTR200133 December 11, 2010 0:17 Char Count= 0

Copyright c© Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

4 JOURNAL OF HEAD TRAUMA REHABILITATION

items loaded above this criterion. To get further indica-tions from Rasch analysis about these 8 items, all wereretained into their corresponding factors. The first factorincluded items related to Depressive Mood; the secondfactor encompassed items related to Cognitive Dysfunc-tion; and the third included items related to Poor Socialand Emotional Self-regulation. Table 1 shows the compo-sition of rotated factors.

Rasch analysis

Rasch analyses were performed for the 3 factoranalysis–derived subscales. The unrestricted (partial-credit) model was adopted for the 3 subscales since alikelihood ratio test (P < .001) showed that the ratingscale model was less suitable due to variable thresholddistances across items. First, threshold ordering was in-spected to check whether category responses were work-ing as intended. A threshold is the transition between 2possible response options. Each threshold has a locationon the logit scale and each item has an average location.For each item, one would expect that with increasingABI severity the probability of selecting each categorywould increase in an ordered fashion from “not at all”to “a little” to “a lot.” Rasch analysis checks this expectedpattern of responses for each item. Only 7 items showeddisordered thresholds indicating that response categoriesdid not work as intended.36 All but 1 belonged to theCognitive Dysfunction factor. Since that can be a sourceof item misfit,41 adjacent categories had to be collapsed.The best schema for items 20 and 43 was collapsing “alittle” and “a lot” categories, and for items 21, 32, 49, 56,and 57 “not at all” and “a little” were collapsed.

Rasch analysis of the Depressive Mood subscale

Fit statistics

Fit to the Rasch model was determined by consider-ing 3 main statistics. The first was a summary chi-squareof item-trait interaction. This statistic indicates whetherthe hierarchical ordering of the items on the subscale re-mains the same at different levels of the latent trait. TheDepressive Mood subscale initially did not fit the Raschmodel as indicated by the significant chi-square valuethat revealed lack of invariance of the items across thelatent construct. In addition, item fit and person fit statis-tics were examined. These statistics assess the residual ordivergence between the expected value and the actual.For each item, this statistic is based on the standardizedresiduals of the responses of all persons to the item. Bothstatistics are transformed to an approximate Z score andhence a mean of 0.0 and an SD of 1.0 would indicate aperfect fit to the model. In both cases, residuals rangingwithin ±2.50 and nonsignificant chi-square values (afterBonferroni adjustment) were acceptable. Since misfit of

items indicates a lack of the expected probabilistic re-lation between the individual item and other items inthe scale, item misfit may indicate that an item does notcontribute to the latent trait in question. Misfit for items1, 6, 63, 18, and 41 were indicated by residuals outsidethe criterion range and significant chi-square values. Thecontents of items 18 (feeling sad) and 41 (crying easily) werecombined into a subtest or “superitem” that eliminatedits misfit. Item 1 (headaches) with an extreme positiveresidual of 4.7 and item 6 (others do not understand yourproblems) with residual of 2.8 were removed accordingto that misfit, which seemed to show a lack of contribu-tion to the construct of Depressive Mood. Conversely,item 63 (having problems in general) was deleted becauseof its high negative residual of −2.7 that indicated re-dundancy. After these changes the subscale achieved asatisfactory fit to the model (Table 2). However, an SD ofperson fit was above 1.3, meaning that some participantshad patterns of performance that did not show internalconsistency.42 A close inspection of these participant re-sponses revealed that 32 subjects had residuals outsidethe criteria of ±2.5 (13 from France, 12 from Spain, and7 from the United Kingdom), 71.8% of them are fromthe TBI group.

Differential item functioning

To accomplish our second objective, we ran an anal-ysis of DIF by nationality and 4 more “person factors”(gender, etiology, time since injury, and age) as possiblesources of item misfit. Age was entered as 3 categories(≤25; 26–45; ≥46) and months since injury also as 3categories (≤12; 13–36; ≥37). An ANOVA of person-item deviation residuals with person factors and classintervals as factors was used. No DIF by any factor wasfound, adding support for the construct validity of theDepressive Mood subscale.

Unidimensionality

This was checked using a PCA of the person residu-als. Two subsets of items were defined by positive andnegative loadings on the first residual component af-ter the “Rasch factor” (analogous to the first principalcomponent)43 had been removed. These 2 subsets wereseparately fitted to the Rasch model and the person es-timates obtained. A paired t test was used to compareperson estimates as the best way to reject or confirmunidimensionality.44 The criterion for percentage of ttests outside the confidence interval at 5% should notexceed 5%.37

Significant differences were found only on 13 of the359 person estimates given by the 2 subsets when com-pared with person estimates given by the full subscale.This value represented a percentage lower than 5% and

LWW/JHTR HTR200133 December 11, 2010 0:17 Char Count= 0

Copyright c© Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Cross-cultural Validation of the EBIQ 5

TABLE 1 Factor loading

Items RF1 RF2 RF3

12. Feeling lonely, even with others 0.72418. Feeling sad 0.68230. Feeling lonely 0.65631. Feeling inferior to other people 0.633

9. Hopeless about the future 0.62917. Hiding your feelings from others 0.61447. Feeling of worthlessness 0.60663. Having problems in general 0.55811. Being confused 0.53948. Lack of interest in hobbies outside the home 0.53453. Feeling life is not worth living 0.52625. Having your feelings easily hurt 0.51751. Feeling tense 0.51358. Preferring to be alone 0.51159. Difficulty in making decisions 0.490

6. Others do not understand your problems 0.48232. Sleep problems 0.46429. Lack of interest in hobbies at home 0.44460. Losing contact with your friends 0.44333. Feeling inferior to other people 0.40116. Faintness or dizziness 0.38561. Lack of interest in current affairs 0.374

1. Headaches 0.37138. Lack of interest in your surroundings 0.34041. Crying easily 0.294

2. Get things done on time 0.67128. Problems with household chores 0.59354. Forgetting appointments 0.58815. Having to do things slowly 0.580

8. Unable to plan activities 0.56942. Difficulty finding your way in new surroundings 0.54926. Feeling unable to get things done 0.534

7. Everything is an effort 0.51821. Difficulty managing your finance 0.51546. Forgetting the day of the week 0.488

4. Trouble remembering things 0.48535. Difficulty in communicating what you want 0.48245. Lack of energy or being slowed down 0.48149. Needing help with personal hygiene 0.458

5. Difficulty participating in conversations 0.44855. Leaving others the initiative in conversations 0.43822. Trouble concentrating 0.43736. Unsure what to do in dangerous situations 0.43352. Acting inappropriately in dangerous situations 0.43223. Failing to notice other people’s mood 0.42656. Loss of sexual interest or pleasure 0.40720. Needing to be reminded about hygiene 0.33743. Being inclined to eat too much 0.30810. Having temper outbursts 0.71734. Shouting at people in anger 0.71444. Getting into quarrels easily 0.69919. Being “bossy” or dominating 0.67124. Feeling anger against other people 0.60727. Annoyance or irritation 0.59813. Mood swings without reason 0.57557. Throwing things in anger 0.54837. Being obstinate 0.54162. Behaving tactlessly 0.525

3. Reacting too quickly to what others say or do 0.51140. Mistrusting other people 0.48214. Feeling critical of others 0.45950. Restlessness 0.45239. Thinking only on yourself 0.392

Abbreviations: RF, rotated factor; RF1, item loading on rotated factor Depressive Mood; RF2, item loading on rotated factor Cognitive Dysfunction;RF3, item loading on rotated factor Poor Social And Emotional Self-regulation.

www.headtraumarehab.com

LWW/JHTR HTR200133 December 11, 2010 0:17 Char Count= 0

Copyright c© Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

6 JOURNAL OF HEAD TRAUMA REHABILITATION

TABLE 2 Item location order for the Depressive Mood subscale

Items Location SE Fit residual Chi square P

9. Hopeless about the future −0.391 0.08 −0.84 .04411. Being confused −0.344 0.087 −0.193 .34551. Feeling tense −0.312 0.085 0.447 .81559. Difficulty in making decisions −0.283 0.085 −0.38 .21917. Hiding your feelings from others −0.261 0.081 1.113 .21725. Having your feelings easily hurt −0.257 0.081 −0.856 .73047. Feeling of worthlessness −0.233 0.08 −1.295 .07530. Feeling lonely −0.232 0.085 0.212 .67660. Losing contact with your friends −0.192 0.08 1.73 .09318+41. Feeling sad and Crying easily (combined item) −0.162 0.06 0.547 .83931. Feeling inferior to other people −0.093 0.08 −0.262 .63833. Feeling uncomfortable in crowds −0.087 0.079 1.462 .49048. Lack of interest in hobbies outside the home −0.001 0.081 −0.318 .90429. Lack of interest in hobbies at home 0.151 0.083 0.689 .78258. Preferring to be alone 0.192 0.086 1.01 .77061. Lack of interest in current affairs 0.216 0.084 1.291 .38712. Feeling lonely, even with others 0.243 0.087 −1.538 .14753. Feeling life is not worth living 0.464 0.085 −1.254 .37232. Sleep problems 0.791 0.13 0.527 .36038. Lack of interest in your surroundings 0.794 0.092 −0.16 .73763. Having problems in general Misfit

6. Others do not understand your problems Misfit16. Faintness or dizziness Misfit

1. Headaches Misfit

Bold figure shows nonsignificant P values after Bonferroni adjustment.

then local independence of items and unidimensionalityof the subscale can be assumed.45

Person Separation Index

This is an estimate interpreted in a manner similar tothe Cronbach alpha reliability coefficient.26 However,Rasch analysis calculates the reliability in terms of thenumber of strata on the basis of their disorder severitythat can be distinguished in the distribution of respon-dents. Person separation index (PSI) refers to the abilityof the subscale to differentiate persons on the measuredvariable and indicates how many ranges there are in themeasurement continuum. Person separation index andthe Cronbach alpha are very close in value when the per-sons and items are well aligned. This index is acceptableat 0.8 (corresponding to a reliability coefficient of 0.8)because it is possible to distinguish 3 strata of persons(high, average, or low) separated with 95% confidenceinterval.46

A PSI of 0.90 was found indicating a good reliabilityof the Depressive Mood subscale. This value means agood separation of items along the construct and there-fore a sufficient power to discriminate among 4 groupsof respondents on the basis of their Depressive Moodseverity.46

Targeting

This refers to the extent to which the item symptomseverity has adequately targeted the disorder severity ofthe people in the sample. As the mean of items is the-oretically placed at the 0.0 point of the common logitscale, mean person location and SD will indicate thetargeting of the subscale.

Mean person location outcome was –0.455 (SD =1.23) indicating that the average Depressive Mood sever-ity of the sample is below the average of depressive symp-toms severity reflected by the items. This value is closeenough to the central zero logit to support a conclusionof “good targeting.”

Item calibration

Rasch analysis makes a calibration of items basedon likelihood of endorsement (symptom severity). In-spection of item location order of the DepressiveMood subscale is a way to assess its construct validity(Table 2).

Rasch analysis of cognitive Dysfunction subscale

A satisfactory fit to the model was achieved after delet-ing item 35 showing residual of 3.11 (Table 3). A uniformDIF by country was found for item 36 (being unsure what

LWW/JHTR HTR200133 December 11, 2010 0:17 Char Count= 0

Copyright c© Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Cross-cultural Validation of the EBIQ 7

TABLE 3 Item location order for the Cognitive Dysfunction subscale

Items Location SE Fit residual Chi square P

4. Trouble remembering things −1.393 0.089 −0.864 .51215. Having to do things slowly −1.212 0.089 −1.069 .17622. Trouble concentrating −0.978 0.086 −1.45 .1347. Everything is an effort −0.898 0.088 −0.56 .758

45. Lack of energy or being slowed down −0.677 0.086 0.049 .70755. Leaving others the initiative in conversations −0.655 0.082 2.357 .00626. Feeling unable to get things done −0.569 0.085 −1.555 .4762. Get things done on time −0.556 0.088 −1.843 .014

21. Difficulty managing your finance −0.452 0.120 −0.238 .5968. Unable to plan activities −0.398 0.084 −1.532 .0505. Difficulty participating in conversations −0.214 0.083 0.063 .458

42. Difficulty finding your way in new surroundings −0.147 0.082 −0.118 .91746. Forgetting the day of the week −0.142 0.082 −0.333 .29828. Problems with household chores −0.098 0.083 2.275 .05354. Forgetting appointments −0.015 0.084 −1.152 .68423. Failing to notice other people’s mood 0.336 0.087 0.277 .82936. Unsure what to do in dangerous situations 0.382 0.088 1.225 .16352. Acting inappropriately in dangerous situations 0.417 0.089 2.219 .09656. Loss of sexual interest or pleasure 0.854 0.140 0.079 .49343. Being inclined to eat too much 1.539 0.156 0.48 .24949. Needing help with personal hygiene 2.028 0.180 0.813 .01920. Needing to be reminded about hygiene 2.847 0.240 0.166 .41935. Difficulty in communicating what you want Misfit

Bold figure shows nonsignificant P values after Bonferroni adjustment.

to do in dangerous situations). This means that at the samelevel of disorder there is a constant difference betweenthe 3 country groups in the probability of endorsing thisitem across the trait (ANOVA main effect). The resultsshowed that the Spanish subsample had a higher proba-bility of endorsing item 36 than the others. Item 56 (lossof sexual interest or pleasure) showed uniform DIF by age,the bias being toward the older group that was less likelyto endorse this item than the other 2 age groups. A PSIof 0.88 allows distinguishing at least 3 strata of person-level symptoms reporting (mild, moderate, and severe).A mean person location of −0.536 (SD = 1.17) showedthat targeting of this subscale was good.

Rasch analysis of poor social and emotionalself-regulation

Significant chi-square probabilities indicated misfitsfor items 27 (annoyance or irritation) and 44 (getting intoquarrels easily). These items were removed after an un-successful try to combine them into a superitem. The 13remaining items fit the model, and the subscale was uni-dimensional (Table 4). However, 6 items showed DIF bycountry. This was uniform for items 10, 13, 24, 37, and50 and nonuniform for item 3. A PSI of 0.82 indicatedthat 3 strata of persons can be separated. Targeting to thesample was worse than other subscales (mean = −0.696;SD = 1.17).

DISCUSSION

The first aim of the study was to explore the EBIQ’sfactor structure using factor analysis and the overall psy-chometric properties of the items grouped on the factorsapplying Rasch analysis. Our second aim was to performa cross-cultural validity assessment by checking DIF dueto nationality factor in a sample of participants with ABIfrom 3 different cultures.

In relation to the missing data, according to previ-ous findings7 items 36 (unsure what to do in dangeroussituations) and 52 (acting inappropriately in dangerous situa-tions) were frequently unmarked. “Dangerous situations”might be an unclear and subjective concept that a largenumber of participants with ABI may feel unable to as-sess. In addition, items 39 (thinking only of yourself) and56 (loss of sexual interest or pleasure) might be reviewed formissing responses in the self-rating form. This could behard to disclose for participants and valid answers mightperhaps only be obtained in the close relative version ofthe EBIQ.

Factor structure of the questionnaire

The factors identified in the current analysis are verysimilar to the 3 main domain-based subscales found inthe original study of the EBIQ1 and those from previousstudies that used factor analysis.3,21–23 It must be takeninto account that our French subsample was extracted

www.headtraumarehab.com

LWW/JHTR HTR200133 December 11, 2010 0:17 Char Count= 0

Copyright c© Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

8 JOURNAL OF HEAD TRAUMA REHABILITATION

TABLE 4 Item location order for the Poor Social And Emotional Self-regulation subscale

Items Location SE Fit residual Chi square P

37. Being obstinate −0.741 0.083 1.15 .58910. Having temper outbursts −0.562 0.083 −1.908 .037

3. Reacting too quickly to what others say or do −0.499 0.082 0.552 .40750. Restlessness −0.395 0.082 1.42 .52440. Mistrusting other people −0.222 0.086 0.639 .84434. Shouting at people in anger −0.192 0.082 −1.211 .03214. Feeling critical of others −0.158 0.085 1.702 .55413. Mood swings without reason −0.157 0.085 −0.55 .33024. Feeling anger against other people 0.064 0.087 −1.565 .02219. Being “bossy” or dominating 0.248 0.085 0.586 .67762. Behaving tactlessly 0.275 0.093 −0.565 .21939. Thinking only on yourself 0.444 0.089 1.343 .02457. Throwing things in anger 1.895 0.183 −0.861 .32544. Getting into quarrels easily Misfit27. Annoyance or irritation Misfit

Bold figure shows nonsignificant P values after Bonferroni adjustment.

from the sample of Teasdale and colleagues1 and De-loche and colleagues.3 This might partially explain thesimilarities with the findings of these studies. However,the structure of the 3 factors that we obtained is alsosimilar to that identified by others21 with independentsamples of patients and even with healthy controls.23

The number of items of each factor is greater than thatin previous studies because we adopted a loading fac-tor of 0.40 instead of 0.50.3,21,22 The labels of the first2 factors (Depressive Mood and Cognitive Difficulties) wereadopted from others, while the third factor has been re-named Poor Social and Emotional Self-regulation since thisseems to summarize the content better than previouslabel (see Table 5).

Psychometric properties of the EBIQ factors

Rasch analysis offers several statistics for evaluatingthe psychometric properties of each factor as a poten-tial subscale. Regarding the Depressive Mood factor we

TABLE 5 Fit statistics and reliability and unidimensionality indices for the subscales

Item-trait Item fit Person fit Person Unidimensionalityinteraction residual residual separation independent

Subscale χ 2 (df); P mean (SD) mean (SD) index t test (95% CI)

Depressive Mood 102.103 (100); .42 0.097 (0.962) −0.162 (1.428) 0.90 3.62% (1.9–6.1)Cognitive Dysfunction 150.941 (110); .005a −0.032 (1.237) −0.207 (1.144) 0.88 4.93% (2.9–7.6)Poor Social And 90.718 (65); .01a 0.056 (1.224) −0.102 (1.140) 0.82 4.20% (2.3–6.8)

EmotionalSelf-regulation

Abbreviation: 95% CI, 95% confidence interval.aNonsignificant P values after Bonferroni adjustment.

found 4 items showing misfit whose contents do notreflect the latent construct. After deleting these itemsthe factor achieved all requirements of the Rasch modelfor measurement. For assessing construct validity, an in-spection of the calibration of items showed a properhierarchical order (Table 2) starting on the “easier” itemto endorse at the negative top of the logit scale (item9, Hopeless about the future) until the more “difficult” atthe positive bottom (item 38, Lack of interest in your sur-roundings). However, as with other instruments for theassessment of depression after ABI, some items mightoverlap with cognitive or somatic symptoms (item 11,Being confused or item 59, Difficulty in making decisions).47

This might explain the high SD found on the personfit residuals. Items 18 and 41 were combined into a su-peritem as the better way to retain these items into thesubscale. However, combining items increases user bur-den for scoring, thus rewording both items in a newsingle item might be a worthwhile approach in futurestudies. Also, the content and calibration of items on

LWW/JHTR HTR200133 December 11, 2010 0:17 Char Count= 0

Copyright c© Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Cross-cultural Validation of the EBIQ 9

the Cognitive Dysfunction factor support its constructvalidity as a subscale. In this way, the easier item to en-dorse was a memory deficit (item 4, Trouble rememberingthings) corresponding to the most frequently reportedcognitive symptom after ABI48,49 and the last items in-dicate an expected severe disability for basic living ac-tivities. Finally, psychometric findings for the Poor So-cial and Emotional Self-regulation factor indicate thatmost of the items work as a subscale with a unidimen-sional latent construct validated by scientific literature.50

In summary, Fit statistics, Item location order and Uni-dimensionality support validity of the 3 factors. An ap-propriate targeting to the sample and a PSI ranging from0.83 to 0.90 indicate a useful reliability from a clinicalpoint of view because it allows for discriminating at least3 severity levels (mild, moderate, and severe) of the latentconstruct of each subscale.38

As mentioned above, our subscales of the EBIQare very similar to those identified by Teasdale andcolleagues1 and Martin and colleagues21 (Depression,Cognitive, and Impulsivity/Social Difficulties sub-scales). Responses to those subscales of 226 participantsthat partially overlap with our British subsample7 wereanalyzed using Rasch analysis. Only the Depression sub-scale of Martin and colleagues21 deviated from modelexpectation. However, after Bonferroni adjustment eventhat subscale achieved Rasch model requirements.7

Cross-cultural validation of the EBIQ subscales

Differential item functioning by country factor fromRasch analysis was applied to assess cross-cultural valid-ity of the items on the EBIQ subscales. On the CognitiveDysfunction subscale only item 36 (being unsure what todo in dangerous situations) showed DIF by country. Asmentioned above, the rate of missing data for this itemmight indicate some comprehension difficulties; thus,this finding has to be taken with caution.

On the Poor Social and Emotional Self-regulationsubscale, 6 items showed DIF by country (3, 10, 13,24, 37, and 50). According to others, self-control or self-regulation is a key cross-cultural factor.33 Different self-regulations due to culture influence might lead to dif-ferent rating of social and emotional behaviors. Despitethe fact that this subscale met the other Rasch model re-quirements, the presence of DIF by country means thatculture might contribute to the scores on these items.Therefore, when pooling data from different countries,

items showing DIF should be removed or split. An itera-tive “top-down purification” splitting approach for itemsshowing uniform DIF has been applied elsewhere.29,51

Since this is the first cross-cultural study of the EBIQand taking into account the presence of confoundingsample characteristics other than nationality, these find-ings should to be taken in a preliminary and cautionarymanner.

In relation to others factor coded for the DIF analysis,no differences by gender, time since injury, or diagnos-tic was found. However, item 56 (loss of sexual interest orpleasure) showed DIF by age, being older than 46 whoyielded greater probabilities of endorsement.

Limitations and future trends

A limitation of this study comes from pooling data ofindividuals with TBI and stroke. This was addressed bychecking that DIF by diagnostic was not present. In addi-tion, Martin et al21,22 obtained very similar (but not iden-tical) items loading on the factors from separate samplesof participants with TBI and stroke. A second limitationis related to the sample. Although Rasch analysis allowsfor the detection of DIF within the current sample size,future studies should attempt to replicate these resultsusing greater sample size. Furthermore, methodologicalcontrol over confounding factors might be increased byavoiding the risks associated with retrospective recruit-ment and data reuse. Some improvements to the EBIQproposed herein might be tested in future researches in-cluding rewording of unclear concepts, testing of somedichotomous items, and designing of distinct versionsfor individuals with ABI and relatives.

CONCLUSION

Three factors of the EBIQ labeled Depressive Mood,Cognitive Dysfunction, and Poor Social and Emo-tional Self-regulation were extracted using factor analy-sis. After removing only 7 items, Rasch analysis showedfit to the model, unidimensionality, construct valid-ity, and good reliability of the 3 factors. Accord-ing to the Rasch model, the 3 factors achieved thepsychometric characteristics to be used as subscales ofthe EBIQ. However, only the Depressive and Cogni-tive subscales had cross-cultural validity for pooling datafrom the 3 samples of this international study.

REFERENCES

1. Teasdale T, Christensen A, Willmes K, et al. Subjective experiencein brain-injured patients and their close relatives: a EuropeanBrain Injury Questionnaire study. Brain Inj. 1997;11(8):543–563.

2. Deloche G, North P, Dellatolas G, et al. Le handicap des adultescerebroleses: le point de vue des patients et de leur entourage.Ann Readapt Med Phys. 1996;39(1):1–9.

3. Deloche G, Dellatolas G, Christensen A. The European Brain

www.headtraumarehab.com

LWW/JHTR HTR200133 December 11, 2010 0:17 Char Count= 0

Copyright c© Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

10 JOURNAL OF HEAD TRAUMA REHABILITATION

Injury Questionnaire. In: Christensen A-L, Uzzell BP, eds. Inter-national Handbook of Neuropsychological Rehabilitation. New York,NY: Kluwer Academic/Plenum Publishers; 2000:81–92.

4. Christensen A, Svendsen H, Willmes K. Subjective experi-ence in brain-injured patients and their close relatives: Euro-pean Brain Injury Questionnaire studies. Acta Neuropsychologica.2005;3(1/2):60–68.

5. Sopena S, Dewar B, Nannery R, Teasdale T, Wilson B. The Eu-ropean Brain Injury Questionnaire (EBIQ) as a reliable out-come measure for use with people with brain injury. Brain Inj.2007;21(10):1063–1068.

6. Souza L, Braga L, Filho G, Dellatolas G. Quality-of-life: childand parent perspectives following severe traumatic brain injury.Dev Neurorehabil. 2007;10(1):35–47.

7. Bateman A, Teasdale TW, Willmes K. Assessing construct va-lidity of the self-rating version of the European Brain InjuryQuestionnaire (EBIQ) using Rasch analysis. Neuropsychol Reha-bil. 2009;19(6):941–954.

8. Engberg AW, Teasdale TW. Psychosocial outcome followingtraumatic brain injury in adults: a long-term population-basedfollow-up. Brain Inj. 2004;18(6):533–545.

9. Mathiesen BB, Weinryb RM. Unstable identity and prefrontalinjury. Cogn Neuropsychiatry. 2004;9(4):249.

10. McCrimmon S, Oddy M. Return to work following moderate-to-severe traumatic brain injury. Brain Inj. 2006;20(10):1037.

11. Bjorkdahl A, Lundgren Nilsson A, Stibrant Sunnerhagen K. Thestructural properties of the European Brain Injury Question-naire. J Stroke Cerebrovasc Dis. 2004;13(3):122–128.

12. Holm S, Schonberger M, Poulsen I, Caetano C. Patients’ and rel-atives’ experience of difficulties following severe traumatic braininjury: the sub-acute stage. Neuropsychol Rehabil. 2009;19(3):444–460.

13. Williams W, Evans J, Willson B. Outcome measures for survivorsof acquired brain injury in day and outpatient neurorehabilita-tion programmes. Neuropsychol Rehabil. 1999;9(3):421–436.

14. Boman I, Lindstedt M, Hemmingsson H, Bartfai A. Cogni-tive training in home environment. Brain Inj. 2004;18(10):985–995.

15. Caracuel A, Verdejo-Garcıa A, Vilar-Lopez R, et al. Frontal be-havioral and emotional symptoms in Spanish individuals withacquired brain injury and substance use disorders. Arch Clin Neu-ropsychol. 2008;23(4):447–454.

16. Coetzer R, Rushe R. Post-acute rehabilitation following trau-matic brain injury: are both early and later improved outcomespossible? Int J Rehabil Res. 2005;28:361–363.

17. Dewar B, Wilson BA. Cognitive recovery from encephalitislethargica. Brain Inj. 2005;19(14):1285–1291.

18. Svendsen H, Teasdale T. The influence of neuropsychologicalrehabilitation on symptomatology and quality of life follow-ing brain injury: a controlled long-term follow-up. Brain Inj.2006;20(12):1295–1306.

19. Schonberger M, Humle F, Zeeman P, Teasdale TW. Patient com-pliance in brain injury rehabilitation in relation to awarenessand cognitive and physical improvement. Neuropsychol Rehabil.2006;16(5):561–578.

20. Svendsen H, Teasdale T, Pinner M. Subjective experience in pa-tients with brain injury and their close relatives before and aftera rehabilitation programme. Neuropsychol Rehabil. 2004;14:495–515.

21. Martin C, Viguier D, Deloche G, Dellatolas G. Subjective expe-rience after traumatic brain injury. Brain Inj. 2001;15(11):947–959.

22. Martin C, Dellatolas G, Viguier D, Willadino-Braga L, De-loche G. Subjective experience after stroke. Appl Neuropsychol.2002;9(3):148.

23. Santos ME, De Sousa L, Castro-Caldas A. Avaliacao da ex-periencia subjectiva em pessoas com lesao cerebral: Adaptacaopara a populacao portuguesa do European Brain Injury Ques-tionnaire (EBIQ). Analise Psicologica. 2001;19(2):219–236.

24. Rasch G. Probabilistic Models for Some Intelligence and AttainmentTests. Chicago, IL: University of Chicago Press; 1980.

25. Wilson M. Constructing Measures: An Item Response Modelling Ap-proach. London: Lawrence Erlbaum Associates; 2005.

26. Bode RK, Heinemann AW, Semik P. Measurement propertiesof the Galveston Orientation and Amnesia Test (GOAT) andimprovement patterns during inpatient rehabilitation. J HeadTrauma Rehabil. 2000;15(1):637–655.

27. Lawton G, Lundgren-Nilsson A, Biering-Sorensen F, et al. Cross-cultural validity of FIM in spinal cord injury. Spinal Cord.2006;44(12):746–752.

28. Embretson SE, Reise SP. Item Response Theory for Psychologists.Mahwah, NJ: Lawrence Erlbaum Associates; 2000.

29. Tennant A, Penta M, Tesio L, et al. Assessing and adjustingfor cross-cultural validity of impairment and activity limitationscales through differential item functioning within the frame-work of the Rasch model. Med Care. 2004;42(1):1–37.

30. Tesio L. Measuring behaviours and perceptions: Rasch analysis asa tool for rehabilitation research. J Rehabil Med. 2003;35(3):105–115.

31. Prigatano G, Ogano M, Amakusa B. A cross-cultural study onimpaired self-awareness in Japanese patients with brain dysfunc-tion. Neuropsychiatry Neuropsychol Behav Neurol. 1997;10(2):135–143.

32. Chen C. Transcultural expression of subcortical vascular disease.J Neurol Sci. 2004;226(1/2):45–47.

33. Puente A, Agranovich AV. The cultural in cross-cultural neu-ropsychology. In: Goldstein G, Beers S., eds. ComprehensiveHandbook of Psychological Assessment. Vol 1: Intellectual and neu-ropsychological assessment. Hoboken, NJ: John Wiley & Sons;2004:321–332.

34. Linacre J. Sample size and item calibration stability. Rasch MeasTrans. 1994;7(4).

35. Temkin NR, Corrigan JD, Dikmen SS, Machamer J. Socialfunctioning after traumatic brain injury. J Head Trauma Rehabil.2009;24(6):460–467.

36. Hagquist C, Bruce M, Gustavsson J. Using the Rasch model innursing research: an introduction and illustrative example. Int JNurs Stud. 2009;46(3):380–393.

37. Tennant A, Conaghan P. The Rasch measurement model inrheumatology: what is it and why use it? When should it beapplied, and what should one look for in a Rasch paper? Arthri-tis Care Res. 2007;57(8):1358–1362.

38. Linacre J. Understanding Rasch measurement: optimizing ratingscale category effectiveness. J Appl Meas. 2002;3(1):85–106.

39. Andrich D, Lyne A, Sheridan B, Luo G. RUMM2020. Perth,Australia: RUMM Laboratory; 2003.

40. Gorsuch R. Exploratory factor analysis: its role in item analysis.J Pers Assess. 1997;68(3):532–560.

41. Davidson M. Rasch analysis of three versions of the OswestryDisability Questionnaire. Man Ther. 2008;13(3):222–231.

42. Wright BD, Masters G. Rating Scale Analysis. Chicago, IL: MESAPress; 1982.

43. Smith RM, Miao CY. Assessing unidimensionality for Raschmeasurement. In: Wilson M, ed. Objective Measurement: Theoryinto Practice.Vol 2. Norwood NJ: Ablex; 1994:316–327.

44. Tennant A, Pallant JF. Unidimensionality matters! Rasch MeasTrans. 2006;20(1):1048–1051.

45. Smith R. Fit analysis in latent trait measurement models. J ApplMeas. 2000;1(2):199–218.

46. Fisher WJ. Reliability statistics. Rasch Meas Trans. 1992;6(3):238.47. Dikmen SS, Bombardier CH, Machamer JE, Fann JR, Temkin

LWW/JHTR HTR200133 December 11, 2010 0:17 Char Count= 0

Copyright c© Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Cross-cultural Validation of the EBIQ 11

NR. Natural history of depression in traumatic brain injury. ArchPhys Med Rehabil. 2004;85(9):1457–1464.

48. Dikmen S, Corrigan J, Lewin H, et al. Cognitive outcomefollowing traumatic brain injury. J Head Trauma Rehabil.2009;24(6):430–438.

49. Tatemichi TK, Desmond DW, Stern Y, Paik M, Bagiella E. Cog-nitive impairment after stroke: frequency, patterns, and rela-

tionship to functional abilities. J Neurol Neurosurg Psychiatry.1994;57:202–207.

50. Tucker DM, Luu P, Pribram KH. Social and emotional self-regulation. Ann N Y Acad Sci. 1995;769:213–239.

51. Tennant A, Pallant J. DIF matters: a practical approach to testif Differential Item Functioning makes a difference. Rasch MeasTrans. 2007;20(4):1082–1084.

www.headtraumarehab.com