Rohling’s Interpretive Rohling’s Interpretive Method: How Can a Flexible Method: How Can a Flexible Battery Perform Like a Battery Perform Like a Fixed Battery Fixed Battery Martin L. Rohling, Ph.D. Martin L. Rohling, Ph.D. Associate Professor Associate Professor Department of Psychology Department of Psychology University of South Alabama University of South Alabama
138
Embed
Rohlings Interpretive Method: How Can a Flexible Battery Perform Like a Fixed Battery Martin L. Rohling, Ph.D. Associate Professor Department of Psychology.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Rohling’s Interpretive Method: Rohling’s Interpretive Method: How Can a Flexible Battery How Can a Flexible Battery
Perform Like a Fixed BatteryPerform Like a Fixed Battery
Martin L. Rohling, Ph.D.Martin L. Rohling, Ph.D.
Associate ProfessorAssociate Professor
Department of PsychologyDepartment of Psychology
University of South AlabamaUniversity of South Alabama
January 13, 2007 Rohling - CCPN Orlando, FL2
Clinical vs. Mechanical DiagnosisClinical vs. Mechanical Diagnosis
Much research has been conducted since Meehl (1954) found clinical judgment to be less accurate than mechanical or “actuarial” judgement
e.g., Dawes, Faust, & Meehl (1989); Filskov (1981); Garb (1989); Garb (1994); Garb (1998); Grove et al. (2000); Sawyer (1966); and Wedding & Faust (1989)
Such results influential in causing NPs to turn to different versions of the HRB (Russell, 1998).
“Batteries” have been defined as the method by which one can avoid the clinical errors highlighted by Meehl an others, using “actuarial” rules for diagnosis (Russell, 1995; Russell et al., 2005).
January 13, 2007 Rohling - CCPN Orlando, FL3
Rohling’s Interpretive Method Rohling’s Interpretive Method (RIM): Development History(RIM): Development History
Conducted several meta-analysis with Dr. Laurence Binder at the Portland, OR – VA
The last of these focused on the residual cognitive effects of mild head injury. Binder, Rohling, & Larrabee (1997)
Binder et al. grouped effect sizes (ES) into domains of neuropsychological functioning based on factor analytic studies. e.g., Leonberger, Nicks, Larrabee, & Goldfader (1992)
January 13, 2007 Rohling - CCPN Orlando, FL4
RIM Generated fromRIM Generated fromMeta-Analytic ProceduresMeta-Analytic Procedures
Meta-analysis (MA) combines effect sizes (ES) across samples assuming that they all sample the population M for the particular effect of interest. Common method ES calculation is a standardized
mean difference score (e.g., Glass’ delta). delta = difference between con. & exp. group’s M’s
divided by con. group’s SD. delta analogous to Z score - linear equivalent of T
score used in clinical neuropsychology
January 13, 2007 Rohling - CCPN Orlando, FL5
RIM Generated fromRIM Generated fromMeta-Analytic ProceduresMeta-Analytic Procedures
Binder et al. (1997) combined ES’s generated from various tests into cognitive domains.
Why not similarly combine ES’s, or T scores, from a single patient into cognitive domains in the same way that it is accomplished in MA. Each test score is treated as a ES that reflects the
individual’s ability within a domain. ES can be combined based on homogeneity of
variance, so as to avoid combining apples and oranges.
January 13, 2007 Rohling - CCPN Orlando, FL6
Introduction to the RIM AnalysisIntroduction to the RIM Analysis
Flexible battery (multiple measure) use: Is the most frequently cited model of assessment
among neuropsychologists. Only 7% of neuropsychologists use a fixed
battery (Rabin et al, 2006, ACN). Regarding the suitability, practicality, and
usefulness of any fixed battery: “We know of no batteries that fully satisfy these
Advantages of Flexible BatteryAdvantages of Flexible Battery
Dynamic & responsive to clinician’s needs Covers 1 or many domains “Flexible”, can be adapted for each patient Can “oversample” domains Well suited for hypothesis-driven approach
January 13, 2007 Rohling - CCPN Orlando, FL8
Potential Problems with aPotential Problems with aFlexible BatteryFlexible Battery
Inflated error rates Multicollinearity Weighting decision problems Unknown veracity/reliability of sets of tasks Human judgment errors
Present a set of procedures that allows for a quantitatively-based comparison of an overall battery of measures. Non-specific to battery measures
themselves. Can be used for any individual patient.
Demonstrate importance and practicality of use of established statistical indices. (e.g., alpha, beta, effect size).
January 13, 2007 Rohling - CCPN Orlando, FL13
Today’s Intent (cont’d)Today’s Intent (cont’d)
Present a data format for any set of measures to be inspected at: Global level (OTBM) Domain level (DTBM) Test measure level (ITBM)
Present a series of calculations to assist in the generation of these indices.
Present Steps in conjunction with clinical judgment from an informed position.
January 13, 2007 Rohling - CCPN Orlando, FL14
Common RIM Domains of Common RIM Domains of FunctioningFunctioning
Symptom Validity (SV) Tests Emotional / Personality (EP) Measures Meta-Cognition, Pain, or other self-ratings Estimated Premorbid General Ability (EPGA) Test Battery Means
Brief of RIM Steps:Brief of RIM Steps:• There are 24 steps to the RIM process
17 calculation steps:Advice on design of the batteryCalculation of summary statisticsGeneration of graphic displays
7 interpretative steps.• Detail a systematic procedure for use of the
statistical summary table and graphic displays to: Assess and verify summary data. Identify strengths/limitations of current data. Obtain a reliable diagnosis. Develop tx plans based on sound judgments.
• We briefly review each step in just a moment.
January 13, 2007 Rohling - CCPN Orlando, FL18
Support for the RIM ProcessSupport for the RIM Process
Rational support/reasoning: Reduce clinical judgment errors. The RIM is a Process, not a program. Rather, the RIM is a way of formalizing
thinking & interpretation of individual case data.
This is operationalizing what many flexible battery clinicians are already doing in their head.
January 13, 2007 Rohling - CCPN Orlando, FL19
Support for the RIM Process:Support for the RIM Process:Specific AdvantagesSpecific Advantages
Psychometric properties at level with fixed, co-normed batteries, without their limitations.
Flexibility of test selection.
Flexibility of theoretical view of cognition (domain structure)
January 13, 2007 Rohling - CCPN Orlando, FL20
Support for the RIM Process:Support for the RIM Process:Specific AdvantagesSpecific Advantages
Quantitatively support your conclusions and interpretations Statistical evaluation Measure of confidence in findings Measure of limitations of findings Ability to present data at different
levels of interpretation Greater defensibility
January 13, 2007 Rohling - CCPN Orlando, FL21
The RIM has a Set of The RIM has a Set of Procedure or Specific StepsProcedure or Specific Steps
January 13, 2007 Rohling - CCPN Orlando, FL22
RIM Steps 1-4: Summary DataRIM Steps 1-4: Summary Data
1) Design & administer battery. Use well standardized recently normed tests.
2) Estimate premorbid general ability. Use Reading (WTAR), Regression (OPIE-III), &
academic records (rank, SAT, ACT).
3) Convert test scores to a common metric. We recommend T scores, but z or SS OK too.
4) Assign scores to domains. Factor analysis to support assignment
(Tulsky et al., 2003)
January 13, 2007 Rohling - CCPN Orlando, FL23
RIM Steps 5-8: Summary DataRIM Steps 5-8: Summary Data
5) Calculate domain M, sd, & n.6) Calculate test battery means (TBM).
Overall TBM – All scores, large N & high power. Domain TBM – Avoids domain over weighting.
(e.g., attention & memory). Instrument TBM – One score per norm sample.
7) Calculate p for heterogeneity. Have you put “apples & oranges” together?
8) Determine categories of impairment. Recommend using of Heaton et al. (2003).
January 13, 2007 Rohling - CCPN Orlando, FL24
RIM Steps 9-12: Summary DataRIM Steps 9-12: Summary Data
9) Determine % of test impaired. Analogous to Halstead Impairment Index
# scores below cutoff / total # of of scores
10) Calculate ES for all summary stats. Use Cohen’s d = (Me – Mc) / SD pooled
11) Calculate confidence interval for stats. 90% CI = 1.645 x SEM
12) Upper limit of performance for impair. Look for overlap between 90% CI of EPGA
(lower) & Summary Stats (upper)
January 13, 2007 Rohling - CCPN Orlando, FL25
RIM Steps 13-17: Summary DataRIM Steps 13-17: Summary Data
13) Conduct one-sample t tests. Use EPGA as reference point
14) Conduct a between-subjects ANOVA. Looking for strengths & weaknesses
15) Conduct power analyses. Only needed for those NS differences
16) Sort scores for visual inspection.
17) Graphically display summary statistics.
January 13, 2007 Rohling - CCPN Orlando, FL26
RIM Steps 18-20: InterpretationRIM Steps 18-20: Interpretation
18) Assess battery validity. Examine the Symptom Validity scores. Caution in accepting low power results. Look at heterogeneity of summary stats.
1. Normative sample unrepresentative of patient.2. Scores assigned to wrong domain. 3. Inconsistent performance on construct measures.
19) Examine influence of psychopathology. Examine scores for heterogeneity.
20) Check OTBM, DTBM, & ITBM impaired.
January 13, 2007 Rohling - CCPN Orlando, FL27
RIM Steps 21-24: InterpretationRIM Steps 21-24: Interpretation
21) Examine strengths/weaknesses looking at:1. Confidence intervals overlap.2. Results from one-sample t tests.3. Results of ANOVA.4. %TI show differences otherwise not evident.
23) Explore Type II errors –need more tests?24) Examine sorted T-scores
Look for patterns missed by summary stats.
January 13, 2007 Rohling - CCPN Orlando, FL28
RIM Sample Case 1: Obvious TBIRIM Sample Case 1: Obvious TBI Age: 37 Handed: Left Race: Euro-American Sex: Female Ed: 14 years Occup: Nursing Marital: Sep. 10 yrs Living: Camper in
parent’s backyard
Reason for Referral:TBI in head-on boat accident. Propeller hit pt in right parietal-occipital lobe (LOC = 7 days; GCS = 3). Eval. to determine capacity for medical & financial decisions, parenting skills, occupational prognosis, & disability status. Significant emotional, behavioral, occupational, and social problems pre-TBI.
January 13, 2007 Rohling - CCPN Orlando, FL29
RIM Sample Case 1: Obvious TBIRIM Sample Case 1: Obvious TBI
0
10
20
30
40
50
60
70
80
90
100S
V
EP
EP
GA
OT
BM
DT
BM
ITB
M
VC
PO
EF
AM
L
VM
L
AW PS
PM
LA
SP
January 13, 2007 Rohling - CCPN Orlando, FL30
RIM Sample Case 1: Obvious TBIRIM Sample Case 1: Obvious TBIRIM Summary Statistics: T-scores, variances, ES, & power.
Column # 1 2 3 4 5 6 7 8 9 10 11 12 R o w
Cognitive Domain M sd n Hetero. p value Classify % TI ES CI PreM
LOCLOC nn ESES TT Mean Diff.Mean Diff.(EPGA – OTBM)(EPGA – OTBM)
Trauma ControlTrauma Control 121 .00 48.1 ---
G1: < 1hrG1: < 1hr 299 -.11 46.5 1.5
G2: 1-24 hrG2: 1-24 hr 152 -.22 44.2 3.8
G3: 1-6 dayG3: 1-6 day 99 -.33 42.7 5.6
G4: 7-13 dayG4: 7-13 day 56 -.68 38.1 10.2
G5: 14-28 dayG5: 14-28 day 46 -1.29 29.4 19.1
G6: > 28 dayG6: > 28 day 43 -1.49 27.7 20.6
January 13, 2007 Rohling - CCPN Orlando, FL33
Return to Work Study: OTBM’s for Return to Work Study: OTBM’s for 4 Groups of TBI Survivors4 Groups of TBI Survivors
GroupGroup nn MM SDSD ESES
DisabledDisabled 17 32.8 6.4 -2.29
UnemployedUnemployed 96 39.5 6.1 -1.01
Below PreviousBelow Previous 32 43.3 4.6 -.36
At PreviousAt Previous 137 45.1 5.2 -.45
January 13, 2007 Rohling - CCPN Orlando, FL34
RIM Sample Case 1: Obvious TBI RIM Sample Case 1: Obvious TBI Normal Distribution of T ScoresNormal Distribution of T Scores
0
5
10
15
20
25
Per
cen
t
0 10 20 30 40 50 60 70 80 90 100OTBM
January 13, 2007 Rohling - CCPN Orlando, FL35
RIM Sample Case 2: Subtle DiabetesRIM Sample Case 2: Subtle Diabetes
Reason for Referral: 2 yrs dangerous work habits. Eval to see if atrial fib & Type II diabetes impairs cognition. Hospitalized “TIA-like” Sx. Admitted to problems for 20 yrs, cardiac dysrhythmia & bradycardia, pacemaker, blood sugar difficult to manage, & family Hx of heart disease & diabetes.
Age: 55 Handed: Right Race: Euro-American Sex: Male Ed: 13 years Occup: Mechanic Marital: Married 20 yr Living: at home w/wife
January 13, 2007 Rohling - CCPN Orlando, FL36
RIM Sample Case 2: Subtle DiabetesRIM Sample Case 2: Subtle Diabetes
0
10
20
30
40
50
60
70
80
90
100S
V
EP
EP
GA
OT
BM
DT
BM
ITB
M
VC
PO EF
AM
L
VM
L
AW PS
PM LA
SP
January 13, 2007 Rohling - CCPN Orlando, FL37
RIM Sample Case 2: Subtle DiabetesRIM Sample Case 2: Subtle DiabetesRIM Summary Statistics: T-scores, variances, ES, & power.
Column # 1 2 3 4 5 6 7 8 9 10 11 12 R o w
Cognitive Domain M sd n Hetero. p value Classify % TI ES CI PreM
RIM Sample Case 2: Subtle Diabetes RIM Sample Case 2: Subtle Diabetes Normal Distribution of T ScoresNormal Distribution of T Scores
0
5
10
15
20
25
Perc
en
t
0 10 20 30 40 50 60 70 80 90 100OTBM
Histogram
January 13, 2007 Rohling - CCPN Orlando, FL39
RIM Critiques: Concern 1RIM Critiques: Concern 1
The method of calculating the standard deviations (SDs) for summary statistics and domain scores is incorrect. Since many of the remaining steps of the
RIM depend on the use of these SDs, this error is magnified in the subsequent steps.
SDs statistically can not exceed 9.99 and are more likely to be around 6.4
January 13, 2007 Rohling - CCPN Orlando, FL40
Response 1: RIM Response 1: RIM MsMs 4 Datasets 4 Datasets
0
5
10
15
20
25
30
Perc
ent
20 25 30 35 40 45 50 55 60 65 70Mean OTBM WAIS-R
Mean OTBM for Dataset 1 (WAIS-R)
0
5
10
15
20
25
30
Perc
ent
20 25 30 35 40 45 50 55 60 65 70Mean OTBM Meyers
Mean OTBM for Dataset 3 (Meyers)
0
5
10
15
20
25
30
Pe
rce
nt
20 25 30 35 40 45 50 55 60 65 70Mean OTBM Green
Mean OTBM for Dataset 2 (Green)
0
5
10
15
20
25
30
Perc
ent
20 25 30 35 40 45 50 55 60 65 70Mean OTBM HRB
Mean OTBM for Dataset 4 (HRB)
January 13, 2007 Rohling - CCPN Orlando, FL41
Inter-Individual Inter-Individual Ms Ms && SDs SDs
N Mn SD
Dataset 1 Psych Pts
WAIS-R 457 43.2 7.2
WAIS 150 45.0 9.1
Dataset 2 (Green) 904 44.8 7.3
Dataset 3 (Meyers) 1,734 42.0 7.3
Dataset 4 (HRB) 114 42.8 6.8
Total 4 Samples 3,359 43.1 7.4
January 13, 2007 Rohling - CCPN Orlando, FL42
Response 1: RIM Response 1: RIM SDsSDs 4 Datasets 4 Datasets
0
5
10
15
20
25
Perc
en
t
0 2 4 6 8 10 12 14 16 18 20WAIS-R OTBM-SD
OTBM-sd for Dataset 1 (WAIS-R)
0
5
10
15
20
25
Pe
rce
nt
0 2 4 6 8 10 12 14 16 18 20Meyers OTBM-SD
OTBM-sd for Dataset 3 (Meyers)
0
5
10
15
20
25
Perc
en
t
0 2 4 6 8 10 12 14 16 18 20Green's OTBM-SD
OTBM-sd for Dataset 2 (Green)
0
5
10
15
20
25
Pe
rce
nt
0 2 4 6 8 10 12 14 16 18 20HRB OTBM-SD
OTBM-sd for Dataset 4 (HRB)
January 13, 2007 Rohling - CCPN Orlando, FL43
Intra-Individual Intra-Individual MsMs & & SDsSDs
N Mn SD % > 9.99
Dataset 1 Psych Pts
WAIS-R 457 6.8 2.0 7%
WAIS 150 7.4 2.2 10%
Dataset 2 (Green) 904 11.4 2.9 65%
Dataset 3 (Meyers) 1,734 11.9 2.9 56%
Dataset 4 (HRB) 114 10.6 2.4 61%
Total 4 Samples 3,359 10.8 2.8 50%
January 13, 2007 Rohling - CCPN Orlando, FL44
RIM Critiques: Concern 2RIM Critiques: Concern 2
More false-positives then clinical judgment. Palmer et al. (2004) expressed concern
that We failed to distinguish “statistical” from
“clinical” significance. This failure is a critical error that precludes
the prudent clinician from using the RIM.
January 13, 2007 Rohling - CCPN Orlando, FL45
Response 2: RIM vs. Manual Response 2: RIM vs. Manual Detecting Differences – Overall %Detecting Differences – Overall %
% of Total S’s MANUAL METHOD
RIM t TEST VIQ-PIQ: NS VIQ-PIQ: Sig. Marginal M’s
VIQ-PIQ: NS 54% 23% 78%
VIQ-PIQ: Sig. 1% 21% 22%
Marginal M’s 55% 45% 100%
January 13, 2007 Rohling - CCPN Orlando, FL46
Response 2: RIM vs. Manual Response 2: RIM vs. Manual Detecting Differences – Detecting Differences – ESsESs
Means (SDs) MANUAL METHOD
RIM t TEST VIQ-PIQ: NS VIQ-PIQ: Sig.Marginal
M’s
VIQ-PIQ: NS .38 (.30) .80 (.41) .50 (.39)
VIQ-PIQ: Sig. 1.58 (.82) 1.70 (.86) 1.69 (.85)
Marginal M’s .40 (.37) 1.22 (.80) .90 (.71)
January 13, 2007 Rohling - CCPN Orlando, FL47
Response 2: RIM vs. ManualResponse 2: RIM vs. ManualDetecting Differences ScoresDetecting Differences Scores
Means (SDs) MANUAL METHOD
RIM t TEST VIQ-PIQ: NS VIQ-PIQ: Sig. Marginal M’s
VIQ-PIQ: NS 3.9 (2.5) 13.2 (3.7) 6.7 (5.2)
VIQ-PIQ: Sig. 6.7 (0.8) 19.0 (6.5) 16.9 (8.5)
Marginal M’s 4.0 (2.5) 15.9 (6.0) 9.3 (7.4)
January 13, 2007 Rohling - CCPN Orlando, FL48
RIM Critiques: Concern 3RIM Critiques: Concern 3
Clinicians who use the RIM will: Idiosyncratically assign scores to cognitive
domains. This will result in low inter-rater reliability in
analysis & diagnosis.
January 13, 2007 Rohling - CCPN Orlando, FL49
RIM Critiques: Concern 4RIM Critiques: Concern 4
Scores on domains are unit weighted, which introduces error.
Willson & Reynolds (2004) said scores load on multiple domains. Assignment to domains & weights depend on: Battery of tests administered. Patients whose test scores are being
examined.
January 13, 2007 Rohling - CCPN Orlando, FL50
Response 4: Cross-Valid. Unit WtsResponse 4: Cross-Valid. Unit Wts
Conducted 4 multiple reg. on 457 pts’ WAIS-R. Split sample in ½ - assess shrinkage.
Regressed patients’ verbal subtests onto PIQ. Generated ideal weights for the 1st ½ of sample.
Used wts to predict PIQs in the 2nd ½ of sample. Pre-PIQs regressed on actual PIQs 2nd ½ sample.
Also, generated weights for the 2nd ½ of sample. Use Pre-PIQ’s regress on actual PIQs 1st ½ sample.
Repeated, except performance subtests predict VIQ split sample ½ & generate same statistics as before.
January 13, 2007 Rohling - CCPN Orlando, FL51
Response 4: Cross-Valid. Unit WtsResponse 4: Cross-Valid. Unit Wts
Purpose of these procedures: How much variance in wts. is sample specific. Amount of shrinkage using cross-validated wts. Shrinkage error compared to error introduced by
using “unit wts” vs. “ideal wts.” Results: 98% of the variance accounted for with
unit wts. Compared to ideal weights. Support use of unit wts. Rather than ideal wts.
See, Dawes, R. M. (1979).
January 13, 2007 Rohling - CCPN Orlando, FL52
RIM Critiques: Concern 5RIM Critiques: Concern 5
Multiple measures used to generate composite scores: Results in less accurate estimates of
the cognitive domains.
January 13, 2007 Rohling - CCPN Orlando, FL53
Response 5: Estimate FSIQ Using Response 5: Estimate FSIQ Using Scaled Score Means’sScaled Score Means’s
Diff. between 1 predictor and additional predictors
No overlap in normative samples. Worst-case condition, generally administer
instruments (e.g., WAIS-III) with OTBMs generated from “co-normed” variables. Meyers & Rohling test-retest reliability of .86.
When different norms used, often gave same instruments (e.g., AVLT or RCFT)
No instrument used OTBM-1 included OTBM-2 Heaton et al.’s (2001) - schizophrenic pts.
Obtained a test-retest reliability of .97. Comparing 2 identical batteries, not worst-case.
January 13, 2007 Rohling - CCPN Orlando, FL60
RIM Critiques: Concern 8RIM Critiques: Concern 8
The RIM will result in an undue inflation of clinicians’ confidence. Such overconfidence results in more error in
a interpretation, not less.
January 13, 2007 Rohling - CCPN Orlando, FL61
RIM vs. Tulsky et al. (2003): Case 1RIM vs. Tulsky et al. (2003): Case 1WAIS & WMS Battery Full Battery
Index or Domain ScoresDisc.
Model RIM
Model # RIM Model #
S EPGA1 (WTAR) 103 102 1 106 5
U Overall TBM2 (FSIQ) (112) 104 18 96 I 70
M Domain TBM2 --- 105 6 98 7
M Inst TBM2 (FSIQ/GMI) --- 104 2 93 I 14
1 Verbal Comp (VCI) 120 * 118 * 3 110 6
2 Percept Organ (POI) 121 * 117 * 3 107 5
3 Attent/Work Mem (WMI) 105 105 2 95 I* 9
4 Process Speed (PSI) 93 ** 95 ** 2 86 I* 5
5 Aud Mem & Learn (AMI) 97 ** 97 ** 4 81I***** 18
6 Vis Mem & Learn (VMI) 94 95 4 98 13
7 Executive Function (EF) --- --- 0 100 I* 15
8 Psycho-Motor (PM) --- --- 0 111 6
January 13, 2007 Rohling - CCPN Orlando, FL62
RIM vs. Tulsky et al. (2003): Case 2RIM vs. Tulsky et al. (2003): Case 2WAIS & WMS Battery Full Battery
Index or Domain Scores Disc. Model RIM Model # RIM Model #
S EPGA1 (WTAR) 125 120 1 117 5
U Overall TBM2 (FSIQ) (119) 103 18 96 I 53
M Domain TBM2 --- 105 6 95 I 7
M Inst TBM2 (FSIQ/GMI) --- 106 2 93 I 11
1 Verbal Comp (VCI) 124 122 3 117 5
2 Percept Organ (POI) 95 97 3 97 3
3 Attent/Work Mem (WMI) 108 108 2 102 7
4 Process Speed (PSI) 98 92 2 83 5
5 Aud Mem & Learn (AMI) 111 110 4 99 I 9
6 Vis Mem & Learn (VMI) 104 103 4 89 I 10
7 Executive Function (EF) --- --- 0 94 I 14
8 Psycho-Motor (PM) --- --- 0 77 6
January 13, 2007 Rohling - CCPN Orlando, FL63
Summary of the Rohling Summary of the Rohling Interpretive Method of Statistical Interpretive Method of Statistical
Analysis of Individual Analysis of Individual Neuropsychological Test DataNeuropsychological Test Data
January 13, 2007 Rohling - CCPN Orlando, FL64
Summary of RIM StepsSummary of RIM Steps
24 total steps to the process 17 calculation steps
Battery Design Calculation of summary statistics Generation of graphic displays
7 interpretative steps Use of summary table and graphic displays to:
Assess and verify summary data Identify strengths/limitations of current data Obtain a reliable diagnosis Develop tx plans based on clinical judgments.
January 13, 2007 Rohling - CCPN Orlando, FL65
Summary of RIM AdvantagesSummary of RIM Advantages
Formulize thinking interpretation of data: Operationalize what you already do.
Reduce judgment errors thru RIM Process. Take advantage of psychometric properties
at level with fixed, co-normed batteries. Allows flexibility of test selection. Allows flexibility of theoretical view of
cognition (e.g., domain structure)
January 13, 2007 Rohling - CCPN Orlando, FL66
Summary of RIM Advantages cont’dSummary of RIM Advantages cont’d
Gives Quantitative support for your conclusions and interpretations Statistical evaluation Measure of confidence in findings Measure of limitations of findings Ability to present data at different levels of
interpretation
Equals greater defensibility
January 13, 2007 Rohling - CCPN Orlando, FL67
Our RIM Cautions/ConcernsOur RIM Cautions/Concerns
Does not “replace” clinical judgment, rather, informs clinical judgment. This still means CJ errors are possible.
Susceptibility T-Scores to distrib. deviance
Process, not program Pre-morbid ability estimates
Domain selection, test placement
January 13, 2007 Rohling - CCPN Orlando, FL68
RIM is Not Alone Out There!
Dawn Flanagan, Ph.D., at St. Johns University in New York independently developed a similar method The Cattell-Horn-Carroll (CHC) Cross Battery Approach.
Second edition of Essentials of Cross-Battery Assessment (Flanagan, Ortiz, & Alfonso, in press) is due out in March, which explains her method, along with co-authors Some of her work can also be found on the website by
Dumont-Willis.
Published Research Findings Published Research Findings Using the RIMUsing the RIM
1) RIM vs. HRB
2) Variance Accounted for by SVT
3) Effect of Depression on NP Results
4) Prediction of Employment after TBI
January 13, 2007 Rohling - CCPN Orlando, FL70
RIM of HRB: OTBM vs. HIIRIM of HRB: OTBM vs. HII
Heaton et al.’s (1991) HRB norms for OTBM T Score (M=50, sd=10)
OTBM r with HII = -.79 (p < .0001) 62% variance account.
Over predicts low Under predicts high
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
HII
20 25 30 35 40 45 50 55 60
OTBM
January 13, 2007 Rohling - CCPN Orlando, FL71
RIM of HRB: OTBM vs. GNDSRIM of HRB: OTBM vs. GNDS
OTBM r with GNDS = -.87 76% variance acc. OTBM neither under
nor over predicts across range of GNDS
Intercept impairment is T Score = 46.0 Reitan & Wolfson (1993)
(GNDS = 29)0
10
20
30
40
50
60
70
80
GNDS
20 25 30 35 40 45 50 55 60
OTBM
January 13, 2007 Rohling - CCPN Orlando, FL72
RIM of HRB: OTBM’s RIM of HRB: OTBM’s Relationship to Global IndicesRelationship to Global Indices
INDICES OF FUNCTIONINDICES OF FUNCTION Correlation CoefficientCorrelation Coefficient
Halstead Impairment IndexHalstead Impairment Index .79
Average Impairment RatingAverage Impairment Rating .90
Global Neuro. Deficit ScaleGlobal Neuro. Deficit Scale .87
RIM: Domain TBMRIM: Domain TBM .99
RIM: Instrument TBMRIM: Instrument TBM .95
RIM: % Tests ImpairedRIM: % Tests Impaired .96
January 13, 2007 Rohling - CCPN Orlando, FL73
RIM of HRB: Diagnostic RIM of HRB: Diagnostic Classification Using the HIIClassification Using the HII
Rohling, M. L., Green, P., Allen, L. M., & Iverson, G. L. (2002). Depressive symptoms and neurocognitive test scores in patients passing symptom validity tests. Archives of Clinical Neuropsychology, 17, 205-222.
January 13, 2007 Rohling - CCPN Orlando, FL80
Mood Group AssignmentMood Group Assignment
Patients classified into 2 subgroups From entire sample, 420 passed all SVTs
Sample split based on BDI Low-Depressed 25%ile on BDI (< 10)
n = 178, M = 6 (3) High-Depressed 75%ile on BDI (> 25)
n = 187, M = 31 (6)
January 13, 2007 Rohling - CCPN Orlando, FL81
Depression Study ParticipantsDepression Study Participants
All 365 patients referred for evaluation for compensation-related purposes
All diagnostic groups included 53% Head injury referrals 22% Medical referrals 14% Psychiatric referrals 11% Other neurological
Age = 42 (11); Ed = 13 (3); Sex = 64% males; Non-English = 18%; Handedness = 9% Left
January 13, 2007 Rohling - CCPN Orlando, FL82
Results Mood & Validity StatusResults Mood & Validity Status
107
(30%)
68
(19%)
159
(44%)
27
(7%)
SVT Status
Mood BDI
NonDep 25%ile
Depress 75%ile
Genuine Exaggerating
175(48%)
186(52%)
266(74%)
95(26%)
January 13, 2007 Rohling - CCPN Orlando, FL83
Results: Sample Split by ValidityResults: Sample Split by Validity
55% of High-Dep; 33% of Low-Dep group Memory ratings related to mood (r = .55) Mood not correlated with cognition (r = .06) Memory ratings related to cognition (r = .15) Group means correlated with Green’s .94
all patient (High-D, Low-D, Gen, & Exag).
January 13, 2007 Rohling - CCPN Orlando, FL87
SVT Results Test Score
WMT CARB TOMM N Mean OTBM
Mean Pain
Pass Pass Pass 340 0.24 -0.30
Pass Fail Pass 14 -0.06 -0.09
Fail Pass Pass 74 -0.54 0.14
Fail Fail Pass 35 -0.74 0.57
Fail Pass Fail 17 -0.96 0.39
Fail Fail Fail 38 -1.39 0.67
Effect if Pain on OTBMEffect if Pain on OTBM
January 13, 2007 Rohling - CCPN Orlando, FL88
Effect if Pain on OTBMEffect if Pain on OTBM Pain Rating Status
Rohling’s Interpretive Method: Use Rohling’s Interpretive Method: Use of Meta-Analytic Procedures for of Meta-Analytic Procedures for
Single Case Data AnalysisSingle Case Data Analysis
Martin L. RohlingQuestions & Comments Welcome!Questions & Comments Welcome!
January 13, 2007 Rohling - CCPN Orlando, FL105
CT/MRI Data
Participant Demographic Information Variable Sample Sizes (N = 124) Gender
Male 82 Female 42
Ethnicity Caucasian 119 Other 5
January 13, 2007 Rohling - CCPN Orlando, FL106
CT/MRI
Diagnostic Groups Sample Size MVA/TBI 47 Blow to Head 32 LCVA 24 RCVA 21
January 13, 2007 Rohling - CCPN Orlando, FL107
CT/MRI
1. All were Right Handed.
2. All were followed by Dr. Meyers through hospitalization and rehabilitation.
3. None were involved in litigation.
4. All passed internal validity checks.
January 13, 2007 Rohling - CCPN Orlando, FL108
CT/MRI
CT/MRI Location Left Frontal 59 Left Parietal 37 Left Temporal 34 Left Occipital 6 Right Frontal 40 Right Parietal 42 Right Temporal 31 Right Occipital 3
January 13, 2007 Rohling - CCPN Orlando, FL109
CT/MRI
All were given MNB CT/MRI data coded for injury reported on
MRI/CT at the time of injury Present = 1 Absent = 0
January 13, 2007 Rohling - CCPN Orlando, FL110
January 13, 2007 Rohling - CCPN Orlando, FL111
January 13, 2007 Rohling - CCPN Orlando, FL112
CT/MRI
Independent Sample 1-tailed t-test on each lobe
On CT/MRI report Present = 1 Absent = 0
January 13, 2007 Rohling - CCPN Orlando, FL113
CT/MRI Data
January 13, 2007 Rohling - CCPN Orlando, FL114
Brain Regions Involved in the Performance of WAIS-III Arithmetic
January 13, 2007 Rohling - CCPN Orlando, FL115
Brain Regions Involved in the Performance of the Boston Naming Test
January 13, 2007 Rohling - CCPN Orlando, FL116
Brain Regions Involved in the Performance of the Rey-CFT Copy
January 13, 2007 Rohling - CCPN Orlando, FL117
Brain Regions Involved in the Performance of the AVLT Total Score
January 13, 2007 Rohling - CCPN Orlando, FL118
CT/MRI
NP tests generally behaved as expected A more “Systemic” or “Domain” like
approach better at explaining results Construct of “Executive Function” not
Motor and Sensory Finger Tapping Dominant Hand Finger Tapping Non-Dominant Hand Finger Localization Dominant Hand Finger Localization Non-Dominant Hand
January 13, 2007 Rohling - CCPN Orlando, FL123
January 13, 2007 Rohling - CCPN Orlando, FL124
January 13, 2007 Rohling - CCPN Orlando, FL125
January 13, 2007 Rohling - CCPN Orlando, FL126
January 13, 2007 Rohling - CCPN Orlando, FL127
Commonality of Reduced O2
January 13, 2007 Rohling - CCPN Orlando, FL128
Domain Consistency
N = 936 Passed all validity checks No missing data Not involved in litigation
Calculated Domain M’s Regression used to predict Domain M’s