Alex Mitchell www.psycho-oncology.info Department of Cancer & Molecular Medicine, Leicester Royal Infirmary Department of Liaison Psychiatry, Leicester General Hospital Portugal 2010 Portugal 2010 WORKSHOP Day 1 Science of Screening: Definitions, analysis, screening tools, case-finding tools, prevalence, link with physical concerns
89
Embed
[Workshop] The science of screening in Psycho-oncology (Oct10)
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Alex Mitchell www.psycho-oncology.info
Department of Cancer & Molecular Medicine, Leicester Royal Infirmary
Department of Liaison Psychiatry, Leicester General Hospital
Portugal 2010Portugal 2010
WORKSHOP Day 1
Science of Screening:Definitions, analysis, screening tools, case-finding tools, prevalence, link with physical concerns
WORKSHOP Day 1
Science of Screening:Definitions, analysis, screening tools, case-finding tools, prevalence, link with physical concerns
Schedule Day 1Schedule Day 1
930-10.00 – Introduction, groups and issues
10.00-11.00 – T1 Basic science of screening
Break
11.30 – 12.30 – Group task #1
Lunch
1.30-2.30 – T2 Symptoms, Burden, Help, Needs in Cancer
Break
3.00 – 4.00 – Evaluation of a screening paper
10 Questions10 Questions1. How do we understand screening studies2. Can we design a screening study3. Which instrument works best4. Which is the most popular tool5. How good are clinicians alone6. Can the DT be improved7. Is screening effective in clinical practice8. What are the barriers to successful implementation9. How can screening be improved10. Do somatic symptoms interfere with the diagnosis
*Age-adjusted to the 2000 US standard population.Source: US Mortality Data 1960-2005, US Mortality Volumes 1930-1959,National Center for Health Statistics, Centers for Disease Control and Prevention, 2008.
0
20
40
60
80
10019
30
1935
1940
1945
1950
1955
1960
1965
1970
1975
1980
1985
1990
1995
2000
2005
Lung & bronchus
Colon & rectum
Stomach
Rate Per 100,000
Prostate
Pancreas
LiverLeukemia
Cancer Death Rates* Among Women, US,1930-2005
*Age-adjusted to the 2000 US standard population.Source: US Mortality Data 1960-2005, US Mortality Volumes 1930-1959,National Center for Health Statistics, Centers for Disease Control and Prevention, 2008.
0
20
40
60
80
10019
30
1935
1940
1945
1950
1955
1960
1965
1970
1975
1980
1985
1990
1995
2000
2005
Lung & bronchus
Colon & rectum
Uterus
Stomach
Breast
Ovary
Pancreas
Rate Per 100,000
0
10
20
30
40
50
60
70
80
90
100
Melanom
aBrea
st (fe
male)
Urinary
bladde
r
Prostat
e
Colon
All site
s
Rectum
Non-H
odgkin
lymph
oma
Ovary
Leuk
emiaLu
ng and
bron
chus
Pancre
as
1975-19771984-19861996-2004Change
5 Year Survival in US Cancers
Distress Thermometer – PooledProportion
18 .4 %
12 .9 %
11.2 %12 .3 %
8 .1%
11.9 %
5.0 %
2 .8 % 2 .6 %
7.7% 7.2 %
0.0%
2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
16.0%
18.0%
20.0%
Zero One Two Three Four Five Six Seven Eight Nine Ten
Insignificant SevereModerateMildMinimal
p124
50%
94.2%
37.4%
8 yrs N= 9282 NCS‐R
N=23 studies; 50% some treatment 33% minimal treatment N=19 studies; 30% 1 in 1/12; 10% 3 in 3 months
T1. Basic Science of ScreeningT1. Basic Science of Screening
Accuracy (aka convergent validity)The degree of approximation (veracity) to a robust comparator
Validity (aka criterion validity)The degree of approximation (veracity) to a criterion reference
PrecisionThe degree of predictability (low SD) in the measure
Stage Type Purpose Description
Pre-clinical Development Development of the proposed tool or test
Here the aim is to develop a screening method that is likely to help in the detection of the underlying disorder, either in a specific setting or in all setting. Issues of acceptability of the tool to both patients and staff must be considered in order for implementation to be successful.
Phase I_screen
Diagnostic validity Early diagnostic validity testing in a selected sample and refinement of tool
The aim is to evaluate the early design of the screening method against a known (ideally accurate) standard known as the criterion reference. In early testing the tool may be refined, selecting most useful aspects and deleting redundant aspects in order to make the tool as efficient (brief) as possible whilst retaining its value.
Phase II_screen
Diagnostic validity Diagnostic validity in a representative sample
The aim is to assess the refined tool against a criterion (gold standard) in a real world sample where the comparator subjects may comprise several competing condition which may otherwise cause difficulty regarding differential diagnosis.
Phase III_screen
Implementation Screening RCT; clinicians using vs not using a screening tool
This is an important step in which the tool is evaluated clinically in one group with access to the new method compared to a second group (ideally selected in a randomized fashion) who make assessments without the tool.
Phase IV_screen
Implementation Screening implementation studies using real-world outcomes
In this last step the screening tool /method is introduced clinically but monitored to discover the effect on important patient outcomes such as new identifications, new cases treated and new cases entering remission.
Development of Diagnostic Tests
Concepts: Se Sp PPV NPVConcepts: Se Sp PPV NPV
Accuracy 2x2 TableAccuracy 2x2 Table
Depression
PRESENT
Depression
ABSENT
Test +ve True +ve False +ve PPV
Test ‐ve False ‐Ve True ‐Ve NPV
Sensitivity Specificity Prevalence
Reference StandardDisorder Present
Reference StandardNo Disorder
Test+ve A B
A/A + BPPV
Test-ve C D
D/C + DNPV
Total A / A + CSn
D / B + DSp
Basic Measures of AccuracyBasic Measures of Accuracy
Sensitivity (Se) a/(a + c) TP / (TP + FN)
A measure of accuracy defined the proportion of patients with disease in whom the test result is positive: a/(a + c)
Specificity (Sp) d/(b + d) TN / (TN + FP)A measure of accuracy defined as the proportion of patients without disease in
whom the test result is negative
Positive Predictive Value a/(a+b) TP / (TP + FP)A measure of rule‐in accuracy defined as the proportion of true positives in
those that screen positive screening result, as follows
Negative Predictive Value c/(c+d) TN / (TN + FN)A measure of rule‐out accuracy defined as the proportion of true negatives in
those that screen negative screening result, as follows
Concepts => FiguresConcepts => Figures
Graphical – Screening principles
Non-Depressed
Depressed
# ofIndividuals
# ofIndividuals
Severity of Depression
Graphical – Screening principles
Non-Depressed
Depressed
# ofIndividuals
Cut-Off
# ofIndividuals
Severity of Depression
HighLow
Graphical – Screening principles
Non-Depressed
Depressed
# ofIndividuals
Cut-Off
# ofIndividuals
Severity of Depression
HighLow
High Sensitivity >>>>
<<<< high Specificity
Graphical – Screening principles
Non-Depressed
Depressed
# ofIndividuals
Cut-Off
# ofIndividuals
Severity of Depression
HighLow
High Sensitivity >>>>
<<<< low Specificity
Can This Help establish a syndrome?Can This Help establish a syndrome?
Example: A Clear Disease [#1]Example: A Clear Disease [#1]
Disorder
Number ofIndividuals
False +veFalse +ve
True -veTrue -ve
Point of Partial Rarity
Test Result
No Disorder
False -veFalse -ve
True +veTrue +ve
Example: A Probable Syndrome [#2]Example: A Probable Syndrome [#2]
Disorder
Number ofIndividuals
False +veFalse +ve False -veFalse -ve
True -veTrue -ve
True +veTrue +ve
MMSE Cognitive Score
No Disorder
Example: A Normally Distributed Trait [#3]Example: A Normally Distributed Trait [#3]
Disorder
Number ofIndividuals
False +veFalse +ve False -veFalse -ve
True -veTrue -ve
True +veTrue +ve
MMSE Cognitive Score
No Disorder
Example: DementiaExample: Dementia
Disease?Syndrome?Trait?
Hubbert et al (2005) BMC GeriatricsHubbert et al (2005) BMC Geriatrics
MMSE scores for dementia (n=72)and non-dementia (n=2735)
Huppert et al BMC Geriatrc 2005
Example: DepressionExample: Depression
DiseaseSyndromeTrait
Thompson et al (2001) n=18,414 HADS-DThompson et al (2001) n=18,414 HADS-D
Definition 1:The additional ability of a test to rule‐in or rule‐out
compared with the baseline ratePPV minus PrevalenceNPV minus prevalence
Definition 2:The additional of a test to rule‐in or rule‐out compared
with the unassisted ratePPV test minus PPV no test (assuming equal prevalence)
LR+ test minus LR+ no test
AUC test minus AUC no test
Reciprocal MeasuresReciprocal Measures
Number Needed to Diagnose (NND)1 / (Youden's J)
Number Needed to Predict (NNP)1 / (PSI)
Number Needed to Screen (NNS)1/(FC‐FiC)
-0.10
0.00
0.10
0.20
0.30
0.40
0.50A
nger
Anx
iety
Dec
reas
ed a
ppet
ite
Dec
reas
ed w
eigh
t
Dep
ress
ed m
ood
Dim
inis
hed
conc
entr
atio
n
Dim
inis
hed
driv
eD
imin
ishe
d in
tere
st/p
leas
ure
Exce
ssiv
e gu
ilt
Hel
ple
ssne
ss
Hop
eles
snes
s
Hyp
erso
mni
a
Incr
ease
d ap
peti
te
Incr
ease
d w
eigh
t
Inde
cisi
vene
ss
Inso
mni
aLa
ck o
f re
acti
ve m
ood
Loss
of
ener
gy
Psyc
hic
anxi
ety
Psyc
hom
otor
agi
tati
on
Psyc
hom
otor
cha
nge
Psyc
hom
otor
ret
arda
tion
Slee
p di
stur
banc
e
Som
atic
anx
iety
Thou
ghts
of
deat
h
Wor
thle
ssne
ss
Rule-In Added Value (PPV-Prev)Rule-Out Added Value (NPV-Prev)
Accuracy of Tests: Visual Post-test ProbabilitiesAccuracy of Tests: Visual Post-test Probabilities
0% 100%25% 75%
Very unlikely Very likelylikelyunlikely
2 Questions
Overall
PHQ-2
WHO5 (1+3)
1 Question3% - (37) - 63% = 60%
3% - (16) - 32% = 29%
3% - (16) - 32% = 29%
10% - (22) -50% = 54%
32% - (37) - 96% = 64%
Henckel et al (2004) Eur Arch Psychiatry Clin Neuros
CIDI (computer) Any Depression
Henckel et al (2004) Eur Arch Psychiatry Clin Neuros
CIDI (computer) Any Depression
Arroll B et al (2003) BMJ
CIDI (computer) Mj Depression
CIDI (computer) Mj Depression
Murphy JM, Berwick DM, Weinstein MC, Borus JF, Budman SH, Klerman GL 1987 : Performance of screening and diagnostic tests: Application of Receiver Operating Characteristic ROC analysis. Arch Gen Psychiatry 44:550-555
Receiver Operating Characteristic
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Pre-test Probability
Post
-test
Pro
babi
lity
Depression Present (Routine)
Depression Absent (Routine)
Depression Scales +ve (Median)
Depression Scales -ve (Median)
Prior Probability
PPV=0.41
NPV=0. 97
Prevalence of 0.15
Group Work #1Group Work #1
930-10.00 – Introduction, groups and issues
10.00-11.00 – T1 Basic science of screening
Break
11.30 – 12.30 – Group task #1
Lunch
1.30-2.30 – T2 Symptoms, Burden, Help, Needs in Cancer
Break
3.00 – 4.00 – Evaluation of a screening paper
Cancer Mj Depression vs NonMjCancer Mj Depression vs NonMj
Clinicians diagnosis using DSMIV vs SCAN/PSE
50 people with depression
200 without depression
Clinicians using DSMIVClinicians using DSMIV
IF: Clinicians diagnosed 50 cases with depressionIF: Their specificity was 95%
Q. What was the sensitivity?Q. What was the prevalence?Q. What was the PPV?Q. What was overall accuracy
Test vs Major DepressionTest vs Major Depression
DepressionOn SCAN
DepressionABSENT
Test +ve(Clinician)
40 10 50
Test -ve 10 190
50 200
Sensitivity80%
PPV 80%
Specificity95%
NPV 95%
Prevalence 0.20%
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Pre-test Probability
Post
-test
Pro
babi
lity
NH NAs+
NH NAs-
Baseline Probability
Cancer Mj+Mn Depression vs Non Cancer Mj+Mn Depression vs Non
Clinicians diagnosis using DSMIV vs SCAN/PSE
50 people with depression
200 without depression => 50 had minor depression
=> Answer 2=> Answer 2
Test vs Major DepressionTest vs Major Depression
DepressionOn SCAN
DepressionABSENT
Test +ve(Clinician)
50 0 50
Test -ve 50 150 200
100 150
Sensitivity50%
SN-OUT
PPV 100%
Specificity100%
SP-IN
NPV 40%
Prevalence 66.7%
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Pre-test Probability
Post
-test
Pro
babi
lity
NH NAs+
NH NAs-
Baseline Probability
Likelihood RatiosLikelihood Ratios
Likelihood Ratio for Positive Tests The chance of testing positive among those with the condition; divided by the chance of testing positive among those without the condition Sensitivity / (1 - Specificity) [ TP / (TP + FN) ] / [ FP / (FP + TN) ]
= PPV / Prevalence
Likelihood Ratio for Negative Tests The chance of testing negative among those with the condition; divided by the chance of testing negative among those without the condition Specificity / (1 – Sensitivity)[ FN / (FN + TP) ] / [ TN / (TN + FP) ]
= NPV / Prevalence
T3. Symptoms, Help, Needs in CancerT3. Symptoms, Help, Needs in Cancer
Clinician Opinion
Patient Opinion
Psychosocial
Complications
Help
Seeking
Symptoms
Recognized
Intervention
Offered
Help
Accepted
Complication
Resolves
Lag
time
Lag
time
Lag
time
Lag
time
Lag
time
years months weeks weeks days
Cancer
OnsetCancer
Progress
Lessons?
462 (42%)Meetable Needs
1093 (100%)Population
388 (84%)Aware of Need
172 (44%)Requested Help
80 (47%)Needs Met
462 needs
17.3%
322 DSMIV
25%
T4. How Common is Distress?T4. How Common is Distress?
Clinician Opinion
Patient Opinion
Requires depressed mood for most of the day, for most days (by subjective account or observation) for at least 2 years
The symptoms cause clinically significant distress OR impairment in social, occupational, or other important areas of functioning.
Requires persistently low mood two (or more) of the following six symptoms:
(1) poor appetite or overeating (2) Insomnia or hypersomnia(3) low energy or fatigue (4) low self-esteem (5) poor concentration or difficulty
making decisions (6) feelings of hopelessness
DSM-IV Dysthymic disorder
Acute: if the disturbance lasts less than 6 months Chronic: if the disturbance lasts for 6 months
These symptoms cause marked distress that is in excess of what would be expected from exposure to the stressor OR significant impairment in social or occupational (academic) functioning
Requires the development of emotional or behavioral symptoms in response to an identifiable stressor(s) occurring within 3 months of the onset of the stressor(s). Once the stressor has terminated, the symptoms do not persist for more than an additional 6 months.
DSM-IV Adjustment disorder
2 weeksThese symptoms cause clinically important distress OR impair work, social or personal functioning.
Requires two to four out of nine symptoms with at least at least one from the first two (depressed mood and loss of interest).
DSM-IV Minor Depressive Disorder
2 weeksThese symptoms cause clinically important distress OR impair work, social or personal functioning.
Requires five or more out of nine symptoms with at least at least one from the first two (depressed mood and loss of interest).
DSM-IV Major Depressive Disorder
2 weeks unless symptoms are unusually severe or of rapid onset).
At least some difficulty in continuing with ordinary work and social activities
Requires two of the first three symptoms (depressed mood, loss of interest in everyday activities, reduction in energy) plus at least two of the remaining seven symptoms (minimum of four symptoms)
ICD-10 Depressive Episode
DurationClinical SignificanceSymptoms
Depression
13%
20%
57%
48%
38%
18%
Anxiety
Adjustment Disorder
N=11N=4
N=10
Comment: Slide illustrates meta-analytic rates of mood disorder
Prevalence of depression in Palliative settings
20 studies involving 2655 individuals
16.9% (95% CI = 13.2% to 21.0%)
13.0% (95% CI = 11.6% to 14.5%) for MDD
p572
Proportion meta-analysis plot [random effects]
0.0 0.2 0.4 0.6
combined 0.17 (0.13, 0.21)
Maguire et al (1999) 0.05 (0.01, 0.14)
Akechi et al (2004) 0.07 (0.04, 0.11)
Kadan-Lottich et al (2005) 0.07 (0.04, 0.11)
Love et al (2004) 0.07 (0.04, 0.11)
Wilson et al (2004) 0.12 (0.05, 0.22)
Chochinov et al (1997) 0.12 (0.08, 0.18)
Wilson et al (2007) 0.13 (0.10, 0.17)
Kelly et al (2004) 0.14 (0.06, 0.26)
Chochinov et al (1994) 0.17 (0.11, 0.24)
Le Fevre et al (1999) 0.18 (0.10, 0.28)
Breitbart et al (2000) 0.18 (0.11, 0.28)
Meyer et al (2003) 0.20 (0.10, 0.35)
Minagawa et al (1996) 0.20 (0.11, 0.34)
Lloyd-Williams et al (2001) 0.22 (0.14, 0.31)
Hopwood et al (1991) 0.25 (0.16, 0.36)
Desai et al (1999) [late] 0.25 (0.10, 0.47)
Payne et al (2007) 0.26 (0.19, 0.33)
Lloyd-Williams et al (2003) 0.27 (0.17, 0.39)
Jen et al (2006) 0.27 (0.19, 0.36)
Lloyd-Williams et al (2007) 0.30 (0.24, 0.36)
proportion (95% confidence interval)
Prevalence of depression in Oncology settings
57 studies involving 9195 individuals across 12 countries.
The prevalence of depression was 17.3% (95% CI = 13.8% to 21.2%),
13.0% (95% CI = 11.6% to 14.5%) for MDD
p572
Proportion meta-analysis plot [random effects]
0.0 0.3 0.6 0.9
combined 0.1730 (0.1375, 0.2116)
Colon et al (1991) 0.0100 (0.0003, 0.0545)
Massie and Holland (1987) 0.0147 (0.0063, 0.0287)
Hardman et al (1989) 0.0317 (0.0087, 0.0793)
Derogatis et al (1983) 0.0372 (0.0162, 0.0720)
Lansky et al (1985) 0.0455 (0.0291, 0.0676)
Mehnert et al (2007) 0.0472 (0.0175, 0.1000)
Katz et al (2004) 0.0500 (0.0104, 0.1392)
Singer et al (2008) 0.0519 (0.0300, 0.0830)
Sneeuw et al (1994) 0.0540 (0.0367, 0.0761)
Pasacreta et al (1997) 0.0633 (0.0209, 0.1416)
Lee et al (1992) 0.0660 (0.0356, 0.1102)
Reuter and Hart (2001) 0.0761 (0.0422, 0.1244)
Grassi et al (2009) 0.0826 (0.0385, 0.1510)
Grassi et al (1993) 0.0828 (0.0448, 0.1374)
Walker et al (2007) 0.0831 (0.0568, 0.1165)
Kawase et al (2006) 0.0851 (0.0553, 0.1240)
Coyne et al (2004) 0.0885 (0.0433, 0.1567)
Alexander et al (2010) 0.0900 (0.0542, 0.1385)
Love et al (2002) 0.0957 (0.0650, 0.1346)
Ozalp et al (2008) 0.0971 (0.0576, 0.1510)
Morasso et al (2001) 0.0985 (0.0535, 0.1625)
Costantini et al (1999) 0.0985 (0.0535, 0.1625)
Silberfarb et al (1980) 0.1027 (0.0587, 0.1638)
Desai et al (1999) [early] 0.1111 (0.0371, 0.2405)
Morasso et al (1996) 0.1121 (0.0593, 0.1877)
Prieto et al (2002) 0.1227 (0.0825, 0.1735)
Ibbotson et al (1994) 0.1242 (0.0776, 0.1853)
Payne et al (1999) 0.1290 (0.0363, 0.2983)
Kugaya et al (1998) 0.1328 (0.0793, 0.2041)
Alexander et al (1993) 0.1333 (0.0594, 0.2459)
Gandubert et al (2009) 0.1597 (0.1040, 0.2300)
Razavi et al (1990) 0.1667 (0.1189, 0.2241)
Akizuki et al (2005) 0.1797 (0.1376, 0.2283)
Leopold et al (1998) 0.1887 (0.0944, 0.3197)
Devlen et al (1987) 0.1889 (0.1141, 0.2851)
Berard et al (1998) 0.1900 (0.1184, 0.2807)
Joffe et al (1986) 0.1905 (0.0545, 0.4191)
Berard et al (1998) 0.2100 (0.1349, 0.3029)
Maunsell et al (1992) 0.2146 (0.1605, 0.2772)
Grandi et al (1987) 0.2222 (0.0641, 0.4764)
Evans et al (1986) 0.2289 (0.1438, 0.3342)
Spiegel et al (1984) 0.2292 (0.1495, 0.3261)
Golden et al (1991) 0.2308 (0.1353, 0.3519)
Fallowfield et al (1990) 0.2565 (0.2054, 0.3131)
Hosaka and Aoki (1996) 0.2800 (0.1623, 0.4249)
Kathol et al (1990) 0.2961 (0.2248, 0.3754)
Green et al (1998) 0.3125 (0.2417, 0.3904)
Jenkins et al (1991) 0.3182 (0.1386, 0.5487)
Burgess et al (2005) 0.3317 (0.2672, 0.4012)
Hall et al (1999) 0.3722 (0.3139, 0.4333)
Morton et al (1984) 0.3958 (0.2577, 0.5473)
Baile et al (1992) 0.4000 (0.2570, 0.5567)
Passik et al (2001) 0.4167 (0.2907, 0.5512)
Bukberg et al (1984) 0.4194 (0.2951, 0.5515)
Massie et al (1979) 0.4850 (0.4303, 0.5401)
Ciaramella and Poli (2001) 0.4900 (0.3886, 0.5920)