Quality of Life Measures: Mental Health Services and Consumer Satisfaction
Post on 03-Jan-2016
26 Views
Preview:
DESCRIPTION
Transcript
Quality of Life Measures:
Mental Health Services and Consumer Satisfaction
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
For further information, please contact: Dennis McBride: (253) 756-2335 | dmcb@uw.edu
Catherine M. Wilson: (253) 761-7573 | cmw27@uw.edu
Part 1
Consumer Satisfaction Surveys
The Mental Health Statistical Improvement Program(MHSIP)
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
For further information, please contact: Dennis McBride: (253) 756-2335 | dmcb@uw.edu
Catherine M. Wilson: (253) 761-7573 | cmw27@uw.edu
MHSIP Emerges 1976 – MHSIP Advisory Group (developed data standards for public
mental health systems) 1987 – National Research Institute (NRI) to Support MHSIP 1989 – MHSIP Policy Group (Refined Data Standards for Mental Health
Systems) 1996 – MHSIP Consumer-Oriented Report Card (Developed base
survey we now use). National Association of State Mental Health Program Directors (NASMHPD) was created to further develop the Application of MHSIP.
1997 – Five State Feasibility Project (refined survey) 1998-2001 – 16 State Study on Mental Health Performance Measures
(refined survey) 2001- 2014 – Current Survey
Goal & Audiences
The consumer surveys are designed to examine service quality issues related to Washington State’s delivery of state-funded mental health services.
Using Survey Data for:Federal ReportingPerformance Indicator Work GroupsProvider Agencies and RSNs (e.g., EQROs)Consumer Groups
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
Federal Uses• Data Infrastructure Grants (DIGS)• National Outcome Measures (NOMS)• Government Performance and Results Act (GPRA)• Uniform Reporting System (URS).
The MHSIP Consumer Survey is now being implemented in 55 states/territories for the adult survey and 54 states/territories for youth surveys.
Surveys to Meet CMS Requirements
Survey Year
Statewide Sample Based Outcomes Survey
1998
Children with Special Needs 1999
MHSIP Adult Consumer Survey (10 surveys)
2002-2013
MHSIP Child and Family Consumer Survey (9 surveys)
2002-2013
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
2013 MHSIP Survey Samples
• Persons receiving at least one hour of service within a 6 month time frame: May-Oct 2012
Adult Survey: 18+ years oldYouth Survey: 13-20 years oldFamily Survey: <13 years old
• Stratified Random Sample (Stratified by RSN, Age, Minority Status
• Probability Proportionate to Size (Size adjusted for small RSNs)
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
WIMHRT’s CATI Lab System
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
Sources of Data
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
Contact Informati
on
ProviderOne Data System
Research and Data Analysis
(DSHS)
ACES Barcode data – Economic
Services Administration
(DSHS)
Selecting Participants
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
49,746
5,646
1,334
30,311
3,924
924
Adult Consumer Survey (ACS)
Sample Frame
Drawn
Sample
Respondent Sample
Child & Family Consumer Survey (CFCS)
Sample Frame
Drawn Sample
Respondent Sample
F=604Y=320
ACS & CFCS Disposition
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013 Adult
Disposition N (%)
Family/YouthDisposition
N (%)
Completions 1344 (23.6) 924 (23.5)
Refusals 647 (11.5) 260 (6.6)
Incorrect Numbers 2073 (36.7) 1015 (25.9)
No MH Services 46 (0.8) 54 (1.4)
Language Barrier 114 (2.0) 9 (0.2)
Unavailable 310 (5.5) 86 (2.2)
Deceased 33 (0.6) 1 (0.03)
No Answer 932 (16.5) 1534 (39.1)
Other 157 (2.8) 41 (1.04)
TOTAL 5646 3924
Response & Cooperation Rates Over Time
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
0102030405060708090
100ACS % Coopera-tion Rate
ACS % Response Rate
CFCS % Coopera-tion Rate
CFCS % Response Rate
2004 Data CFCS % Cooperation Rate N/ACFCS % Response Rate N/A
States’ MHSIP Completion Rates 2008
State Sample Method
Survey Method
Adult N Adult % Y/F N Y/F %
Texas Stratified Random
Mail Out 334 19% 389 18%
Wisconsin* Random Mail Out 803 36% 524 39%
Vermont Random Mail Out 737 36% 251 20%
Washington State
Stratified Random
Telephone/Mail Out
1500 24% 906 33%
Illinois Random Mail Out 561 19% 586 17%
Oregon Total Population
Mail Out 3240 24% Combined
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
Source: Sampling and the MHSIP Consumer Surveys: Techniques, Models, Issues. Mary E. Smith & Vijay Ganju (June, 2008).
*Removes non-contacts from the denominator.
Total number of attempts/calls: 32,628
ACS: 22,023CFCS: 10,605
Average number of attempts/calls per completion: 14.45
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
2013 Calls & Completions
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
Total number of attempts/calls: 341,078
ACS: 231,995CFCS: 109.083
Total number of completions: 24,005
ACS: 15,592CFCS: 8,413
Total Calls & Completions
Representativeness
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
Age Hours Age Hours0
10
20
30
40
50
Respondent Sample Drawn Sample Sample Frame
Female % Minority % Female % Minority %0
10203040506070
ACS CFCS
Adult/Youth Perception of Outcomes
As the result of services I received:
1. I am better at handling daily life.2. I get along better with family members.3. I get along better with friends and other
people.4. I am doing better in school and/or work.5. I am better able to cope when things go wrong.6. I am satisfied with my family life right now.
Scoring: 1-Strongly Disagree; 2-Disagree; 3-Undecided; 4-Agree; 5-Strongly Agree THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013 Alpha = 0.90
Scale
Scale Construction
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
Scale ACS CFCSGeneral Satisfaction .87 .92
Appropriateness/Quality of Services .87 .91
Participation in Treatment Goals .56 .70
Perceived Outcomes .91 .90
Perception of Access .82 .48
Satisfaction with Staff NA .84
Cultural Sensitivity of Staff NA .84
NOMS Functioning* .87 NA
NOMS Social Connectedness* .79 .81
Stigma* .86 .85(Youth only)
*Added in 2007
ACS Scales
General Sat-isfaction
Quality of Service
Participation in Treatment
Perception of Access
Perceived Outcomes
0%
25%
50%
75%
100%
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Strongly Agree
Agree
Undecided
Disagree
Strongly Disagree
Average ScorePerc
ent S
atisfi
ed
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
CFCS Scales
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
0%
25%
50%
75%
100%
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Strongly Agree
Agree
Undecided
Disagree
Strongly Disagree
Average Score
Perc
ent S
atisfi
ed
Other Relationships
• Demographics and Total Service Hours• Living Situation by Age, Gender, & Ethnicity• Employment Status by Age, Gender, & Ethnicity• Employment Status by Age, Gender, & Ethnicity• Medical/Insurance Information by Age, Gender, &
Ethnicity• Satisfaction Scales by Age, Gender, & Ethnicity• Satisfaction Scales by RSN
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
What two things do you like the most about the services you received?
Services14%
Support21%
Therapy/Counseling11%
Medications1%
Access6%
Staff1%
Professional Staff22%
Environment1%
No Response21%
General Positive 1% General Negative
1%
ACS First Response
Services
Support
Therapy/Counseling
Medications
Access
Staff
Professional Staff
Environment
No Response
General Positive
General Negative
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
Open-Ended Responses: Adult – Like Most
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
Support:
”The support that they give me. If I need someone to talk with, they are
there. If not, they call right back. They are good listeners and very
understanding .”
”You are not rejected on account of anything you might be
experiencing/feeling. If there’s one counselor that you’re just not clicking
with, they will rotate you to another. They’ll bear with you and just keep
going until you can identify your feelings and handle them.”
What two things do you like the least about the services you received?
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
Services16%
Support21%
Therapy/Counseling8%
Medications1%
Access7%
Staff2%
Professional Staff17%
No Response25%
General Positive 2% General Negative
1%
CFCS First Response
Services
Support
Therapy/Counseling
Medications
Access
Staff
Professional Staff
No Response
General Positive
General Negative
Open-Ended Responses: Family – Like Least
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
Access:
”The location and difficulty in getting after school appointments .”
”Long waiting list and it takes months to see a psychiatrist.”
”I liked least that they are short of time. They had so few choices
of time. My child would miss school for appointments.’
STIGMA
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
ACS Perceived Stigma by Gender/Age
Female Male 18 to <21 21 to <41 41 to <61 61 to <760%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0
0.5
1
1.5
2
2.5
3
3.5
4
Most Stigmatized (3.5-4)
More Stigmatized (2.5-3.49)
Less Stigmatized (1.5-2.49)
Least Stigmatized (1-1.49)
Average ScorePerc
ent S
tigm
atize
d
Consumer Satisfaction Surveys 2013 ACS Perceived Stigma by Gender/Ethnicity
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
ACS Perceived Service Outcomes by Stigma
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
ACS Consumer Satisfaction Surveys 2012 -13
Service Perception Scores Stigma Scale
Satisfaction with Services -.124
Appropriateness and quality of Services
-.175
Participation in Treatment Goals
-.166
Perceived Outcome of Service
-.241
Perception of Access to Service
-.189
NOMS Functioning Scale -.232
NOMS Social Connectedness Scale
-.312
N = 2578 p < .001
Correlations: Service Perception by Stigma
Part 2
Trends
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
ACS Perceived Stigma by Year
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
2007 2008 2009 2010 2011 2012 20130%
25%
50%
75%
100%
0
0.5
1
1.5
2
2.5
3
3.5
4
Most Stigmatized (3.5-4)More Stigmatized (2.5-3.49)Less Stigmatized (1.5-2.49)Least Stigmatized (1-1.49)Average Score
Perc
ent S
tigm
atize
d
Trends in Mean ACS Satisfaction Scores Over Time
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
2002 2004 2006 2007 2008 2009 2010 2011 2012 20133.2
3.3
3.4
3.5
3.6
3.7
3.8
3.9
4
4.1
4.2
General Satisfation Quality of ServiceParticipation in Treatment Perception of AccessPerceived Outcome
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
Trends in Mean CFCS Satisfaction Scores Over Time
2002 2005 2007 2008 2009 2010 2011 2012 20133.4
3.6
3.8
4
4.2
4.4
General Satisfation Satisfation with StaffPerception of AccessParticipation in Treatment Cultural Sensitivity of StaffAppropriateness of Services Perceived Outcome
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
Functioning/Connectedness Scores Over Time
2007 2008 2009 2010 2011 2012 20133.5
3.6
3.7
3.8
3.9
4
4.1
4.2
4.3
ACS Social ConnectednessACS Functioning CFCS Social ConnectednessCFCS Functioning
States’ Adult MHSIP Scores 2010
ACS Measures Texas Wisconsin Vermont Washington State
Illinois Oregon
Access to Services
78.40% 74.40% 82.20% 68.20% 84.60% 74.00%
Quality/Appropriateness of Services
83.70% 79.60% 84.60% 80.70% 83.20% 82.90%
Outcome of Services
57.20% 57.20% 69.30% 61.50% 67.30% 57.20%
Participation in Treatment
71.00% 67.30% 76.90% 71.90% 85.90% 66.80%
Overall Satisfaction
89.00% 76.70% 84.40% 77.10% 85.70% 80.40%
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
Source: 2010 CMHS Uniform Reporting System Output Tables
Note: Scores represent the average of those positively agreeing or strongly agreeing with each scale.
Summary of Findings: ACS
• Satisfaction scores on most scales remain high, with at least 70% of consumers on most scales saying they are satisfied or very satisfied with services.
• Since 2002, satisfaction score trends have shown slight but steady improvement over time.
• Across years, Perceived Outcomes has remained the lowest, with more than one third of consumers being undecided or dissatisfied in this area.
• Consumers continue to be most satisfied with Appropriateness and Quality of Services.
• Perceived Stigma affects nearly half of consumers and has remained relatively constant since 2007.
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
• Satisfaction scores on most scales remain high, with at least three quarters of consumers on most scales saying they are satisfied or very satisfied with services.
• Since 2002, all areas of youth and family service satisfaction have continued to improve.
• Cultural Sensitivity of Staff and Satisfaction with Staff continue to be the highest rated areas.
• Perceived Outcomes is consistently the lowest rated area.
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
Summary of Findings: CFCS
HTTPS://DEPTS.WASHINGTON.EDU/PBHJP/DEV/PROJECTS-PROGRAMS/PAGE/MENTAL-HEALTH-STATISTICS-IMPROVEMENT-PROGRAM-CONSUMER-SATISFACTION-SURVEYS
THE WASHINGTON INSTITUTE FOR MENTAL HEALTH RESEARCH & TRAINING
Consumer Satisfaction Surveys 2013
top related