1 1 Research Agenda for Research Agenda for Consumer Operated Services Consumer Operated Services to Achieve Evidence Based to Achieve Evidence Based Practice Status Practice Status Phyllis Solomon, Ph.D. Phyllis Solomon, Ph.D. Professor Professor School of Social Policy & Practice School of Social Policy & Practice University of Pennsylvania University of Pennsylvania 2 ソロモン先生講演会 ソロモン先生講演会 当事者運営サービス( 当事者運営サービス( COS COS )を、 )を、 科学的根拠に基づく実践プログラム 科学的根拠に基づく実践プログラム ( ( EBP EBP )に発展させるための研究的課題 )に発展させるための研究的課題 フィリス・ソロモン フィリス・ソロモン, Ph.D. , Ph.D. ペンシルバニア大学 ペンシルバニア大学 社会政策・社会実践学部教授 社会政策・社会実践学部教授 (翻訳:大島巌、日本社会事業大学) (翻訳:大島巌、日本社会事業大学) 2007 2007 年8月1日 年8月1日 於、日本社会事業大学 於、日本社会事業大学
38
Embed
Research Agenda for Consumer Operated Services to Achieve ...27slecture_aug1.2007.pdf1 1 Research Agenda for Consumer Operated Services to Achieve Evidence Based Practice Status Phyllis
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
11
Research Agenda for Research Agenda for Consumer Operated Services Consumer Operated Services to Achieve Evidence Based to Achieve Evidence Based
Practice Status Practice Status Phyllis Solomon, Ph.D.Phyllis Solomon, Ph.D.
ProfessorProfessorSchool of Social Policy & PracticeSchool of Social Policy & Practice
University of PennsylvaniaUniversity of Pennsylvania
–– Indivs. ID as consumersIndivs. ID as consumers–– Administration & primary activities Administration & primary activities
independent of independent of mhmh provider orgs.provider orgs.–– Consumers control Board of Directors, staff, & Consumers control Board of Directors, staff, &
What are Consumer What are Consumer Partnerships?Partnerships?
Services provided by consumersServices provided by consumersConsumers self identify as having mental Consumers self identify as having mental illnessillnessAdministration, budgeting shared by nonAdministration, budgeting shared by non--consumersconsumersControl shared w/ nonControl shared w/ non--consumersconsumers
Theoretical Basis for Consumer Theoretical Basis for Consumer Operated ServicesOperated Services
Consumers who share common Consumers who share common experiences of mental illness & receipt of experiences of mental illness & receipt of services from MH systems provide support, services from MH systems provide support, encouragement, hope, & adviceencouragement, hope, & advice
–– serve as role models serve as role models
88
当事者運営サービス当事者運営サービス(COS)(COS)のの理論的基盤理論的基盤
精神疾患や精神保健福祉システムからサー精神疾患や精神保健福祉システムからサー
ビスを受けるという経験を共有する当事者が、ビスを受けるという経験を共有する当事者が、
相互に支援と励まし、希望、助言を提供する相互に支援と励まし、希望、助言を提供する
–– 役割モデル(お手本になる)役割モデル(お手本になる)
5
99
Theoretical Basis for Consumer Theoretical Basis for Consumer Operated ServicesOperated Services
Use of consumers as means to change Use of consumers as means to change behavior & skills of those who share behavior & skills of those who share experience of mental illnessexperience of mental illness
Increased ability to engage others in Increased ability to engage others in serviceservice
Credibility of servicesCredibility of servicesFinancial resourcesFinancial resourcesOrganizational supportOrganizational supportStability of servicesStability of servicesSustainability as part of MH service Sustainability as part of MH service delivery system delivery system
What is an Evidence Based What is an Evidence Based Practice (EBP)?Practice (EBP)?
Practice established as Practice established as ‘‘effectiveeffective’’ through through scientific research according to a clear set scientific research according to a clear set of explicit criteria of explicit criteria (Miller & Streiner, 2004)(Miller & Streiner, 2004)
Practice, based on expert or consensus Practice, based on expert or consensus opinion, re available evidenceopinion, re available evidence–– expected to produce specific clinical outcome expected to produce specific clinical outcome
–– measurable change in client status measurable change in client status (SAMHSA) (SAMHSA)
What is Emerging Best What is Emerging Best Practices?Practices?
Treatments & services that are promising Treatments & services that are promising but less documented than EBPs but less documented than EBPs (President(President’’s s
New Freedom CommissionNew Freedom Commission))
Existing research positive, but limited in Existing research positive, but limited in rigor & quantity rigor & quantity –– need more convincing need more convincing empirical evidenceempirical evidence
What Type of Evidence What Type of Evidence Establishes EBP?Establishes EBP?
Pyramid of EvidencePyramid of Evidence
–– Expert panel review of research evidenceExpert panel review of research evidence–– MetaMeta--analytic studiesanalytic studies–– RCTsRCTs–– Narrative reviews Narrative reviews –– Single RCTs, Multiple QuasiSingle RCTs, Multiple Quasi--Exp., Lg. multiExp., Lg. multi--site, single group site, single group
designdesign–– QuasiQuasi--experimental experimental –– Single group pre/post testSingle group pre/post test–– Pilot & Case studiesPilot & Case studies–– Correlational & Descriptive studies Correlational & Descriptive studies –– Qualitative studies: observational, interview, & focus groupsQualitative studies: observational, interview, & focus groups(Adapted from SAMHSA, 2006) (Adapted from SAMHSA, 2006)
Summary of the EvidenceSummary of the EvidenceEarly studies demonstrated consumer capabilityEarly studies demonstrated consumer capabilityNo detrimental effects from consumer providing No detrimental effects from consumer providing serviceserviceNo detrimental effects to consumers providing No detrimental effects to consumers providing servicesservicesLimited research Limited research –– most descriptive studies, few most descriptive studies, few RCTsRCTsSometimes difficult to distinguish COS from Sometimes difficult to distinguish COS from Consumer PartnershipConsumer Partnership
Marginally more effectiveMarginally more effective
Generally as effective as nonGenerally as effective as non--consumer in consumer in providing conventional servicesproviding conventional services
Shows promise Shows promise –– PresidentPresident’’s New s New Freedom Commission considers COS Freedom Commission considers COS ““Emerging Best PracticeEmerging Best Practice””
Challenges to Evaluating COSChallenges to Evaluating COS
Stronger designs conducted w/ Stronger designs conducted w/ conventional service provision conventional service provision –– not not unique to consumersunique to consumers–– Would not expect much difference in Would not expect much difference in
outcomesoutcomes–– Anecdotal evidence of consumer provider Anecdotal evidence of consumer provider
services services –– estab. rapport & trust, estab. rapport & trust, ↑↑empathy, empathy, ↑↑ knowledge of survival skillsknowledge of survival skills
Challenges to Evaluating COSChallenges to Evaluating COS
•• What is unique re COS as compared to What is unique re COS as compared to Conventional services? And how to Conventional services? And how to measure it?measure it?
Primary Requirement for Establishing Primary Requirement for Establishing EBPEBP
Well defined program model, i.e., specified Well defined program model, i.e., specified program elementsprogram elementsProgram manualProgram manualRepeatability of program model by others Repeatability of program model by others than developerthan developerFidelity assessment of program modelFidelity assessment of program model
2828
EBPEBPを確立するために必要な主な要件を確立するために必要な主な要件
よく定義されたプログラムモデル:すなわち、特定のよく定義されたプログラムモデル:すなわち、特定の
プログラム要素プログラム要素
プログラムのマニュアルプログラムのマニュアル
開発者以外の人たちによって、プログラムモデルが開発者以外の人たちによって、プログラムモデルが
再現されること再現されること
プログラムモデルのフィデリティ評価(忠実度、適合プログラムモデルのフィデリティ評価(忠実度、適合
度評価)度評価)
15
2929
Process for Developing Manual Process for Developing Manual
Manual Manual –– describes a single programdescribes a single programManual includes:Manual includes:–– Theoretical basis of program Theoretical basis of program –– literature reviewliterature review–– Guidelines for therapeutic relationshipGuidelines for therapeutic relationship–– Specific techniques & content Specific techniques & content –– maybe curriculummaybe curriculum–– Suggested sequencing of activitiesSuggested sequencing of activities–– Strategies for special problemsStrategies for special problems–– Implementation issuesImplementation issues–– Termination issues Termination issues
3030
マニュアルを開発するプロセスマニュアルを開発するプロセス
マニュアル:マニュアル: 単一のプログラムを記述する単一のプログラムを記述する
マニュアルには、以下のものが含まれる:マニュアルには、以下のものが含まれる:
––プログラムの理論的基盤:文献レビュープログラムの理論的基盤:文献レビュー
––援助関係のためのガイドライン援助関係のためのガイドライン
––特定の技術と内容特定の技術と内容 (カリキュラムのようなもの)(カリキュラムのようなもの)
––推奨される活動の順序づけ推奨される活動の順序づけ(sequencing of activities)(sequencing of activities)
––特別な特別な問題に対する方策問題に対する方策
––実施上の問題実施上の問題
––終結終結の問題の問題
16
3131
Process for Developing ManualProcess for Developing Manual
Intervention manual lay core elements, Intervention manual lay core elements, when & how to deliverwhen & how to deliverStructural elements of program, e.g., Structural elements of program, e.g., caseload size, staff qualificationscaseload size, staff qualificationsLocation of serviceLocation of serviceSocial work interventions difficult to Social work interventions difficult to manualizemanualize-- occur in multiple locations, with occur in multiple locations, with multiple providers & recipientsmultiple providers & recipientsManuals vary greatly Manuals vary greatly
Process For Developing ManualProcess For Developing Manual
Review existing written material on programReview existing written material on programDraft documentDraft documentInput from experts Input from experts –– providers, consumer, providers, consumer, families, etc.families, etc.Focus groupsFocus groupsIndepthIndepth interviewsinterviewsObservations Observations –– site visit programssite visit programsReview by expertsReview by experts
Process of Developing Fidelity Process of Developing Fidelity AssessmentAssessment
Manuals form basis to assess fidelity of Manuals form basis to assess fidelity of interventioninterventionFidelity assessment Fidelity assessment –– degree to which degree to which intervention conducted as plannedintervention conducted as plannedFidelity measures Fidelity measures –– tools to assess tools to assess adequacy of implementation of program adequacy of implementation of program modelmodel
3636
フィデリティ評価を開発するプロセスフィデリティ評価を開発するプロセス
マニュアルは、介入のフィデリティを評価する基礎をマニュアルは、介入のフィデリティを評価する基礎を
形成する形成する
フィデリティ評価:フィデリティ評価: 介入が計画されたとおり実施され介入が計画されたとおり実施され
ている程度ている程度
フィデリティ測定:フィデリティ測定: プログラムモデルの実施が適切でプログラムモデルの実施が適切で
あるかどうかを評価する用具あるかどうかを評価する用具
19
3737
Steps for Developing Fidelity MeasureSteps for Developing Fidelity Measure
Define purpose of fidelity scaleDefine purpose of fidelity scaleAssess degree of model developmentAssess degree of model developmentIdentify model dimensionsIdentify model dimensionsDetermine if appropriate fidelity scales Determine if appropriate fidelity scales already existalready existFormulate fidelity scale planFormulate fidelity scale planDevelop items Develop items
3838
フィデリティ測定を開発するステップフィデリティ測定を開発するステップ
フィデリティ尺度の目的を明確にするフィデリティ尺度の目的を明確にする
プログラムモデル発展の程度を評価するプログラムモデル発展の程度を評価する
モデルの次元を特定するモデルの次元を特定する
既存の適切なフィデリティ尺度があるかどうかを見定既存の適切なフィデリティ尺度があるかどうかを見定
めるめる
フィデリティ尺度の計画を考案するフィデリティ尺度の計画を考案する
項目を開発する項目を開発する
20
3939
Steps for Developing Fidelity MeasureSteps for Developing Fidelity Measure
Choose data collection sources & methodsChoose data collection sources & methodsDetermine item orderDetermine item orderDevelop data collection protocolDevelop data collection protocolTrain interviewers/ratersTrain interviewers/ratersPilot scalePilot scaleAssess psychometric propertiesAssess psychometric propertiesDetermine scoring & weighting of itemsDetermine scoring & weighting of items
4040
フィデリティ測定を開発するステップフィデリティ測定を開発するステップ
データ収集のデータ収集の情報源情報源と方法を選択すると方法を選択する
項目の順番を定める項目の順番を定める
データ収集のプロトコルを作成するデータ収集のプロトコルを作成する
面接者や評価者の研修面接者や評価者の研修
予備的尺度予備的尺度
心理測定学的特性を評価する心理測定学的特性を評価する
得点法や項目の重み付けを決定する得点法や項目の重み付けを決定する
21
4141
Development of Fidelity Measure for Development of Fidelity Measure for Consumer Run Drop In CentersConsumer Run Drop In Centers
Reviewed literature on consumer run Reviewed literature on consumer run organizations & advocacy organizations & advocacy –– published & published & unpublishedunpublishedDeveloped list of critical ingredientsDeveloped list of critical ingredientsExperts mailed survey Experts mailed survey –– rate critical rate critical ingredients ingredients –– agree, disagree, neither & agree, disagree, neither & additional criteria Wave Iadditional criteria Wave IWave II Wave II –– experts experts –– process to select least process to select least & most essential ingredients& most essential ingredients
Development of Fidelity Measure of Development of Fidelity Measure of Consumer Run Drop In CentersConsumer Run Drop In Centers
Developed instrumentDeveloped instrumentTeam conducted site visits Team conducted site visits –– consumers consumers completed questionnaires, director completed questionnaires, director interviewed, & site observations madeinterviewed, & site observations madePrograms rated by teamPrograms rated by teamInvestigators conducted a number of Investigators conducted a number of validation procedures validation procedures
Process Process –– Belief SystemsBelief SystemsGroups empowerment, practice skills, recovery Groups empowerment, practice skills, recovery orientation, etcorientation, etc
Process Process –– Opportunity Role StructureOpportunity Role StructureConsumer involvement, consumer choice, & decisionConsumer involvement, consumer choice, & decision--making, nonmaking, non--hierarchial structure, etchierarchial structure, etc
ProcessProcess –– Social SupportSocial SupportMember retention, respect for diversity, sense of community, Member retention, respect for diversity, sense of community, selfself--help & reciprocity, etc help & reciprocity, etc
Differences in Service Environments of Differences in Service Environments of Consumer Run DropConsumer Run Drop--in Centers (CDIs) & in Centers (CDIs) &
ClubhousesClubhousesGreater control & involvement in CDIs (i.e., program Greater control & involvement in CDIs (i.e., program decisions re: financial operations & staffing) decisions re: financial operations & staffing)
Clubhouses more instrumental services & activities Clubhouses more instrumental services & activities (meals, laundry services, job & housing finding, (meals, laundry services, job & housing finding, system level advocacy, etc) than CDIssystem level advocacy, etc) than CDIs
Importance & availability of social & recreational Importance & availability of social & recreational services more important for Clubhouses than CDIsservices more important for Clubhouses than CDIs
Clubhouses substantially more resources than CDIsClubhouses substantially more resources than CDIs(Mowbray, et al., 2005)(Mowbray, et al., 2005)
COSP Fidelity Assessment Common COSP Fidelity Assessment Common IngredientsIngredients Tool (FACIT)Tool (FACIT)
Program StructureProgram StructureConsumer OperatedConsumer OperatedParticipant responsiveParticipant responsiveLinkage to other supportsLinkage to other supports
Impact of Environment on Impact of Environment on Consumer OutcomesConsumer Outcomes
Greater involvement & control in COS greater gains Greater involvement & control in COS greater gains in personal empowerment & social functioning in personal empowerment & social functioning (Segal & Silverman, 2002)
Greater use of COS greater increase in wellGreater use of COS greater increase in well--being being (Teague, et al., 2005)(Teague, et al., 2005)
Greater participation larger social network, impr. self Greater participation larger social network, impr. self concept, dec. hosp. concept, dec. hosp. (Corrigan, et al, 2005)(Corrigan, et al, 2005)
Attendance assoc. w/inc.Attendance assoc. w/inc. social functioningsocial functioning ((YanosYanos, et al, , et al, 2001)2001)
Impact of Environment on Consumer Impact of Environment on Consumer OutcomesOutcomes
Opportunities to participate in decisions about care Opportunities to participate in decisions about care result in positive outcomes (result in positive outcomes (Segal & Silverman, 2002) Segal & Silverman, 2002)
Quality of participation (meaningful involved) result Quality of participation (meaningful involved) result in short term positive outcomes in short term positive outcomes ––quality of life, daily quality of life, daily activities, & community integration activities, & community integration (Nelson & (Nelson & LomoteyLomotey, , 2006) 2006)
COS participation associated with recovery & COS participation associated with recovery & empowermentempowerment ((CooriganCoorigan, 2006), 2006)
Environmental acceptance result in engagement of consumers Environmental acceptance result in engagement of consumers who donwho don’’t want or have had negative experiences w/ t want or have had negative experiences w/ conventional MH servicesconventional MH services
Important Features of COSImportant Features of COS
Organizational Structure & Process of Organizational Structure & Process of COS programs important to client COS programs important to client outcomesoutcomes
Research on uniqueness of COS is Research on uniqueness of COS is underdeveloped underdeveloped (Goering, et al., 2006)(Goering, et al., 2006)
する研究は未発達であるする研究は未発達である(Goering, et al., 2006)(Goering, et al., 2006)
30
5959
Neglect of Human Neglect of Human Social InteractionsSocial Interactions
EvidenceEvidence--based practices described by based practices described by program structures program structures (i.e., staffing, case (i.e., staffing, case loadload……))–– have overlooked ingredients of helping have overlooked ingredients of helping
processes that occur w/in each practice & processes that occur w/in each practice & which behavioral sciences research has which behavioral sciences research has shown to be related to how people change shown to be related to how people change and grow. and grow. (Anthony, 2003, p. 1)(Anthony, 2003, p. 1)
–– それぞれの実践の中で発生し、行動科学的研究によそれぞれの実践の中で発生し、行動科学的研究によりどのように人が変化し成長するかということに関連りどのように人が変化し成長するかということに関連しているということが示唆された、援助プロセスの要しているということが示唆された、援助プロセスの要素を見過ごしている素を見過ごしている (Anthony, 2003, p. 1)(Anthony, 2003, p. 1)
31
6161
Researching the Helping Researching the Helping Process of ConsumersProcess of Consumers
EvidenceEvidence--based practice broaden to incorporate based practice broaden to incorporate ““empirically derived helping processes (relational empirically derived helping processes (relational variables, skill teaching strategies, hope variables, skill teaching strategies, hope engendering techniques) that are fundamental to engendering techniques) that are fundamental to peoplepeople’’s growth and change, and which may s growth and change, and which may underlie most evidence based structuresunderlie most evidence based structures”” (Anthony, (Anthony, 2003, p. 2) 2003, p. 2)
Study Peer support Study Peer support –– which is not clinical support, which is not clinical support, nor is it being friendsnor is it being friends
Going beyond studying Going beyond studying whatwhat consumers as consumers as providers do, but providers do, but howhow they do itthey do it
Importance of Consumer Social Importance of Consumer Social Interaction in Service ProvisionInteraction in Service Provision
Psychotherapy research has found:Psychotherapy research has found:
Client Client ––therapist relationship factors greater effect therapist relationship factors greater effect on outcomes of therapy than specific interventionon outcomes of therapy than specific intervention
Research Reviews conclude 30% of therapeutic Research Reviews conclude 30% of therapeutic change due to therapist effects & 15% to specific change due to therapist effects & 15% to specific interventionintervention
Dodo bird verdict Dodo bird verdict –– all psychotherapies deserve the all psychotherapies deserve the prizeprize
Importance of Consumer Social Importance of Consumer Social Interaction in Service ProvisionInteraction in Service Provision
Do treatment cure disorders or do Do treatment cure disorders or do relationships heal people?relationships heal people?Consumers consistently rank alliance with Consumers consistently rank alliance with providers as one of the most important providers as one of the most important aspects of services aspects of services (Ware, 2004)(Ware, 2004)
Importance of Consumer Social Importance of Consumer Social Interaction in Service ProvisionInteraction in Service Provision
Conclusion that in addition to Conclusion that in addition to whatwhat providers do, providers do, how how they do it impacts outcomesthey do it impacts outcomes
Assume how consumers serve other consumers Assume how consumers serve other consumers differs from nondiffers from non--consumersconsumers
Need to study social interaction process of service Need to study social interaction process of service delivery in both consumers & nondelivery in both consumers & non--consumers consumers
Are there differences in processes? Are there differences in processes? e.g., more recovery or not?e.g., more recovery or not?
No need for more RCTs of conventional No need for more RCTs of conventional service delivery by consumersservice delivery by consumers–– Likely get same resultsLikely get same results–– Learned consumers capable of conventional Learned consumers capable of conventional
MH MH servserv. provision. provision–– Not question that needs to be addressed in Not question that needs to be addressed in
US US –– may differ in Japanmay differ in JapanDetermine effectiveness of COS Determine effectiveness of COS ––consumer initiativesconsumer initiatives
RCTs on consumer initiatives RCTs on consumer initiatives –– not feasiblenot feasible–– Unclear what comparison condition isUnclear what comparison condition is–– Voluntary nature Voluntary nature –– few will attend few will attend –– failed study of failed study of
self help group by Kaufman self help group by Kaufman –– 16% in both exp & 16% in both exp & con attended self helpcon attended self help
–– Those who attend COS Those who attend COS –– do differ somewhat from do differ somewhat from those who do not those who do not -- use for different reasonsuse for different reasons
–– Users of COS overlap with conventional service Users of COS overlap with conventional service users users –– many COS participants use conventional many COS participants use conventional MH MH servserv. . –– Few conventional Few conventional mhmh users attend users attend COS COS
Conclusion Conclusion –– Future ResearchFuture Research
More research on environmental context & More research on environmental context & organizational process, climate, & structureorganizational process, climate, & structureResearch on interpersonal process of consumer Research on interpersonal process of consumer service provision & comparison to nonservice provision & comparison to non--consumer service provisionconsumer service provisionExamination of these factors (environmental Examination of these factors (environmental context, org. process, climate, & structure, context, org. process, climate, & structure, interpersonal process) on impact of outcomes interpersonal process) on impact of outcomes for COS participantsfor COS participants
Conclusion Conclusion –– Future ResearchFuture Research
More longitudinal research More longitudinal research
More qualitative researchMore qualitative research
Case for building evidence from other than RCTsCase for building evidence from other than RCTs
Focus on different outcomes Focus on different outcomes –– recovery, empowerment, recovery, empowerment, social functioning, well being quality of life rather than social functioning, well being quality of life rather than rehosprehosp. symptom reductions. symptom reductions