Class 5: Evidence Based Interventions and logic models UTA SSW, SOCW 6371 Community & Administrative Practice UTA school of social work Dr. Dick Schoech.

Class 5:Class 5: Evidence Based Interventions Evidence Based Interventions and logic models and logic models

UTA SSW, UTA SSW, SOCW 6371SOCW 6371Community & Administrative PracticeCommunity & Administrative Practice

UTA school of social work UTA school of social work Dr. Dick SchoechDr. Dick Schoech

Copyright 2009 (permission required before use)Copyright 2009 (permission required before use)

Suggest printing slides for class using: Print | Handouts | 3 slides per page | grayscale optionsSuggest printing slides for class using: Print | Handouts | 3 slides per page | grayscale options

Overview of Sessions To DateOverview of Sessions To Date

1.1. Generalist macro practice history, change process, Generalist macro practice history, change process, roles, levels of interventionroles, levels of intervention

2.2. Theories, perspectives, values Theories, perspectives, values

3.3. Problem definition, theories of needProblem definition, theories of need

4.4. Capacities and needs assessmentCapacities and needs assessment

5.5. Evidence based interventions & logic modelsEvidence based interventions & logic models

6.6. Program descriptions, community linkagesProgram descriptions, community linkages

Learning Objectives for TodayLearning Objectives for Today

Understanding evidence & decision makingUnderstanding evidence & decision making Understanding evidence based practice (EBP) & Understanding evidence based practice (EBP) &

similar concepts (evidence informed practice is the similar concepts (evidence informed practice is the term used at the UT Arlington SSW)term used at the UT Arlington SSW)

Understand difference between EBP & optionsUnderstand difference between EBP & options Understand traditional EBP perspectiveUnderstand traditional EBP perspective Understand traditional EBP process Understand traditional EBP process Understanding logic models and their useUnderstanding logic models and their use Understand a performance mana EBP approachUnderstand a performance mana EBP approach

Traditional Approaches to Decision MakingTraditional Approaches to Decision Making Common sense approach to decision making – what’s Common sense approach to decision making – what’s

common to me makes sense – can be very limitingcommon to me makes sense – can be very limiting

Expert approach (but who is an expert)Expert approach (but who is an expert)

Theory approach (too vague for detailed decisions)Theory approach (too vague for detailed decisions)

Often if lacking evidence, use the bureaucratic Often if lacking evidence, use the bureaucratic structure (1:8 ratio staff to supervisors) to control for structure (1:8 ratio staff to supervisors) to control for variance, but expensive (e.g., CPS)variance, but expensive (e.g., CPS)

Decision making too complex for management control Decision making too complex for management control so Street Level Bureaucrats (Lipsky)so Street Level Bureaucrats (Lipsky)

More Scientific Approaches to Decision MakingMore Scientific Approaches to Decision Making

Best practicesBest practices = finding & using best possible way of doing = finding & using best possible way of doing something something established by history, utilization review, CQI, committees, etc. established by history, utilization review, CQI, committees, etc.

Data miningData mining = searching agency data for trends, predictions, = searching agency data for trends, predictions, performance on objectives and standards, etc. performance on objectives and standards, etc.

Data visualization = Data visualization = display data in an easy-to-understand format so display data in an easy-to-understand format so decisions are obvious from the datadecisions are obvious from the data

Research = Research = application of scientific principles to practice application of scientific principles to practice Program evaluation = Review process, outcome, and impact of intervention, e.g., Program evaluation = Review process, outcome, and impact of intervention, e.g.,

pre-post client measures, focus groups, key informant interviews, public opinion pre-post client measures, focus groups, key informant interviews, public opinion surveys surveys

Clinical trials = experimental designs with randomly assigned control group & Clinical trials = experimental designs with randomly assigned control group & control for intervening variables. Look for significance & effect size control for intervening variables. Look for significance & effect size

Fidelity = A measure of how well the intervention was replicated. For example, Fidelity = A measure of how well the intervention was replicated. For example, dosage is often important. If research shows that 15 training sessions are needed dosage is often important. If research shows that 15 training sessions are needed and clients on the average only attend 10 sessions, then fidelity is compromised.and clients on the average only attend 10 sessions, then fidelity is compromised.

Classic EBP PerspectiveClassic EBP Perspective

Placing the client’s benefits first, evidence-based practitioners adopt a Placing the client’s benefits first, evidence-based practitioners adopt a process of lifelong learning that involves continually posing specific process of lifelong learning that involves continually posing specific questions of direct practical importance to clients, searching questions of direct practical importance to clients, searching objectively and efficiently for the current best evidence relative to objectively and efficiently for the current best evidence relative to each question, and taking appropriate action guided by evidence. each question, and taking appropriate action guided by evidence. http://www.evidence.brookscole.com/http://www.evidence.brookscole.com/

Evidence-based practiceEvidence-based practice is "a total process beginning with knowing is "a total process beginning with knowing what clinical questions to ask, how to find the best practice, and how what clinical questions to ask, how to find the best practice, and how to critically appraise the evidence for validity and applicability to the to critically appraise the evidence for validity and applicability to the particular care situation. The best evidence then must be applied by particular care situation. The best evidence then must be applied by a clinician with expertise in considering the patient's unique values a clinician with expertise in considering the patient's unique values and needs. The final aspect of the process is evaluation of the and needs. The final aspect of the process is evaluation of the effectiveness of care and the continual improvement of the process" effectiveness of care and the continual improvement of the process" ((DePalma, 2000).).

EBPEBP ProcessProcess1.1. Ask answerable questions about the Ask answerable questions about the

decision/problem/situation decision/problem/situation

2.2. Track down the best evidence to answer the questions Track down the best evidence to answer the questions

3.3. Appraise the validity, impact, and applicability of the Appraise the validity, impact, and applicability of the evidence evidence

4.4. Determine if something needs to or can be done Determine if something needs to or can be done

5.5. Inform those involved and consider their opinions and Inform those involved and consider their opinions and expectations expectations

6.6. Integrate all forms of evidence into an action plan Integrate all forms of evidence into an action plan

7.7. Monitor and evaluate the processes and outcomes of Monitor and evaluate the processes and outcomes of the intervention the intervention

Understanding logic models and Understanding logic models and their limitationstheir limitations

Logic Model is tool for:Logic Model is tool for: Explicating and communicating program theory, Explicating and communicating program theory,

assumptions, rationale (Here is how we see it)assumptions, rationale (Here is how we see it) Assessing program feasibility (Is program needed)Assessing program feasibility (Is program needed) Developing programs (How does program work)Developing programs (How does program work) Performance management (Tie outputs to mission) Performance management (Tie outputs to mission) Program evaluation (Assess if program works & why)Program evaluation (Assess if program works & why) knowledge building (capture why things work)knowledge building (capture why things work)

Review logic models in templateReview logic models in template Reading by Savaya gives examples alsoReading by Savaya gives examples also

Problems with Current EBP ApproachProblems with Current EBP Approach Originated in medical field, not typical SW practice Originated in medical field, not typical SW practice Evidence in academic journals, not easy to useEvidence in academic journals, not easy to use Evidence is old--need evidence in real timeEvidence is old--need evidence in real time Requires new generation of trained social workersRequires new generation of trained social workers No time to track down evidenceNo time to track down evidence Difficulty of interpreting complex researchDifficulty of interpreting complex research Focuses on individual, not organization’s roleFocuses on individual, not organization’s role Evidence always lost, no “organizational learning”Evidence always lost, no “organizational learning” Few practitioners evaluation practice as per EBPFew practitioners evaluation practice as per EBP Few consider evidence when select interventions Few consider evidence when select interventions No evidence that EBP is better than other optionsNo evidence that EBP is better than other options May be culturally destructive, May be culturally destructive, e.g., e.g., Native Americans powwow Native Americans powwow

to solve family problems vs . EBPto solve family problems vs . EBP

Key EBP Changes NeededKey EBP Changes Needed

Web support e.g. help desk or travel agent exampleWeb support e.g. help desk or travel agent example

FROM: individual practitioner searching down relevant FROM: individual practitioner searching down relevant research for each caseresearch for each case

TO: organization with tools that make data readily TO: organization with tools that make data readily available so that workers can monitor their performance available so that workers can monitor their performance on well-researched, outcome based, practice on well-researched, outcome based, practice standards. standards.

Source: Schoech, D., Basham, R., and Fluke, J. (2006). A Technology Enhanced EBP ModelSource: Schoech, D., Basham, R., and Fluke, J. (2006). A Technology Enhanced EBP Model. . Journal of Evidence-Based Social Work, 3Journal of Evidence-Based Social Work, 3(3/4), 55-72.(3/4), 55-72.

CPS Example: Performance Mana EBPCPS Example: Performance Mana EBP

IndicatorIndicator1.1. Length of time to achieve Length of time to achieve

reunification reunification

2.2. Stability of foster care Stability of foster care placement placement

3.3. Foster care re-entries Foster care re-entries

4.4. Length of time to achieve Length of time to achieve adoption adoption

5.5. Recurrence of maltreatmentRecurrence of maltreatment

6.6. Incidence of child abuse Incidence of child abuse and/or neglect in foster careand/or neglect in foster care

StandardStandard1.1. At least 76.2% are reunified in less At least 76.2% are reunified in less

than twelve months from the time of than twelve months from the time of the latest removal from homethe latest removal from home

2.2. At least 86.7% had no more than two At least 86.7% had no more than two placement settingsplacement settings

3.3. Less than 8.6% re-entered foster care Less than 8.6% re-entered foster care within 12 months of a prior foster care within 12 months of a prior foster care episodeepisode

4.4. At least 32% exited care in less than At least 32% exited care in less than 24 months from the time of the latest 24 months from the time of the latest removal from homeremoval from home

5.5. Less than 6.1% had another Less than 6.1% had another substantiated or indicated report substantiated or indicated report within six monthswithin six months

6.6. Less than .57% were the subject of Less than .57% were the subject of substantiated or indicated substantiated or indicated maltreatment by a foster parent or maltreatment by a foster parent or facility stafffacility staff

Illustration of DEMOS EBP ApproachIllustration of DEMOS EBP Approach

Advantages: Performance Mana Advantages: Performance Mana EBPEBP

Quicker to implementQuicker to implement Staff correct data that impact their workStaff correct data that impact their work Responsibility shifts from the worker to the Responsibility shifts from the worker to the

organization, consistent with Deming TQM theoryorganization, consistent with Deming TQM theory Questions to ask are in organizational Questions to ask are in organizational

performance standards tied to agency goalsperformance standards tied to agency goals Shift from bureaucracy to decision supportShift from bureaucracy to decision support Instantly availableInstantly available User customized models & interfaces (e.g., User customized models & interfaces (e.g.,

my.yahoo)my.yahoo)

Disadvantages of ApproachDisadvantages of Approach Infrastructure is requiredInfrastructure is required

Agreement on performance standards is Agreement on performance standards is requiredrequired

Costly in terms of technology – human Costly in terms of technology – human services can more easily justify hiring services can more easily justify hiring people than ITpeople than IT

EBP requires a cultural changeEBP requires a cultural change

Discussion & QuestionsDiscussion & Questions