Julie R. Brown Director, Milwaukee Child Welfare Partnership for Professional Development (MCWPPD)

Presentation Author, 2006

Training for resultsAn instructional model for developing basic safety

intervention skill in new child welfare staff

Julie R. BrownDirector, Milwaukee Child Welfare Partnership for

Professional Development (MCWPPD)Helen Bader School of Social Welfare

University of Wisconsin-Milwaukee

Helen Bader School of Social Welfare

Milwaukee Child Welfare Partnership for Professional Development (MCWPPD)

• Serves 500+ staff of Bureau of Milwaukee Child Welfare (BMCW) and 800+ licensed foster families

– Largest university-based partnership in WI

• BMCW is state agency in otherwise county-based state

– Public/private partnership

– CPS=public

– Ongoing services=contracted private agencies

2

Safety Intervention Training Academy

Designed to prepare new staff to fulfill fundamental role in assuring child safety…

in accordance with WI state standards (ACTION for Child Protection Model)…

at a basic level of proficiency.

3

Why a new approach?• Response to weaknesses of “survey”

model – Overview of many important topics; limited skill

building

– Limited focus on learning (vs. “covering” content)

– Risks clouding job purpose in litany of concepts, tasks, activities

– Low information retention

– Leap to on-the-job application too great 4

Helen Bader School of Social Welfare

Vision for Academy model

Skill-based

Instructional strategies chosen,

evaluated and revised to promote

demonstrable skill

Professional/Ethical

Roots practice in professional standards

Protected environment for skill practice and feedback before working with real

families

Strategic

Focus on safety intervention

centers initial job definition and

grounds subsequent

learning

Rigorous

Clear standards for competency at

each level

Consistent evaluation

tools/processes

Basic competency required to move from one level to

the next

Relationally informed

Built on collaboration,

models collaboration

Builds relationships with supervisors and

peers

Promotes cultural competency

5

Academy structure

Formal Instruction

Classroom application

Field application

Skill evaluation

• Statewide requirements• Local case planning model

• Drills/preparation• Practice with actors• Peer and instructor feedback

• Structured activity• Field instructor feedback

• Proficiency of phase content• Readiness for next

phase/casework

6

Learning Phases: Layers of increasingly complex application

Assume case load with close supervision

Apply in field casework with trained field mentor/supervisor

Apply concepts classroom exercises

Learn safety intervention

concepts

7

Key roles and relationships

• UW-MCWPPD staff

• Collaborate with agency leadership

• Design, plan, manage, instruct

• Develop Training Supervisors as performance coaches

• Agency partners– Training supervisors

• Lead field application, coaching, mentoring

– Agency leadership

• Collaborate, advise on process

• Give feedback on results

8

Helen Bader School of Social Welfare

Skill Evaluation• End-of-phase evaluations

– test skills taught and practiced –nothing new at evaluation!

• Evaluation panels score performance– Consistent evaluation rubrics

– UW-M, Training Supervisor and agency leadership representative

– Pass/no pass (70% cut off)

•

9

Helen Bader School of Social Welfare

Skill evaluation as learning strategy

• Designed to mirror performance required on-the-job

• Provide performance feedback critical to building confidence

• For example…response to “no pass” – Participant self-assessment guided by panel

– Customized re-teaching/re-evaluation plan

10

Helen Bader School of Social Welfare

Presentation Author, 2006

Outcome and process measures

Helen Bader School of Social Welfare

Outcome measures

• Competence in safety intervention

• End-of-phase evaluations

• 6-month and 12-month post-graduation re-evaluations

• Ongoing feedback from participants and leadership

• Emotional adjustment/lack of burnout

– Maslach Burnout Inventory (3-, 6- and 12-month administrations)

12

Helen Bader School of Social Welfare

Process measures

13

• Ongoing monitoring and refinement of instructional processes

• Participant feedback– “Formal” collections at 6- and 12-months post-

graduation; Informal collections ongoing

• Ongoing collaboration with leadership

Helen Bader School of Social Welfare

Participant profiles• 289 Graduates to date

• 60%2 or fewer years of experience

• 88% femaleGender

• 56%--27 years old or younger• 24%--27 to 33 years old

Age

• 24% BSW• 24% MSW• 43% BA/BS in allied field

Education

14

Presentation Author, 2006

Significant and suggestive results

15

Passing rate at each phase--significant

16

Helen Bader School of Social Welfare

One-sample binomial test; sign<.05

Significant differences in mean scores between evaluation periods

Phases and significancelevels

Phase 2

Phase 3 Phase 4 6 months 12 months

Phase 2 .002 .000 .029

Phase 3 .000

Phase 4 .038 .018

6 months

12 months

17

Related-samples Wilcoxon Signed Rank TestSign level <.05

Hypothesis• Drop off between Phase 4 and 6 month follow-up

suggests weak, sporadic or inconsistent reinforcement on-the-job

– Led to renewed emphasis on developing supervisory skill

• Initial grasp of safety intervention concepts significant to subsequent application

• Initial application to “real” case significant to subsequent application on-the-job

18

No significant relationships between scores at any phase and…

Any profile variable (age, education, experience, cohort, etc.)

Job function

Employing agency

Training supervisor

Number of supervisors

Leaving employment within first 12 months

19

Maslach Burnout Inventory (MBI) • Used to identify emotional adaptation to the job

– Potential for burnout precursor to turnover (McMurtry, et al.)

• Subscales measure…

– Emotional exhaustion (EE)

– Depersonalization (DP)

– Personal Accomplishment (PA)

• 3-, 6- and 12 month administrations

20

MBI results: Initial comparisons to national social service scores

Administration and Subscale

UW-MCWPPD sample medians

National medians

3-month EE 12.00* (n=229) 21.350

6-month EE 25.00* (n=91)

12-month EE 20.50 (n=20)

3-month DP 4.00* (n=225) 7.460

6-month DP 8.00 (n=92)

12-month DP 6.00 (n=19)

3-month PA 37.00* (n=228) 32.750

6-month PA 36.00* (n=92)

12-month PA 37.00* (n=20)

21One-sample Wilcoxon Signed Rank Test; sign<.05

Caveats

• Diminishing sample sizes over administrations significantly tempers conclusions

• Sample sizes reflect– Staff turnover– Logistical complications—now mostly

solved

22

Further questions…• Adjustment to full case load may account for

sharp increases in EE and DP between 3 and 6 months

– Is leveling off/decrease at 12 months a significant change?

• Significant differences between local findings and national norms

– Workforce related? Training related? Organizational culture related?

• Stable PA scores – Work demands “externalized” such that

confidence and self-efficacy are maintained?23

6- and 12-month participant feedback

• Assessing danger threats/applying danger threshold

• Assessing parental protective capacity• Engaging families

Most success applying on-

the-job

• Applying danger threshold• PCFA (Case planning)• Creating safety plans• Differentiating risk from safety• Engaging families

Most difficulty applying

24

More 6- and 12-month feedback

• Application/practice with actors• Group discussion/application to case

examples• Learning safety concepts• Evaluation panels

Most helpful teaching techniques/classes?

• Regular case reviews with practice advice

• Continuing to challenge and teach us• More coordination among UW-M,

agencies and court on “safety” vs. incident-based approach

Support needed now from UW instructors

or agency supervisors?

25

Continuous improvement processes

Outcome/process assessment—1st in 2014

Biannual norming sessions

Quarterly planning retreats

Biweekly “check in” meetings

26

Other lessons learned….

• Relationships with agency leadership are critical for launching and sustaining model– Trust and transparency help manage

mutual vulnerability– Role clarity helps to…and is always a

work in progress!• Integration of field practice requires

specific structure and oversight27

28

Contact Information:

• Julie R. Brown

• jrbrown @uwm.edu

• (414) 964-7412

Helen Bader School of Social Welfare