Top Banner
Evaluating Transition-to-Work Programs: Utilizing a Cross-Systems Approach Julie J. Christensen, PhD, LMSW Susan Hetherington, PhD Strong Center for Developmental Disabilities
48

Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

Apr 25, 2018

Download

Documents

hoanghuong
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

Evaluating Transition-to-Work Programs: Utilizing a Cross-Systems Approach

Julie J. Christensen, PhD, LMSWSusan Hetherington, PhD

Strong Center for Developmental Disabilities

Page 2: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

Education

Employment

Health

Recreation

TransitionAge

Transition

AdultsandAging

UCEDDStrong Center

forDevelopmentalDisabilities

Note: Children’s Services, Transition, and Aging/Adults cross over all four focus areas. Solid lines show direct relationships, while dashed lines denote emerging or planned opportunities for collaboration.

Page 3: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

Education

Employment

Health

Recreation

ChildandYouth

Transition

AdultsandAging

UCEDDStrong Center

forDevelopmentalDisabilities

Note: Children’s Services, Transition, and Aging/Adults cross over all four focus areas. Solid lines show direct relationships, while dashed lines denote emerging or planned opportunities for collaboration.

NYSProject SEARCHTM

NYSPartnerships inEmployment Systems

Change

Employment First!

Employment FirstStateLeadership MentorProgram (ODEP)

OPWDDPathways toEmployment Evaluation

Technical Assistance,Advocacy &Outreach

ShelteredWorkshopConversion

Page 4: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

Addressing the employment gap

4

Page 5: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

5

Challenges and barriers

• Disability related factors• Lack of education and training

• Inadequate transportation options• Need for accommodations on the worksite

• Attitudinal factors and stigma• Low expectations

Page 6: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

6

Systems issues

• Differing priorities• Conflicting goals and regulatory guidance

• “Language” barriers• Inadequate funding & payer of last resort

approach

Page 7: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

7

Promoting Employment First!

A national movement that promotes employment for all people with disabilities

Objectives of Employment First!:• Person-centered planning• Building infrastructure capacity• Collaborations that increase employment• Developing strategic partnerships with business

Page 8: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

8

E1 and raising expectations

Adopting an Employment First philosophy in NYS

Belief that employment is attainable for all

Need to assess for and train towards least restrictive employment options

Integrated, competitive employment as Gold Standard:

• 20+ hours per week at prevailing wage• Fully integrated within work environment,

utilizing natural support

Page 9: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

9

Background: Employment options

Fully supported work environment

Independent work environment

Sheltered workshopsWork activity centers

Individual or groupsupported employment

Competitive employment

Page 10: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

The “Gold” Standard

10

Fully supported work environment

Independent work environment

Sheltered workshopsWork activity centers

Individual or groupsupported employment

Competitive employment

Defining competitive employment:• 20+ hours per week• Prevailing wages with commensurate benefits• Fully integrated environment

Page 11: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

Lots of Work to Do

Fully supported work environment

Independent work environment

Sheltered workshopsWork activity centers

Individual or groupsupported employment

Competitive employment

Source: National Core Indicators (2013-2014)

21%12%

79%88%

0%

20%

40%

60%

80%

100%

18-34(N=150)

All Ages(N=432)

Employed in the community (paid)

Not Employed

Page 12: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

WIOA (2014)l Enhanced collaboration between core

partnersl Expansion of education and training optionsl Increased emphasis on youth workforce

trainingl Consistent accountability measures

Page 13: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

13

Project SEARCHTM: A collaborative model

© CCHMC 1/3/06

Vocational Rehabilitation

School District

Business

Supported Employment Agency

Developmental Disabilities Agency

Page 14: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

14

Project SEARCHTM: Key concepts

• Business led collaboration with education, rehabilitation and long term support

• Braided funding

• Immersion and impact • Training in real work settings

• Low risk, low cost for business • Hiring students who are “good fit”

Page 15: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

Annual school schedule

15

Page 16: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

Current programs in Monroe County, NYl 4 programs

l University of Rochester Medical Center l City of Rochesterl Wegmansl Delmonte Hotel Group (adult program)

Page 17: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

Project outcomes: The good newsl Promising results in first six years

l 144 participantsl Completion rate: 85%

l 79% of program graduates transitioned into competitive, integrated employmentl 57% hired by partner host business

l Graduates work an average of 23 hours per week at $8.51/hour at hire (range: $7.25 = $12.43)

Page 18: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

Monroe County, NY outcomesl Graduates who maintain employment are

growing in their positions

18

Page 19: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

Monroe County, NY outcomesl Maintaining employment over time

(job retention)

19

0%10%20%30%40%50%60%70%80%90%

12 months 24 months 36 months 48 months

% PS Grads Remaining Employed

OPWDD Job Retention Data

Avg. Rate of Employment for People with Disabilities

Page 20: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

Monroe County, NY outcomesl Impact on income for students receiving SSI

20

SSI = $808/month

23 hours @ $8.51 = $196 x 4 = $784/mo. (wages)

$784 – ($65 + $20) = $699 (adjusted income)$699/2 = $350 (subtracted from SSI)$808 – $350 = $458 (new SSI amount)

$784 + $458 = $1,242/month

Debunking myths:• Reduction in SSI has NO IMPACT on Medicaid health benefits

(up to earning $46,000 annually)• Loss of job or hospitalization allows for AUTOMATIC REINSTATEMENT of SSI

Page 21: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

NYS Project SEARCH

21

Programs in operation (n=14)

Programs in development (n=2)

Page 22: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

22

Goal of program evaluation

Strengthen program • Improved outcomes

• Enhanced cross-systems collaboration and communication

• Increased fiscal efficiency

Page 23: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

23

Program evaluation approach

Process and outcome evaluation

Continuous and ongoing feedback and assessment

Data from multiple stakeholders (e.g., student teacher, job coach, business supervisor, family)

Page 24: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

24

Objectives of program evaluation

Identify factors that contribute to success

Identify potential barriers to success

• Why participants don’t complete• Why participants who complete don’t pursue

competitive employment

• Why participants who obtain competitive employment are unable to maintain employment over time

Page 25: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

25

Program evaluation tools

Weekly internship assessment (completed by teacher and job coach)

Self-assessment (completed by student)

JOBS scale (completed by internship site supervisor)

Internship rotation progress reviews (3x/year)

Page 26: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

26

Weekly internship assessment

Completed by teacher and job coach

Reviewed at the end of each month• Opportunity to learn from different perspectives

Action plan developed to address areas of needed improvement• Includes both instructional and job coaching

supports

Weekly internship assessment

Page 27: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

27

Student internship assessment

Completed at the end of each month

Opportunity to compare self-rating to assessments completed by teacher and job coach• Encourage self-awareness and honest self-

reflection• Practice receiving feedback• Preparation for performance appraisal

Page 28: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

28

JOBS scale (Brady & Rosenberg, 2002)

Completed by internship supervisor at the end of each internship rotation

Standardized measure of readiness for competitive employment

JOBS scale

Page 29: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

29

JOBS scale (Brady & Rosenberg, 2002)

Reviewed by team (including VR counselor)• Opportunity to learn from business perspective• Compared against teacher/job coach weekly

assessments

Measure of progress towards goal of competitive employment• Identify potential barriers to employment and

develop action plan

JOBS Scale Score Sheet

Page 30: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

30

Internship progress review (3x/year)

Includes student, family, education team, job coach/adult agency, VR, DD, MSC, other appropriate supports

Goal:• Discuss potential for readiness for competitive

employment by end of program year• Develop appropriate Employment First! exit plan • Address logistics and systemic barriers to

employment (e.g., referral to outside agencies for supports, regulatory issues with independent living)

Page 31: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

31

It’s time for some group work!

Page 32: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

32

Let’s practice with the tools!

RED = Student

ORANGE = Instructor

YELLOW = Job coach

GREEN = Business (internship supervisor)

BLUE = VR and DD

PURPLE = Family

Page 33: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

33

Let’s practice with the tools!

In your groups, conduct a mid-year review:

• Student, instructor, job coach, site supervisor to each share strengths and challenge areas

• Family to share desires and concerns for student

• VR & DD to listen to all input

Page 34: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

34

Let’s practice with the tools!

As a group, discuss:

• Is this student ready to begin job development?

• What challenge areas does the student need to address during the upcoming internship?

• What barriers remain that might inhibit the ability to transition into competitive employment?

• What resources can assist with these barriers?

Page 35: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

35

Let’s practice with the tools!

Page 36: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

36

Case study: Gary

20yo w/ CP

Utilizes motorized wheelchair independently; has some ability to walk with supports

Interest in food services or office work

Page 37: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

37

Case study: Gary

Weekly teacher and job coach assessments initially focused on mobility concerns and quantity/speed of work

Supervisor JOBS assessment identified significant behavioral concerns

Student acknowledged behavior concerns on self-assessment

Page 38: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

38

Case study: Gary

Mid-year review• Agreement that student had work skills to be

competitively employed• Behavioral concerns as significant barrier to

employment

Page 39: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

39

Case study: Gary

Mid-year review•Agreement that student does not yet have necessary skills to independent living•Group home regulations as barrier to pursuit of competitive employment

• Pushing workshop placement (applications due in March)

•VR and DD not in support of workshop placement•Adult agency acknowledged that vocational goal might not be met within necessary timeframe

Page 40: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

40

Case study: Gary

Mid-year review outcomes:• Student and family willing to pursue outside

counseling to address behavior concerns• Group home willing to work with partners

explore other paths to employment beyond workshop placement

• VR, DD and adult agency to develop back-up plan for options if competitive employment not obtained by housing deadline

Page 41: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

41

Let’s compare!How did your group do?

(Understanding that you had a brief period of time and limited information…)

Page 42: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

42

Summary: Benefit of evaluation

Promotes opportunities for increased conversation and group learning across systems

Page 43: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

43

Additional thoughts

What happens when there is disagreement between teacher and job coach?• Most frequently teacher gives higher marks

(awarded for effort vs. evaluated against employer expectations)

• Opportunity to cross-train

Page 44: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

44

Additional thoughts

What happens with the business’ assessment does not agree with the team’s assessment?• Opportunity for discussion and adjustments:

• Is there a breakdown in communication?• Is the business being too critical (holding to

different standard)?• Is the business not being critical enough (not

holding student accountable)?

Page 45: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

45

Additional thoughts

What if the student does not agree with the assessment (from teacher, job coach, or business supervisor)?• Teachable moment related to realistic self-

assessment• Helps student accept constructive criticism and

buy-in to improvement plan• No surprises on exit interview or if not provided

a positive reference

Page 46: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

46

Questions & discussionStrong Center for Developmental Disabilities (UCEDD)

University of Rochester Medical Center 601 Elmwood Ave., Box 671

Rochester, NY 14642

Julie Christensen, Director of Employment Programs585-273-3713

[email protected]@uiowa.edu

Susan Hetherington, Co-Director (SCDD)585-275-6608

[email protected]

Page 47: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

47

Page 48: Evaluating Transition-to-Work Programs: Utilizing a …apse.org/wp-content/uploads/2016/07/Evaluating-Transition-to-Work...l Wegmans l Delmonte Hotel Group (adult program) Project

48

Appendices

Evaluation Tools