-
FORMATIVE EVALUATION:PRINCE EDWARD ISLAND
DISABILITY SUPPORT PROGRAM
Final Report
July, 2003
Prepared for:
Province of Prince Edward IslandDepartment of Health and Social
Services
Charlottetown, Prince Edward Island
Charlottetown • Prince Edward Island
In association with
LexEcon Consulting
-
Formative Evaluation:Prince Edward Island Disability Support
Program
Table of Contents
Executive Summary . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . i
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 11.1 Purpose and Organization of this
Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . 11.2 Program Description and Context . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . 3
1.2.1 Program Description . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 31.2.2 Intent . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 41.2.3 Goals and Objectives . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 5
1.3 Program Structure, Resources and Components . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.3.1
Program Structure . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
61.3.2 Resources . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 71.3.3 Program Components . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 8
1.4 Scope of the Evaluation . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 101.5 Methodology and Limitations . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 11
2. Trends in Disability Support Initiatives . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . 142.1 Background . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . 142.2 The Vision . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 152.3 Federal Role in
Disability . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 162.4
Definition of Disability . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 172.5 Types of Disability Supports . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 172.6 PEI PALS Disability Statistics . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . 192.7 The Standard Rules: A Tool to Mobilize
International Action . . . . . . . . . . . . . . . . . . . . . . .
. 212.8 Recent Policy Positions . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . 222.9 Disability Policy Success . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 242.10 Conclusion . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 25
3. Participant and Program Activity Profile . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 283.1 Participant Profile . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 283.2 Program Activity Profile . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . 33
-
Formative Evaluation:Prince Edward Island Disability Support
Program
4. Findings and Discussion . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 394.1 Benchmarking Framework . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 394.2 Program Rationale and Model . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 414.3 Program Targeting, Eligibility and
Participation . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 41
4.3.1 Program Targeting . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 414.3.2 Eligibility . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . 41
4.3.2.1 Financial Assessment . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
414.3.2.2 De-Linking of Disability Support and Financial Assistance
. . . . . . . . . . . . . . . . 454.3.2.3 Eligible Impairments . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 454.3.2.4 Eligible Supports . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . 46
4.4 Program Design and Pre-Implementation . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474.5
Program Implementation and Current Status . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . 50
4.5.1 Case File Transition . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 504.5.2 Partnerships . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 514.5.3 Communications . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 524.5.4 Administrative Processes . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 53
4.5.4.1 Assessment . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
534.5.4.2 Case Planning . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
544.5.4.3 Financial Support . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
574.5.4.4 Service Delivery Standards . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 584.5.4.5
Staff Competencies, Training and Development . . . . . . . . . . .
. . . . . . . . . . . . . . 584.5.4.6 Support Coordination . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 604.5.4.7 Community Service Providers . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 614.5.4.8 Corporate Culture . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
624.5.4.9 Case File Data Management . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 634.5.4.10
Financial Accountability . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . 644.5.4.11 Review
Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 64
4.6 Program Expenditures . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . 65
5. Program Impacts, Effects and Cost-Effectiveness . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
675.1 Key Success Factors . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . 675.2 Participants’ Satisfaction . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 68
5.2.1 Current Active Files . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 685.2.2 Closed DSP Files . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 73
5.3 Community Service Providers . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 745.4 Employment Outcomes . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 745.5 Unintended Impacts and Effects . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . 775.6 Excluded/Ineligible Populations . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 775.7 Scope and Scale of Support . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . 775.8 Cost-Effectiveness . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . 785.9 Key Performance Indicators . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . 80
-
Formative Evaluation:Prince Edward Island Disability Support
Program
6. Summary Observations . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 826.1 Current Program Status Compared to Intent . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 826.2 Implementation . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 836.3 Achievements . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . 846.4 Gaps in Program Access and
Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . 846.5 Program Relevance . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . 85
Appendices: A: Other DSP Descriptions Used by GovernmentB:
Disability Support Program Sources and Uses of FundingC: Document
Review ListD: List of Key Informant InterviewsE: References –
Section 2: Trends in Disability Support InitiativesF: Case Audit
Report
-
Formative Evaluation:Prince Edward Island Disability Support
Program
i
Executive Summary
1 Has the Disability Support Program been implemented as
intended?
2Are the Disability Support Program’s core activities (case
planning, referrals and financial support) and types of services
consistent with the program’s mandate?
3 What are the types and profiles of DSP interventions?
4 What is the profile of DSP participants?
5What impact has the separation of disability supports from
income support programming had upon employment barriers for DSP
participants?
6 What impact have DSP interventions had upon participant
employment and employability by type of intervention and
disability?
7 What impact have DSP interventions had upon participant income
and participant dependence upon welfare assistance support?
8 What impact have DSP child disability supports had upon
families with extraordinary child-rearing needs?
9 What impact have DSP interventions had in enhancing
participant independence and improving their quality of life?
10 How satisfied are DSP participants with service delivery
including reasons for dissatisfaction?
11 How effective has the support planning component of the DSP
been in establishing partnerships with other disability support
agencies?
12 Has the DSP produced any unintended impacts and effects?
13 Under the DSP, are there gaps in service delivery including
eligible persons not served, waiting lists and lack of access to
interventions?
14What key performance indicators would be useful for ongoing
assessment of the efficiency and effectiveness of the Disability
Support Program?
Program and Participant Profile
Implementation
Impacts, Effects and Cost-Effectiveness
Prince Edward Island Disability Support Program Formative
Evaluation
Evaluation Primary Questions
Exhibit 1
PurposeBaker Consulting Inc., in association with LexEcon
Consulting, was retained by the Prince EdwardIsland Department of
Health and Social Services to conduct a formative evaluation of the
province’sDisability Support Program (DSP). A formative evaluation
is a “project in progress” snapshotevaluation designed to inform
stakeholders of a program’s strengths and weaknesses at a
specificpoint in time with a goal of identifying opportunities for
improving efficiency and effectiveness.
The formative evaluation of the Disability Support Program was
centred on 14 primary questionsdeveloped by key program
stakeholders:
-
Formative Evaluation:Prince Edward Island Disability Support
Program
ii
MethodologyThe data collection methodology employed for this
evaluation was broad-based, and included:
• focus groups with disability support workers and disability
support administrators;• a literature review conducted by
Department of Health and Social Services (DHSS) staff
focusing on current practices in government-based disability
support programming;• a case audit of 30 DSP case files covering
six randomly selected from each of five health
region administrative offices;• roundtable sessions with
representatives of organizations in the disability support
community;• public input sessions;• focus groups with people
with disabilities;• expert and key informant interviews, primarily
one-to-one in-person;• a review of DHSS documents pertaining to the
development and implementation of the
DSP;• quantitative telephone surveying conducted among:
< DSP participants and/or a proxy with an active DSP
file;< former DSP participants and/or a proxy where service was
completed and the
file closed;< individuals who applied to the DSP and were
deemed to be ineligible for
financial or other reasons; and< individuals where a partial
DSP application or file was opened but never
completed or an application was approved but service was never
accessed; and• analysis of socio-demographic data of current and
former DSP participants maintained
in the program’s electronic administrative database.
Participant ProfileThe participant profile was limited to 773
open files as at 30 April 2003 where the programparticipant was
receiving recurring monthly financial support. Case distribution by
health regionwas 16% West Prince, 23% East Prince, 44% Queens, and
16% Kings. Gender distribution was 54%male/46% female while mean
age was 33 years. Seventy percent (70%) of the adult cases
werereceiving government income support in addition to the DSP
financial supports.
Participants under the age of 18 years accounted for slightly
less than one-quarter (23%) of the casefiles with young adults (18
to 29 years) accounting for 19%, adults aged 30 to 49 years 39%,
adults40 to 59 years 35%, and adults aged 60+ years 5%.
Among these cases files, 26% were coded as a physical
disability, 64% a developmental orintellectual disability, 10% both
physical and developmental/intellectual, and 0.1% mental
illness.DSP administrators use percent functioning and behaviour
scores to arrive at an overall level offunctioning of either very
low, low, medium or high. The mean percent functioning score for
all
-
Formative Evaluation:Prince Edward Island Disability Support
Program
iii
Recurring Monthly Financial Support by Major Category
41.3 $262 $15021.4 $95 $51
8.7 $159 $1071.4 $90 $404.4 $661 $576
15.2 $137 $1001.8 $99 $505.8 $101 $59
100.0 $154 $78
Community Living SupportsCommunity Participation
SupportsEmployment & Vocational SupportsGrandfathered
FSPIBIRespiteSupport CoordinationTechnical Aids & Assistive
DevicesTotal
% ofTotalSum Mean Median
PAYMENT
Exhibit 2 DSP active files receiving recurring monthlyfinancial
support (30APR03)
cases was 68.7% while the mean behaviour score was 23.8/30.
Overall levels of functioning are 38%high, 29% medium, 23% low and
10% very low.
Based on age, disability and level of functioning, there was
little homogeneity between healthregions.
There are three main DSP service categories: 1) Adult Disability
Supports (AD); 2) Child DisabilitySupports (CD); and Employment and
Vocational Supports (EV). More than half the files (52%) wereAdult
with another 23% being both Adult and Employment and Vocational,
and 24% being Child.Only 2% of the files were strictly Employment
and Vocational.
DSP financial supports are coded under approximately 48 line
items. Based on the 773 recurringpayment files analysed, community
access support is the most utilized support item, representing69%
of all files. This is followed by social/recreational
transportation appearing in 54% of all files,and extended respite
appearing in 40% of all files. The size of payments being made
rangedconsiderably both within and between regions. Overall,
supervision was the largest expense item,representing 15% of the
total monthly provincial expense at that point in time. The top
five expenseitems from this summary were:
1. supervision (15%);2. personal care (12%);3. extended respite
(10%);4. vocational alternative support (9%); and5. behavioural
care (8%).
Individual support items are aggregated into major categories.
Exhibit 2 below summarizes thesecategories as percent of total
financial support at that point in time along with the mean and
medianpayment being made.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
iv
Low Medium High1 2 3 4 5
1. Program and Policy Design and Development1.1 A sound and
accepted philosophic base exists.
1.2 Well thought out program rationale exists.
1.3Detailed and current statistical and marketplace information
about people with disabilities exists to promote sound analysis and
policy development and to accurately assess progress in meeting
policy and programming goals.
1.4 There is meaningful involvement by the disability community
in the development, implementation and direction of policies where
they are affected by the outcome.
1.5 There is a cross-disability emphasis in policy and
initiatives reflecting the inclusion of people with all types of
disabilities.1.6 Initiatives are community-based and
community-responsive.
1.7 Program coverage provides a full continuum of supports.
1.8 Persons with disabilities and/or their families will not be
impoverished in order to access publically-funded supports.
1.9 Disability support includes the provision of counselling,
guidance and planning in assisting people with disabilities and
their families in making informed choices.
1.10 Conditions of adequate resources and capacity exist. The
provision of services is fair in that a minimum baseline of support
is provided.
1.11Where people with disabilities or their families are the
purchasers of goods and services using publically-funded money, the
funding is adequate to pay fair market value for the level of
assistance required.
1.12 Services are based on need, appropriateness and
value-for-money.
1.13 Key results indicators have been determined and are
consistent with the program rationale.
2. Program Implementation
2.1 Government provides tangible evidence of leadership.
2.2A single point of entry exists for people with disabilities
to receive information or supports. Communication strategies fully
inform potential participants of the scope of support
available.
2.3Mechanisms are in place to ensure consistency, collaboration
and coordination across and within support providers, including
provincial departments and agencies, to ensure that policy and
initiatives interact for optimum impact and efficiency.
2.4 Program delivery staff are professionally competent and
well-trained.
2.5 The needs of people with disabilities and their families are
assessed in a professional, caring, equitable and timely
manner.
2.6 An effective and equitable appeal process exists.
3. Program Performance and Evaluation
3.1 Adherence to policy and program guidelines is regularly
monitored.
3.2 Policy and initiatives are modified as necessary based on
regular monitoring of impacts and outcomes and program
evaluations.
3.3 Service delivery performance is regularly monitored against
established standards and both standards and performance are
modified as necessary.
Key Success FactorsA benchmark framework for program success was
developed by the consulting team which wasthen used to subjectively
“score” the program to date. This framework is not unlike the
disabilitylens concept. The disability lens is increasingly being
used as a tool by policy and programdevelopers and analysts for
identifying and clarifying issues affecting people with
disabilities, andaddressing the impacts of policy and program
decisions on the disability community. Scoring thefactors shows
strength in the front-end “rhyme and reason” and implementation
planning butweakness in areas of actual implementation.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
v
Aggregate Negative Response Aggregate Positive Response
-38.4
-26.2
-21.4
58.4
70.7
76.1
Barriers
Independence
Quality of Life
-40.0 -20.0 0.0 20.0 40.0 60.0 80.0 100.0
Key Conditions of Living Measures
DSP Telephone Survey
Exhibit 4
Participant SatisfactionBased upon results of a telephone survey
among current and former DSP participants and/or anappropriate
proxy, recipients of DSP supports are largely a satisfied
clientele. Among currentparticipants and/or a proxy, 56% reported
that the impact of DSP supports in the past year hadmade the
participant’s life “better,” while 39% reported life to be “about
the same.” Thirty-fourpercent (34%) reported that “important new
supports” had been received in the past year, while27% reported
that changes in the past year had made it easier for the
participant do things like go towork, go shopping or attend
social/recreational activities. Among this 27%, almost
three-quartersattributed the changes to financial support received
through the DSP.
To gain insight into the conditions of living being experienced
by DSP participants, a set of surveyquestions were developed around
three dimensions: quality of life, independence and
barriers.Aggregate positive and negative responses to these
questions are illustrated below, showing a clearpositive skew.
The importance of the Disability Support Program is reflected in
the 61% of respondents (75%participants and 57% proxies) who said
that it would be difficult for the participant to stay where heor
she was presently living without the DSP supports that were being
received.
In terms of DSP staff service quality, survey participants were
generally very positive as thesummary of staff- and service-related
questions in the chart on the next page shows.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
vi
Yes No Other% % %
Does participant/proxy know who their DSW is 91.3 7.9 0.8
From among survey participants who responded YES:
Can participant/proxy talk to DSW whenever he/she needs 86.4 4.5
9.1
Does DSW understand participant's disability 81.9 5.1 13.0
Does DSW understand participant's needs 80.7 6.9 12.4
Does DSW do the things they say they will 81.7 5.1 13.2
Has DSW helped participant/proxy decide what supports and
services to use 66.1 20.1 13.8
Has DSW helped participant feel better about being able to do
things for himself/herself 57.5 23.4 19.1
Is participant/proxy happy overall with quality of help received
from his/her DSW 80.1 8.3 11.6
Has particpant/proxy been unhappy in the past year about any DSP
support 17.7 78.9 3.4
From among survey participants who responded YES:
Did participant tell DSW or anyone else 85.6 10.0 4.4
Was participant/proxy happy about how the complaint was fixed
20.8 75.3 3.9
Disability Support Program Telephone Survey
Disability Support Worker Service Quality Questions
Exhibit 5
Current Program Status Compared to IntentThe intent of the
Disability Support Program is summarized as follows:
1. to make access to publically-funded disability support more
equitable;2. to strengthen the social component (i.e. action
planning) of disability support;3. to increase employment-based
financial independence among people with disabilities;4. to make
publicly-funded disability supports available to a broader base of
Island
residents;5. to give disability support consumers more choice
and purchasing control; and6. to achieve a holistic approach to
service delivery with the aim of funding and providing
the mix of services that gives the best overall result.
Disability Support ProgramProgram Status Compared to Intent
1. To make access to publicly-funded disability support
moreequitable
It could be argued that government has “shuffled the deck” with
respect toequality of access with negligible net gain. Initiatives
in this area wereconstrained by a strong corporate culture of cost
containment/costavoidance and considerable opportunity appears to
exist to enhanceequality of access to publicly-funded disability
support.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
vii
Disability Support ProgramProgram Status Compared to Intent
2. To strengthen the socialcomponent (i.e. action planning)of
disability support.
The Disability Support Program is promoted as “a social program
with afinancial component.” The reality of the situation is that
the DSP is afinancial program with a social component. It would
appear that amajority of applicants to the program have a clear
idea of what theirwants/needs are and see little benefit in the
social component. That beingsaid, there are sound examples of
families who had a strong need forplanning support and clearly
benefited through their contact with the DSP.
3. To increase employment-based financial independenceamong
people with disabilities.
Employment-based financial independence is dependent upon
anindividual obtaining and/or retaining permanent part-time or
full-timeemployment. It would appear that some success has been
achieved, butlargely among people with disabilities who are higher
functioning.Whether this success can largely be attributed to the
DSP or the federal-provincial cooperative Employment Assistance for
People with Disabilities(EAPD) agreement is unclear. The vision of
DSP developers to movepeople with disabilities off government
income support to self-supportingfinancial independence does not
appear to have happened to the degreeanticipated.
4. To make publicly-fundeddisability supports availableto a
broader base of Islandresidents.
This has been an unquestionable success.
5. To give disability supportconsumers more choice andpurchasing
control.
This has been a qualified success as success is partially
tempered by:• a clear indication that a material segment of the
disability community
does not want this responsibility; and• the fact that service
resources and capacity are lacking in some
areas of need and in some regions of the province.
6. To achieve a holisticapproach to service deliverywith the aim
of funding andproviding the mix of servicesthat gives the best
overallresult.
In order for the Disability Support Program to be a centre-point
fordisability support planning requires that a strong network of
partnershipsexist both within the disability community and
cross-government. It wouldbe anticipated that:• the assets that
government has in established community service
providers would be leveraged;• the knowledge and experience of
the disability community would be
incorporated into program service and policy monitoring
andenhancement; and
• some type of formal protocol or framework would exist
forcoordinating disability supports and a continuum of support
acrossgovernment.
This does not exist with the Disability Support Program and
servicedelivery appears to be largely centred on: 1. identify the
need; and 2. fundthe need if eligible. It is material, however,
that much of this situationappears to be consumer driven.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
viii
ImplementationDevelopers of the Disability Support Program faced
a considerable challenge integrating threedisparate programs into
one new program with a new approach to service delivery requiring
afundamentally new way of thinking. Pre-implementation was
thoroughly planned but pre-implementation execution exhibited a
notable weakness in the degree of inclusion of the
disabilitycommunity in program development and in
pre-implementation communication with the disabilitycommunity.
Given the number of case files requiring transition, the
vagaries of the new program, and theobvious learning curve
required, the implementation time frame was likely ambitious given
theresources available. This contributed to the creation of initial
case files and case management thatwere outside the vision of the
program. Material implementation challenges include:
• service delivery inconsistency across the province in terms of
assessment, policyinterpretation and support funding;
• a corporate culture of cost containment/cost avoidance which
manifested itself in aconservative approach to service delivery;
and
• community-based shortages in resources and capacity for
certain types of disabilitysupports.
Fully understanding the seriousness of service delivery
variances is difficult as no effective meansof benchmarking,
monitoring and improving/correcting service delivery is in
place.
Material weaknesses of program implementation include:• a lack
of communication with the disability community and a failure to
provide
adequate program information;• failure to engage the disability
community in the review/modification/enhancement of
policy in a meaningful manner; and• failure to establish
mechanisms to ensure consistency, collaboration and
coordination
across and within support providers, including government.
AchievementsThe Disability Support Program has successfully
established itself as a flexible, person-centred,needs-based
program that is independent of income support programming. The
program hasbroadened access to government-funded disability
supports for Islanders and has undoubtedlyimproved quality of life
and community participation for many.
EmploymentEmployment outcomes are unclear, partly because of the
difficulty many people with disabilitieshave finding permanent
full-time or part-time work as opposed to on-the-job-training
experiencesor short-term work projects. It is also likely that some
of the apparent successes through job creation
-
Formative Evaluation:Prince Edward Island Disability Support
Program
ix
partnerships would have occurred regardless of whether the DSP
existed or not.
It is understood that to this point in time, specific outcomes
with respect involving people withdisabilities in the labour market
and moving people with disabilities off income support have notbeen
achieved. From the DSP telephone survey, only one-third of the
participants were reported ashaving had paid employment of any kind
in the past year with 70% of this cohort (24% of all
surveyrespondents) being reported as having regular paid
employment. Among this regular paidemployment group, the mean
number of hours worked “last week” was 15.
On a positive note, among those who had some type of paid
employment in the past year, it wasreported that:
• 23% started paid employment because of supports received
through the DSP;• 16% started better paying employment because of
supports received through the DSP;
and• 18% started new paid employment that was more “fun” because
of supports received
through the DSP.
Gaps in Program Access and ServiceNotable probable gaps in
program access identified include:
• the model used to calculate “net income” which is not an
accurate reflection of disabilitysupport need;
• the “18 to 24" age parameter for the inclusion of
parental/guardian income for incometesting which does not exhibit
strongly defensible rationale;
• family relationships for the inclusion of family income for
income testing, particularlywith respect to guardians, but also in
regard to a married or common law spouse who isnot a parent of the
person with a disability;
• the inequality of grandfathering supports while new applicants
are denied access tothese supports without access to any
alternative source of service;
• the lack of a vision and action plan that will see equality of
access to publicly-fundeddisability supports for all people with
disabilities; and
• the review process which does not adequately represent the
interests of DSP applicantsor participants.
Notable gaps in program service include:• lifetime government
funding ceilings;• early intervention supports for children;•
disability supports for the more severely disabled, particularly
children;• a system framework that is responsive to a continuum of
planning and supports as
individuals transition from one stage of life to another;•
respite; and
-
Formative Evaluation:Prince Edward Island Disability Support
Program
x
• transportation, particularly employment-related;
Cost-EffectivenessAt this point in time it is very difficult to
measure the cost-effectiveness of the Disability SupportProgram.
This is largely due to the lack of quantitative benchmarks for
measuring cost-effectiveness. The challenge to measuring
cost-effectiveness is compounded by the limitedknowledge that
government has of the overall disability support needs of the
disability communityin Prince Edward Island and there is a
fundamental need to establish quantitative researchprotocols for
measuring outside the DSP “box.”
Through this evaluation process, initiatives such as the
telephone survey have explored ways ofmeasuring dimensions such as
independence, barriers, quality of life and program
satisfactionamong DSP participants. This would be a good starting
point for establishing benchmarks forfuture measurements. Exhibit
59 on page 80 provides a single-page comparative summary by
healthregion for a number of participant profile, satisfaction and
financial activities examined during thisevaluation.
Program RelevanceThe need for an initiative such as the
Disability Support Program is strongly supported by theevaluation.
As it exists, however, it is an innovative financial service with a
social planningcomponent and not the reverse situation that was
envisioned. However, this is not necessarily badin and of itself as
there is an obvious community need and the effectiveness and impact
of theprogram could be enhanced noticeably with some policy and
service delivery adjustments andadditional resources.
If, however, the Disability Support Program is to be a holistic
centre-point for disability supportplanning, then there is likely a
much more challenging task ahead. The reasoning for this is that
anall-encompassing provincial policy framework is needed to:
• establish responsibility, authority and accountability that
ensures a coordinatedapproach in the development of disability
supported-related policies, legislation,programs and services;
and
• ensure the cohesive and seamless delivery of
disability-related programs to people withdisabilities.
Through this central point, effective cross-government
personalized planning can then be deliveredand supports coordinated
such that people with disabilities can reach their full potential
at allstages of their life. The Government of Prince Edward Island
has exhibited innovation andleadership in establishing the
Disability Support Program. It may wish to continue along this
pathby taking on this challenge.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
1
1. Introduction
1.1 Purpose and Organization of this ReportResponding to an open
competitive call for proposals (PEIGOV RFP#223), Baker Consulting
Inc., inassociation with LexEcon Consulting, was awarded a contract
by the Province of Prince EdwardIsland’s Department of Health and
Social Services (DHSS) to conduct a formative evaluation of
theprovince’s Disability Support Program (DSP). For this
assignment, the consulting team reported tothe PEI Disability
Support Program Evaluation Management Team, consisting of:
• Brian BertelsenCoordinator of Disability SupportsSocial Policy
Development, Department of Health and Social Services
• Bob CreedManager of Disability Support Programs and Employment
Enhancement ProgramsQueens Health Region
• Jean FallisDirector of Social Supports and Senior
ServicesSocial Policy Development, Department of Health and Social
Services
• Bethany MacKayCase Management SupervisorChild and Family
Services, West Prince Health
• Olive MoaseHealth System Researcher, Evaluation ServicesSocial
Policy Development, Department of Health and Social Services
• Anne-Marie SmithManager of Strategic Planning &
CommunicationsCorporate Services, Department of Health and Social
Services
• Jill WakelinDisability Supports Client Services
OfficerDisability Supports and Services, Social Policy Development,
Department of Health and SocialServices
Within the context of this assignment, an evaluation can be
described in general terms as a processwhereby judgement is made
about the impact, value, worth or merit of a program, activity,
productor entity through systematic investigation and analysis. A
formative evaluation is a “project inprogress” snapshot evaluation
designed to inform stakeholders of strengths and weaknesses at
a
-
Formative Evaluation:Prince Edward Island Disability Support
Program
2
1 Has the Disability Support Program been implemented as
intended?
2Are the Disability Support Program’s core activities (case
planning, referrals and financial support) and types of services
consistent with the program’s mandate?
3 What are the types and profiles of DSP interventions?
4 What is the profile of DSP participants?
5What impact has the separation of disability supports from
income support programming had upon employment barriers for DSP
participants?
6 What impact have DSP interventions had upon participant
employment and employability by type of intervention and
disability?
7 What impact have DSP interventions had upon participant income
and participant dependence upon welfare assistance support?
8 What impact have DSP child disability supports had upon
families with extraordinary child-rearing needs?
9 What impact have DSP interventions had in enhancing
participant independence and improving their quality of life?
10 How satisfied are DSP participants with service delivery
including reasons for dissatisfaction?
11 How effective has the support planning component of the DSP
been in establishing partnerships with other disability support
agencies?
12 Has the DSP produced any unintended impacts and effects?
13 Under the DSP, are there gaps in service delivery including
eligible persons not served, waiting lists and lack of access to
interventions?
14What key performance indicators would be useful for ongoing
assessment of the efficiency and effectiveness of the Disability
Support Program?
Program and Participant Profile
Implementation
Impacts, Effects and Cost-Effectiveness
Prince Edward Island Disability Support Program Formative
Evaluation
Evaluation Primary Questions
Exhibit 6
specific point in time with a goal of identifying opportunities
for improving efficiency andeffectiveness.
The Disability Support Program has not been evaluated since its
official launch in October, 2001.The program is considered by
program management to be in a formative stage at this point in
timeand typical areas of evaluation interest include:
• program rationale and implementation;• participant profiles;
and• initial impacts, effects and cost-effectiveness.
The formative evaluation of the Disability Support Program was
centred on 14 primary questionsdeveloped by key program
stakeholders. These questions are summarized below.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
3
This report is organized as follows:• Section 1: Introduction
provides background information on: a) the nature of the
assignment; b) the DSP, its intent, goals and objectives; and c)
the scope of the evaluation,the methodological approach and
limitations encountered.
• Section 2: Trends in Disability Support Initiatives is a
summary report on a literature reviewconducted by DHSS staff and is
intended to bring to the evaluation a global perspective ofcurrent
practices in government-based disability support programming.
• Section 3: Participant and Program Activity Profile provides
readers with a statisticaldescriptive analysis of DSP participants
who were receiving recurring monthly financialassistance as at 30
April 2003.
• Section 4: Findings and Discussion reports on the findings of
the evaluation through pre-implementation planning, implementation
and the current status of a number of programcomponents.
• Section 5: Program Impacts, Effects and Cost-Effectiveness
draws summary conclusionsabout the success of the program to date
in terms of impact, outcomes, program satisfactionamong the
disability community and value-for-money.
• Section 6: Summary Observations summarizes key program
accomplishments to this pointin time along with program strengths
and weaknesses.
1.2 Program Description and Context1.2.1 Program DescriptionThe
Disability Support Program is described in policy as:
“...a program designed to assist Islanders who have a qualifying
disability overcomebarriers, to attain a satisfactory quality of
life, and to strive to achieve financialindependence...The DSP may
assist children and youth with disabilities by offeringsupport to
them and their parent(s)/guardian(s)...The DSP is a social program
witha financial component. The program offers support planning
assistance to helpdevelop a plan of action and to help determine
supports that are necessary. Theprogram may also provide referrals
to other agencies where complimentary servicesmay be obtained.”
Disability Support Program policy: Section 1.1: Program
Description
Other program descriptions used by government can be found in
Appendix A.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
4
1.2.2 IntentGovernment’s intent in developing and implementing
the Disability Support Program is notcrystalized in any single
program document. Several pieces of communication, however,
provideinsight into government’s intent which can then be
summarized. The pieces of communicationinclude the following:
“...by providing disability supports to citizens who have a
qualifying physical orintellectual disability that are totally
separate from income support programming....provides disability
supports for people with physical and intellectual
disabilitieswhile still allowing these individuals to earn
income...Persons with disabilities andtheir families will be fully
involved and central to the decision making process. It isour
intent to provide a range of disability related supports and
services, based onthe needs of the individual rather than on
entitlement.”
“Ministerial Statement: Disability Support Program.” Hon. Jamie
Ballem. May 3,2001.
“In the past, Islanders received disability supports through
several programs. Thelevel of support was sometimes different from
program to program, or region toregion. People who received support
through the Welfare Assistance Program couldreceive less support if
they became employed. And often, there was not enoughflexibility in
the programs to respond to a person’s individual needs and help
themreach their full potential.”
“Questions and Answers on the New Disability Support Program.”
October, 2001.
“Government is now developing a Provincial Disability Supports
Program toimprove access to disability supports, provide more
assistance with disability costs,and promote equitable access
outside of income support programming. Enhancedassistance for
children will focus on early interventions to help in
overcomingbarriers and to ensure that children with disabilities
participate fully in PrinceEdward Island society.”
“Prince Edward Island Disability Support Program Communication
Update #1.”March, 2001.
“• Program to be needs based vs. entitlement • Program to have
parameters and an expectation of contribution from individuals
and/or families • Program to be a social program with a
financial component ($3000. month maximum) • Program to provide
flexibility • Program to separate disability supports from income
support programming”
Presentation to the Federal, Provincial and Territorial
Ministers of Social ServicesMeeting. Moncton: November, 2002.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
5
“This is an opportunity we have to provide support for folks
that haven’t had it inthe past. It’s an encouragement for folks
that haven’t been able to participate in theworkforce.”
Hon. Jamie Ballem quoted in Program offers more financial
independence for disabled. TheGuardian. Monday, May 7, 2001:
A3.
From these statements, the intent of the program can be
summarized as:1. to make access to publicly-funded disability
support more equitable;2. to strengthen the social component (i.e.
action planning) of disability support;3. to increase
employment-based financial independence among people with
disabilities;4. to make publicly-funded disability supports
available to a broader base of Island
residents;5. to give disability support consumers more choice
and purchasing control; and6. to achieve a holistic approach to
service delivery with the aim of funding and providing
the mix of services that gives the best overall result.
1.2.3 Goals and ObjectivesP GoalsThe goals of the Disability
Support Program have been taken from PEI Disability Support
Program:Staff Familiarization Sessions (September, 2001):
• To promote opportunities which will allow persons with a
disability to achieve their desiredlevel of independence.
• To work towards enhancing quality of life.
P ObjectivesThe core objectives of the Disability Support
Program are found in policy:
“a) Child Disability Supports – To provide a range of disability
related supports andservices to families who have children with
qualifying disabilities, according totheir individual needs, to
assist with extraordinary child-rearing support needsdirectly
related to their disability.
b) Adult Disability Supports – To provide a range of disability
related supports andservices to people with qualifying disabilities
according to their individual needs andrequirements to help them
achieve a higher level of independence.
c) Employment and Vocational Supports – To provide a flexible
range of measures thatinclude assessment, training, skills
development and pre-employment in anindividually-focussed program
which will result in competitive, long-termemployment, or supported
employment, if appropriate.”
Disability Support Program policy: Section 1.4: Program
Objectives
-
Formative Evaluation:Prince Edward Island Disability Support
Program
6
Minister of Health andSocial Services
DHSSDisability Support Program
Administration
Health Regions
Senior Managementand Income
Support Administration
Disability Support ProgramSupervisor
Disability Support Worker(DSW)
Disability SupportAdministrator
Exhibit 7 Disability Support Program organizational
structure
In addition, the document review identified the following
targeted program outcomes:“• Establish baseline data which is
specific to Prince Edward Island... • Reduce the number of persons
with a disability receiving income support through the
Welfare Assistance Program by 10% in year one and 15% by the end
of year three. • Have a formal case plan for each program
participant within the first year. • Increase participation in
vocational and employment preparation training by 50% in the
first year. • Increase employment placements and opportunities
by 20% in the first year.”
“A Proposal for a Prince Edward Island Disability Support
Program.” Department ofHealth and Social Services.
1.3 Program Structure, Resources and Components1.3.1 Program
StructureThe organizational structure of the Disability Support
Program is illustrated below. Designed by theDepartment of Health
and Social Services, responsibility for the program, including
policy, restswith the Minister of Health and Social Services.
Funded by the Province of PrinceEdward Island, program delivery
isthe responsibility of the fourprovincial health regions
througheach region’s Income Supportdivision. A Disability
SupportProgram unit at the Department ofHealth and Social Services
providespolicy, case management andadministration support to
theregions.
At the regional level, the DSP isdelivered by a disability
supportworker, supported by a disabilitysupport administrator.
Bothpositions report to a DSP supervisor.The disability support
worker isresponsible for an assigned caseload(intake and case
management) while the administrator provides administrative support
related tocase management and is frequently the front line contact
for information inquiries from the public.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
7
SOURCES OF FUNDINGProvince of Prince Edward Island 5,840,297$
84.3% 6,882,600$ 86.1%Federal government 625,000$ 9.0% 625,000$
7.8%Client contributions 462,780$ 6.7% 485,919$ 6.1%TOTAL SOURCES
OF FUNDING 6,928,077$ 100.0% 7,993,519$ 100.0%
USES OF FUNDING1. Goods and Services
West Prince 636,877$ 9.2% 1,012,900$ 12.7%East Prince 1,740,879$
25.1% 1,822,200$ 22.8%
Queens 2,302,869$ 33.2% 2,624,800$ 32.8%Kings 640,736$ 9.2%
680,000$ 8.5%
Client contributions 462,780$ 6.7% 485,919$ 6.1%Total Goods and
Services 5,784,141$ 83.5% 6,625,819$ 82.9%
2. AdministrationDHSS 237,907$ 3.4% 382,900$ 4.8%
West Prince 230,544$ 3.3% 253,600$ 3.2%East Prince 216,701$ 3.1%
212,100$ 2.7%
Queens 396,387$ 5.7% 369,800$ 4.6%Kings 62,397$ 0.9% 149,300$
1.9%
Total Administration 1,143,936$ 16.5% 1,367,700$ 17.1%
TOTAL USES OF FUNDING 6,928,077$ 100.0% 7,993,519$ 100.0%
Disability Support ProgramSources and Uses of Funding
Actual 2002 - 2003 Budget 2003 - 2004
Exhibit 9 Source: Department of Health and Social Services
Disability Support ProgramDirect Program Staffing (Full-Time
Equivalent Positions)
DHSS West Prince East Prince Queens KingsProvincial Coordinator
1.00Disability Supports Client Services Officer 1.00Preschool IBI
Services 3.00Supervisor 0.50 0.50 0.50 0.50Disability Support
Worker 3.20 3.50 7.00 3.25Disability Support Admin. Support 1.00
0.50 1.00 1.00 1.00Total 6.00 4.20 5.00 8.50 4.75
Exhibit 8 Source: Department of Health and Social Services
1.3.2 ResourcesExhibit 8 below summarizes direct program
staffing associated with the DSP as at July, 2003. Intotal, there
are 28.45 full-time equivalent (FTE) staff currently assigned to
the program.
Exhibit 9 summarizes actual expenditures for fiscal 2002 - 2003
and the 2003 - 2004 budget. A moredetailed summary of 2002 - 2003
can be found in Appendix B.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
8
1.3.3 Program ComponentsPrimary principles identified in policy
establish a context for the delivery of program componentsand the
program overall:
“a) Every person with a disability has the right to seek support
in attempting toovercome barriers, to attain a satisfactory quality
of life, and to achievefinancial independence.
b) The dignity and value of the person should not be compromised
in theindividual’s seeking of services.
c) If a person with a disability earns an income they may still
qualify for assistancethrough this program.
d) Most individuals prefer to be self-sufficient rather than
dependent, and servicesprovided should be directed toward that
end.
e) To facilitate growth toward independence, Disability Support
Workers must avoidtaking on tasks that can be completed by the
individual and/or family.
f) Generally, families are open and prepared to help other
family members, and serviceintervention should not erode this
traditional practice.
g) Individual persons or families usually know what their issues
are and wish to bedirectly involved in making decisions on how best
to resolve those issues.
h) Disability Support Workers are expected to work with the
individual and/or family todevelop a support plan with goals and
actions that outlines the supports necessary toachieve the
goals.
i) In seeking support, the individual and/or family has the
responsibility to complywith the expectations and obligations of
the service process.
j) Information obtained by the Health and Social Services System
while providingservices is confidential and may be shared only with
other government departmentsor private agencies when necessary to
expedite required or requested services onlywhen permission has
been obtained (Refer to Section 3.0, Confidentiality).
k) The quality with which any service is rendered is contingent
upon the attitude, lifeexperience, training and confidence of those
staff who are the primary deliverers ofthe service.
l) Public and community perceptions and views with respect to
any public program arepowerful influences; hence a sensitivity to
these perceptions is necessary.
m) Resources (budget allocations, staff) to deliver the program
are limited; hence, inorder to ensure that those in need are
assisted, program management must managethe program
efficiently.”
Disability Support Program policy: Section 1.3: Primary
Principles
Building on these primary principles, Exhibit 10 on the next
page illustrates the four majorcomponents of the Disability Support
Program:
1. a disability assessment;2. support planning;
-
Formative Evaluation:Prince Edward Island Disability Support
Program
9
Disability Assessment
Financial Assessment(if required)
• Behaviour/Psychosocial• Function
• Independent activities of daily living• Nutrition• Safety
• Community integration
Support Planning
• Identify goals• Identify and prioritize disabilityissues/needs
to achieve goals• Identify support-specific needs
• Identify support providers
• Determine cost of DSP-eligible supports• Determine maximum
level of
funding support• Calculate client contribution
Action Plan• Complete action plan
• Identify support coordinator• Financial contract (if
required)• Record of supports used
MinimumAnnualReview
• Technical aids and assistive devices• Community participation
supports
• Community living supports• Respite
• Modifications• IBI
Exhibit 10
3. a financial assessment, if required; and4. development of an
action plan.
1. Disability AssessmentProgram staff utilize a screening tool
to provide consistency in understanding and assessing
anindividual’s disability and level of functioning. Based on
self-disclosed information andobservation, the screening tool
measures behaviour/psychosocial function, physical
function,independent activities of daily living capability,
nutrition, safety and community integration.
2. Support PlanningThe support planning component establishes a
planning framework for the individual. As part ofthis component,
individual and/or family goals are identified, disability
issues/needs associatedwith achieving these goals are identified
and prioritized, support-specific needs are identified, andsupport
providers are identified.
3. Financial AssessmentThe provincial government may act as
funder of last resort under the Disability Support Programfor
eligible goods and services in one of six broad areas:
1. technical aids and assistive devices;2. community
participation supports;3. community living supports;4. respite;5.
home, vehicle and workplace modifications; and
-
Formative Evaluation:Prince Edward Island Disability Support
Program
10
6. Intensive Behavioural Intervention (IBI).
If eligible DSP-funded goods and services are required, a
financial assessment is conducted whichincludes:
a) determination of the maximum amount of funding support
available based upon theparticipant’s overall level of functioning;
and
b) the calculation of a client contribution amount based upon an
income test.
4. Action PlanBuilding on the disability assessment, support
planning and the financial assessment, if required,the last
component of the DSP is an action plan. This plan is a working
document that identifies thepathways to achieving identified goals,
including:
a) action steps;b) responsibility for each action step;c) time
frames; andd) expected outcomes.
The action plan process includes the identification of a support
coordinator, who may be one or acombination of: the participant; a
direct or extended family member; a friend; a service provider; ora
disability support worker. As the title indicates, the primary role
of the support coordinator is toorganize and coordinate the timing
and delivery of required goods and services. Where a
monthlyfinancial benefit is paid by the DSP, the support
coordinator may also be responsible for thepayment of purchased
goods and services.
If funding is provided under the DSP, the participant is
required to enter into a financial contract(Individual Support
Agreement) with the respective health region. While the keeping of
receipts isnot a requirement under program policy for financial
accountability, participants receiving fundingare expected to
maintain a Record of Supports Used. This form is used by
participants and/or adesignate to track the timing, services,
service providers and costs of goods and services purchasedwith
government funding.
1.4 Scope of the EvaluationOfficially launched in October, 2001,
this is the first evaluation of the Disability Support
Program.While the program is considered to be fully operational,
program management considers theimplementation to be formative in
nature and consequently are seeking to have a formativeevaluation
completed.
The formative evaluation issues and questions cover the areas of
program rationale andimplementation, program and participant
profiles, and initial impacts, effects and cost-effectiveness.
Issues of employment/employability and quality of life are of
particular interest togovernment.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
11
1.5 Methodology and LimitationsThis report presents findings
arrived at from multiple lines of evidence. A summary of
eachmethodology used is given below. The primary evaluation
questions, planned methodologicalapproaches, and analysis and
reporting processes are detailed in a pre-evaluation technical
reportthat was prepared entitled “PEI Disability Support Program
Formative Evaluation: TechnicalReport.”
In executing this formative evaluation, two notable challenges
were encountered by the consultingteam:
1. access to reliable pre- and post-DSP socio-demographic and
funding data on whichcomparative analysis could be done; and
2. access to reliable socio-demographic and funding data on DSP
participants – bothcurrent and former – on which comparative
analysis could be done.
This situation was largely due to the hardware and software
technology used to store and managesocial services data, including
pre-DSP and current DSP records. This technology is very dated
andgovernment is presently in the process of replacing this
technology with a new databasemanagement system.
Limitations of the existing technology, including a lack of
specialized technical support, make itdifficult to define and
export data. The situation was compounded by the fact that DSP
records foreach participant are a combination of electronic files
which reside on the government mainframe,and paper files which
reside in the health regions. In the end, reliable exported data
for analysispurposes was limited to current DSP participants
receiving a monthly support payment.
A secondary challenge was inconsistencies in the coding of
certain record fields, some of whichmade creating unique record
identifiers on exported data difficult for survey and analysis
purposes.Others, such as fields associated with the status of a
file presented some difficulties in beingthoroughly accurate
“slotting” contacts for surveying that was conducted.
Quantitative data was only used for inclusion in this report
after the accuracy and integrity of thedata was confirmed using
data screening and testing techniques.
Finally, as there is no detailed data available about all people
with disabilities in Prince EdwardIsland and their disability
support needs, it is difficult to truly understand the magnitude of
needwithin the province as a whole.
P Disability Support Worker and Disability Support Administrator
Focus GroupsSeparate focus groups were conducted with disability
support workers (two sessions) and disabilitysupport administrators
(one session).
-
Formative Evaluation:Prince Edward Island Disability Support
Program
12
P Literature ReviewA literature review was conducted by DHSS
staff with the intent of bringing to the evaluation aglobal
perspective of current practices in government-based disability
support programming.Section 2: Trends in Disability Support
Initiatives is authored by department staff and reports on
theliterature review findings.
P Case File AuditA review of 30 DSP case files was conducted by
DHSS DSP administrative staff. Reviewers visitedeach of five health
region offices (O’Leary, Summerside, Charlottetown, Montague and
Souris) withsix case file numbers covering adult, child and
employment/vocation cases that were randomlyselected in advance.
Regional office staff had no prior knowledge of the cases selected
prior to thearrival of the reviewer on-site.
P Disability Support Organization Roundtable SessionsFour
roundtable sessions (West Prince, East Prince, Charlottetown,
Kings) were held withrepresentatives of the disability support
community including NGOs and volunteer organizationsin Prince
Edward Island.
P Public Input SessionsFour public input sessions (West Prince,
East Prince, Charlottetown, Kings) were held providingthe public at
large with the opportunity to express opinions about the Disability
Support Program.
P Persons with Disabilities Focus GroupsTwo focus groups
(Summerside, Charlottetown) were conducted with persons with
disabilities.
P Expert and Key Informant InterviewsA total of 36 expert and
key informant interviews were conducted with DHSS and health
regionDSP program administrators, DHSS and health region
management, health care providers, andrepresentatives of the
disability support community. Several interviews were also
conducted withparents of children with disabilities who wished to
remain anonymous. A list of the individualsinterviewed can be found
in Appendix D. These interviews were in-depth, lasting, on
average,slightly less than two hours each.
P Document ReviewA review of DHSS documents pertaining to the
development and implementation of the DisabilitySupport Program was
conducted. A listing of the documents reviewed can be found in
AppendixC.
P Quantitative Telephone SurveyingQuantitative telephone
surveying was conducted among:
• all DSP participants and/or a proxy with an active DSP
file;
-
Formative Evaluation:Prince Edward Island Disability Support
Program
13
• all former DSP participants and/or a proxy where service was
completed and the fileclosed;
• all individuals who applied to the DSP and were deemed to be
ineligible for financial orother reasons; and
• all individuals where a partial DSP application or file was
opened but never completedor an application was approved but
service was never accessed.
A detailed analysis of the telephone survey data can be found in
a separate report, “Prince EdwardIsland Disability Support Program:
Telephone Survey.”
P Administrative Data AnalysisDigital socio-demographic data of
current and former DSP participants were assembled andanalysed to
create a DSP participant profile.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
14
2. Trends in Disability Support InitiativesAuthored by
Department of Health and Social Services staff
2.1 BackgroundThe Prince Edward Island Disability Support
Program (DSP) has removed disability supports fromincome support
programming. PEI is the first jurisdiction in Canada to implement
this newapproach consistent with In Unison: A Canadian Approach to
Disability Issues(Federal/Provincial/Territorial Ministers of
Social Services, 1998) which stated a policy direction
asfollows:
Income programs that separate access to disability supports from
eligibility for financial assistance. Other program approaches that
are currently in operation across Canada offer either generalincome
support for the public including persons with disabilities, or
income support that is solelyfor persons with qualifying
disabilities. It was at the June 1996 First Ministers Meeting that
personswith disabilities were identified as a collective priority
for social policy renewal.
DSP is only one of many disability support programs, services
and funding that is available toIslanders. Examples include:
• Canada Pension Plan disability benefits; • Employment
Insurance sickness benefit; • Canada Study grants for students with
disabilities; • Opportunities Fund;• Disability component of the
Aboriginal HRDC strategy;• Canada Pension Plan - Disability
Vocational Rehabilitation Program; • Workplace equity programs; •
Disability component of the Social Development Partnerships
Program; • Participation and Activity Limitation Survey; •
Disability Tax Credit (Canada Customs and Revenue Agency); •
Employment equity ( Public Service Commission/Treasury Board); •
Workers' Compensation;• Long-term disability insurance;• Motor
vehicle no-fault accident benefits;• Personal injury awards and
settlements;• Income tax credits;• Provincial programs (e.g. early
intervention, home care, education supports);• Non governmental
organizations.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
15
Exhibit 11
2.2 The VisionChoice and Opportunity (1994-1999) - This program
was a strategic initiative for social policy reformconcerning
people with intellectual disabilities which occurred from 1994
-1999. It consisted of apartnership between the Canadian
Association for Community Living, PEI Association forCommunity
Living, Government of Canada and the Province of Prince Edward
Island. Thisinitiative included extensive consultations with the
community and key stakeholders, thecompletion of comprehensive
research and the trial of innovative policies and disability
supports.
In Unison (1998) represented a shift in the social policy
direction for persons with disabilities inaccordance with the
social policy trends seen in Exhibit 11.
The DSP has attempted to adhere to the national direction
outlined within In Unison (1998) wherethe following twelve items
summarize the policy directions regarding full citizenship,
disabilitysupports and employment.
1. Polices that promote access to generic programs and services
for all Canadians, includingpersons with disabilities.
2. Policies and programs that promote greater access to
supports.
3. Policies that separate access to supports from eligibility
for income and other programs.
4. More consumer control, flexibility and responsiveness in the
provision of disabilitysupports.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
16
5. Widespread understanding and application of the concept of
accommodation.
6. Measures that provide more assistance to offset work-related
disability costs.
7. Greater support for community economic development and
self-employment for personswith disabilities.
8. Enhanced employability through better access to education
training and transitionmechanisms.
9. Income programs that reduce financial disincentives to
work.
10. Income programs that separate access to disability supports
from eligibility for financialassistance.
11. Improved coordination of assessment procedures and
rehabilitation between incomeprograms.
12. Income programs that continue to ensure financial assistance
when labour marketparticipation is interrupted or not possible.
In Unison 2000 Persons with Disabilities in Canada - (F/P/T
Ministers Responsible for SocialServices). This report built on the
In Unison (1998) framework and provided a broad view of howadults
with disabilities are faring in comparison with those without
disabilities. It providedexamples of effective practices
implemented in Canada, emphasizing flexible policy solutions tomeet
individualized needs.
2.3 Federal Role in Disability The federal government performs
six roles in disability policy and supports
1. financing to P/T governments through transfer payments;2.
income and tax benefits to individuals and families;3. capacity
building of disability organizations;4. research, evaluation and
innovation;5. rule development and right definition; and6. service
provision and delivery (Source: Prince, 2002, p. 4).
The provision of funds to provincial and territorial governments
or individuals and families occursthrough transfer payments such as
the Canada Health and Social Transfer (CHST), Early
ChildhoodDevelopment (ECD), Labour Market Development Agreements
(LMDA), and EmployabilityAssistance for Persons with Disabilities
(EAPD).
Included are supports transferred to individuals and families
such as Employment Insurance (EI),
-
Formative Evaluation:Prince Edward Island Disability Support
Program
17
tax benefits and supports delivered directly to Aboriginal
Canadians on reserve. Economically, theintergovernmental grants: 1)
fill financial gaps between the federal funding responsibilities
andtaxing powers of the provinces and territories; and 2) ensure
fiscal equity through equalization ofthe capacity of governments in
the union. Socially, the federal spending power adapts public
policyto changing situations and values, or in SUFA terms the
federal transfers “support the delivery ofsocial programs and
services by provinces and territories in order to promote equality
of opportunity andmobility for all Canadians and to pursue
Canada-wide objectives” (Prince, 2002, p. 4). The federalgovernment
also participates with the provinces in funding national disability
organizations,applied research, and multilateral and bilateral
processes consistent with the Social UnionFramework Agreement.
2.4 Definition of DisabilityOver time the definition of
“disability” has changed. The meaning of the word has progressed
fromunscientific to medical, to social, to rights, and more
recently, to a bio-psycho-social definition witha strong
environmental component (Brown, 2001). At present the trend is for
sources to support theidea of incorporating the “environmental
conditions/circumstances” into the definition ofdisability. In
Canada, there are many disability support programs each defining
disabilitydifferently for the parameters of the program. For
example, various federal government programssuch as Canada Pension
Plan (CPP) Disability Benefits, Disability Tax Credit, Employment
Equity,and Veterans Disability Pension each have a different
definition of disability depending on thepurpose and provisions of
the specific program. Currently, the CPP Disability Benefits
Program isthe largest benefit program in Canada with almost 300,000
participants and approximately 70,000applicants annually (Standing
Committee on Human Resources Development and the Status ofPersons
with Disabilities, 2003, Chapter 1, p. 1/10). It has been said that
multiple definitions aredesirable if social policy categories are
to be true to their purposes. “Social policy categories which
arenot sufficiently relevant are unsustainable” because they may
exclude persons who should be included,or include persons who
should be excluded (Mabbett, 2003). The DSP has also established
its’ owneligibility parameters not unlike those of other disability
support programs.
2.5 Types of Disability SupportsEmerging “Supports Paradigm” -
It has been suggested that a “supports paradigm” has been
evolvingthrough the stages of normalization (1970s),
community-based involvement (1980s), and now,quality of life
(1990s). In other words, the paradigm shift has been from a focus
on deficits to a focuson self-determination and inclusion. In the
2000s disability support workers work in partnership tohelp
determine what supports will help people participate in their
community with meaningfulsocial roles that will allow them to
experience greater fulfillment (Thompson et. al., 2002, p.
391).
Case Planning - To move the menu of supports from the program to
the person a planning process isused to 1) identify participant’s
desired life experiences and goals, 2) determine the
participant’sintensity of need across their range of circumstances,
3) develop an individualized support plan, 4)
-
Formative Evaluation:Prince Edward Island Disability Support
Program
18
monitor and assess outcomes, and 5) revise the support plan on a
schedule, or in response tochanged circumstances. Contemporary
principles for support planning include 1) tailoring supportsto
individual needs and preferences, 2) flexibility in provision
across varies circumstances in theperson’s life, 3) allowing for
individual participants to have supports that are especially
importantto them, 4) scheduling systematic assessments of support
needs to guide revision of support plans,and 5) considering
relevant factors “close to a person” such as his/her ethnicity,
culture, language,communication mode, and behaviour characteristics
(Thompson, 2002, p. 390-405).
Supports Associated with Disability Programming - DSP recognizes
the definitions of disabilitysupports as proposed by Caledon
Institute of Public Policy and Roeher Institute. The
CaledonInstitute of Public Policy (Torjman, 2000, pp. 3-5)
indicates that personal supports “refer to a rangeof goods and
services that help offset the effects of a disabling condition”.
Personal supports can beclassified into three streams: 1) technical
aids and equipment, 2) personal services, and 3)brokerage. The
Roeher Institute defined disability supports similarly saying:
“Any good or service which assists a person with a disability in
overcoming barriers tocarrying out every day activities or to
social, political and cultural activities and
economicparticipation. In this definition, there is no fixed set of
disability-related goods and services,rather it is open ended. A
good or a service becomes "disability-related" when it is used
toassist a person or persons in overcoming barriers associated with
a disabling condition”(Roeher Institute, 2002).
“[A]ny good, service or environmental adaptation that assists
persons with disabilities toovercome limitations to carrying out
activities of daily living and in participating in the
social,economic, political and cultural life of the community a)
provided to individuals and theirfamilies and b) delivered to
people with disabilities as a group through broad
communitymeasures” (The Canadian Association for Community Living
and the Council of Canadianswith Disabilities, 2003).
The intent of the DSP is to reflect the vision of these two
well-known policy-related organizationsand offer a range of
supports that include respite, community living, community
participationsupports, technical aids and assistive devices and
planning. The particular “mix” of disabilitysupports required by
each program participant reflects their individual needs as
determined duringthe planning with individuals and families.
The individualized case planning approach allows referrals to
programs and/or services offered bythe sponsor department, Health
and Social Services. These referral programs include Acute
Care,Addiction Services, Adoption Services, Ambulance Services,
Child Protection, Community CareFacilities, Dental Public Health,
Diabetes, Dialysis, Drug Cost Assistance Programs,
EnvironmentalHealth, Family Housing, Financial Assistance, Foster
Care Services, Health Information ResourceCenter, Home Care, Job
Creation/Employment Enhancement, Long Term Care, Mental Health
-
Formative Evaluation:Prince Edward Island Disability Support
Program
19
Services, Out-of-Province Hospital Services, Out-of-Province
Physician Services, PhysicianServices, and Seniors Housing (PEI
Health and Social Services Annual Report, 2002, p. 55).
Clearly,these Health and Social Services programs offered in
combination with programs of otherprovincial government departments
and federal government departments, private companies,
andnon-government organizations create a wide array of essential
supports that Islanders withdisabilities can access regardless of
age or location.
2.6 PEI PALS Disability StatisticsDisability can occur among
people of all ages,sexes, incomes, races, etc. Disability is more
closelyassociated with some demographic characteristicsthan
others.
Age - The prevalence of disability increases withage due to
injury, chronic disease, aging ofpopulation, and poverty.
• Among persons under 15 years, 15-64 years, and 65 and over
years the percentage ofpersons estimated to have a disability is
3.5%, 11.8% and 44.4% of the general population,respectively.
• Almost 90% of the population is disability-free until the
middle years (45-64 years) whenprevalence increases
progressively.
• The increase during the middle years reveals a PEI trend that
is similar to the Canada trendbut with PEI figures that are 3-4%
higher than Canada figures.
• More than one-half of seniors age 75 and over have one or more
disabilities whether theylive in PEI or Canada.
Sex/Gender - The prevalence of disability is slightly higher
among males than females according tothe recent PALS which is
formally termed the Participation and Activity Limitation
Survey(Statistics Canada, 2001).
• 4% of males and 2.5% of females age 0-14 years live with a
disability.
• Yet, 9.4% of males and 10.4% of females age 15-64 years live
with a disability, so that themale prevalence doubles from youth to
adulthood while the female rate quadruples in thesame age
range.
• With 38.5% of males and 42.0% of females age 65+ years having
a disability the prevalenceof disability for each sex quadruples
from adult years to senior years.
The prevalence of disability in Canada was12.4% and on PEI was
14.3% with therange extending from 8.4% (Quebec) to17.1% (Nova
Scotia). 14.3% Islanders was18,970 persons. Source: PALS, 2001.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
20
Employment - According to PALS (2001), the gender gap in
employment narrowed between 1986and 1991. By 1991, youth age 15-24
years had similar prevalence rates for disability with thewomen’s
rate being 96.1% that of the men’s rate. In contrast, among age
55-64 year-olds, the femaleparticipation rate was only 47.4% that
of the male rate. Analyses of Welfare Incomes from theNational
Council of Welfare for the year 2000-2001 which were reported by
the Canadian Councilon Social Development (2002) showed that on PEI
a person with a disability with a welfare incomeattained 55% of the
poverty line (range, low of 40% in Alberta and high of 62%
Ontario). In thesame study, a single employable person attained
only 36% while single parent with one child andcouple with two
children attained 62% and 64%, respectively, if living on PEI.
Education - Educational attainment is one determinant of
employability. Among adults 20-64 yearsof age who responded to the
1996 census, the high school, university, and trade/community
schoolcompletion rates were 33%, 7%, and 26% for persons with
disabilities compared to 50%, 17% and32%, respectively, for persons
without disabilities. Among the aboriginal population, 23%
withdisabilities and 26% without disabilities completed trade
school while 2% with disabilities and 4%without disabilities
completed university (Statistics Canada, 1996). Higher levels of
education aregenerally associated with higher wages whether or not
a disability is present. And, at any level ofeducation wages tend
to be lower for persons with disabilities than persons without
disabilities(Canadian Council on Social Development, 2002). Among
Islanders with disabilities 22% havecompleted Grade 9 or less, 44%
have completed Grade 12 with or without a diploma, and theremaining
34% have completed a trade certificates and university degree -
with an even split betweenthe two groups (Dynamic Research,
2000).
Income and Poverty - Employment affects income and poverty.
Among persons with disabilitiesand age 15 years and over ~25% of
women were poor compared to ~18% of their male counterparts(HALS,
1991). For persons without disabilities the comparable rates were
almost one-half as high --~14% for women and ~11% for men (HALS,
1991). However, the severity of the disability plays alarge role in
determining these figures for adults with the respective percentage
who are pooramong persons with severe, moderate, and mild
disabilities being 30.3%, 23.7% and 17.7%,respectively (HALS,
1991). Severity of disability affects ability to work as well as
hours worked.
Type of Disability - PALS coded 11 different types of
disabilities. Some types occur much morefrequently in the general
population than others.
• Among persons age 15 years and over the three most common
disabilities were mobility,72%; pain, 70%; and agility, 67%.
• Among Canadians age 15+ years the most common disability was
hearing, 30%. All otherdisabilities occur among less than 20% of
the population age 15+ years.
• The actual rates of disability are seeing, 17%; psychological,
15%; learning, 13%; memory,12%; speech, 11%; developmental, 4%; and
unknown, 3%.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
21
Severity of Disability - PALS categorized disabilities into one
of four categories: mild, moderate,severe, and very severe. The
prevalence by type of disability is similar for all ages with
littlevariation between age 5-14, 15-64 or 65 and over years.
• 32-36% have mild disability (among categories 5-14, 15-64 and
65+ years of age).
• 25% have moderate disability.
• 26-28% have severe disability.
• 14-17% have a very severe disability.
At present five general factors account for the increase in
disabilities: aging population, poverty,medical advances, cultural
shifts or emerging conditions, and legislation (Fujiura, 2003, pp.
1-4;Center on Emergent Disability, 1997, pp. 1-16). These factors
will increase the number ofadolescents and young adults with
disabilities and thereby increase the demand for
disabilityservices. The number of seniors will be increasing until
2040. Each year more infants with moreserious disabilities will be
saved due to modern medical technology. Changing conditions in
theworld can instantly increase the number of persons with
disabilities, e.g., war-related violence. Aswell, disability “feeds
on poverty, and poverty on disability” (Rand, 2002, p. 8/13). In
Canada, therate of disability is 30% vs. 11.8% for Aboriginal vs.
general population adults age 15-64 years.Among Aboriginal seniors
age 65 years and over 53% have a disability compared to 44.4% in
thegeneral population (PALS, 2001).
2.7 The Standard Rules: A Tool to Mobilize International
ActionThe “Standard Rules” adopted by the United Nations General
Assembly in 1993 represented amajor outcome of experience gained
during the Decade of Disabled Persons (1983-1992). See Exhibit12.
The rules, though not legally binding, do exert an impact on policy
development nationally andinternationally. Through use the 22 Rules
have become international customary rules that guidenations wishing
to equalize opportunities for persons with disabilities.
Collectively the rules summarizethe message of the World Programme
of Action (WPA) using the human rights perspective whichhad been
developed during the decade.
The rules are composed of four chapters that cover all aspects
of the life of persons with disabilities:1. preconditions for equal
participation;2. target areas for equal participation;3.
implementation measures; and4. the monitoring mechanism.
The DSP follows these guidelines in developing the optimal
combination of elements for anindividual case plan.
-
Formative Evaluation:Prince Edward Island Disability Support
Program
22
Chapter I. Preconditions for Equal ParticipationRule 1.
Awareness-raisingRule 2. Medical careRule 3. RehabilitationRule 4.
Support servicesChapter II. Target Areas for Equal Participation
Rule 5. AccessibilityRule 6. EducationRule 7. EmploymentRule 8.
Income maintenance and social securityRule 9. Family life and
personal integrityRule 10. CultureRule 11. Recreation and
sportsRule 12. ReligionChapter III. Implementation MeasuresRule 13.
Information and researchRule 14. Policy-making and planningRule 15.
LegislationRule 16. Economic policiesRule 17. Coordination of
workRule 18. Organization of persons with disabilitiesRule 19.
Personnel trainingRule 20. National monitoring and evaluation of
disabilityprogrammesRule 21. Technical and economic cooperationRule
22. International cooperationChapter IV. Monitoring
Exhibit 12 Standard rules on the equalization of opportunities
forpersons with disabilities
These Rules require a “multi”approach when applied. They
usemultiple channels, multipleaudiences, multiple strategies,
etc.to effect sustainable change in thequality of life of persons
living withdisabilities, including women,children, elderly, poor,
seasonal,dual or multiple disa