Opportunities for Postsecondary Success Project Center for
Community Partnerships Department of Occupational Therapy Colorado
State University
Opportunities for Postsecondary Success ProjectCenter for
Community Partnerships Department of Occupational TherapyColorado
State UniversityGoal Attainment ScalingAHEAD 2012
Conference7/11/12Cathy Schelly, M.Ed., OTR Assistant
ProfessorDirector, Center for Community PartnershipsPatti Davies,
PhD., OTRProfessorResearch DirectorOpportunities for Postsecondary
Success (OPS)Implementation site: Center for Community Partnerships
(CCP)
OPS Students:CSU & Front Range Community College students
with Aspergers/ASD, TBI, IDTransitioning youth with ID from local
school district (partnership with Project Search)Young adults with
ID seeking community employmentEstablishing measurable goals with
participants: Goal Attainment Scaling
Goal Attainment ScalingGoal attainment scaling (GAS): an
individualized approach for measuring the achievement of goals
(King et al., 1999).Originally developed to assess adults in a
community mental health setting, but has since been applied to
numerous practice areas, including education, health, and social
work (MacKay, Somerville, & Lundie, 1996).
Original goal attainment scaling methodThe original GAS scale
uses a 5-point scale, ranging from -2 to +2, with zero representing
the expected level of performance after intervention.
LevelsKiresuk, Smith, & Cardillo (1994)- 2Much less than
expected outcome- 1Somewhat less than expected outcome 0Projected
level of performance+1Somewhat more than expected outcome+2Much
more than expected outcomeAlternative methodsLevelsKing et al.
(1999)- 2Baseline- 1Less than expected 0Expected level+1Greater
than expected +2Much greater than expectedLevelsSteenbeek et al.
(2007)+2A lot more+1A little more0As expected-1Partially achieved
-2Same as baseline-3WorseLevels1No change2Small
improvement3Interest in goal4Takes some action towards
goal5Maintains new behavior6Committed to achieving goal7Achieves
goalOPS Scale Numerical Level of Goal AttainmentDescriptive Level
of Goal AttainmentI lost ground.0Where I am right now. (Baseline)1
Im getting close to my goal. 2 I reached my goal!3I did even better
than expected!47OPS Goal Attainment Scaling GOAL: Implement
effective study strategies.
Numerical Level of Goal AttainmentDescriptive Level of Goal
AttainmentI lost ground.0Not willing or interested in
discussing/considering effective study strategies.Where I am right
now.1Willing to discuss/consider potential effective study
strategies but not implementing any strategies. Im getting close to
my goal. 2 Implementing effective study strategies with most
difficult course, with cueing and reminders.I reached my
goal!3Implementing effective study strategies with all courses,
with cueing and reminders.I did even better than
expected!4Independently implementing effective study strategies
with all courses.8Applying GAS in practiceIdentify the overall
objective.Identify the specific problem area(s) that the individual
wants/needs to address.Identify the behaviors or events that will
indicate improvement in each area.Determine the methodology to
collect the desired information.Identify outcomes for
scaling.Determine the individuals current status (baseline).
Determine how progress will be documented.What is the overall
goal?Setting your expected level of outcome or the desired goal:For
example, you and the student may agree that developing
self-advocacy skills is the overall program goal.Document the
students current status (baseline).Establish incremental changes in
behavior from baseline to depict improvements or losing ground as
student pursues goal.What are the specific problem areas?
Prioritize areas of concern and formulate SMART goals.Prioritized
goals are given a weight, which is used to convert the scale score
into a standard score.
11What are SMART goals?Specific= a specific goal has a much
greater chance of being achieved. It is the who, what, where, when,
which, and why of the goal.Measurable=establish concrete criteria
for measuring progress.Attainable=help client set goals that are
meaningful to them and that they want to achieve.Relevant=a goal
must represent an objective toward which you are both willing and
able to work.Time-specific=a goal should be grounded within a
time-frame.
Tips for scaling goalsSelect the expected level of
performance.Identify the least favorable outcome and the most
favorable outcome.Identify the intermediate levels of
performance.Set SMART goals.Develop scale with equally interval
levels.Important considerationsWhere is my clients current level of
performance (baseline)?When/how often should I follow-up to
determine and document clients change in status?
What behaviors and events will indicate improvement?Create a
scale that will be useful for measuring changes in performance.For
the student working on self-advocacy skills, this may involve
creating a scale based on the number of times the student attends
his/her professors office hours.For student who is wanting to make
friends in the residence hall (RH), this may involve creating a
scale based on the number of RH activities he participates in.GAS
Goal: Recognize and address issues that result in lack of
persistence spring semester. Numerical Level of Goal
AttainmentDescriptive Level of Goal AttainmentI lost
ground.0Withdraws from all courses spring semester with no warning,
no support.Where I am right now. (Baseline)1Planning on withdrawing
from challenging courses spring semester seeking support but
unwilling to discuss issues that are causing urge to withdraw. Im
getting close to my goal. 2 Willing to discuss issues that cause
the urge to withdraw from challenging courses.I reached my
goal!3Recognizes and begins addressing issues that are causing urge
to withdraw from challenging courses.I did even better than
expected!4Proactively addresses issues that cause urge to withdraw
from challenging courses and persists with all courses.16GAS Goal:
Implement study strategies. Numerical Level of Goal
AttainmentDescriptive Level of Goal AttainmentI lost ground.0Not
willing to develop study strategies.Where I am right now.
(Baseline)1Not participating in Academic Advancement Center (AAC)
to learn study strategies and not using any study strategies, but
willing to consider development of strategies. Im getting close to
my goal. 2 Attending AAC study strategy sessions to learn about and
develop study strategies.I reached my goal!3Fully implementing
study strategies with OPS support during study sessions.I did even
better than expected!4Independently implementing study
strategies.17GAS Goal: Self-advocacy for learning accommodation
needs. Numerical Level of Goal AttainmentDescriptive Level of Goal
AttainmentI lost ground.0Not willing to speak with professor.Where
I am right now. (Baseline)1States a willingness to speak with
professor regarding accommodation needs, but afraid to follow
through. Im getting close to my goal. 2 Made list of learning
accommodation needs in preparation for meeting with with
professor(s).I reached my goal!3With mentor assistance, set up
appointment(s) with professor and kept appointment(s), advocating
for learning accommodation needs and with mentor attending
meeting(s).I did even better than expected!4Independently set up
appointment(s) with professors and kept appointments, advocating
for learning accommodation needs with behind the scenes
encouragement from mentor.18GAS Goal: Completes new tasks when
assigned jobs are finished. Numerical Level of Goal
AttainmentDescriptive Level of Goal AttainmentI lost ground.0Leaves
work area when assigned jobs are finished.Where I am right now.
(Baseline)1Sits down and stops working when assigned jobs are
finished, even when additional tasks are not completed. Im getting
close to my goal. 2 Asks job coach what to do next when assigned
jobs are finished and works on additional tasks as directed.I
reached my goal!3Asks supervisor and/or natural support staff if
there are additional tasks to be done and then works on these tasks
as directed.I did even better than expected!4Recognizes additional
tasks after assigned jobs are finished and independently completes
these tasks.19Process we useDevelop relationship with
student.Connect student with mentor.Gather information re: needs,
wants of student, and areas where there are challenges (assessment
process).Develop semester goal with student review with team to
ensure that goal is SMART.Enter goal into GAS database.Begin goal
attainment activities, documenting incremental changes.At end of
semester, regroup and establish new goal(s) for next semester.CCP
Assessment ProcessFunctional, community-referenced processIdentify
strengths, challenges, barriers in all domainsIdentify support
needsFormulate student/participant driven goal(s)
Challenges/Support NeedsEvaluation
Observed Indicator (actual behavior, action or circumstance)
Fatigue (physical and cognitive)Student describes inability to
stay focused and awake in afternoon classes. Student complains that
fatigue is unrelenting, even after 8 hours of sleep each night.
Student states that mornings are best for being alert and
focused.Confusion (short-term memory challenges)Student spends
unusually long periods of time studying for exams and completing
assignments, but complains of inability to remember content even
after reading numerous times.Assessment process is
ongoingAssessment is the critical starting point that paves the way
to successful planning, educational mentoring, and provision of
necessary supports that facilitate success.Functional assessment
continues after the student is working towards goals to continually
guide planning, solve problems, improve academic performance and,
ultimately, persistence, retention and graduation.Same guidelines
apply for students pursuing employment/careers.
Canadian Occupational Performance Measure (COPM)Under
consideration, to complement GAS.Assesses individuals perceived
occupational performance in life areas.Process:Conduct
semi-structured interview to identify wants/needs of client.Based
on wants/needs, create goals.Supplement interview information
through direct observation, environmental assessment, or
standardized tests.Benefits of COPMMeasures clients perceived
progress in goal attainment between pre-COPM, intervention and
post-COPM (designed as outcome measure).Easy to administer ~20
minutes.Demonstrated reliability and validity.Similar to GAS,
useful with a wide variety of clients in many different
settings.Can weight and prioritize goals (again, consistent with
GAS).
Audience DiscussionWhat type of assessment process(es) are you
using?What are the advantages of your assessment process?What are
the disadvantages, if any, of your assessment process?What
population do you serve?Practice session on writing goalsReview GAS
worksheets.Develop goals as per worksheet, OR, for student you are
working with.Connect goals to domain areas.Develop SMART
goals.DiscussionActual goals that we established with our
students/participantsHow will data be collected?What process will
your group implement to collect and analyze data?
How will information be collected and who will collect it?Role
of the Transition CoordinatorRole of the MenteeRole of the
MentorOur GAS ResearchExciting results from pilot study fall 2011
and spring 2012Noted progress in following
areas:AcademicsSelf-advocacySocializationPersistenceRetentionGraduationEmploymentPreliminary
DataParticipants - total 2312 students at CSU5 students at FRCC6
students enrolled in Project Search
Number of goals written total 4315 in Fall semester 2011 28 in
Spring semester 2012
Students enrolled in Project Search6 students
Goals written for a 3 month internship
8 goals written 1.3 goals on average per student
83% of the students made steady improvement on their goals
during their internships
16% of the students displayed a decline in their GAS score at
the end of the internship
100% of students were hired after completing their
internships
Students in College (FRCC) and University (CSU)17 students
Goals written for a semester
35 goals written 1.8 goals on average per student
47% of the students made steady improvement on their goals
across the semester
24% of the students displayed a decline in their GAS score at
the end of the semester
Changes in Goal Attainment Scores Before and After InterventionT
ScoreMeanStandard DeviationPre Intervention38.122.23Post
Intervention58.907.92
Changes in Goal Attainment Scores Before and After
InterventionANOVAStatistic: F(1,20) = 136.17, p < .0005
Effect Size:Partial = .87Converting Goal Attainment Scales
Scores to T ScoresSum the GAS score for the goal(s)If a participant
only has one goal the sum will be the score for that one goal.If a
participant has more than one goal the score for each goal will be
added together.Convert the summed score to a T ScoreFormula in
Kiresuk & Sherman (1968)More precisionTables in Kiresuk, Smith,
& Cardillo (1994)Less precision
Advantages of Using T ScoresHelps to equate scores across
participants when they have different number of goals.
Provides variability to pre-intervention scores which is
necessary for conducting statistical analyses.
Goals can be Weighted To reflectRelative value of each goal
based on intervention outcomesRelative importance of each goal
based on participants perspectivesRelative to the difficulty of the
goalWeights do not need to sum to a particular fixed
totalExamples1, 2, 3 or 2, 4, 6 or 10, 20, 30Equal weighting of
goals lose little information
Goal Attainment Scaling as an Intervention ToolWeighting and
selecting goals based on clients desires
Setting goals in collaboration with client
Using improvement on GAS scores as motivation for the client to
work hard
Can be used in combination with the COPM Determine goal
areasWeight goals
Goal Attainment Scaling as an Research ToolThe evaluation and
establishment of goals may be conducted by someone different than
the person providing the intervention.
Follow-up assessment on goals may be conducted by someone other
than the person providing the intervention.
Consider comparing intervention programs or using a control
group.
Consider using other evaluation tools in addition to GAS.
Goal: University GraduationSuccess!
Goal: College GraduationSuccess!
Advantages of using GAS with the OPS projectProvides a
quantitative measure of outcome.Can be used to compare a students
progress over time.Can be used to compare performances across
clients in the same program but who have different skills,
challenges (Ottenbacher & Cusick, 1989).Allows for a
collaborative approach-working with the student to set realistic
goals.Provides flexibility for measuring diverse outcomes (Brown,
2009).
Limitations of GASScaling can be a time-consuming process.Biases
can occur in goal setting, scaling, and rating.Temptation to modify
goals throughout the course of intervention.NOTE: Good goals are
really hard to write!!
ReferencesKing, G.A., McDougal, J., Palisano, R.J., Gritzan, J.,
& Tucker, M.A. (1999). Goal attainment scaling: its use in
evaluating pediatric therapy programs. Physical and Occupational
Therapy in Pediatrics, p. 31-52.Kiresuk, T. & Sherman, R.
(1968). Goal attainment scaling: A general method of evaluating
comprehensive mental health programs. Community Mental Health
Journal, 4 (6), 443-453. Kiresuk, T.J., Smith, A., & Cardillo,
J.E. (1994). Goal Attainment Scaling: Application, Theory, &
Measurement. Hilldale, N.J.: Lawrence Erlbaum Associates. Mailloux,
Z., May-Benson, T.A., Summers, C.A., Miller, L.J., Brett-Green, B.,
Burke, J.P., et al. (2007). The Issue Is-Goal Attainment Scaling as
a measure of meaningful outcomes for children with sensory
integration disorders. American Journal of Occupational Therapy,
61, 254-259. McKay, G., Somerville, W., & Lundie, J. (1996).
Reflections on goal attainment scaling (GAS): cautionary notes and
proposals for development. Education Research, 38,2,
161-172.Ottenbacher, K.J. & Cusick, A. (1990) Goal attainment
scaling as a method of clinical service evaluation. American
Journal of Occupational Therapy, 44, 6, 519-525.Steenbeek, D.,
Ketelaar, M., Galama, K., & Gorter, J.W. (2007). Goal
attainment scaling in pediatric rehabilitation: a critical review
of the literature. Dev Med Child Neurol, 49, 550-556.Turner-Stokes,
L. & Williams, H. (2010). Goal attainment scaling: a direct
comparison of alternative rating methods. Journal of Clinical
Rehabilitation, 24, 66-73.