Developmental Evaluation … an emerging approach, discipline , methodology, disposition… Rita Ndagire Kizito CTLM 30 March 2016
Developmental Evaluation… an emerging approach, discipline ,
methodology, disposition…
Rita Ndagire KizitoCTLM 30 March 2016
• Developmental evaluation is an emerging approach to evaluating innovations (or
interventions) …that are in a continuous state of unpredictable change and development (Patton, 2011; Reynolds, 2014).
• Builds on Patton’s (1978) earlier concept of ‘utilisation-focused’ evaluation.
• A form of design thinking that situates evaluation differently ,as “ not simply part of quality assurance, but as a key design, feedback and planning tool”
(Leonard, Fitzgerald & Riordan, 2015)
• Outcomes and goals are already known
• Seeks to establish if progress is being made towards these goals
Conventional Evaluation(Formative or Summative Evaluation
Developmental Evaluation
• Seeks to ‘design’ the goals
• Asks what works for whom, and in what circumstances?
(Leonard, Fitzgerald & Riordan, 2015)
Conventional Evaluation
Plan Act
(Formative Evaluation)
(Summative Evaluation
Evaluate
DevelopmentalEvaluation
Evaluate
Time
Plan
Act
Evaluate
“these two types of thinking are seen to be mutually exclusive, but developmental evaluation is about holding them in balance” ( Gamble, 2006)
Developmental evaluation
merges
Evaluation
…about critical thinking
Development
…about creative thinking
Which type of evaluation is appropriate ?
“what ? So what? Now what ?”
Forms of evaluation
Object of
evaluation
Developmental
(In the initial stages
and along the way)
Formative
(along the way)
Summative
(at the end or
end of a
section)
Processes
and
procedures
Have we misjudged
something?
Has something
unexpected
occurred?
Do we need to
change our
assumptions?
Is there
anything we can
adjust to
improve what is
happening?
Delivery - Did
things happen
the way we
planned for
them to
happen?
Quality- how
well did things
get done?
(The next five slides are from a presentation by Prof Jeff Jawitz, 2015, UCT)
Jeff Jawitz, 2015
Object of
evaluation
Developmental
(In the initial
stages
and along the
way)
Formative
(along the way)
Summative
(at the end or
end of a section)
Participant
satisfaction
and
experience
How well did
we understand
participant
expectations?
Can we identify
and address some
of the negative
aspects of
participant
experience?
Can we identify
and take
advantage of
positive aspects of
participant
experience?
Did participants
report positive or
negative
experiences of the
course?
Jeff Jawitz, 2015
Object of
evaluation
Developmental
(In the initial
stages
and along the
way)
Formative
(along the
way)
Summative
(at the end or
end of a section)
Participant
performance
and
achievements
(learning?)
How well did we
judge the
capabilities of
the participants?
How well have
we been able to
stretch
participants in
terms of
extending their
capabilities and
understanding?
Participant
engagement –
to what extent
did
participants
engage with
elements of
the
programme
- what was the
quality of
engagement?
Did participants
achieve the levels of
performance that we
expected them to
achieve?
Jeff Jawitz, 2015
Object of
evaluation
Developmental
(In the initial
stages
and along the
way)
Formative
(along the
way)
Summative
(at the end or
end of a
section)
Project
effectiveness
How realistic are
we in expecting
significant results
from the
effective design
and delivery of
projects within
the institutional
context?
Can we
improve
support for
project
design?
What levels of
effectiveness
of
participant’s
project was
achieved?
Jeff Jawitz, 2015
Object of
evaluation
Developmental
(In the initial
stages
and along the
way)
Formative
(along the
way)
Summative
(at the end or
end of a
section)
Task: Select a few items that you wish to evaluate related to your project. Identify three questions that reflect the three different evaluation focusses for each of these items.
Jeff Jawitz, 2015
References
• Jawitz Jeff ( 2015). Presentation at TAU Fellowship workshop• Leonard, S. N., Fitzgerald, R. N., & Riordan, G. (2015). Using
developmental evaluation as a design thinking tool for curriculum innovation in professional higher education. Higher Education Research & Development, 1-13.
• Patton, M.Q. (1978). Utilization-focused evaluation. Beverley Hills, CA: Sage.
• Patton, M.Q. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York: Guilford Press
• Reynolds, M. (2014). Equity-focused developmental evaluation using critical systems thinking. Evaluation, 20(1), 75–95. doi:10.1177/1356389013516054 .