Program Evaluation Essentials. WHAT is Program Evaluation?
Post on 16-Dec-2015
222 Views
Preview:
Transcript
Program Evaluation Program Evaluation EssentialsEssentials
WHAT is Program WHAT is Program Evaluation?Evaluation?
Think/Draw/PairThink/Draw/Pair
Share your image with the person Share your image with the person sitting next to you and notice common sitting next to you and notice common themesthemes
Draw the first image that comes to mind when you hear or see the word evaluation
Label your drawing with a word or two that describes the image
Evaluation defined…Evaluation defined…
• Systematic determination of the merit or worth of an object. (Scriven, 1967)
• Utilization-focused program evaluation: done for and with specific intended primary users for specific, intended uses. (Patton, 2008)
• What? So what? Now what? (Patton, 2008)
• Numerous additional definitions.
How is evaluation different How is evaluation different
from research?from research?• Similarities:
o Describe objects studied
o Understand the relationships between variables
o Use qualitative and quantitative methods
How is evaluation different How is evaluation different from research?from research?
• Differences:
o Purpose
o Utility
o Context
o Judgmental quality
Common misconception #1Common misconception #1
Evaluation is only about numbers (quantitative data)
Common misconception #2Common misconception #2
RESEARCH is about numbers; evaluation is NOT about numbers.
Instead, evaluation is about words, stories, and anecdotes (qualitative data)
Rather. . .Rather. . .
Research Research ANDAND evaluation evaluationuse use BOTHBOTH
quaquantntitative itative ANDAND quaquallitative methodsitative methods
Evaluation is a professionEvaluation is a professionguided by standardsguided by standards
The Joint Committee Program Evaluation Standards
Utility
Feasibility
Propriety
Accuracy
WHY would I use WHY would I use Program Evaluation in Program Evaluation in my organization?my organization?
Top 10 reasons you Top 10 reasons you might use evaluation…might use evaluation…
To establish the impact of a program
To see if a program achieved its outcomes
To make informed decisions about a program, policy, product
To determine the need for a program
To improve a program
To figure out how a program can be or has been To figure out how a program can be or has been implementedimplemented
To determine a program’s merit, worth, or valueTo determine a program’s merit, worth, or value
To examine To examine howhow a program works a program works
To distinguish what works from what doesn’tTo distinguish what works from what doesn’t
And thus, to make the world a better placeAnd thus, to make the world a better place
Top 10 reasons you Top 10 reasons you might use evaluation…might use evaluation…
What's in it for me?What's in it for me?
• Useful skills Useful skills
• Support from others Support from others
• Ability to impress Ability to impress people people
• Good food Good food
• It’s fun!It’s fun!
What's in it for my What's in it for my organization?organization?
• Clarifies purpose & direction
• Develops common language
• Provides opportunities for collaboration
• Contributes to credibility & helps secure funding
• Promotes motivation & excitement
• Promotes organizational learning & improvement
• Increases effectiveness
• Furthers good decision making
What evaluation What evaluation isis and and what you and your what you and your
organization organization getget from from evaluation depend on evaluation depend on
the purpose you the purpose you establish for the establish for the
evaluationevaluation
Evaluation PurposesEvaluation Purposes
Two main purposes:• Valuing (Michael Scriven)
• Decision-making (Daniel Stufflebeam and Michael Quinn Patton)
Accountability Development
More recently discussed purposes:• Evaluation for learning (Torres & Preskill, 2001; Preskill, 2008)
• Evaluation for social betterment (Mark & Henry, 2003; 2004)
• Evaluation for deliberative democracy (House & Howe, 2000)
HOW can I begin use of HOW can I begin use of evaluation?evaluation?
So it all starts with a So it all starts with a Question…Question…
What do you want to know about your program?
How does your program work?
What is the impact of your program?
What opportunities do you have to
expand your program to serve others?
How or to what degree do the resources and activities meet the needs of stakeholders?
Answering the question Answering the question requires…requires…
Systematic collection and analysis of datao This might mean using quantitative measures,
qualitative measures, or both
Possible ways to collect Possible ways to collect data for an evaluationdata for an evaluation
Quantitative:
o Surveyso Participant
assessmentso Cost-benefit analysiso Statistical analysis of
existing program datao Some kinds of record
and document review
Qualitative:
o Focus groupso Interviewso Observationso Appreciative inquiryo Case studieso Some kinds of record
and document review
What are the best What are the best methods for your methods for your
evaluation??evaluation??It all goes back to the question you started with…
Some data collection methods are better than others at answering your questions
And some tools are more appropriate for the audience you need to collect information from
The Evaluation ProcessThe Evaluation Process
1.1. Determine the object (program, process, Determine the object (program, process, etc.) for evaluation and available resourcesetc.) for evaluation and available resources
2.2. Develop guiding evaluation question(s)Develop guiding evaluation question(s)
3.3. Design the evaluation, including methodsDesign the evaluation, including methods
4.4. Implement the evaluation: collect Implement the evaluation: collect information, analyze and interpret findingsinformation, analyze and interpret findings
5.5. Report and use evaluationReport and use evaluation
Questions?Questions?
This presentation was developed for This presentation was developed for the Minnesota Evaluation the Minnesota Evaluation
AssociationAssociation
June, 2009June, 2009
All rights reservedAll rights reserved
Presentation prepared by (in alphabetical order) Presentation prepared by (in alphabetical order) Edita Bucinca, Ann Mavis, Laura Pejsa, Cindy Reich, Edita Bucinca, Ann Mavis, Laura Pejsa, Cindy Reich,
Denise Roseland, and Robert TornbergDenise Roseland, and Robert Tornberg
Under the direction of Dr. Jean A. King, University of MinnesotaUnder the direction of Dr. Jean A. King, University of Minnesota
top related