Evaluation on a Shoestring Participatory Techniques for the Evaluator with Minimal Funding.

Evaluation on a Shoestring

Participatory Techniques for the Evaluator with Minimal Funding

Overall goals for the morning

Learn how to use 11 interactive techniques

Study frameworks for participatory evaluation practice

Discuss how to overcome barriers to participatory evaluation

Reflect on how to apply these ideas to your own evaluation practice

The morning’s agenda

Seven techniques before the breakLecturette: Frameworks and

psychological principlesFour more techniques before lunch

Throughout the morning: Continuing reflection on your own practice

Types of techniques

For responding to set content

(#1-5)For generating information

(#6-10)For organizing information

(#11)

The basic tasks of inquiry

1. Framing questions

2. Determining an appropriate design

3. Identifying a sample

4. Collecting data

5. Analyzing data and presenting results

6. Interpreting results

7. “Reporting”

Important point

You can use

these techniques THROUGHOUT

the evaluation process

[Not just at the beginning. . .]

Why use participatory techniques?

How can participatory techniques help evaluators?

My participatory principles

1. Building people’s capacity to think evaluatively matters.

2. Participation in evaluations should be a learning experience.

3. It is essential to involve people actively in evaluations.

How participation helps

People invest in the evaluation process and outcomes

It makes evaluation less scarySome will learn evaluation skills

It is fun!

Technique #3- Statement 1

State specialists should include an evaluation tool for each program they introduce to county staff.

Strongly Disagree Agree Strongly

Disagree Agree

Technique #3- Statement 2

State specialists should be responsible for compiling overall evaluation results that will lead to the creation of impact statements for ND reporting.

Strongly Disagree Agree StronglyDisagree Agree

Technique #3- Statement 3

County staff should be required to enter all county survey results into a system that will allow state staff to aggregate data and produce impact reports.

Strongly Disagree Agree StronglyDisagree Agree

Debrief strategies for responding to set content

#1- Voicing variables

#2- Fist-to-five

#3- Belief sheet

#4- Dot voting

#5- Corners

Strategies for generating information

#6- Three-step interview

#7- Data dialogue

#8- Making metaphors

#9- Check-in

#10- Graffiti/carousel

Technique #6- Three-step interview

Three roles create three steps: Interviewer Interviewee/respondentRecorder

The interview process is structured to build on social psychological principles of cooperation

Technique #7- Data dialogue

A process to use when you cannot afford focus groups

It takes advantage of some of the processes of the three-step interview

Can be useful in community settings

Analysis exercise

In what ways is a data dialogue like a three-step interview?

In what ways is it different?

Conceptual frameworks for participatory techniques

Useful for planning Helpful for analysis

What roles can evaluators play?

A relationship exists between the evaluator and the client, the program staff, and other evaluation stakeholders

The evaluation decision-making and implementation relationship may shift during the study

Interactive Evaluation Quotient

LOW

HIGHEvalu-

ator

Programleaders,

staff

Invo

lvem

ent i

n de

cisio

n m

akin

g an

d im

plem

enta

tion

Participant-directed

Collabor-ative

Evaluator-directed

ZONES

Examples of evaluator roles

Technical expert on research design, measurement, statistics

Facilitator of group interactionCoach of others doing their own

evaluationsOthers?

Types of participant involvement

Mere awarenessPassive support or minimal

participationActive participation in the evaluation

processCommitment to consider and

ultimately use the evaluation results

Social interdependence theory

Goal

Structures Interactions Outcomes

Social interdependence theory

At best Cooperative goal structures (positive interdependence) Promotive (responsive) interaction Constructive outcomes

At worst Competitive goal structures (negative interdependence) Oppositional (obstructive) interaction Destructive outcomes

Structuring cooperative participation

Positive interdependenceIndividual accountabilityPromotive (face-to-face) interactionSocial skillsGroup processing

(Adapted from Johnson & Johnson, 2000)

Structuring positive interdependence

1. Identify a common purpose2. Create shared benefits or

consequences3. Provide one set of materials4. Assign complementary and

interconnected roles. . . plus

Structuring positive interdependence

5. Designate an outside force to motivate people to coordinate efforts

6. Arrange the workspace purposefully

7. Have the group establish a shared identity

Strategies for generating information

#6- Three-step interview

#7- Data dialogue

#8- Making metaphors

#9- Check-in

#10- Graffiti/carousel

Strategy for organizing information

#11- Concept

formation

Techniques #10 and #11

Can be done on the wall, informally, and is then called graffiti

Can be done on flipchart paper passed among groups and is then called carousel

Always coupled with Technique #11- Concept formation

Technique #8- Making metaphors

A is worth 1000 words

tall impressive

BIGmagnificent terrifying

High Not like home awesome sun struck

Technique #9- Check-in

What is the most important idea you have learned this morning?

Thank you, and good luck!

Jean A. King

[email protected]