An overview of selected informants Harm-Jan Fricke October 2011 [email protected].

An overview of selected informants

Harm-Jan FrickeOctober 2011

[email protected]

Q&I of DE depends particularly on◦ Quality of organisation and management of work◦ Quality of ways of working with audiences, partners,

beneficiaries

Specific to DE are issues of: Coherence:

◦ between values and actions, between actions and relevance to the own organisation and the audience/beneficiaries/partners

Partnerships:◦ explicitly defined and operated

Participation:◦ In methods used and in participants defining work done

Perspectives:◦ Plurality of viewpoints

Project/ programme designbased on quality standards

Monitoring& evaluation(a) M&E instruments, (b) M&E practice

Official policies: quality standards translated into grant guidelines,

funding criteria Organisational Quality Management

Sharing good practices/ lessons learnt within the org. & with networks; peer reviews (e.g. Norway), etc.

Work on quality frameworks (a) in the organisation, (b) in networks, (c) for policy

level

Attention to the context in which work takes place: the relationship between the ‘development agenda’ and the ‘learning agenda’

Distinguish between process and impact evaluation: show progress against defined milestones or progress markers

Suggested criteria and indicators at output level:◦ # of schools/groups/individuals participating in a programme/course/event◦ # of sessions/meetings held◦ % of participants who complete a programme◦ # of publications/resources/webpages produced ◦ # of visits to a website ◦ # of posts to a blog, facebook page etc.◦ # of people who join an organisation

Criteria and indicators at outcome level:◦ % of participants who take an action/ join relevant activist groups or campaigns◦ % of participants who go on to further study/training in the area◦ % of participants who change their consumer behaviour and/or lifestyle choices ◦ Progress towards ‘markers’ of desired behavioural changes

Output criteria Indicator

Cooperation with DES is enhanced Number of meetings with the DES

Number of times DE is mentioned at the cross-departmental working group

Educational policies contain DE messages Number of educational policies containing dev ed messages

Quality Research Number of research articles on DE

Outcome criteria Indicator

Cooperation with DES is enhanced

(Enhancement is defined as increased awareness and buy-in from people at senior management level and increased commitment to integrate messages)

Senior members of DES attend DE events and are open to conversations and dialogue

DE becomes visible in DES communications and policies

Educational policies contain DE messages Quality and depth of dev ed messages contained in policies

Quality Research Indicators for research should follow academic standards (peer-reviews, quality of publications, etc.) but also bear in mind that relevance for practice is an important indicator of success for quality DE research.

Looking for quality in DE means the following: awareness of the relationship DE action -

context, culture, reality, power relationships (society, north-south relationships, institution, funding,…)

To act in coherence with our values. Quality on 2 main levels,

◦ Level 1 : the 3 main groups of actors involved in a DE action : participants, educators (and the institution to which they belong) and partners

◦ Level 2 : the 3 main phases in the setting up and development of any DE action: action design, action process or processes, results and outcomes

“Successful advocacy efforts are characterized not by their ability to proceed along a predefined track, but by their ability to adapt to changing circumstances”

Attribution of effect and of effectiveness is difficult Therefore:

◦ Assess the (policy-social) context within which interventions take place

◦ Use a "spread betting" approach to advocacy, and thus to evaluation since success or pay-off is rarely due to any one project/intervention alone

◦ Adopt a Long Time Horizon◦ Pay Attention to Policies’ Impact on Politics, Not Just on

Target Populations◦ Evaluate organisations as a whole, not the single project

◦ Clarify concepts and definitions: Advocacy v. Campaigns Advocacy v. Awareness Raising Advocacy/Campaigning v. Education

Q&I shown through:◦ Strategic approach to do with for instance:

Choice of topic/theme/issue; Problem analysis Stakeholder analysis Definition of objectives Critical path/Intended change process

◦ Participation including to do with: Vision Values Reflection Theory of change made explicit

Criteria:◦ Participation of audience/target group and

relevance of issues to learners’ needs◦ Methodology to promote engagement,

motivation, critical reflection◦ Engagement as outcome: with skills useful in

various situations

Issues to address include: Plan for impact Address relevance Show coherence with vision Be based on explicit values Use participatory approach Show different perspectives Stimulate reflection (on what is being

gained) Be sensitive to gender and other power

relations

Does it engage the user in questioning, discussion and action?

Is it relevant and educational

Does it show different perspectives on the issue – including minority and southern

Does the resource enforce/reinforce stereotypes?

Is it relevant to the target group Is it accessible to the user Is the information accurate and up-to-date

OECD: Relevance Effectiveness Efficiency Sustainability Impact

DG DevCo: the above plus Coherence Added value

Given the foregoing andgiven the DEEEP objective (a pan-European

quality framework on Development Education):

What can the Q&I group do next?◦ What is needed?◦ What is useful?