Dr Tristram Hooley, University of Derby - Evaluating access and participation

Presentation to Developing Your Access Agreements Conference February 2014

Tristram HooleyInternational Centre for Guidance StudiesUniversity of Derbywww.derby.ac.uk/icegs

Evaluation

Higher Education Outreach to Widen ParticipationToolkits for practitioners

• The Office For Fair Access (OFFA) has specified that access agreements should set out how institutions plan to monitor and evaluate WP activity and specify that resources should be set aside for this purpose (OFFA, 2012: 24).

• HEFCE has also stressed the importance of evaluating WP activity, arguing that it is essential for the effective development of institutions’ strategies and practices in this area (HEFCE, 2010, Annex A).

• Evaluation should therefore be important to all HE providers, clearly acknowledged in WP strategies, and apparent in WP practice.

Understanding evaluation

• When people seek to evaluate what they are doing as part of an attempt to learn and improve, they are usually undertaking a formative evaluation, so called because it is undertaken to inform what is done while the activity is still in progress.

‘We would like to find out how to do these things better’

• When people evaluate to make a judgement on the value of a particular activity and to draw out what has been learnt, it is usually a summative evaluation; so called because it attempts to create a summary of what has been achieved and what the impacts have been.

‘We would like to find out how well these things work’

What is evaluation?

Evaluation enables us to:

• examine what we do

• think about how we can improve it

• decide on whether it was worth doing

• provide others with a summary to help them to understand what was done.

Why evaluate?

Evaluation as part of reflective practice

Levels of impact

Results

Behaviour

Learning

Reaction

Types of data for evaluation

Too little evaluation and impact work on widening participation is published.

Writing up your evaluation for broader circulation is an important way to support the development of the sector. • Self publication• Journal publication• Partnership with academics• Using external consultants

Publish or perish!

• What is your organisations approach to evaluating WP activity?

• Is it embedded or occasional?

• Is it part of an organisational approach to all activities or just specifically related to WP?

• What are the advantages/disadvantages of this approach?

Activity: Understanding evaluation

Planning an evaluationR Reasons and

purposes

What are the Reasons and Purposes for this evaluation?Examples: planning, accountability, developing, to gain resources

U Uses

What will be the Uses of the evaluation?Examples: staff development, learning from good practice, strategic planning, PR, completion of Table 7 in the annual monitoring return to OFFA/HEFCE

F Foci

What will be the Foci for our evaluation?Examples: range of activities, emphasis, aspects of evaluation, identification of priority areas

D Data and evidenceWhat will be the Data and Evidence?Examples: qualitative (interviews, case studies), quantitative (questionnaire)

A AudienceWho will be the Audience?Examples: HEFCE, senior management, other stakeholders

T TimingWhat will be the Timing?Examples: once a year, at year end

A AgencyWho will be the Agency conducting the evaluation?Examples: staff developer, course co-ordinators, external facilitator

• Working in pairs using the RUFDATA framework consider one activity which you might want to evaluate.

• Whilst completing the activity consider how you might adopt this approach to inform the evaluation activities in your organisation.

Activity: Planning an evaluation activity using RUFDATA

In small groups think about the data which is currently gathered within your organisation and how this could be utilised to support your evaluation activity. What new data might need to be gathered to support your evaluation in terms of input, participation and output data

Activity: Sources of data

In your groups or individually, prioritise three areas, activities or actions to take forward with regards to evaluating your widening partnership activity.

Activity: Taking evaluation work forward

Dent, P., Garton, E., Hooley, T., Leonard, C., Marriott, J. and Moore, N. (2013). Higher Education Outreach to Widen Participation: Toolkits for Practitioners. Evaluation, 2nd. Edition. Bristol: HEFCE.

Hooley, T., Marriott, J. and Wellens, J. (2012). What is Online Research?: Using the Internet for Social Science Research. London: Bloomsbury Academic.

Hughes, D., Bowes, L., Hartas, D. and Popham, I (2001). A Little Book of Evaluation. Sheffield: CSNU.

Moore, N., Christopoulos, M., Dimitriou, K. and Hutchinson, J. (2011). Researching the Aimhigher Greater Manchester Graduate Officer Programme. Derby: International Centre for Guidance Studies.

Moore, N. and Hooley, T. (2011). Building a Progression Culture: Exploring Learning Organisations' Use of The Progression Matrix. Derby: International Centre for Guidance Studies.

Bibliography

Dr Tristram Hooley

Reader in Career Development

International Centre for Guidance Studies

University of Derby

Tel: 01332 591268

Email: [email protected]

Twitter: @pigironjoe

Blog: http://adventuresincareerdevelopment.wordpress.com/

Contact details