“Review of Student Evaluation of Teaching at WCPPE” Catherine O’Brien PCUTL Presentation Group Project Jan 2012.

“Review of Student Evaluation of Teaching at WCPPE”Catherine O’Brien PCUTL Presentation Group Project Jan 2012

The Wales Centre for Pharmacy ProfessionalEducation (WCPPE) works:

"To motivate and support the pharmacy team, through the provision of learning and development opportunities, to enhance their competence to deliver high quality patient services in Wales."

WCPPE

How/What ?

1.Literature Review – COB 2.Current state: process & test v best practice3.Future State: proposal & test v best practice4.Costing (!!) & Implementation5.Review

Student Evaluations of TeachingHarvey (2001)2 main functions

– Internal information to guide improvement– External information for potential students and other stakeholders.

levels– Institution– Faculty– Programme– Module– Teacher Appraisal

Student Evaluations of TeachingWCPPE context• Learners – Graduate CPD, non graduate –

different expectations and needs from UG• Range of formats including online and VC• Non CU facilities• Large number of learning facilitators • Repeated courses• Learner – development in learning

Satisfaction Cycle, Harvey 2001

Stake-holder determined questions

Question distribution

Feedback to stake-holders

Implement and Monitor

Action Plan

Consultation

Report with action

Analysis of results

?

X

X

X

X

X?

Workshop

Objectives of the process• What is the internal and external purpose of the SET for WCPPE?

• What are we trying to achieve?

• What areas do we want to evaluate?

• What does the current WCPPE process look like and what are the strengths and weaknesses?

Workshop

Design of questionnaire ( Rowley, 2003) • Will one size fit all for face to face and e-format and for various styles?

• Should they be tailored to the learning outcomes?

• Do we take a quantitative or qualitative approach?

• How do we optimise response rate?

WorkshopDesign of questionnaire- contd• Should they be tick boxes or free text?

• How do we design in relation to the relative value of the responses?

• Do we require students to identify themselves or should there be anonymity?

• What aspects of teacher performance should be evaluated?

WorkshopData Collection and analysis and action• When is the right time to get them filled in, immediate or on return to work place?

• Who should collate and analyse data?

• Who is responsible for action?

• What should the new process be and how do we close feedback loop?

ResultsIts hard- balance size and aspects covered

Focus on face to face courses–Venue –Teacher performance–Qualitative approach – free text –Style and content and application to practice

2 forms –Learning facilitator - BOS–Learners – paper

Review Harvey- Satisfaction Cycle

Stakeholder determined questions

Feedback to stake-holders

Implement and Monitor

Action Plan

Consultation

Evaluation to Moodle for each course and more…

Standing item on meetings.

Developing KPI

Add to agenda with Welsh Government and other stakeholders.

Consulted Virtual Opinion Panel but could do more.

Harvey- Satisfaction Cycle (2)

Stakeholder determined questions

Question distribution

Feedback to stakeholders

Implement and Monitor

Action Plan

Consultation

Report with action

Analysis of results

Impact for WCPPE• Renewed focus on SET as a tool within WCPPE (not just ticking boxes) • Improved quality of courses as feedback is used.• Feedback is contextualised by Learning Facilitators. • Freed administrative time to market courses.• Gained experience to apply to other learning resources. • Peer review of learning facilitators.

Current thoughts and conundrums

• Will we maintain the cycle and review?• How do we engage stakeholders in questionnaire design in future?• Move to online? • Link completion& Certificate

• What is the picture across CU?• How do we consider SETs in the context of CPD across CU?

References

Lee Harvey, (2001) “Student Feedback”, “A report to the Higher Education Funding Council for England”

Jenifer Rowley, (2003) “Designing student feedback questionnaires”, Quality Assurance in Education, Volume 11 Iss: 3, pp 142-149.