UCD Fellows in Teaching and Academic Development Exploring the Practice of Assessment in First Year Suzanne Guerin – School of Psychology Jonathan McNulty.

UCD Fellows in Teaching and Academic Development

Exploring the Practice of Assessment in First Year

Suzanne Guerin – School of Psychology

Jonathan McNulty – School of Medicine & Medical Science

Michael Staunton – School of History & Archives

Introduction

• Assessment is central to teaching and learning practice and policy (Goos, Gannaway & Hughes, 2011)

• Evidence of increased awareness and variation

• Continues to be an area for improvement (Craddock & Mathias, 2009)

• Developments can create challenges for key stakeholders (Brew, Riley & Walta, 2009)

• Little research into provision of assessment, choice of methods and factors that influence practice (Craddock & Mathias, 2009)

Aim and Justification for Study• Aim of this study is to explore the practice of assessment in

first year teaching

• Underpinned by centrality of assessment, esp. in first year, in UCD’s education strategy; One of the key aims of which is:

‘To foster early and lasting student engagement’,

which includes : ‘A review and reform of the structure, outcomes, assessment and remediation strategies for first year, and in particular the first semester, to support the transition from second- to third-level and to adapt to the different needs of different students; The further development of approaches to engage and support students, especially in their first year, including small group learning, peer-mentoring, academic advice and mentoring, specific supports for the development of transferable skills and information literacy, and general welfare supports‘.

Justification

• We need to examine practice against UCD Teaching & Learning assessment principles (O’Neill & Noonan, 2011):– regular, low stakes assessment with opportunity for

feedback on their progress– in-class self and/or peer review– collaborative learning (peer & group work, project

work)– redesign of learning sequence, blended learning

opportunities– Active/task-based learning with more authentic

assessments (subject/discipline identity)– Awareness of student work-load

Research Questions

• What factors currently contribute to assessment practice in first year modules?

• What supports and obstacles are evident in assessment practice?

• To what extent are assessment practices aligned with the UCD assessment principles?

Overarching design

1. Module review (Autumn 2011-Spring 2012)

2. Survey of staff (Autumn 2012)

3. Case studies (Spring-Summer 2013)

Phase 1: Module Review

• Aim: to establish the types of assessment used in first year modules and key patterns

• Examined institutional data provided in Module Descriptors, which contain the core information provided to students

• Content analysis (Downe‐Wamboldt, 1992) of information from 2011-2012

• 627 first year modules, drawn from 36 schools in the Colleges of Arts and Celtic Studies, Business and Law, Engineering and Architecture, Health Sciences, Human Sciences, and Science

Module review: Key Findings

• Average module size = 149 (Range 10-647)

• Average number of assessments = 2.5 (Range 0-8)

• Only 11.8% of modules included reference to assessment in the module descriptor text

Module review: Assessment Type

Module review: Assessment Timing

Phase 2: Designing a Survey of Staff

• Aim of survey is to examine practice of assessment in more detail

• Series of key stakeholder interviews conducted to inform design of the survey

• Seven individuals (four male, three female) from administration, academic and management positions

• Thematic analysis (Braun & Clarke, 2006) used to identify main themes

Phase 2: Interview Findings

• Recognised importance of assessment – as both a positive and negative influence:

“I think the assessment can be a hindrance [on engagement]. On the other hand, of course, it keeps the guys who are not genuinely excited going because they know they are going to be

assessed.”

Phase 2: Elements of Assessment

Discussion: module review and interviews

• Average number of assessments lower than previously identified and fewer modules had three or more assessments (O’Neill & Noonan, 2011).

• Variation in methods reflects Craddock & Mathias (2009) though clear emphasis on exams.

• Interviews identified a broad range of influencing assessment choices reflects previous research (Harris & James, 2006; Havnes & McDowell, 2008).

• Patterns suggest limited impact of the UCD assessment principles (O’Neill & Noonan, 2011).

14

Staff survey: online questionnaire

• Survey of all staff involved in delivery of first year modules

• Survey examines:– Current assessment practice in first year

modules– Related workload for staff and students– Familiarity with and attitudes towards the

UCD Assessment Principles– Factors influencing assessment practice

Staff Survey: Methodology

• Online survey of staff

• Targets staff listed as coordinating Level 1 UG modules in 2011-2012 and 2012-2013

• Initial email to Heads of School and Heads of Teaching & Learning in all schools

• Invitation email sent by Prof. Bairbre Redmond to stress institutional importance of the survey

• Follow up email after two weeks (due week of 26/11/12)

Staff Survey: Preliminary Analysis

• 52 responses to date, 61.5% completed

• 44% male, 44% female, 12% missing data

• Years teaching in UCD:– Range 1-35 yrs, Average = 10.9 (SD = 7.35)

• Number of Stage 1 UG modules coordinated– Range 1- 5, Mode = 1

• Number of Stage 1 UG modules taught– Range 1- 5, Mode = 1

• Teaching & Learning Responsibilities: 65.4%

What College are you based in?

Assessment Types Examined

Types of Assessment Used

Module review: Assessment Type

Types of Assessment Used• Evidence of group assignments, self- and peer- assessment,

active / task-based assessments, learning journals, etc.

• Assessment FOR learning

Additional Questions on Assessment

• What proportion of the module does the assessment account for?

• How useful are the methods of assessment?

• What is the associated workload for staff and for students?

• Definition and use of formative assessment.

• Awareness and views of UCD assessment principles

• Factors influencing assessment practice

• Views on strengths and limitations of current module descriptor system

Phase 3: Case studies: sampling criteria• Stratified sampling:

– Module size– Number of assessments above and below the mean– Traditional v non-traditional forms of assessments– Involvement in T & L

e.g. 4 large modules, half with assessments above the mean, half below

4 small modules, half with assessments above the mean

• one case study per block (n=8)

Case studies: data collection

• Documentary material e.g. module guides

• Student feedback (via module coordinators)

• Focus groups with students

• Interviews with module coordinators

Data analysis:

• anonymous cross case analysis (public),

• underpinned by individual case analysis (private)

Thank you

• Suzanne Guerin – [email protected]

• Jonathan McNulty – [email protected]

• Michael Staunton – [email protected]

UCD Fellowships funded under the Strategic Innovation Fund II