Embedding Institutional Curricular Priorities in the First Year Embedding Institutional Curricular Priorities in the First Year A Case Study from RGU Roger.

Embedding Institutional Embedding Institutional Curricular Priorities in the Curricular Priorities in the

First Year First Year

A Case Study from RGU

Roger McDermott, Gordon Eccleston, Garry BrindleySchool of Computing

The Robert Gordon UniversityAberdeen, UK

[email protected]

2HEA-ICS: 2008

Plan of TalkPlan of Talk Introduction

Institutional Priorities: Where do they come from? The RGU context

The School of Computing The Evaluation Problem

Adapting the Kirkpatrick model

3HEA-ICS: 2008

Institutional Curricular PrioritiesInstitutional Curricular Priorities

External: Social/Political – QA/QE Scottish Funding Council Quality Assurance Agency for Higher Education

Subject Benchmarks ELIR – Enhancement Led Institutional Review Higher Education Academy Scottish Enhancement Themes

Environmental Demographic change Technological change

4HEA-ICS: 2008

The “Ideal” FY CurriculumThe “Ideal” FY Curriculum

First Year Enhancement Theme 2008 report on Curriculum Design for the First Year. Kathy Bovill, Kate Morss and Catherine Bulley (QMU)

tried to identify key features of “an ideal first year”. Work was based on investigation of perspectives from

academic staff students

Also consulted previously published literature, case studies.

5HEA-ICS: 2008

Ideal Curriculum (Literature)Ideal Curriculum (Literature) Orientation of students to increase social and academic

engagement, ‘connectedness’ to university, sense of direction and future career

Development of learning skills Student-centred, active learning, through problem-

based, project-based and group learning Collaborative learning or learning communities to

enhance transferable skills and lend a sense of belonging

Formative assessment and feedback Progressive skills development Time and structures for reflecting on learning

6HEA-ICS: 2008

Ideal Curriculum (Staff)Ideal Curriculum (Staff) Co-ordinated programme level approach Small group work Problem-based learning Student choice Early formative feedback Use most experienced staff to teach first year students Involving students in curriculum design Opportunities for personal contact between students/staff Clear communication between staff and students about all

elements of the curriculum

7HEA-ICS: 2008

Ideal Curriculum (Student)Ideal Curriculum (Student) More attention on assessment and timely feedback More challenging work Students being involved in curriculum design in a role

that is ‘more than just feedback’ Student participation in designing timetabling and

curriculum structure

8HEA-ICS: 2008

The Old RGU First YearThe Old RGU First Year

Asse

ssmen

t

APPLICATION SOFTWARE

Asse

ssmen

t

INTRO TO OOP

Asse

ssmen

t

WEBTECHNIQUES

Assessment

Asse

ssmen

t

SYSTEMS ANALYSIS

Asse

ssmen

t

Asse

ssmen

t

COMPUTER SYSTEMS

OOP TECHNIQUES

Assessment

• GRAPHICS• BUSINESS

QUANTITATIVEMETHODS

ROUTE MODULE

PE

RS

ON

AL

TU

TO

RIA

L S

YS

TE

M

3D PROG WITH ALICE

MATHS &

STATS

JAVAWITH

BLUE-J

Asse

ssmen

t

APPLICATION SOFTWARE

Asse

ssmen

t

INTRO TO OOP

Asse

ssmen

t

WEBTECHNIQUES

Assessment

Asse

ssmen

t

SYSTEMS ANALYSIS

Asse

ssmen

t

Asse

ssmen

t

COMPUTER SYSTEMS

OOP TECHNIQUES

Assessment

• GRAPHICS• BUSINESS

QUANTITATIVEMETHODS

ROUTE MODULE

PE

RS

ON

AL

TU

TO

RIA

L S

YS

TE

M

3D PROG WITH ALICE

MATHS &

STATS

JAVAWITH

BLUE-J

9HEA-ICS: 2008

What was wrong with the Old What was wrong with the Old Structure?Structure?

Disenchantment in sem 1 disengagement in sem 2. Little daily contact between personal tutors and tutees Little provision for students to learn study skills, group/

team-working. 12 week first semester teaching session

Relentless teaching schedule Diagnostic assessment and remediation difficult to

schedule due to time constraints.

10HEA-ICS: 2008

An Integrated Curriculum An Integrated Curriculum The structure of the new Foundation Year should place strong emphasis on the idea that, as a discipline, Computing is seen as a unified and self-supporting whole.

Introductory teaching should, wherever possible, reflect and convey this notion of integration to the student.

As a consequence, prominence should be placed on the concept of an Integrated CurriculumIntegrated Curriculum.

11HEA-ICS: 2008

An Integrated CurriculumAn Integrated CurriculumThis integration should manifest itself at structural and pedagogical levels, as well as that of student-support:

Integration of teaching across more fluid thematic boundaries

Integrative assessment Partner-module assessment Synoptic assessment Integration of Student-Support

12HEA-ICS: 2008

The New Foundation Year The New Foundation Year

13HEA-ICS: 2008

30 credit themes which run over two semesters greater flexibility to manage a balanced and engaging

teaching style can deal specifically with issues of transition.

Refocus of subject content on three major themes: Software Design and Development, Problem-Solving and Modelling, and Information Systems.

Main PointsMain Points

14HEA-ICS: 2008

2-semester 15 credit module to address soft-skills, collaborative and group-working, PDP, promote employability skills and foster study skills to encourage independent learning.

Used (but monitored!) portfolio-based assessment to allow students to build up work over two semesters

Promoted the introduction of joint assessment opportunities between the major themes and the “collaborative working” module.

Built time for remediation into the timetable at the start. Used Social Software – Blogs, Wikis, Virtual Social

Spaces – to provide a vehicle for collaboration.

Other PointsOther Points

15HEA-ICS: 2008

Other PointsOther Points Strict attendance monitoring and used participation in

formative assessment as a Learning Objective.

Set up a dedicated Foundation Year Teaching Team, composed of enthusiastic and approachable staff with proven teaching ability to oversee the academic delivery.

Use the same staff to provide pastoral, academic and remediation support for students.

16HEA-ICS: 2008

AssessmentAssessmentAssessment was accomplished by coursework. Group projects – extremely popular with students.

introduction of structured peer and self-assessment for the collaborative work,

greatly enhanced the degree of peer socialisation in the cohort and seemed to promote the “connectedness”.

2-semester modules Sem 1 used mainly for “quasi-formative” assessment

(passed through participation in the process) Summative assessment was reserved for Sem 2.

17HEA-ICS: 2008

ResultsResultsConcern that long modules simply allow students to fail 30 credits at a time rather than 15 credits. Evidence from 2007-8 suggests that this fear may be

allayed by rigorous module administration. Individual occasions of assessment were used to demonstrate and record competence and achievement

in a variety of outcomes across the different themes. Subsequent assessment occasions also allowed students to exhibit competence in prior learning outcomes which they may not have acquired at their first

attempt.

18HEA-ICS: 2008

ResultsResults For First Year students who were not failed for non-

submission, the 2007-8 pass rates for the three 30 credit modules, based on a similar size cohort (~75), were, on average, 5% higher than the corresponding 2006-7 figure for equivalent pairs of 15 credit modules.

The headline figure for non-submissions for the 30 credit modules were somewhat higher than those for pairs of 15 credit modules, but the narrative changed from that of slow disengagement throughout the 2006-7 session, to one where students did not submit because they failed to engage from the outset.

19HEA-ICS: 2008

ResultsResults The majority of these were non-progressing students

referred from the previous first year cohort who simply did not attend after the second or third week (and who, for administrative reasons, could not be withdrawn).

Few students progressively disengaged throughout the year and none did so solely in the second semester. This is in contrast to previous years where failure in

the first semester modules was a major trigger for disengagement in the second semester.

20HEA-ICS: 2008

The Evaluation ProblemThe Evaluation Problem

What criteria for success? Institutional Drivers

Achievement rates Retention rates

What about assessing student learning? What is the nature of the evidence? How do you provide such evidence?

What about enhancement of career skills?

21HEA-ICS: 2008

The Evaluation ProblemThe Evaluation ProblemKirkpatrick’s Model In order to classify areas of evaluation in business

training, Donald Kirkpatrick (1959) created what is still one of the most widely used models. His four levels of evaluation are: Level 1: Reaction - a measure of satisfaction Level 2: Learning - a measure of learning Level 3: Behavior - a measure of behavior change Level 4: Results - a measure of contribution to the

organisation Some dispute over whether this is a model or an

evaluation taxonomy.

22HEA-ICS: 2008

The Evaluation ProblemThe Evaluation Problem

Kirkpatrick’s Four Level Evaluation ModelThe model seeks to measure: Reaction of student - what they thought and felt about the

training Learning - the resulting increase in knowledge or capability Behaviour - extent of behaviour and capability

improvement and implementation/application Results - the effects on the business or environment

resulting from the student/staff performance.

23HEA-ICS: 2008

Kirkpatrick’s ModelKirkpatrick’s ModelLevel 1: Reaction of Students to the first year experience Affective Reactions

Did the student enjoy the first year? Did they engage with the material? Do they consider the material relevant? Did they like the operational set-up? Did they feel that level of effort required to make the

most of the learning was sustainable? Do they feel confident about the practicability and

potential for applying the learning?

24HEA-ICS: 2008

Kirkpatrick’s ModelKirkpatrick’s ModelLevel 2: Learning Evaluation Increase in Knowledge

How closely did the lecturer’s aims and objectives for students on the course match with the actual learning experience of the students?

How far has the student’s level of competence or expertise in the subject improved over the period of the first year?

How far is a positive result for one student reflected across the cohort.

25HEA-ICS: 2008

Level 3: Application of Learning Change in Behaviour

Did the students put their learning into effect in subsequent tasks?

Were the relevant skills and knowledge used appropriately?

Was there any measurable change in the activity and performance of the students and was any change in behaviour and new level of knowledge sustained?

Was the student able to transfer their learning to another person?

Is the student aware of their change in behaviour, knowledge, skill level?

Kirkpatrick’s ModelKirkpatrick’s Model

26HEA-ICS: 2008

Level 4: Results In the original context, this was the most important (and

hardest to assess). Measures the effect on the organisation or environment

resulting from the improved performance of the trainee. Typical metrics would be key performance indicators

measuring quantifiable aspects of business performance, e.g. numbers of complaints, staff turnover, attrition,

failures, wastage, non-compliance, quality ratings, achievement of standards and accreditations, growth, retention, etc.

Vital to be able to calculate a Return on Investment!

Kirkpatrick’s ModelKirkpatrick’s Model

27HEA-ICS: 2008

Kirkpatrick’s ModelKirkpatrick’s Model Levels 1 – 3 can be seen as evaluation at the individual

or cohort level. Evaluation in these areas can be related to the usual

feedback mechanisms student questionnaires staff and student focus groups statistics relating to student performance

What about Level 4? A new T&L initiative’s contribution to Institutional

success must be a factor in the overall evaluation.