Validation, Annual Review and Periodic Subject Review at the University of Northampton Dr Anne Craven, Head of Quality and Academic Partnerships 25 April.

Validation, Annual Review and Periodic Subject Review

at the University of Northampton

Dr Anne Craven,Head of Quality and Academic Partnerships

25 April 2014

Validation

External Context

• The University is required to meet the expectations set down in the QAA’s UK Quality Code for Higher Education

• Of particular relevance are:– Chapter B1: Programme design, development a

nd approval (October 2013)– Chapter B8: Programme monitoring and review

(October 2013)

• QAA Higher Education Review due to take place in 2015 – we will need to provide evidence that we do this

Purpose of Validation

• To assess the quality and standards of proposed new programmes (particularly the curriculum)

• To ensure that the new programmes align with the University’s strategic direction (i.e. Raising the Bar, Ashoka-U and the move to Waterside

• To ensure that the new programmes align with the QAA Subject Benchmark Statements, PSRB requirements and the needs of employers

• To ensure that the Programme Team is appropriately equipped to deliver a high quality student experience

Underlying Principles

• Validation is underlined by the principle of external peer review – absolutely essential

• Student engagement: student Panel members, student focus groups and evidence of student involvement of the programme development

• The importance of the appropriate consideration of all ‘Education with Others’ provision

• Strategic fit and direction with the University’s strategic plan (particularly in relation to employability, social enterprise and internationalisation)

• Proportionality and responsiveness• Accountability – ultimately Senate is responsible

Key Validation Issues

• This is not an exhaustive list:– Evidence of a real market for the programme and its fit

with the strategic direction of the School and University– The programme design and curriculum (academic level,

progression between levels, diversity of assessment strategies, coherence of modules and the Award Map

– The provision of appropriate resources (staffing, books, on-line resources, laboratories, specialist equipment etc)

– The preparation of students for employment– The quality of the student experience overall (i.e. access

to Personal Academic Tutors, support for international students etc)

First Steps

• School based planning process• Development approval

– University Management Team– If collaborative provision: approval of both institution and

programme– Throughout the process

• Due Diligence• Assessment of risks

Validation Processes

• From 2013-2014 the majority of validations will be held ‘on-line’ – using an iterative process

• TUNDRA 2 used as a forum to conduct ‘on-line’ consultation• Still a robust consideration of a rationale and curriculum

documentation• An External Academic Advisor to provide advice to the

Programme Team from the earliest opportunity• An External Academic Reviewer to review the final documentation

and assure that best practice has been carried out• There will still be an internal Chair to facilitate the process• A QAP Officer will monitor the process• PSRB-related validations might still require an ‘event’ • Student-focus groups will be used to obtain the student view• All Panels to have an External Industry Advisor

Role of Panel members

• To provide their peer academic expertise (academic Panel members) to positively critique the provision being considered

• To identify areas that need further exploration (i.e. issues that may need addressing to ensure the best quality student experience)

• To identify examples of good practice and enhancement that can be disseminated more widely

• To confirm that the standards are equivalent to other programmes at their own institutions (externals)

• To ensure that the programme will meet the needs of employers and enhance the employability of students

• Above all to be ‘Critical Friends’ – constructive not confrontational

Questions to Ask at Validation (some examples)

• Some examples of areas to explore:– The variety of assessment strategies (any innovative

forms?)– Evidence of progression between levels (Learning

Outcomes)– What is the strategic fit of the programme?– What is the strategic direction of travel?– How will the employability of students be enhanced?– Is there a real and sustainable market for the students?– Will there be adequate resources (books, journals, staff

etc) to provide a quality experience?– How will the student voice be heard?

Other Considerations

• Lifelong Learning (PT study, timetabling, equivalent qualifications)

• Accreditation of prior learning

• Internationalisation; collaborative provision

Sources of Information

• Quality Assurance Agency website:– www.qaa.ac.uk

• Quality and Academic Partnerships:– Contact [email protected]– The Validation Handbook being amended to

reflect introduction of ‘on-line’ validation processes from 2013-2014

Annual Review

Annual Monitoring

• Refocused on the ‘module’ and the ‘programme’• Efficient and risk-based approach• Refocus to action planning• Philosophy of continuous improvement• More intelligent use of institutional data• Integration with ‘Raising the Bar’• Ownership of quality at the individual staff and team level

Where next? Role of Programme Leaders in quality enhancement and student experience. Module evaluation (EvaSys). Northampton Online Framework. Programme of Auditing NILE sites.

The DashboardFIELD OF EARLY YEARS 01/08/2011

B/M 90% 

B/M

Red <80% Red <C

HE Level Module Session Code Total Total Pass Pass %% Pass

IndicatorAv Pass Grade

Pass Grade Indicator

4 EYS1011 STD 77 66 86% v B- u

  EYS1012 STD 76 71 93% v C v

  EYS1013 STD 76 74 97% u C+ w

  EYS1013 STDCY 28 28 100% u C+ w

  EYS1014 STD 76 67 88% v C+ w

  EYS1110 LC 15 15 100% u C+ w

  EYS1110 LS 23 23 100% u C v

  EYS1110 STD 26 26 100% u C v

  EYS1110 STDB 27 27 100% u C+ w

  EYS1111 LC 15 15 100% u C+ w

  EYS1111 LS 23 23 100% u C+ w

  EYS1111 STD 27 26 96% u C+ w

  EYS1111 STDB 26 26 100% u C+ w

  EYS1112 LC 15 14 93% v B- u

  EYS1112 LS 23 23 100% u B- u

  EYS1112 STD 26 26 100% u C+ w

  EYS1112 STDB 27 27 100% u C+ w

HE Level 4 Total 606 577 95%   C+  

The Dashboard – Comparability of Session Codes

     Module av. Indicator

Module av. Indicator

Module av. Indicator

  HRM3011 DL 1   0%   

   0%

  HRM3011 STD 43 40 93%  B-

 25 58%

Average (arrow compares field av.)

        47%  

B-

  29%

     Module av. Indicator

Module av. Indicator

Module av. Indicator

  LEI3006 DL 2 2 100%   B+   2 100%  

  LEI3006 DL02 1 1 100%   C-     0%  

  LEI3006 DLBN 5 2 40%   D+     0%  

  LEI3006 DLH 31 17 55%   C     0%  

  LEI3006 DLH02 11 9 82%   D+     0%  

  LEI3006 JAN 90 75 83%   C   23 26%  

  LEI3006 STD 173 150 87%   C+   60 35%  

Average (arrow compares field av.)

    173 150 87%  

C

60 35%

Dashboard

• Red = below threshold. This requires a commentary and a SMART action to remedy and improve.

• Green = exceeding benchmark. This may (where appropriate) require a commentary and a SMART action for the dissemination of good practice.

• Amber = below threshold but normally relates to small numbers (less than 10). This does not require immediate action but a continuing trend will attract an action for improvement.

• Black = between threshold and benchmark– no action required.

Periodic Subject Review (PSR)

External Context

• The University is required to meet the expectations set down in the QAA UK Quality Code for Higher Education

• Of particular relevance to PSR are:

– Chapter B1: Programme design, development and approval (October 2013)

– Chapter B8: Programme monitoring and review (October 2013)

• QAA Higher Education Review (HER) due to take place in 2015 – the University will need to provide evidence that we do this

Purpose of PSR

• To allow Schools and Subject Teams to assess the continued ‘health’ and appropriateness of existing programmes

• To ensure on-going alignment with QAA Subject Benchmark Statements, PSRB and employers needs

• To assess the future strategic direction of the programmes - particularly in relation to Raising the Bar and Ashoka-U

• To test that there are appropriate resources/strategies in place to ensure the effective management of student learning opportunities

• To identify examples of good practice and enhancement to disseminate to the wider University

• To ensure that any Education with Others (EWO) arrangements remain appropriate

• To ensure that public information is correct and accessible

Underlying Principles

• PSR is underlined by the principles of both external and internal peer review – absolutely essential

• The importance of student engagement:

• Recent graduate and (from 2014-2015) a current student on all PSR Panels

• Panels meet with a representative sample of students during the event

• The importance of the appropriate consideration of all ‘Education with Others’ provision

• Strategic fit and direction (and sustainability)

• Currency/coherence of the provision since the last PSR (taking into account changes since the last PSR)

• Ensuring staff research informs the curriculum

Key PSR Issues

• This is not an exhaustive list:

• The appropriateness of the strategic direction of the programmes and their fit with Raising the Bar etc

• The currency of the programmes - particularly in meeting employers, PSRBs and students’ needs

• The identification of actions arising from the Annual Review process and student surveys/evaluations and the extent to which these are being carried out

• The on-going appropriateness of ‘Education with Others’ provision and arrangements (comparability with on-site provision)

• Ongoing development of the programmes – ‘future-proofing’

PSR Panel constitutions

• Panel Chair - normally an Executive Dean or Deputy Dean from a School outside of the Subject area

• Three internal Panel members (two from other Schools outside of the Subject area)

• Employer representative (may be a representative of a Professional/Statutory/Regulatory Body)

• Recent graduate from the same Subject area

• From 2014-2015: a current student from a ‘pool’ of trained student reviewers (will be from outside of the Subject area)

• An Officer – normally from within Quality and Academic Partnerships

PSR Documentation

• Self-Evaluation Document (previously a Critical Appraisal) – a critical reflective document, identifying both good practice and areas to address

• Curriculum documentation – Programme Specifications, Module Specifications and Award Maps

• Subject Team CVs

• Library and Learning Services Supporting Statement

• BIMI data (e.g. enrolment, retention, achievement, NSS)

• Links to indicative reading lists, Programme and Module Guides on TALIS/NILE

• Marketing materials

• External Examiner reports

• Supporting EWO documentation (Memos of Co-operation)

Questions asked at PSR (some examples)

• Assessment strategies – sufficiently varied and innovative?

• Evidence of progression between academic levels (demonstrated by the Learning Outcomes)?

• What does trend analysis from the data such as enrolments and retention reports reveal?

• What is the strategic fit of the programmes?

• What is the strategic direction of travel?

• What is the KIS data telling potential students about the programmes

• Is there evidence of student feedback being addressed?

• How does staff research inform the curriculum?

Sources of Information

• Quality Assurance Agency website:

• www.qaa.ac.uk

• Quality and Academic Partnerships:

• Contact [email protected]

• Periodic Subject Review Handbook

Thank you; any questions?