Top Banner
Graduate Outcomes Measuring success and providing opportunity Welcome #GraduateOutcomes
180

:HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Jul 15, 2018

Download

Documents

hoanghanh
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate OutcomesMeasuring success and providing opportunityWelcome

#GraduateOutcomes

Page 2: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate Outcomes ConferenceAndy Youell, Director of Data Policy & Governance

Page 3: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

History

• 1972 – First Destinations Record (USR)

• 1984 – Exam Results and First Destinations Supplement (DfE)

• 1993 – First Destinations Supplement (HESA)

• 2002 – Destinations of Leavers from HE (HESA)

• 2017 – Graduate Outcomes

Page 4: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Increasing significance…

• Policy, funding and regulation

• Information for HE providers

• Information for prospective students

• Information for current students

• National statistics

• League tables

Page 5: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Reforming the HE data infrastructure

• Graduate Outcomes

• Data Futures

• Data Landscape Steering Group

Page 6: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Aims of today

• Where we are now

• HESA implementation of Graduate Outcomes

• Provider responsibilities and opportunities

• Different perspectives on Graduate Outcomes

• Looking to the future

• Networking and discussion

Page 7: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate OutcomesMeasuring success and providing opportunity

#GraduateOutcomes

Page 8: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

From DLHE to Graduate Outcomes

Dan Cook, Head of Data Policy & Development

Page 9: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 10: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 11: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

The remit

Page 12: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

WorkingStrategic

Review groups

Page 13: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Events

Page 14: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Research reports

Page 15: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Conference

Page 16: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Articles and blogs

Page 17: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

International interest and engagement

Page 18: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Two consultations

Page 19: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

With two rounds of feedback

Page 20: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Tackled difficult issues

Page 21: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Proposed novel solutions

Page 22: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Implementing the model

Page 23: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

How will it operate?

Page 24: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Implementation timeline

Page 25: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Materials currently available

• Information for students

• Guidance on roles and responsibilities of HE providers

• Promotional materials and brand identity

• Record specification for the collection of graduates’ contact details

• Steering Group in operation

• Data protection guidance

Page 26: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Progress (1)

• Survey has been cognitively tested with graduates and published

• Engagement strategy under development

• Running survey through Confirmit system

• Procuring a call centre and supplier to conduct coding

• Developing HESA systems for providers to:− Upload contact details− Personalise the survey (including provider logo)− Access survey data dashboards

Page 27: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Progress (2)

• Design and develop detailed plans for outputs

• Detailed analysis and development of approach to using linked data

• Recruitment to roles at HESA to support survey operations

Page 28: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate OutcomesMeasuring success and providing opportunity

#GraduateOutcomes

Page 29: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Standing up the capability

Doug Sparrow, Project Sponsor

Page 30: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Objectives

• High data quality

• Consistent quality

• Minimise risk

• Minimise complication for the sector and students

• Minimise the overall cost

Page 31: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Driving high quality

• Single integrated best of class survey platform

• Utilising a pre qualified and established framework for procuring the call centre

• Cognitive testing of the question banks

• Intended use of established and familiar SIC and SOC coding expertise

• Utilising the established HESA Liaison and Data Quality functions

Page 32: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Achieving consistent quality

• Split tasks to separate areas of expertise

• Each task applied evenly across all students

• Quality managed centrally

Page 33: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Minimising risk

• Contracted with industry leading companies

• Procuring the contact centre through an established framework

• Recruiting additional internal expertise

• Incorporating the expertise of ONS

Page 34: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Minimising complication

• Single and consistent point of contact for Higher Education Providers - HESA

• Development of a single portal

− Contact detail collection

− Personalisation

− Reporting

Page 35: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Minimising costs

• Procuring expertise not risk

• Structuring the collection and engagement to maximise online responses

• Building on, not replicating existing HESA capabilities

• Continuing to use the expertise throughout the sector to raise awareness

• Delaying the subscription point to 30 November

Page 36: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate OutcomesMeasuring success and providing opportunity

#GraduateOutcomes

Page 37: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Understanding the questionnaire and opt-in

question banksRachel Hewitt, Data Policy & Governance Manager

Page 38: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

How has the Graduate Outcomes survey been developed?

NewDLHE review – consultation one• What data should we collect?

NewDLHE review – consultation two• 84% support for survey design

Cognitive testing

Final version published• June 2018

Page 39: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Questionnaire: core

Page 40: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

‘Traditional’ employment

One or more jobs

Full time/part

time

SOC information Supervision

Contract type

Salary SIC information

Qualification requirement

Job choice Length of employment

Page 41: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

‘Non-traditional’ employment

Running your own business

Self-employment

Developing a creative/artistic/business portfolio

Page 42: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

‘Non-traditional’ employment

Full time/part time SOC information Supervision SIC information

Qualification requirement Job choice Length of

employment

Company funding• Running your own

business

Salary• Self employment

Page 43: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Study, training or research

Full time/part time Qualification type University/college name

Location of study

Page 44: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Early destinations

Number of previous jobs

Number and type of

previous study

Page 45: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Additional measures

Graduate Voice measures• On track• Meaningful• Skills

Subjective wellbeing measures

Page 46: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Questionnaire: opt-in

Page 47: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

What are opt-in question banks?

• Providers can ‘opt-in’ to having additional questions asked to core from a pre-defined set

• Additional cost – more information to follow on prices

• Some may be ‘opted-into’ by other bodies e.g. UKRI, NCTL

• Select opt-in banks through the provider portal

Page 48: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Opt-in banks

How did you find your job?

Net Promoter Score

Would you choose your

course again?

Would you like your careers

service to follow up?

Teacher training Research students

Page 49: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Cognitive testing

Page 50: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Why cognitively test?

To test:

• Understanding of the questions and corresponding response options and whether they are interpreted as intended

• Recall (of what they were doing in the census week) and the strategies they use to recall this information

• Judgement (exploring motivation, sensitivity and social desirability)

• Response (mapping the intended response with available options)

Page 51: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

• 95 cognitive interviews

• LocationsLondonManchesterGlasgowCardiffBelfast

• Type of provider

• Personal characteristics

• Subject areas

• 1094 cognitive interviews

• Top level views provided on experience of completing survey

Online surveyWorkshops/Skype

Cognitive testing

Page 52: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Positives

• Graduate voice measures

• Flow

• Representation of variety of outcomes

• Census week

• Length of survey (47 seconds longer than DLHE)

• Survey name

Page 53: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Lessons learned

• Clarified activities

• Further developed self-employment/running a business

• Handling of those developing a portfolio/working a portfolio career

• Order survey based on most important activity

• Tightening up wording throughout

Page 54: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Next steps

Page 55: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Beyond the 2017/18 survey

• Continue to refine the survey

• No changes between cohorts

• Seeking expertise on developing a creative/artistic/professional portfolio questions

• Building in provider questions

Page 56: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate OutcomesMeasuring success and providing opportunityCoffee break

#GraduateOutcomes

Page 57: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Promoting Graduate Outcomes: changes and challenges

Tammy Goldfeld, Head of the Careers Service

Dr Miriam Firth, Associate Director of Employability and Professional Learning

Page 58: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

The University of Manchester:A Case Study

• Largest single-site university in the UK• Biggest student community: 40,490 students• @25% of UGs are WP

Page 59: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 60: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Strategy: Key Performance Indicator

By 2020 to achieve a positive destinations rate of at least 85% (as measured six months after graduation in the Destinations of Leavers from Higher Education Survey), ensuring that the University is ranked in the upper quartile of Russell Group institutions on this measure.

Page 61: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Widening Participation (WP)

Achieve a year-on-year increase in the % of WP students in positive employment destinations

and

narrow any gap between WP and non-WP students in achieving successful employability outcomes.

Page 62: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Strategy

New: Graduate Outcomes Strategy Group, to replace the outgoing DLHE Strategy Group.

Consists of members from Careers, Planning, Alumni and Faculties

Additional GO Implementation Group, includes IT Services

Page 63: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Practical, operational changes

• Outsourced the final DLHE collection for the first time• New careers events for graduates (Leeds, Manchester,

London)• Updates to Data Protection statements

Page 64: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 65: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 66: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 67: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 68: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

New ambition and KPI Opportunity of better data and analytics – correlations with

demographics (eg WP); student engagement; career decision making; work experience, etc.

Closer relationships with new graduates

To conclude: Opportunities Ahead

Page 69: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 70: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate OutcomesMeasuring success and providing opportunity

#GraduateOutcomes

Page 71: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate Outcomes for Careers Services

Andrew Whitmore, Joint chair of AGCAS GEOD group, University of Manchester

Page 72: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate Outcomes, HEP responsibilities...

• Collect and maintain contact details• Map contacts with graduates…who?

• Return contact details• Inform students, graduates and staff• Data protection compliance• Data monitoringWho will have responsibility for driving this at your institution?

Page 73: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

The impact on Careers Services • Forging complementary relationships with colleagues, e.g.

Academic Planning and Alumni teams• Map contacts with graduates…who?

• Agreeing roles and responsibilities• Check HESA web pages for examples

• Developing new approaches to professional practice - E.g.• "Formal" post graduation support lengthening to15 months and

beyond • targeted support for second jobbers at 15 months and beyond

Page 74: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Working with Students and Graduates (1): Supporting students (Pre-exit)

• Informing them about the new survey• Encouraging them to update contact details• Encouraging them to complete the survey

Page 75: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Working with Students and Graduates (2): Supporting recent graduates

• A new kind of relationship• To beyond immediately after graduation e.g. at 15 months….• Incentivising engagement with Careers Services.

Page 76: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Working with Students and Graduates (3): Continuous communications & support – 15 months

• Post-PGT experience• Digital support• Access to vacancies• Physical support….

• access to guidance and training• access to CPD

Page 77: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Working with Colleagues (1): Informing them

• Senior Management• Planning/Alumni/Student Records• Academics• Data protection staff• Recruitment teams• Careers team

Page 78: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Working with Colleagues (2): Monitoring and Reporting

• HESA Dashboard• Monitoring responses• Marketing to non-respondents• Reporting within your institution

• Senior management• Planning teams• Marketing teams• Records team

Page 79: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Discussion

What are you doing/ planning to do….

• To inform students?• To engage, inform and support graduates?• To work with colleagues to ensure that accurate contact

information will be available?

Page 80: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Last thoughts…

• Check HESA pages• Get on the Graduate Outcomes Jiscmail group• AGCAS regional groups, check• Look out for messages from AGCAS GEOD (Graduate

Employability and Outcomes Data Group )

Page 81: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate OutcomesMeasuring success and providing opportunity

#GraduateOutcomes

Page 82: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Methodological Considerations for the Graduate Outcomes Survey

Salah Merad

Methodology, Office for National Statistics

Page 83: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Outline

• Introduction

• Survey-process overview: the GSBPM• Data Collection

• Non-response and weighting

• Summary

Aim:

to provide some insights into some survey-methodology areas and their implications

83

Page 84: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Introductions

• Me, and my role at ONS and in the GSS

• The Graduate Outcomes Steering Group:• Role

• Input on methods

• National Statistics and the Code of Practice

84

Page 85: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Context

• Comparison with other official statistics:

Similarities with social surveys• Individual responses

• Clustered by course, HE provider

• Non-response

Similarities with business surveys• Surveys of organisations (HE providers)

• range of sizes; variety; contacts

• A lot of auxiliary information available on the frame

85

Page 86: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

GSBPM

• The Generic Statistical Business Process Model• https://statswiki.unece.org/display/GSBPM/GSBPM+v5.0

• https://gss.civilservice.gov.uk/wp-content/uploads/2016/01/Generic-Statistical-Business-Process-Model.pdf

• Specify Needs >>> Design >>>

Build >>> Collect >>> Process

>>> Analyse >>> Disseminate

>>> Evaluate >>>

• Covers entire survey process86

Page 87: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

GSBPM

Methodological areas include:

• sample design

• data collection

• data cleaning (editing, coding, imputation)

• weighting and estimation

• analysis methods, e.g. seasonal adjustment

• disclosure control

• dissemination

... we’ll look at two areas:

data collection and non-response 87

Page 88: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Data Collection

Changes in data collection method and mode:

• Self-administered, web-collection

• Interviewer-administered, phone interview

… leading to potentially different: responses, response levels and patterns, quality of data,

opportunities, help/support available, …, cost

88

Page 89: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Data Collection

Also different questions/topics, questionnaire layout, wording, ordering, routing, approach to coding (classifications).

But not starting afresh.

Lots of experience with DLHE and good practice.

Plus cognitive testing (what works, where are there problems, respondents’ interpretation of questions).

89

Page 90: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Data Collection

Also need to gain respondents’ co-operation.

Traditionally, part of interviewers’ role.

Wording, invitation to respond, advance communication.

Incentives (for respondents, for providers?)

… quality and cost-effectiveness.

90

Page 91: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Non-response

Not everyone responds.

Causes problems:

• smaller samples for analysis

• unrepresentative samples … non-response bias

NR-bias occurs if responses vary between different groups, and the propensity to respond is associated with the response itself.

91

Page 92: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Non-response

Types of missingness (assumptions required)

• Missing Completely at Random … no risk of bias, but unlikely to be realistic.

• Missing at Random … propensity to respond related to other, known characteristics (e.g. sex, age, location), but not the response itself.

Usual assumption in official statistics. Common practice is to impute for item non-response, and (re-)weight for unit non-response.

• Not Missing at Random … no obvious way to proceed.

92

Page 93: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Non-response

Greater risk of NR-bias as the response rate falls.

Some examples:

• Population Census (mandatory). 94% overall, > 80% in every local authority.

• Business surveys (mandatory). Still usually achieve 70-75% by number (and more by size).

• Social surveys (voluntary). Rates are falling, often now 50-60%.

93

Page 94: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

94

0

10

20

30

40

50

60

70

80

90S

N97

SN

98

SN

99

SN

00

SN

01

SN

02

SN

03

SN

04

SN

05

JS06

JS07

JS08

JS09

JS10

JS11

JS12

JS13

JS14

JS15

JS16

JS17

Ho

use

hold

resp

ons

e ra

te (

%),

exc

l. im

put

ed

LFS wave-specific response rates, GB 1997-2018

Wave 1 Wave 2 Wave 3 Wave 4 Wave 5 Total

Page 95: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Non-response

Graduate Outcomes – reference date moving from 6 months to 15 months:

• more time to lose contact

• more time for survey to become less salient

General decline in survey response.

Response-rate targets are used, and need to be practical (will need to be set lower than for DLHE).

95

Page 96: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Non-response

Weighting will be introduced; responding sample made to represent both themselves and the non-responders:

• explicit non-response model?

• calibration to known population totals?

• alternatively, could consider unit imputation (would give complete dataset and help with small groups)?

Huge source of information available for the non-responders (compare with social surveys!), including linked administrative data, giving excellent scope for effective non-response weighting.

96

Page 97: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Non-response

Output statistics will come from use of the weights.

Result will be each responder has a weight, w >= 1; interpret was number of graduates represented.

(Note that GOS is still a census, with high response expected).

Sum of w will be a meaningful number.

Work now to determine the weighting classes/models,

and any measures of accuracy. 97

Page 98: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Conclusions

Graduate Outcomes seems in a good place, methodologically.

A big and rich data source, including for the non-responders.

New and changed methods ahead, which will bring some discontinuities (improvements).

Being well-tested, and well-managed by the steering group.

98

Page 99: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate OutcomesMeasuring success and providing opportunityLunch

#GraduateOutcomes

Page 100: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Improving graduate earnings measures using

LEO data

Alison Judd

Higher Education Analysis

Page 101: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

What is LEO data?The Longitudinal Education Outcomes database (LEO) brings together information on education, labour market, and benefits into a single, secure, linked database.

101

Childcare & Early Years data

School records

Further Education records

Higher Education records

Benefits dataEmployment –P14 & P45 records

Page 102: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

What does LEO bring to HE data?• LEO brings together information on education, labour

market, and benefits into a single linked database.

• Changes current reliance on large scale surveys of graduates– Labour Force Survey– Destinations of Leavers from Higher Education (DLHE) – 6 months & 40 months

• Providing much better data on employment outomes:– More reliable earnings and employment information. The DLHE survey contains

earnings data for approx. 60% of HE leavers and LDLHE just under one-third. – Scale means able to look at smaller course and population groups– Longitudinal – beyond the current 3.5 years of DLHE– More information on prior attainment and background characteristics

Page 103: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Helping improve decision making• Improving student/ parent decision making:

– E.g. Unistats and other information sources– Teaching Excellence Framework.

• Improving policy making– Ensuring system delivers for students– Understanding vfm of different public investments– Supporting social mobility

• Helping providers understand student outcomes– What works?

• Supporting academic research and analysis

Page 104: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

But its not perfect…

104

• Information on where an individual works is not currently available. Geography

• Information on hours worked is not available so it is not possible to identify if an individual works full or part time.

Hours worked

• The database does not include information on what job an individual holds. We are exploring whether we can obtain information on the industry an individual works in (SIC code).

Occupation

• It doesn’t contain everything we would like to know

As with other sources: • Gross outcomes can’t be interpreted as causal• It only covers labour market outcomes• Inevitably, it is backwards looking.

Page 105: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

• 3 year development programme, following SBEE Act

• Initial set of six ‘experimental’ publications to broaden understanding and get user feedback

• Note: Experimental does not mean poor quality!

• Now includes self-employment, Further Education Colleges and International Students

• Working with information providers to see how it can be used alongside other data to improve student choice

• Included as a supplementary metric within TEF

How we are taking it forward

Page 106: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

So what does it show…?

106

Employment outcomes of graduates one, three, five and ten years after graduation Coverage: UK domiciled male and female first degree graduates from English HEIs and FECsCohorts: 2004/05 (10 years after graduation), 2009/10 (5 years), 2011/12 (3 years), 2013/14 (1 year)Tax year: 2015/16 Source: https://www.gov.uk/government/statistics/graduate-outcomes-2015-to-2016

Page 107: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Employment by subject

107Proportion in sustained employment, further study or both by subject five years after graduationCoverage: UK domiciled male and female first degree graduates from English HEIs Cohorts: 2008/09 (5 years), Tax year: 2014/15. Source: https://www.gov.uk/government/statistics/graduate-outcomes-by-degree-subject-and-university

Outcomes similar, with small impact from incl. self-employment.

Page 108: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Earnings after graduation

108Earnings of graduates by sex one, three, five and ten years after graduationCoverage: UK domiciled male and female first degree graduates from English HEIs and FECsCohorts: 2004/05 (10 years after graduation), 2009/10 (5 years), 2011/12 (3 years), 2013/14 (1 year)Tax year: 2015/16. Source: https://www.gov.uk/government/statistics/graduate-outcomes-2015-to-2016

Page 109: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Earnings outcomes by subject (f)

109Earnings of graduates by subject five years after graduationCoverage: UK domiciled female first degree graduates from English HEIs and FECsCohorts: 2009/10 (5 years), Tax year: 2015/16. Source: https://www.gov.uk/government/statistics/graduate-outcomes-2015-to-2016

• Significant variation between and within different subjects.

Page 110: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

110Earnings of graduates by subject five years after graduationCoverage: UK domiciled male first degree graduates from English HEIs and FECsCohorts: 2009/10 (5 years), Tax year: 2015/16. Source: https://www.gov.uk/government/statistics/graduate-outcomes-2015-to-2016

Earnings outcomes by subject (m)• Similar pattern for men, but greater within subject range.

Page 111: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Including self-employment data again makes little different to subject level comparisons

111Earnings by subject five years after graduationCoverage: UK domiciled male and female first degree graduates from English HEIs Cohorts: 2008/09 (5 years), Tax year: 2014/15. Source: https://www.gov.uk/government/statistics/graduate-outcomes-leo-including-self-employment-earnings-data

Page 112: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Course level outcomes (1)

112

We see significant variation across institutions for each subject e.g. yellow dots highlight all Business & Admin

Page 113: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

113

Course level outcomes (2)We see significant variation across subjects for each institution e.g. yellow dots look at Southampton courses

Page 114: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Next Steps

114

Next publicationo 21st June 2018 : Update institution by subject data with the latest tax year.

Commissioned research to control for influencing factors

Working with information providerso Unistatso Which?o Open Data Challenge

Looking at how to enable wider (secure) access for research

Page 115: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Summary

115

• An important asset in understanding students’ labour market outcomes: a more accurate, representative and granular picture than ever before.

• Help understand variation by subject, institution and student characteristic.

• Aim to support students (and their parents), policy makers and institutions in making better decisions.

• But needs to be used carefully, in context and recognising wider influences on student decision making and value of Higher Education.

• As well as further developing the data, keen to work with students, providers, researchers and information providers to maximise its usefulness.

Page 116: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate OutcomesMeasuring success and providing opportunity

#GraduateOutcomes

Page 117: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 118: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 119: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Provider Context

Page 120: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Government Context

Page 121: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate Context

Page 122: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Impact

Page 123: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Making this work

Page 124: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Use of graduate data

• Alumni Office• Faculty• Marketing• Data analysis/Information provision /Research• Data linking• Government tracking

Page 125: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Data protection principles

• Fair, lawful and transparent• Specified, explicit purposes• Adequate, relevant and limited to only what is necessary• Accurate and, where necessary, kept up to date• Kept for no longer than is necessary• Secure

Page 126: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Managing transition /challenges

• Impact of the new survey on the results• Using all of the survey • Open data

Page 127: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Opportunity

• Success determined by the graduate

• Improved use of data to inform student choices

• Greater join up between Faculty and professional services

• Visible benefits to students before they graduate

Page 128: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 129: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 130: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate OutcomesMeasuring success and providing opportunity

#GraduateOutcomes

Page 131: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Visualising Destinations and OutcomesGraduate Outcomes: Measuring success and providing opportunity – 13 and 14 June 2018

Page 132: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Janette Hillicks – Senior Co-Design Manager, Jisc

Presenters

Rhodri Rowlands - Senior Data Visualisation Officer, Jisc

Page 133: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

1. Quick overview of Analytics Labs and Community Dashboards2. Dashboard release – what’s available?3. Graduate Outcomes Dashboard demonstration4. What’s next?

Content

Page 134: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

What isAnalytics Labs?

Page 135: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Analytics Labs is a unique Continuing Professional Development opportunity for participants from across the UK HE sector offered in a supported data processing environment. Teams with a range of expertise in data and visualisation work together

with sector colleagues with an in-depth knowledge of the policy context. This Agile collaboration rapidly results in the creation of Community

Dashboards by the sector for the sector... 246 participants from 95 UK Universities so far…

What is Analytics Labs?

Page 136: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 137: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Analytics Labs - The Approach

Page 138: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 139: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Makeup of a team

Product Owner

Brings an understanding of the policy context and the needs of

users

Analysts

Expertise in data and analysis,

especially from a HEI perspective

Scrum Master

Keeps the project on track

and removes impediments to

progress

Data & Viz Support

Supports the team with specialist

knowledge in tools such as Alteryx

and Tableau

Meta Product Owner

Provides expertise and guidance in the

specific theme

Page 140: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Widening Participation (Inclusion)

As a WP Practitioner/School Outreach Activity Officer When deciding to allocate resources on outreach activity I want to Identify areas/schools in which to concentrate outreach activities So I can ultimately, increase recruitment from selected areas/schools and improve social mobility

Page 141: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

User Story Category Heat MapStudent Journey Widening

ParticipationBenchmarking Equality Destinations

Post Graduate Library Teaching Excellence and Student Outcomes Framework (TEF)

Marketing Research Assessment Exercise (RAE)

Student Experience

Brexit Estates Accommodation Postgraduate Teaching (PGT)

Postgraduate Research (PGR)

Course Development

Retention Course Management

Course Articulation

Library Usage Quality Assurance Value Added Course Offering Planning

Page 142: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

A digital badge records participation against 5 competencies: Participating in Agile development Visualising data Transforming data Digital collaboration Understanding policy and the data

landscape

Analytics Labs - Digital Badge

Page 143: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Community Dashboards – by the sector for the sector

Page 144: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Community Dashboards – can be explored in Heidi Plushttps://www.hesa.ac.uk/services/heidi-plus

Page 145: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

ID Name Release Date

1 Athena Swan & Race Equality Dashboard Dec 162 Destination of Leavers by Activity Dec 163 Destination of Leavers Explorer Dec 164 HE-BCI Part B Explorer Dec 165 University Research Benchmarking Feb 186 Finding Comparable Providers Sep 177 League Table Dashboard Jan 189 School Finder Sep 1711 Single HEI Comparison by FTE Sep 1712 Destination Flow Sep 1722 A-Level Subjects Feb 1825 Costs vs Staff Correlation Feb 1829 Brexit Implications on Research Jan 1831 Estates Sector Benchmarks Jan 1839 Provider Healthcheck Apr 18

Community Dashboard portfolio

Page 146: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Community Dashboard BetasID Name Release Date

1 Age & Workforce Planning Jan 182 Destinations Analysis Jan 183 Financial Indicators Jan 184 Future Course Explorer Jan 185 Market Insight Jan 186 TEF Exploratory Dashboard Jan 187 TEF Metrics Core and Split Metrics Jan 18

Page 147: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Dashboard Demos

Page 148: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 149: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 150: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 151: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 152: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 153: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 154: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 155: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 156: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 157: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

What’s next?

Page 158: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Developing dashboard suites to address: Course Market Research

KS5 subject analysis Undergraduate course provision Student destinations Industry and workforce analysis

Staff Metrics Recruitment, retention and progression Sickness and absence Workforce planning (including ageing workforce) Staff demographics

Special Projects Team - April to August 2018

Page 159: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

About Heidi Plus [email protected]

About Community Dashboards - try them now https://www.jisc.ac.uk/rd/get-involved/try-out-our-community-dashboards

About Analytics Labs - https://www.jisc.ac.uk/rd/projects/business-intelligence-project

Keep in touch: Join our list - www.jiscmail.ac.uk/JISC-HESA-BUSINESS-INTEL Follow our blog - https://businessintelligence.jiscinvolve.org/wp/ Drop us a line at [email protected] entering ‘Analytics Labs' in the subject line

Above details are all available via - https://tinyurl.com/Jisc-BI-Project

Find out more...

Page 160: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate OutcomesMeasuring success and providing opportunity

#GraduateOutcomes

Page 161: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Powering your Voice of the Customer

and Market Research

Programmes

Page 162: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Copyright © 2016 Confirmit. All Rights Reserved. Confirmit Confidential. |162

Who We Are

Offices

Partners

surveys sent in 201712 million page views per day

1.2bnunique usersaccessed report dashboards in 2017

250Kuptimein 2017

99.99%

staff worldwide

450+yearfounded

1996clients in over 100 countries

800+

Page 163: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Copyright © 2016 Confirmit |

Who We Are

163

B2B/High Tech Consumer Electronics

Consumer Services Market Research Financial Services

Pharmaceuticals & Healthcare Leisure & RetailConsumer

Products Government Conglomerates & Consulting

Page 164: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

164

Customer Experience – Employee Engagement – Market Research

Technology with CX Consulting and Services

Empower our Clients

Our Heritage

Copyright © 2018 Confirmit. All Rights Reserved. Confirmit Confidential. |

Page 165: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

165

Our heritage has a lot to do with our solutions and how we work with our clients.

Our Collaborative Heritage

Copyright © 2018 Confirmit. All Rights Reserved. Confirmit Confidential. |

Page 166: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Copyright © 2017 Confirmit. All Rights Reserved. Confirmit Confidential. |

Confirmit Horizons

166

Feature rich, single platform solution

Multi-channel solutions reach audiences effectively

Sophisticated reporting/alerting deliver actionable insight

Flexible and scalable SaaS meets changing business needs

Reliable and secure software providing complete peace of mind

Facilitates process automation reducing costs and increasing productivity

Provides high-level of accuracy to help you identify investment areas

Page 167: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Copyright © 2016 Confirmit |

• Time is short due to busy lives• Surveys fit in around their

lifestyle• Portable rather than fixed

devices• Ever changing mobile and email

addresses• Adopt a stop/start approach to

tasks• Never far from their phones

Challenges of Surveying Graduates

167

Page 168: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Copyright © 2016 Confirmit |168

Reaching GraduatesSolutions That Suit The Graduate

Page 169: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Copyright © 2016 Confirmit |169

Reaching GraduatesSolutions That Suit The Graduate

Page 170: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Copyright © 2016 Confirmit |

Mix Mode Challenges

• Over contact• Continuing to request a response from a graduate that has completed

a survey• Updated contact details• Partially completed surveys• Transition from one mode to another• Response rates

170

Page 171: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Copyright © 2016 Confirmit |

Confirmit Solution

Single contact record• Drives all communication• Determines the frequency of contact• Simplifies update to details• Automatically removed from all contact once completed survey• Easy to remove duplicates

171

Page 172: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Copyright © 2016 Confirmit |

• These are your graduates and it is important:• Not to over contact• Act upon do not contact

requests• Protect their data

• It is your reputation that HESA is upholding

Managing Communication

172

Page 173: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Copyright © 2016 Confirmit |173

Dual branding of surveys and communications to improve responses

Use best practice to enhance Providers reputation

Class leading technology delivers a seamless experience

Consistent survey and approach ensures a fair and equal comparison between Providers

No Gaming of system!

Results

Page 174: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Copyright © 2016 Confirmit |174

Thank You

Page 175: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Graduate OutcomesMeasuring success and providing opportunity

#GraduateOutcomes

Page 176: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Closing remarks

Page 177: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

History

• 1972 – First Destinations Record (USR)

• 1984 – Exam Results and First Destinations Supplement (DfE)

• 1993 – First Destinations Supplement (HESA)

• 2002 – Destinations of Leavers from HE (HESA)

• 2017 – Graduate Outcomes

Page 178: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 179: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv
Page 180: :HOFRPH - hesa.ac.uk · ,qfuhdvlqj vljqlilfdqfh« 3rolf\ ixqglqj dqg uhjxodwlrq ,qirupdwlrq iru +( surylghuv ,qirupdwlrq iru survshfwlyh vwxghqwv ,qirupdwlrq iru fxuuhqw vwxghqwv

Closing remarks