Quality-adjusted Output Measure for the Scottish Education System Richard Murray Economic Adviser Scottish Executive.

Quality-adjusted Output Measure for the Scottish Education System

Richard MurrayEconomic Adviser

Scottish Executive

Structure of Presentation

Section 1: Background– Structure of the Scottish Education System– Context for the work

Section 2: Quality-adjusted Output Index Section 3: Concluding Remarks

– Limitations of the New Measure– Summary

Section 1: Background

Structure of Scottish Education System

Different education system in Scotland compared with the rest of the UK

Primary education:– 7 years of education (P1 to P7)– 2,184 schools & 382,783 pupils

Secondary education:– 6 years of education (S1 to S6)– Compulsory to S4– 381 schools & 312,979 pupils

Structure of Scottish Education System

Comprehensive system Examinations sat in final three years of

secondary education (S4, S5 & S6) No national testing before S4

Context to this Work

Atkinson Review– Specific recommendations for education

Improve the way we record inputsMove towards an output-based approachDevelop measures to track changes in quality

of outputExtend coverage across whole of UK

Section 2: Quality-adjusted Output Index for Education

Rationale for Adjusting for Quality

Why adjust for quality?– To capture improvements in the education

pupils receive

How do you measure quality?– Ideally, the quality of teaching pupils receive– Difficult to measure this, though attainment

provides a useful proxy

Differentiating Output by Attainment

Aim to capture wide spectrum of attainment a pupil receives by end of secondary education

3 categories which capture all levels of attainment at school

Category 1 ISCED 3

Category 2 Between ISCED 2 & 3

Category 3 ISCED 2

Proportion of Pupils Gaining Qualifications

0.00

0.10

0.20

0.30

0.40

0.50

0.60

1997 1998 1999 2000 2001 2002 2003 2004

Pro

po

rtio

n o

f P

up

ils

Ga

inin

g Q

ua

lifi

ca

tio

ns

Category 1 Category 2 Category 3

Weighting the Different Levels of Attainment

Each category of attainment needs to be given a weight, relative to its contribution to total output

Inappropriate to use cost weights Expected future earnings provides a proxy

for the benefit pupils receive from education

Expected Future Earnings by Highest Qualification

Results show that higher qualifications lead to higher expected future earnings

Weightings of 48: 32: 20 for Categories 1: 2: 3

Expected future earnings =

Average earnings by highest qualification

Employment rate by highest qualification

X

Quality-adjusted Output Index

Methodology

Step 1: Differentiate S6 pupils by attainment

Step 2: Apply a weight (based on expected future earnings) to

the attainment categories

Step 3: Calculate the weighted total & convert to index

Quality-adjusted Output Index for Final Year of Secondary Education

Number of Pupils

2000 2001 2002 2003 2004

Category 1 18,486 18,583 18,479 18,494 18,315

Category 2 10,191 11,005 11,100 11,414 11,666

Category 3 28,203 28,024 28,079 28,870 28,531

Total Pupils in Cohort

59,250 60,138 59,998 61,035 60,447

Weighted Total

1,777,500 1,804,621 1,803,780 1,830,318 1,823,082

Output Index 98.5 100.0 100.0 101.4 101.0

Quality-adjusted Output for Education

Only measures the output of one year group (i.e. S6)

Apply the differentiated output across all pupils in primary & secondary education

This assumes quality will be the same for each year group

Quality-adjusted Output for Scottish Education System

Number of Pupils

2000 2001 2002 2003 2004

Category 1 213,402 210,277 207,859 204,239 201,522

Category 2 117,645 124,533 124,850 126,048 128,362

Category 3 325,575 317,116 315,838 318,829 313,922

Total Pupils in Cohort

683,980 680,507 674,867 674,056 665,090

Weighted Total (000s)

20,519 20,421 20,289 20,214 20,059

Output Index 100.5 100.0 99.4 99.0 98.2

What is driving the New Quality Adjusted Output Index?

93

94

95

96

97

98

99

100

101

102

103

104

1997 1998 1999 2000 2001 2002 2003 2004

Quantity Quality Quality-adjusted Output

Pupils Gaining No Qualifications

Believe this group should not have a positive impact on the quality dimension of output

Still captured in the quantity dimension of the quality-adjusted output

Sensitivity Analysis

Examined different combinations of qualifications

Applied different weights by attributing part of the future earnings associated with those who go on to university

Overall, little impact on quality-adjusted output index

Section 3: Concluding Remarks

Limitations of New Measure

Unable to track changes in quality across each year group

Will take a number of years before investment influences the measure

Wider benefits of education are excluded Possibly not giving enough weight to high

attainment category

Summary

New Quality-adjusted output measure for Scottish Education system

Output is differentiated by attainment and weighted by expected future earnings

Output index has declined in recent years, driven by fall in pupil numbers