Using research to get the best value from the Pupil Premium

Using research to get the best value from the Pupil

PremiumSteve Higgins,

School of Education, Durham [email protected]

@stig_01

National Middle Schools’ Forum Conference 2013

21st October 2013Stratford Manor Hotel, Stratford on Avon

Sutton Trust/EEF Teaching and Learning Toolkit

Why we wrote it Best ‘buys’ on average Key messages for spending the Pupil Premium Currently used by about a third of schools

http://educationendowmentfoundation.org.uk/toolkit

The pupil premium Aims:

to close the attainment ‘gap’ between the highest and lowest achieving

to increase social mobility to enable more pupils from disadvantaged backgrounds

to get to the top Universities to provide additional resource to schools to do this

Was £600 in 2012-13 for fsm1 pupils; increased to £900 in 2013-14 and £1300 in 2014-15.

1 any child registered for fsm in the last six years and all looked after children, smaller premium for children of Service families

Resources and learning

Above a minimum threshold – no simple link Conclusion: spending more won’t guarantee

improvement - no simple solution

More money ≠ more learning

Smaller classes? Complex evidence- no clear link with class size and

achievement Experimental trials suggest

Classes need to be less than about 17 for 0.2 effect size…And teachers need to change the way they teach…But support from teaching assistants not as effective

The maths:£900 x 25 pupils x 3 classes with 50% on fsm = £33,750= 1 extra teacher per 3 classesClass size reduction from 25 to 19 – expensive for little gain

One-to-one tuition Highly effective

I hour/ day over at least 6 weeksSupport for class teacher to re-integrateEffect size 0.44

The maths…6 weeks x 5 days x 1 hour = 30 hours4 days teacher time (more effective with an experienced teacher)Approx £700 (ECC models less costly)

Expensive but very effectiveConsider using pairs or triplets?

What should the Pupil Premium ‘buy’? Secondary £5,200 per pupil Primary £4,284 Wide variation

Secondaries £4,000 to £9,000 Primaries £3,000 and £8,000 Middle Schools £3,300 - £8,000 (median £4,100) Excludes Academies and Free Schools

Will £1,300 buy an extra three or four months learning for each pupil eligible for the Pupil Premium?

(In England, data from 2009-10: https://www.gov.uk/government/publications/funding-for-primary-and-service-childrens-education-schools )

The Bananarama Principle It ain’t what you do it’s the way that you do it…

So how do you spend to “get results”? Or, what does the evidence say is a good

investment or a poor investment for learning? It ain’t what you spend it’s the way that you

spend it…

What we tried to do Summarise the evidence from meta-analysis about the

impact of different strategies on learning (tested attainment) As found in research studies These are averages

Apply quality criteria to evaluations: rigorous designs only

Estimate the size of the effect Standardised Mean Difference = ‘Months of gain’

Estimate the costs of adopting Information not always available

Toolkit

Best ‘buys’...

New entry

Toolkit

Good ‘buys’...

New entry

Toolkit

Low range ...

Worst ‘buys’... (on average)

Toolkit

New entry

Revised entry

SummariesWhat is it?How effective is it?How secure is the evidence?What are the costs?What do I need to know?

The methodology Systematic searches for meta-analyses of

interventions and quantitative estimates of impact Effect sizes aggregated (fixed effect model) Converted to months progress Moderator variables summarised

Costs estimated in terms of outlay for schools Variation within toolkit themes greater than

variation between them: it ain’t what you do… Technical appendix available on EEF website

Toolkit

Feed

back

Met

a-co

gniti

on a

nd s

elf-r

egul

atio

nPe

er tu

torin

gEa

rly y

ears

inte

rven

tion

Hom

ewor

k (S

econ

dary

)O

ne to

one

tuiti

onCo

llabo

rativ

e le

arni

ngM

aste

ry le

arni

ngPh

onic

sSm

all g

roup

tuiti

onBe

havi

our i

nter

vent

ions

Digi

tal t

echn

olog

y

Soci

al a

nd e

mot

iona

l asp

ects

of l

earn

ing

Pare

ntal

invo

lvem

ent

Out

door

adv

entu

re le

arni

ngRe

duci

ng c

lass

siz

eSu

mm

er s

choo

lsSp

orts

par

ticip

atio

nAr

ts p

artic

ipat

ion

Lear

ning

sty

les

Exte

nded

sch

ool t

ime

Afte

r sch

ool p

rogr

amm

esIn

divi

dual

ised

inst

ruct

ion

Hom

ewor

k (P

rimar

y)M

ento

ring

Aspi

ratio

n in

terv

entio

nsBl

ock

sche

dulin

gPe

rform

ance

pay

Phys

ical

env

ironm

ent

Scho

ol u

nifo

rmTe

achi

ng a

ssis

tant

sAb

ility

gro

upin

gRe

peat

ing

a ye

ar

-0.4

-0.2

0

0.2

0.4

0.6

0.8

Average Effects

Approaches

Effe

ct s

ize

Homework (Secondary)

Overview of value for money

Cost per pupil

Effe

ct S

ize

(pot

entia

l mon

ths

gain

)

£00

1.0

£1000

Meta-cognition

Peer tutoring

EY intervention

1-1 tutoring

Digital technology

Parental involvement

Summer schools

After schoolIndividualised

learning

Performance pay

Teaching assistants

Smaller classes

Ability grouping

Promising

Could be worth it

Needs careful thought

Feedback

Phonics

Key messages Some things that are popular or widely thought

to be effective are hard to make work well in terms of tested attainment

Ability grouping (setting); After-school clubs; Deployment of teaching assistants; Performance pay

Some things look more ‘promising’Effective feedback; Meta-cognition and self regulation strategies; Early years intervention; Peer tutoring; Small group/intensive tuition; Parental involvement and engagement

Issues and limitations Based on meta-analysis – averages of averages Conversion to ‘months progress’ is a rough

estimate Intervention research is compared with ‘normal’

practice which is varied Not ‘what works’ but what has worked – ‘good

bets’ to support professional enquiry

The challenges (1): implementation These strategies have been shown to be cost-effective

in research studies But when we have tried to implement evidence-based

strategies we have not seen system-wide improvement (e.g. AfL, SureStart)

We don’t know how to get schools/teachers who are not currently doing them to do so in ways that are True to the key principles Feasible in real classrooms – with all their constraints Scalable and replicable Sustainable

The challenges (2) : context

This is what has worked (on average) Where is there leverage for improvement in a

school or class? Will it build capacity? (i.e. is it efficient?)

For learners? For teachers?

Is it inclusive?

For disadvantaged /struggling learners… One ‘intervention’ won’t be enough

Identify areas of greatest needClear focus on improving learning, not (just) behaviourMid and high attaining learners can be disadvantaged too!

Effects will need to be cumulativeWhat will build learning capacity and capability?Need to track and evaluate – our best guesses are not always good enough

Evaluate impact

EEF’s DIY Evaluation Guide: http://educationendowmentfoundation.org.uk/library/diy-evaluation-guide

Establishing a baseline Teacher assessment vs standardised tests? Individual progress vs comparison group? Test score increases or age-standarised progress?

Evidence suggests Teacher Assessment tends to over-estimate progress BUT reliance on standardised tests narrows the curriculum and encourages test preparation

FeedbackMeta-cognitionSelf regulationPeer tutoring

Small group tuitionPhonicsTA support(Parent involvement)(Early years intervention)

One-to-oneSEALBehaviour

But remember….“It ain’t what you do, it’s the way that you do it… that’s what gets results”

Effective identificationAdopting solutions, not bandwagonsHigh quality intervention/supportEvaluate to ensure learning progressQuality counts

Some LinksThe full report can be found on the EEF’s website: http://educationendowmentfoundation.org.uk/toolkit/ The toolkit is recommended by the Department for Education: http://www.education.gov.uk/schools/pupilsupport/premium/b00200492/ppstrategies Official information about the Pupil Premium and LA allocations is available at: http://www.education.gov.uk/schools/adminandfinance/financialmanagement/schoolsrevenuefunding/a00200697/pupil-premium-2012-13 Ofsted’s report is available at: http://www.ofsted.gov.uk/resources/pupil-premium Find out how much each school gets: http://educationendowmentfoundation.org.uk/toolkit/pupil-premium-calculator DIY Evaluation Guide: http://educationendowmentfoundation.org.uk/library/diy-evaluation-guide

Questions…

Using research to get the best value from the Pupil

Premium: workshop session Steve Higgins,

School of Education, Durham [email protected]

@stig_01

National Middle Schools’ Forum Conference 2013

21st October 2013Stratford Manor Hotel, Stratford on Avon

‘Best value’ Identify target group Improve effectiveness of provision/support Evaluate impact Track target pupils Adjust plan in response to tracking Use funding to achieve wider outcomes?

Low cost

High Impact

Low Impact

High cost

Preparing for Ofsted Communicate your strategy

Teachers Governors Website

Track target pupils Record data on spending Evaluate accurately Improve plan Justify/argue!

Tracker linksOfsted resources: http://www.ofsted.gov.uk/resources/pupil-premium-analysis-and-challenge-tools-for-schools Leading Learner blog:http://leadinglearner.me/2013/10/10/pupil-premium-analyser-and-tracker/ NAHT: http://www.naht.org.uk/welcome/advice/advice-home/governance-and-infrastructure-advice/pupil-premium-reporting-2012-2013/ TES: http://www.tes.co.uk/teaching-resource/Track-Impact-and-Spend-of-Pupil-Premium-6121277/ Deepings School (CfBT Academy): http://www.deepingschool.org.uk/162/pupil-premium http://www.deepingschool.org.uk/uploads/asset_file/How%20are%20the%20Pupil%20Premium%20pupils%20doing.pdf

For every complex problem there is a solution that is simple, neat…

and WRONG!

H.L. Mencken 1880-1956