Integrity of education systems – a framework for assessment

INTEGRITY OF EDUCATION SYSTEMS (INTES): A framework for assessment

Mihaylo Milovanovitch, Directorate for Education, OECD

Presentation outline

The task at hand

The conceptual framework

Implementation and outcomes

How to go for it?

EDUCATION CORRUPTION

Occurrence

Analysis and policy reaction?

Criminalise & punish

Prohibit & enforce

Teach & mobilise

Reactive/punitive:Focus on cases

Preventative: Focus on

opportunityPreventative:Focus on origin

Milovanovitch

?

Milovanovitch

Act

ive c

orru

ptio

n

Pass

ive c

orru

ptio

n

why?

why?

LIMITATIONS: The “vicious circle” – corruption occurrence and impact

Phase I: Causality links

Level 1: System

shortcomings

Level 3: Corruption occurrence

Phase II: Measuring

impactLevel 2: Preventive framework

How to go for it? INTES demand-focused approach

Lets assume: good education systems have no corruption

How to go for it? INTES demand-focused approach

But:What is a good system?

• Quality of outcomes• Equity of access and success• Teachers and professors• Management

• Prevention and detection

Link of demand and corruption: qualityMean reading score in PISA 2009 adjusted for country's socio-economic profile and the Global Corruption

Barometer (extent do you perceive the education system in this country to be affected by corruption)

Source: INTES Note: Excluding South Korea and Japan

1.52.02.53.03.54.0350

400

450

500

550

UK

f(x) = − 32.3864520104333 x + 564.313301709915R² = 0.217359784415101

Global Corruption Barometer/ Transparency International : To what extent do you perceive the the ed-ucation system in this country to be affected by corruption? (1- 5 strongest)

Mea

n re

adin

g sc

ore

in P

ISA

200

9 ad

just

ed fo

r cou

ntri

es' s

ocio

-ec

onom

ic p

rofil

e

OECD average

Link of staff policy and c.: teachersShare of all those in top quarter of PISA reading performance who want to become teachers and the Global

Corruption Barometer (extent do you perceive the education system in this country to be affected by corruption)

1.52.02.53.03.54.0-0.3

0.2

0.7

1.2

1.7

2.2

1.0282973374398

0.8310311962103650.8185890257558790.8144485206935520.747965853007871

0.897549353301566

1.14964349842233

1.0328222685323

1.22172458450248

1.39438153310105

0.474238114587566

0.8659188034188050.953139228093116

0.540419430005378

1.79353280059468

1.1933095197596

0.676928584213353

1.564796132828921.53509062776304

1.014452674105431.12960036395853

1.447416947998521.34046345811052

1.10806613053533

0.737347356532789

1.341449047712551.36523762047192

1.23497365900383

1.74801870539011

1.33881558983821

2.11023321897634

1.11881188118812

0.869971450817546

1.7588338548424

1.387772694118431.32263101767215

1.178016853039381.058685911416

1.44636564394117

1.60767565896266

1.01055881107514

f(x) = − 0.271070169185046 x + 1.91875719231091R² = 0.12032153596668

Global Corruption Barometer/ Transparency International : To what extent do you perceive the the ed-ucation system in this country to be affected by corruption? (1- 5 strongest)

Rati

o Sh

are

of a

ll th

at w

ant t

o be

com

e te

ache

rs/t

hose

in to

p qu

arte

r of

read

ing

perf

orm

ance

How to go for it? INTES demand-focused approach

Integrity:

The INTES assessments consider integrity in education to be the consistent application of such actions, values, methods and principles which lead to:

• Fair access to education; • Better  quality of education;• Fair and professional treatment of staff, and sound

management;• Successful prevention and detection of

malpractice/corruption

Origins of corruption demand:

Corruption addresses a persisting need of actors and stakeholders in education for a specific education service, which the system is not or is not properly providing. It is caused by this need, by an opportunity situation, or by both.

Systems in which stakeholders feel that one or more of these “deliverables” is/are not satisfactory, have an integrity

problem, and are prone to corruption

How to go for it? Origins of corruption related demand

System failing to deliver intended outcomes (system

with integrity issues)

Demand for fairer access to education;

Demand for better quality of education;

Demand for fair and professional treatment of staff, including sound management;

Failure to ensure successful prevention and detection

Level 2: Preventive framework

Budget monitoring

Regulatory framework

Civil society and media

Accountability mechanisms

Level 3: Corruption incidence

Corruption incidence

Corruption incidence

Education corruption – hypothetical sequence of occurrence

Opportunity, spotted by

need or greed

Shortcoming 1 Shortcoming 2 Shortcoming 3

Level 1:

Education system shortcomings Sub-

dimensions

Key area 1: teachers

Sub-dimensions

Key area 2: assessment

Sub-dimensions

Key area 3: XYMilovanovitch

How to go for it?

 Instances of corruption

Good grades and exam passes obtained through bribes Exam questions sold in advance Exam cheating countenanced or facilitated by education staff  Removing the consequences of failing exams (re-admitting students under false names) School places (for schools perceived to be “better”) ‘auctioned’ out to the highest bidder. Private tutoring from class teacher outside school hours given to paying pupils  Staff recruitment, promotion and postings influenced by factors not related to suitability, such as bribes or sexual

favours or political affiliation High absenteeism (i. e. due to low motivation), affecting de facto student-teacher ratios. Private tutoring, reducing teachers’ motivation in ordinary classes Inflated or adjusted student numbers (including numbers of special needs pupils) quoted to obtain better funding Sub-standard educational material purchased due to manufacturers’ bribes, instructors’ copyrights, etc. Embezzlement of funds intended for materials, school buildings, etc. Licences and authorisations for teaching obtained on false grounds via corrupt means Sale/purchase of diplomas/qualifications Special attention given in class to favoured pupils tied to influence or payments Private tutoring

Demand situations addressed

Instances of corruption

Demand related to equity (access) –

DA

Good grades and exam passes obtained through bribes Exam questions sold in advance Exam cheating countenanced or facilitated by education staff  Removing the consequences of failing exams (re-admitting students under false names)

Demand related to quality (better quality

education) – DQ

School places (for schools perceived to be “better”) ‘auctioned’ out to the highest bidder. Private tutoring from class teacher outside school hours given to paying pupils 

Demand related to staff (prof. recognition,

placement, fair treatment) –

DS

Staff recruitment, promotion and postings influenced by factors not related to suitability, such as bribes or sexual favours or political affiliation

High absenteeism (i. e. due to low motivation), affecting de facto student-teacher ratios. Private tutoring, reducing teachers’ motivation in ordinary classes Inflated or adjusted student numbers (including numbers of special needs pupils) quoted

to obtain better funding

Opportunity – O

Sub-standard educational material purchased due to manufacturers’ bribes, instructors’ copyrights, etc.

Embezzlement of funds intended for materials, school buildings, etc. Licences and authorisations for teaching obtained on false grounds via corrupt means Sale/purchase of diplomas/qualifications Special attention given in class to favoured pupils tied to influence or payments Private tutoring

Factors creating demand

Demand addressed Instances of corruption

? DA

Good grades and exam passes obtained through bribes Exam questions sold in advance Exam cheating countenanced or facilitated by education staff  Removing the consequences of failing exams (re-admitting students under

false names)

? DQ

School places (for schools perceived to be “better”) ‘auctioned’ out to the highest bidder.

Private tutoring from class teacher outside school hours given to paying pupils 

? DS

Staff recruitment, promotion and postings influenced by factors not related to suitability, such as bribes or sexual favours or political affiliation

High absenteeism (i. e. due to low motivation), affecting de facto student-teacher ratios.

Private tutoring, reducing teachers’ motivation in ordinary classes Inflated or adjusted student numbers (including numbers of special needs

pupils) quoted to obtain better funding

? O

Sub-standard educational material purchased due to manufacturers’ bribes, instructors’ copyrights, etc.

Embezzlement of funds intended for materials, school buildings, etc. Licences and authorisations for teaching obtained on false grounds via

corrupt means Sale/purchase of diplomas/qualifications Special attention given in class to favoured pupils tied to influence or

payments Private tutoring

Milovanovitch

INTES assessment

Stakeholder interviews;

media reports

Surveys (PISA; national and international

corruption perception surveys)

Data and information

grid; national data and

indicators

Sources for country assessments

Textbooks and learning material

Infrastructure

Curriculum and teaching time

Classroom climate

Learning outcomes

Budget formulation

Execution and disbursement, cash

flows

Private investments

Education standards

Continuous assessment

Promotion exams

Transition to tertiary education

Performance incentives

Parental and private involvement

Vocational schools

Autonomy and decentralisation

Standardised entrance exams

Other admission criteria

Assessment and examinations

Coursework

Awarding of credentials

Research: Ethical standards (interest of

research)

Research: Process (data/results)

Salaries

Motivation

Supply

Professionalism – Teacher TrainingPre-School

Primary and secondary education

Tertiary Education

Level I - System level demand analysis Pre-university education and tertiary education

Teachers

Quality of learning environment

Assessment

Funding

Education coverage and provision

Governance: system management

University admission

Academic work

Quality assurance

Staff career management

Financial management

Governance: academic boards

Understanding of academic integrity

Internal quality assurance

Accreditation system

Recruitment and appointment

Career development

Salaries

Motivation

Funds

SpendingRepresentation and

competencies

Milovanovitch

Level I Example

Quality of learning environment

Textbooks and learning material

Infrastructure

Curriculum and teaching time

Classroom climate

Learning outcomes

Process of textbook production: selection; procurement; delivery

Renewal of textbooks - frequency? Who initiates it?

Textbooks suppliers in the past 5 years, per level and subject

Freedom of schools and teachers to choose materials?

Shortage of learning materials? Comparison with PISA index on quality of educational resourcesAffordability of textbooks: Price? Renting schemes? Complaints from parents on affordability and from teachers on supply?

curriculum and syllabuses for key subjects (mathematics, literature and reading, foreign language, chemistry, physics geo), incl. hours

Are assessment outcomes used to make curricular decisions?

Is there curriculum autonomy at classroom level and if yes, to what extent?

PISA index on disciplinary climate

Abuse of authority: any reported cases of abuse? How many in the past 5 years and for what? Describe disciplinary measures undertaken

Student teacher ratios - primary, secondary, initial and secondary VET

Classroom diversity: ESCS; within school variance

PISA index of teacher-student relations

PISA scores

Repetition and Drop-Out Rates

Is the condition of school infrastructure known and recorded? Is this information available? What is the condition of school infrastructure?

Decision process for capital investments - building of schools and kindergartens: criteria, funding sources, initiative?

How many schools were built in the past 5 years? Where, and why there?

Responsibilities for maintenance of school infrastructure

Level II – Enabling factors and preventive framework

Integrity climate

Regulatory framework

Accountability mechanisms

Budget monitoring and evaluation

Human resource management

Transparency

Corruption perception index – Transparency International

OECD SIGMA – horizontal integrity assessment

Competitiveness Index – World Economic Forum

Development partner assessments

National corruption assessments

Sector corruption assessment

Investment and business climate ratings

Anti-corruption agencies

Legislation: criminalisation of corruption

Legislation: public sector integrity

Anti-corruption strategies: national

Anti-corruption strategies: sectoral

Anti-corruption strategies: projects

Off-budget funding

Expenditure evaluations

State audit

Internal school audits (if applicable)

Staffing policies

DeterrentsPublic involvement

Inspections

School boards

Leadership

External quality assurance

Parliamentary control

Whistleblowers

Milovanovitch

Integrity analysis sample

Private tutoring by the same teacher

Irregular payments for access

Misuse of authority

Forced purchase of learning materials

Staff postings through payments

Lack of learning in class

Low salary levels

Incoherence of transition criteria

Flawed textbook production

Weak/malfunctioning school inspections

Lack of parental involvement

Flawed system of staff appointment

Sh

ort

com

ing

s

Corruption incidence

Demand addressed

Access to education

Better quality

Greed and opportunity

Demand related to staff

Milovanovitch

The INTES Assessment cycle

Information gathering and desk research

Site visits

Consultations and fine tuning

Submission of draft report

Dissemination

Milovanovitch

INTES: The short-term task at hand

• Introduce a framework for assessing integrity of education systems in view of corruption prevention.

• Provide countries which carried out an INTES assessment with a tool for follow-up work and tailored, sector-level recommendations on how to address shortcomings causing corruption.

INTES: Outcomes and beneficiaries

Outcomes: • Assessment report with analysis of integrity

related shortcomings and policy recommendations which is– A tool for country level follow up– Capacity building for specialised bodies – A guideline for adjustment of legislative and strategic

framework and law enforcement mechanisms towards greater sector relevance

• Beneficiaries: – Ministries of Education– Bodies involved in designing and implementing anti-

corruption policies and measures– Lawmakers– Stakeholders in education– International partners

Thank you

[email protected]

www.oecd.org/edu/nme