Policy-making in French VET systems : trends, performance management, challenges

Policy-making in French VET systems: trends, performance management, challenges

Governance and effective Vocational Education and Training policies: the role of evidence, Torino, 23-24 Nov 2011

Robert Fouchet, Dean of the Institute of Public Management at the Paul Cézanne University in Aix-en-Provence (Aix-Marseilles III)

The LOLF* and its local applications[*Loi organique relative aux lois de finances – Constitutional Bylaw on Budget Acts]

• Origins of the LOLF• The new budget architecture

• Performance management tools• Enhancing transparency

Needs

OutputsObjectives Resources Activities

Intermediateeffects

(Results)

Long-range effects

(Impacts)Socio-economicproblems

Efficiency

Effectiveness

Usefulness and sustainabilityAdapted from Pollitt & Bouckaert, 2004

Relevance

3

Environment: Users and context

Organisation: Mission, Programme, Action

Measuring

Incorporating

Using

The Bouckaert & Halligan framework for analysis

4

Two strands

Inputs Processes Deliverables Impacts

Macro

Meso

Micro

Coverage

Dep

th The stakes of the analysis of coverage:What elements are measured? What dimensions are scrutinised? What is the time horizon of the analysis?Which beneficiaries are taken into account?When is monitoring carried out?

The stakes of the analysis of depth: Is monitoring global? How are hierarchical aspects taken into account? What is the management dialogue strategy?

Quest for more efficient expenditure: Greater knowledge of content of expenditure Identification of room for manoeuvre Transition to a performance management approach

1959 Order => new budget constitution

Origins of the LOLF

• Existing commitments• Discussion focusing on

resources• Priority given to consumption• Partitioning and

compartmentalisation of budget according to nature of charges, etc.

• Justification from the first euro• Matrix classification• Indicators and accountability• Transparency and

parliamentary scrutiny• Strengthen strategic thinking

on public finances

5

Principles of the LOLF

Performance (quest for more efficient public expenditure) is the key feature of the LOLF

Article 1

The budget acts take into account a defined economic equilibrium as well as the objectives and results of the programmes that they determine.

Article 7

A programme covers appropriations for implementing an action or a consistent set of actions coming under the same ministry and involving both specific objectives, defined in the public interest, and expected results subject to review.

Article 48 Spring budget policy debate: list of objectives and indicators planned for the following year’s budget bill.

Article 51

Annual performance plans annexed to the budget bill detailing for each programme: the presentation of its actions, associated costs and goals, and results obtained and

expected for coming years measured by a justified choice of accurate indicators.

Article 54

Annual performance reports annexed to the budget review bill: explanation of discrepancies between forecasts and actual sums reported in relation to

objectives, expected and actual results, indicators and associated costs.

6

A matrix classification

Action n°1 Action n°2Action n°3

Programme X

Nomenclature par nature

Programme Y

Programme Z

Mission A

Le programme : Le programme : cadre de cadre de

spécialisation des spécialisation des créditscrédits

Action n°1 Action n°2Action n°3

Mission B

Dotations des pouvoirs publics

Dépenses de personnel

Dépenses de fonctionne-ment

Charges de la dette

Dépenses d’investis-sement

Dépenses d’interven-

tion

Dépenses d’opérations financières

7 titres et 18 7 titres et 18 catégories de catégories de dépenses fixés dépenses fixés par l’article 5par l’article 5

Action n°1 Action n°2

Un budget détaillé au Un budget détaillé au niveau des actions à niveau des actions à

présenter dans les annexes présenter dans les annexes au PLF (bleus ministériels)au PLF (bleus ministériels)

Action n°1 Action n°2Action n°3

Programme X

Nomenclature par nature

Programme Y

Programme Z

Mission A

Le programme : Le programme : cadre de cadre de

spécialisation des spécialisation des créditscrédits

Action n°1 Action n°2Action n°3

Mission B

Dotations des pouvoirs publics

Dépenses de personnel

Dépenses de fonctionne-ment

Charges de la dette

Dépenses d’investis-sement

Dépenses d’interven-

tion

Dépenses d’opérations financières

7 titres et 18 7 titres et 18 catégories de catégories de dépenses fixés dépenses fixés par l’article 5par l’article 5

Action n°1 Action n°2

Un budget détaillé au Un budget détaillé au niveau des actions à niveau des actions à

présenter dans les annexes présenter dans les annexes au PLF (bleus ministériels)au PLF (bleus ministériels)7

4 Formal presentation to Parliament Commitments made: annual performance plans (APP),

attached to the budget bill. The APP describes the action forming the programme, measures the respective costs, defines the objectives and provides the indicators for measuring results

Results achieved: annual performance reports (APR), attached to the budget bill

APP and APR have the same presentation and the same structure so that forecasts can be compared with actual results

Vote at first reading on bill n-1 before examining the budget bill n+1

4 A written formulation of objectives: – in programme operating budgets (POB) and, where applicable, operating units (OU)– in the documents binding operators to ministries

Performance management tools: accountability

8

Objectives for citizens, users and taxpayers

Performance management tools: objectives and indicators

Point of view Objective Example

Citizen socio-economic effectiveness

to reduce smoking

User quality of service provided

to reduce the statutory period for responding

Taxpayer management effectiveness

to reduce tax management costs

9

Examples of quality of service objectives-indicators

Programme Objectives Indicators

Legal justice To deliver decisions in reasonable time in civil matters

– Average time for processing proceedings by type of jurisdiction– Average age of stock by type of jurisdiction

Development, town planning and public works

To provide users and local authorities with a high-quality service in granting planning permits

Degree of satisfaction of elected representatives in the framework of an evaluation scale identifying specific criteria for preparing bills for applying rights in land.

Land and maritime transport

To improve transport infrastructure upkeep and performance

Rate of availability of rail network (time required for restoration after incident).

Access and return to employment

To improve the effectiveness of the match between supply and demand for employment, taking the different needs into account

Proportion of businesses that have been satisfied overall with the applicants referred to them by the ANPE [Agence Nationale pour l’Emploi – National Employment Agency].

Objectives setting out the expected quality of the service provided to users (in the strict sense), i.e. capacity of the service to satisfy beneficiaries.

11

Performance objectives stated as operational objectives that can be monitored by the services

National performance objectives

Performance objectives limited

Intermediate objectives

to regions(POB)

to particular fields of application:networks, mechanisms, etc.

activity or output objectivesprocess objectives or implementation methods:measures to improve organisation, management, etc.

Objectives influencing the monitoring of services

12

The different types of objectives and indicators

Performance objectives in the production chain

Benefit forCITIZENS:socio-economic effectiveness

Benefit for USERS:quality of service

Benefit forTAXPAYERS:resource management effectiveness

resources activities outputs results long-rangeimpacts

13

Other objectives (excluding performance)

in the production chain

Volume, consumption, allocation of resources objectives

resources activities outputs results long-rangeimpacts

Volume or allocation of activity objectivese.g.: No. of hours

Volume or allocation of output objectivese.g.: No. of certificates issued

Objectives for implementingcourses or forms of actione.g.: No. of action plans, management improvement plans, etc.

General objectives:Socio-economic impacts depending on many factors: e.g.: rate of unemployment

Intermediate objectives

14

Objectives and indicators in budget documents

resources activities outputs results long-range impacts

Generalobjectives

Intermediate objectives Performance

objectives

POB APP

To justify the choice of performance objectives

15

The different types of objectives and indicators

Performance, management monitoring and assessment

socio-economic effectiveness

quality of service

management effectiveness

resources activities outputs results long-range impacts

Public policy assessment

Management monitoring

Performancemeasurement

Define/redefine objectives, indicators,

targets, policy instruments

Initial diagnosis Analysis of results

Implementation Results

Formalising of objectives in APP at national level and POB at

operational level

Performance-driven

monitoring

Transition from a resource-driven approach to a result-driven approach

– Infra-annual (management monitoring; management dialogue)– Annual (APR)– CIAP [Comité Interministériel d'Auditdes Programmes] audits, Court of Auditors audits, inspections, public policy assessment

4An iterative approach

16

Implementation of the LOLF

First milestone:Construction of platform

Second milestone: Prefiguring

T3 T4 T1 T2 T3

2003 2004T4 T1 T2 T3 T4

2005T1

2006

Third milestone: Implementation

• Structuring of missions, programmes, actions

• 2004 experiments identified

• Accounting framework• Structuring of budget-

accounting classification• Guidelines of new

implementation processes

• Communication plans

• Presentation of appropriations in LOLF format

• Performance objectives and indicators

• Ministerial formulation of budget-accounts classification

• Appraisal and development of experiments

• Accounting procedures• Training plans

• 2006 budget in LOLF format

• Breakdown and job monitoring arrangements

• New management and organisation methods

• Opening balance• Stakeholders trained for

the new budget implementation process and new accounting

17

The LOLF removes obstaclesSummary: changes brought about by the LOLF

18

Lifelong vocational training and guidance policies

A decentralised view

20

Lisbon StrategyBologna Process

State

Regional level

Citizens, operators, administrations, businesses,

universities

Law 29/11/11 on lifelong vocational training and guidance

LOLF

Parliament Prime Minister Information and guidance

representative Ministries Conseil National de la

Formation Professionnelle tout au long de la vie (CNFPTLV) [National Council for Lifelong Vocational Training]

Fonds paritaire de sécurisation des parcours professionnels [Joint Fund for Rendering Career Paths Secure]

Association nationale pour la formation professionnelle des adultes (AFPA) [National Association for Adult Vocational Training]

Contrat de plan régional de développement des formations professionnelles (CPRDFP) [Regional Vocational Development Plan Contract]

Annual agreements for applying the CPRDFP

Regional Council Regional Director Decentralised Administrations Education Offices Comité de Coordination Régional de

l’Emploi et de la Formation Professionnelle (CCREFP) [Regional Coordinating Committee for Employment and Vocational Training]

CESR [Conseil économique et social de la région – Regional Economic and Social Committee]

Employee and employer representatives

Context and aims of the new vocational training and guidance systemContext New aims of national and

local policies

21

Gradual enlargement of target audiences of vocational training policies

Young people from 16-25 years of age (Law of 20 December 1993)Anyone who wishes to acquire and update knowledge to further their vocational development

Recognition in law of a lifelong public guidance service

Further decentralisation of powers at regional level

2002: all target audiences, transfer to the region of matters connected to the validation of prior learning2004: transmission of health and social training, vocationally-oriented training in artistic and cultural fields

Signature of CPRDFP in 2011New means of involvement connected to the BTS [Brevet de Technicien Supérieur – Advanced Vocational Training Certificate], secondary school technology pathways, vocationally-oriented training in SE

Law of 29 Nov 2009 A new ‘public lifelong guidance service

to guarantee all persons access to free, full and objective information on occupations, training, certification, job opportunities and remuneration levels, as well as access to high-quality, networked and personalised guidance services’

Creation of the Conseil National de la Formation professionnelle tout au long de la vie (CNFPTLV) [National Council for Lifelong Vocational Training]:

Mission: ‘to promote nationwide dialogue between the State, the regions, the social partners and other stakeholders to define the multiannual guidelines and annual priorities for initial and continuing vocational training policies, and to design and monitor the implementation of such policies’

Context and aims of the new vocational training and guidance system

22

Principal guidelines of the 2011 CNFPTLV:

1. To take account of regional needs, the economic context, the labour market, economic and social change, and the development of occupations and qualifications

2. To propose the provision of a coherent and coordinated lifelong information and guidance service

3. To provide universal access to a first level of qualification

4. To develop 0n-the-job training and apprenticeships

5. To safeguard career paths

6. To ensure greater clarity in training provision, which must be adapted to requirements

7. To ensure better coordination of vocational training policies

8. To develop expertise and observation tools for use by decision-makers

Drafting of CPRDFP at regional level: Provence-Alpes-Côte d’Azur

23

CCREFP

– Prefect, Directors of Education, Regional Directors

– Regional Council Chairman, Vice-Chairman responsible for VT– Employer

organisations and chambers of commerce–

Representatives of employee organisations – Chairman of

the CESR [Conseil économique et social de la région – Regional Economic and Social Committee]

Organisation:

Cabinet Semaphor

es

The opportunity to innovate: ‘We believe that the objectives of the Law on Lifelong Vocational Training and Guidance of 24 November 2009 fall well below the expectations of vocational training stakeholders. This is why we are going to adopt a more ambitious approach that brings the social partners together.’ (CR Chairman)

Mandate

Functioning– Quarterly meetings– Discuss and act upon committee proposals– Monitor CPRDFP implementation– Ensure links with Job Centres and the AFPA [Association nationale pour la formation professionnelle des adultes – National Adult Vocational Training Association]

Implementation:‘This will be […] a compartmentalised CPRDFP, with different contracts according to the partners, [it] will be a living document, a true programming document, with specific financial commitments for each signatory. Those who can commit will sign it’

Operators:– administrations, agencies, universities, training and support bodies, etc.

Matters addressedVocational paths and continuity,

employment and regional economic development: training, social and

economic investmentReception service, information, guidance, support and

advice: help people to obtain vocational qualifications for

sustainable employment… … …

– Experts –Otherinvitedbodies

5 Committees

Diagnosis– Often unquantified elements that can easily be transposed to other French regions

– Data sometimes old (chosen because they are available), in-depth analysis often being relegated to target-based agreements after the CPRDFP has been signed

– Diagnostic processes without using a methodical forward-looking approach

– Involvement of reliable institutions in the diagnostic stage (Observatoire Régional des Métiers [Regional Labour Market Observatory], CARIF Espace Compétences [public interest group])

General observations on the formulation and implementation of lifelong vocational training and guidance policies

24

Sources: CNFPTLV, CCREFP Rhône-Alpes, CCREFP PACAFormulation

– Continuous development of national priorities and law

– Further decentralisation and increase in number of forums in an already crowded landscape

+ Better integration of local context and uptake of innovations in policy implementation

+ Dialogue highlighting needs and knowledge at grassroots level

– Lack of work by Pôle Emploi [public employment service] and university vice-chancellors

Decision-making– A process carried out as a matter of urgency

– Authorities’ role sometimes insufficiently defined

– Inconsistent political and institutional support, making it difficult to match vocational training paths and levels with employment and skills needs

+ Mainstreaming: coordination between sectoral approaches and vocational sectors, regional approaches and analysis in terms of target audiences and training and work integration paths

– Diversity of parties and complexity of stakes involved, making it difficult to define objectives that can be put into operation

ImplementationPoor feedback at this point at this time

– Stakeholders’ involvement in the CPRDFP is not examined as a legally certain contractual commitment

+ Diversity of operators makes it difficult to apply objectives and to formalise agreements in the framework of the CPRDFP

– Unequal attention paid to output development, monitoring and evaluation

Monitoring– The complexity of the stakes and funding mechanisms raise real challenges in defining, calculating and monitoring indicators. This dimension of the CPRDFP must be professionalised.

– Difficult financial commitments in multiannual projects in the context of changing ‘rules’.

Professionalisation in the French higher education system after the Bologna reforms:

expectations and reality• European professionalisation and commitments

• Contracting training provision in France: the four-year contract (principles, processes)

• Expectations and reality of the process in terms of professionalisation• Expectations and reality of indicators of professionalisation

Professionalisation in higher education since BolognaBologna, 1999 Leuven, 2009 (Bologna in

2020)

26

A system of easily readable and comparable degrees which includes the introduction of a joint diploma supplement to enhance transparency;

A system essentially based on two cycles: a first cycle of at least three years, which is advantageous to the labour market, and a second cycle (Master’s) requiring completion of the first cycle;

Employability – the parties involved must cooperate to raise initial qualifications and renew a skilled workforce, and to improve the provision, accessibility and quality of careers and employment-related guidance services. In addition, work placements embedded in study programmes and on-the-job learning must be encouraged;

Improve data collection: create multi-dimensional transparency tools. These tools should be based on comparable data and adequate indicators and should incorporate the principles of the Bologna Process relating to quality assurance and recognition;

Ministry/universities contracting process: four-year training

27

Accreditation of national degrees by the Ministry on the proposal of universities: General degrees Vocational degrees Master’s degrees

Role of the AERES [Agence d’évaluation de la recherche et de l’enseignement supérieur – Research and Higher Education Quality Assurance Agency] as a quality assurance agency

Authorisation in waves on four-year contracts, then five- or even six-year contracts

Ministry/universities contracting process: four-year training

28

Process: Negotiation of training provision (preliminary descriptors, set-up descriptors,

self-test: SWOT, intra-university independent assessment, RNCP [Répertoire national de certifications professionnelles – National Register of Vocational Qualifications] descriptors, annexes describing degree)

Vote in component boards Vote in CEVU [Conseil d’Etudes et de la Vie Universitaire – Advisory Board for

University Life and Studies] then in CA [Conseil académique – Academic Council]

Referral to AERES Return of assessments to training providers (A+, A, B, C) Responses to AERES with or without changes to accreditation files Forwarding to DGES-IP DGES-IP opinion and notes (favourable/unfavourable, reservations,

recommendations) Responses of training providers with or without changes Universities/Ministry negotiation Authorised training provision.

Four-year training and professionalisation

Hopes Realities

29

Length of contract: Bringing into line with

labour market realities

Quality assurance thanks to accreditation procedures

Too short/too long: four years (assessment after less

than two years) or six years (time to respond).

A very long process (2 years): January 2010: launch 15 October 2010: forwarding to Ministry April 2011: AERES opinion October 2011: DGESIP opinion December 2011: accreditation

Indicators of professionalisation not highly valued in the accreditation file: multiplicity of other indicators: staff, attractiveness, costs of tutorial equivalent hours, part of subject/crosscutting education (degree)

Four-year training: indicators of professionalisation

Indicators used The reality of professionalisation

30

Occupation codes: RNCP: ROME [Répertoire opérationnel

des métiers et des emplois – Register of Occupations and Trades] codes

Nomenclature spécialités de formation: NSF [training classification]

Subjects sector codes: SISE-DGESIP

Employment integration figures: National survey 30 months after

obtaining a Master’s.

Upskilling advice with professionals

Realities of the working world becoming more complex and changing

Sectoral and subject approaches ill-suited to the realities of the working world

Unrepresentative method (number of respondents by specialism, lack of information on the match between training and occupations)

Internal surveys not well developed or difficult to compare (different methods)

Not well developed (except for IUT [University Institutes of Technology], Licences pro [vocational degrees], apprenticeship and some components/training)