The Hungarian system of ex post and on-going evaluation focusing on Structural Funds Kinga Kenyeres, Evaluation Division 6-7 May, 2010 National Development Agency
Dec 26, 2015
The Hungarian system of ex post and on-going evaluation focusing on Structural Funds
Kinga Kenyeres, Evaluation Division 6-7 May, 2010
National Development Agency
The present situation of evaluation – Evaluation
• Evaluation is a centralized activity in the National Development Agency
• Separated financial source for evaluation (Implementation OP)
• Capacity building, cost planning based on The Evaluation Plan,
• Strong cooperation with Managing Authorities• In the focus of the activity:
– Project management• ToR• Legal affairs (public procurement procedures)• Project managing of outputs (Inception Report,
Semi-final Report, Final Report)• Communication about evaluational results
– Messages towards policies – Development Policy Academy - Conferences
every 3 months. Goal: extending evaluation culture, sharing experiences
The present situation of evaluation – Evaluation Diviso
• Evaluation is a centralized activity in National Development Agency
• Separated financial source for evaluation (Implementation OP)
• Capacity building, cost planning based on The Evaluation Plan,
• Strong cooperation with Managing Authorities• In the focus of the activity:
– projectmanagement• ToR• Legal affairs (public procurement procedures)• Project managing of outputs (Inception Report,
Semi-final Report, Final Report)• Communication about evaluational results
– Messages towards policies – Development Policy Academy – Conference
every 3 month. Goal: extending evaluation culture, sharing experiences
Mission
Evaluation on using of the Cohesion Fund and the Structural Funds
Provide feedback on evaluation results towards Units of execution and planning,
Active role in using the EU funds more effectively
Goals
1. The Evaluation activity• Provides information for decision-making• Provides suitable methods for
evaluational topics• Perfoms high quality• Is cost-effective2. Transforms the results of the
evaluations into messages and forwards them to Stakeholders
Decision-making support
• Policy planning, level of program coordination– Takes into account the results, impacts of
former measures (Ist National Development Plan, 2004-2006),
– Horizontal disciplines– The ex ante evaluation of OP-modification– Support to ActionPlanning
• Supporting execution– Fine tuning of measures– Practical recommendations in order to improve
processes and institutional activities „Service” approach
Rising demand for on-going evaluations
Valid Methodology• Ex post evaluation
– Econometrical methods based on databases and primer data collection – impact assessment
– Document analysis, interviews – helps to understand the results of the impact assessment, takes them into a context
• On-going evaluations– Monitoring database analyses– Process- and organizational analyses
• Ex ante evaluation– Support to the planning procedure– The goal is impoving the methods of
former impact assessment
Professional credibility, objectivity
High quality, cost efficiency
Evaluation Framework1.roundSelecting evaluators (Autumn 2008 – spring 2009)• Public procurement procedure for 4 lots
within the framework of a 4-year-contract• Technical, financial eligibility on the basis of the
strict qualification of experts• Comittee for decision including colleagues from
Managing Authorities
Evaluation Framework - 4 LOTs
1.Environment (includes: strategic environmental assessment)
2.Infrastructure (transport, energy) 3.Economic and social development
(education, health, employment, R+D and innovation)
4.Operative evaluations (includes: system of institution development)
High quality, cost efficiency
2. roundProcurement of Evaluations through
the Evaluation Framework (Spring 2009 – end of 2013)
• In 4 LOTs: 6 evaluator consortiums compete with each other for one specific evaluation task
• Call for proposal of evaluators based on well-defined TOR and defined number of expert-days
• The price is the only criterion for selection.
High quality, cost efficiency
High Professional Responsibility of Evaluation Unit
Communication of evaluational results 1. Built in the evaluation process
• Cooperation with Managing Authorities• Professional discussions in professional level and in management
level – ToR– Evaluation– Project Managing Committee
2. Following the evaluation process
• Publish evaluational final reports, open opportunity to comment on NDA homepage
• Conferences/ workshops
• Inform European Commission and other member states: Executice Summary in English
• Inform Monitoring Committees
The goal is to communicate clear messages
Reaching the goals
Evaluation Plan– 3-year plan –review every year
– Capacity building and cost plan based on Evaluation Plan.
– Evaluation Plan is based on the OP execution chapter and the Interactive Executing Handbook of NDA.
– Eval Plan has been accepted by the Implementation OP Monitoring Committee
– Inputs• Legal requirements • Information demand of the organization• Requirements of other institutional stakeholders• OP Monitoring Committe opinion• European Commission opinion• Currently: requirements of the forthcoming Hungarian EU
presidency– 2010 - shift from ex post evaluations to on-going ones
Rising rate of on-going evaluations, because– Rising interest in facts of implementation and
coordination– The findings of ex post evaluations cannot be used
easily during implementation Pros:
– Increasing overall demand for evaluation– Stakeholders are more and more concerned in
having proper quality evaluations Cons:
– The very sophisticated methodology of ex post evaluations cannot be used
– We have more experience in ex post evaluations
Evaluations of the Division
The balance between ex post and on going evaluations is a MUST!
Evaluations in the Evaluation Division
The Division launched 11 evaluations in 2009 (9 ex posts and 2 on-goings) and 12 in 2010 (8 on-goings and 4 ex posts)
Párhuzamosan futó értékelések száma
02468
10121416
Párhuzamosanfutó értékelésekszáma
Evaluations in the Evaluation Division
2010 értékelései téma területek szerint
1
1
1
1
1
1
1
2
3
4
4
5
6
6
9
0 1 2 3 4 5 6 7 8 9 10
Energy
Horizontal - equal opportunity
Horizontal - sustainability
Horizontal - handicapped people
Health
Social inclusion
Partnership
Monitoring support
Institutional
Infrastructure
Environmental issues
Labour policy
Public administration
Combined
Economy
2010 értékelései téma területek szerint
Our on-going evaluations:• Evaluation on the financing system of the
Intermediate Bodies of the NSRF - finished
• Evaluation of the action planning process
• Evaluation of the selection procedures of the NSRF
• OP mid-term reviews– Strategy-focused mid-term evaluation of the
implementation – Provides advice to optimize further
implementation
Evaluations in the Evaluation Division