Evaluation of NSRF Health Care Development Projects Norbert Kiss Hétfa Research Institute – Corvinus University of Budapest Balázs Váradi Budapest Institute – Eötvös Loránd University 30 April 2013
Dec 29, 2015
Evaluation of NSRF Health Care Development Projects
Norbert KissHétfa Research Institute – Corvinus University of Budapest
Balázs VáradiBudapest Institute – Eötvös Loránd University
30 April 2013
Evaluation of NSRF Health Care Development Projects 2
Contents
• General overview and evaluation of health care developments
• Geographic targeting of health care development projects
• Evaluation of developments of outpatient care providers
• Modernisation of the central medical equipment register
• Evaluation of lifestyle development programmes
3
Overview of health care development
• Methodology:– Inclusion of health care constructions (from SMIS)– Health care providers as beneficiaries (from SMIS)– Assignment of current funding of the National Health Insurance
Fund (OEP) to levels of health care
• Main conclusions:– Large share of health care constructions in total funding– Smaller amounts from ROPs (accessibility for the disabled,
medical tourism), EEOP, EAOP (e.g. e-health)– EDOP, ROPs, SROP (lifestyle development): private providers
were also eligible (app. HUF 11.5 billion)– Connections between SIOP and ROPs (statistical regions and
„health care planning and management areas”)– Public providers were typically supported from one source only– HR developments (HUF 8,5 billion): only a fraction of the wages
in the health care sector, longer term impacts can only be achived at very specific areas
Evaluation of NSRF Health Care Development Projects
4
Regular OEP funding (2011) Development programs
Funding (billion HUF)
% Funding (billion HUF)
%
Primary129.1 17.1% 15.0 4.5%
Outpatient143.2 19.0% 93.5 27.5%
Inpatient446.3 59.3% 199.9 59.1%
Ambulance25.3 3.4% 15.3 4.5%
Blood supply8.2 1.1% 3.2 1.0%
Health promotion0.5 0.1% 11.4 3.4%
Total752.7 100.0% 336.3 100.0%
Targeting of the development programs (by progressivity – levels of health care)
Evaluation of NSRF Health Care Development Projects
5
Evaluation of geographical targeting
• Methodology:– Health care development projects (from SMIS database)
• Assignment to progressivity levels and to the population of microregions (LAU1) (at institutional level)
– Current („regular”) OEP funding• Assignment to progressivity levels • Assignment to microregions (by using OEP data)
– Avoidable mortality by microregions (J. Vitrai & M. Bakacs)
• Assessment of the targeting of regular and development funds in terms of avoidable mortality and the complex developmental indicator, using Lorenz curves– What percentage of the sources were allocated to the
microregions with greatest needs
Evaluation of NSRF Health Care Development Projects
6
Avoidable mortality by microregion (J. Vitrai & M. Bakacs)
Evaluation of NSRF Health Care Development Projects
7
Development projects are better targeted than regular funding
0.1
.2.3
.4.5
.6.7
.8.9
1S
our
ces
0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1Health level
Regular funding(2011) Programs(2008-12)
Evaluation of NSRF Health Care Development Projects
8
Primary care projects are the best targeted
0.1
.2.3
.4.5
.6.7
.8.9
1S
our
ces
0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1Health level
Primary regular funding(2011) Primary programs(2008-12)
Evaluation of NSRF Health Care Development Projects
9
However, outpatient care targeting is not bad either
0.1
.2.3
.4.5
.6.7
.8.9
1S
our
ces
0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1Health level
Outpatient funding(2011) Outpatient programs(2008-12)
Evaluation of NSRF Health Care Development Projects
10
Targeting of inpatient care development projects
0.1
.2.3
.4.5
.6.7
.8.9
1S
our
ces
0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1Health level
Inpatient regular funding(2011) Inpatient programs(2008-12)
Evaluation of NSRF Health Care Development Projects
11
Geographical targeting of development projects: quantitative indicators
Overall funding Outpatient care Inpatient care Primary care
How much was allocated to the least healthy half of the population? (in % of the total)
Development projects 60.1 74.4 56.5 79.2
Regular funding 48.1 43.9 49.9 46.7
Ratio of sources allocated to the worst and best deciles (in terms of health status)
Development projects 0.62 0.38 0.73 0.18
Regular funding 1.04 1.29 0.94 1.14
Evaluation of NSRF Health Care Development Projects
12
Positioning of the outpatient care development projects
Did the development projects meet the local needs?• Methodology: document analysis, statistical analysis • Key findings:
– Feasibility studies for the tender used different methodologies.
– Short run-up period– Accessibility in terms of time improved:
• App. 310,000 people got access to basic level outpatient care in 20 minutes
• App. 150,000 people got access to advanced level outpatient care in 25 minutes (4 basic specialities + cardiology, traumatology)
• Results are similar when accessibility is defined by either car or bus
• Bus timetables are not in harmonised with office hours
– Low scale of economy, modest capacity utilisationEvaluation of NSRF Health Care Development Projects
13
Improvement in the accessability of basic level outpatient care
No access to basic level care in 20 min
Access to basic level care in 20 min
Place of development (no access to basic level care in 20 min)
Place of development (access to basic level care in 20 min)
Development created access to basic level care in 20 min
Basic level access obtained by other reason
Evaluation of NSRF Health Care Development Projects
14
Changes in the utilisation of services
How have the projects affected the utilisation of health care services?• Methodology: comparing the number of cases between the
population of the developed microregions (= „treated microregions”) vs. the population of control regions– Matching methods, combined with difference in differences– Panel data analysis with fixed effect estimation
• Key findings (SIOP 2.1.2):– Solid increase (25-30%) of utilisation in almost each
medical specialities regardless of the estimation methodology
– Rise in utilisation is 36-43% among people above 60, while 13-23% among younger people
– Standardized number of cases has increased to the level of similar regions
Evaluation of NSRF Health Care Development Projects
15
Short term health impacts
Has the improved access to outpatient care improved the care of chronic patients (e.g. identifying chronic conditions, utilisation of necessary tests and consultations)?Have the additional outpatient care capacities contributed to the reduction of sick leave days?
• Methodology: statistical analysis• Key findings:
– Sick leave days and number of cases dropped by 4-6% (SIOP 2.1.2-3)
– Annual eye screening of diabetic patients: 8% increase (no significant effects on other types of screenings and laboratory tests)
Evaluation of NSRF Health Care Development Projects
16
Economic sustainability
Are the developed outpatient care centers in microregions sustainable on the long term?
• Methodology: case studies, expert interviews• Key findings:
– The original business plans and sustainability calculations became practically outdated at the time of contracting
– No common responsibility of the microregion for the sustainment of the center in several cases
– Problems with additional revenue creating services – The utilisation of the physical infrastructure cannot be
efficient due to the low scale of economy– It is a consequence of the financing system that these
centers cannot break even – or only with great difficulties – Aspects of equity
Evaluation of NSRF Health Care Development Projects
17
Modernisation of the central medical equipment register: the IT system
• Adequate hardware and software environment for the central system
• Data collection methodology uses up-to-date technology• Automatic uploading + web-based interface• Client side interface development was also initiated, 2
major IT system suppliers implemented it
• Preliminary interviews with data suppliers and users • The content of reporting has been expanded
(compulsory+optional)• Information gathering processes of data suppliers have
not been modelled and analysed• Problems with the nomenclature and master data
Evaluation of NSRF Health Care Development Projects
18
Modernisation of the medical equipment register: Efficiency and effectiveness
• „Accurate” reporting practices would lead to increase of administrative burden
• Thus few providers thrive to report „accordingly”• No legal consequences, and there is a conflict of interests
• Data migration between the old and new systems• Validation with sampling
• No change in the number of providers with accurate reports (app. 10%)
• Limited number of users of the database
• Recommendation:– Analysing data gathering processes of providers– Connection to the licensing of providers
Evaluation of NSRF Health Care Development Projects
Lifestyle development programmes in 2009: overview of applications
• App. 250.000 participants (SMIS-monitoring)
Source: SMIS-reporting (10th Jan 2013)
19
Calls for applicationsNo.
applications received
No. projects selected
Grant awarded (HUF)No. of
contractsValue of contracts
(HUF)Payments made (HUF)
SROP - 6.1.2./A-09/1 1406 170 1 554 559 581 HUF 170 1 554 559 581 HUF 1 436 364 923 HUF SROP - 6.1.2./A-09/1-CHR 689 135 1 252 546 799 HUF 135 1 252 546 799 HUF 1 068 866 420 HUF SROP - 6.1.2./LAMR-09/1 50 14 764 442 802 HUF 14 764 442 802 HUF 638 503 871 HUF SROP - 6.1.2./LAMR-09/2 13 1 19 645 825 HUF 1 19 645 825 HUF 12 954 979 HUF
Primary scenes of activitiesSROP -
6.1.2./A-09/1SROP - 6.1.2./A-
09/1-CHR
Micro-region/municipality 12% 15%
Education institutions 35% 24%
Workplaces 34% 46%
Target group specific scenes 12% 6%
Other 7% 9%
Evaluation of NSRF Health Care Development Projects
Geographic targeting of the lifestyle development projects
20
0.1
.2.3
.4.5
.6.7
.8.9
1C
umu
lativ
e va
lue
of fu
ndi
ng
0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1Health status
SROP 6.1.2. Allocation of funds
Evaluation of NSRF Health Care Development Projects
Adequateness of content and impact of lifestyle development projects
• Motivation for individual and/or community level lifestyle changes were in the focus
• Capacity building, changes in the local environment to promote healthy lifestyle were given lower priority
• Difficulties with identifying and following the real professional content of the projects – assessing whether the content has met the needs is also difficult
• Quality assurance of the professional content was missing from the selection procedure (except for LAMR applications)
• Non-LAMR health plans were often only formally adequate
• Recommendations and resources of the (central) research and methodological development institutions were rarely used
• „Copy-paste” type content was typical in communication materials
• Long lasting impacts are not achievable (according to expert opinions)
• Low value of grants – high administrative burden
21Evaluation of NSRF Health Care Development Projects
Contribution to improvement of local conditions
• Not clearly demostrated how the health plans are connected to the local needs and demands
• Health plans, prepared during the projects, have not become a central part of local health development planning
• Methodological support was provided for LAMRs • Inclusion of partners: mostly from the social sector• Partnership agreements were typically made for the
duration of the projects• Occasionally „simple and inexpensive” solutions were less
preferred („this is the time when we have the money…”)• Equal opportunity appeared as a horizontal objective • The practice of „one size fits all” was strong
• A large part of the experiences have been incorporated into the newer calls in 2011-2012
22Evaluation of NSRF Health Care Development Projects