Top Banner
82

WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Oct 07, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were
Page 2: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

WHO Evaluation Office

Country Office Evaluation - Thailand

Evaluation brief 2017

Context

Country office evaluations were included in the Organization-wide evaluation workplan for 2016-2017, approved by the Executive Board in January 2016. They encompass the entirety of WHO activities during a specific period and aim to provide findings, recommendations and lessons that can be used in the design of new strategies and programmes in-country.

Objectives and scope the Evaluation

The main purpose of this evaluation was to identify and document best practices and innovations of WHO in Thailand on the basis of its achievements over the period 2012-2016. These included not only results achieved by the WHO Country Office (WCO) but also contributions at the regional and global levels to the country programme of work.

Key findings and conclusions

Question 1: Were the strategic choices made in the Country Cooperation Strategy (CCS) the right ones to address Thailand’s health needs and coherent with government and partners’ priorities?

The priorities identified in both the CCS 2012-2016 and the CCS 2017-2021 were strategic to address Thailand’s major health needs and were coherent with government and partners’ priorities expressed in the United Nations Partnership Framework. Overall, the CCS 2012-2016 introduced a major shift from a fragmented approach through many small projects to a much stronger focus around five priorities and three main activities. This shift has been further strengthened in the design of the CCS 2017-2021 which includes only five main strategic priorities. The CCS provides the strategic framework for WHO’s work in and with Thailand. However, the priorities and activities therein do not necessarily cover the totality of the WCO’s contribution to health in Thailand. There is a discrepancy between the WHO programme and funding structure, to which WCO workplans must conform, and the priorities elaborated in the CCS, making it challenging for the WCO to develop its workplans in line with the CCS priorities.

WHO’s intellectual and social capital. The CCS 2017-2021 provides a unique opportunity for both the Royal Thai Government (RTG) and WHO to engage in a strategic partnership of a new kind where funding is

no longer the main commodity but the means by which both partners contribute their respective added value. Building on its well-established and recognized intellectual capital, WHO now has to strengthen its positioning in terms of social capital and branding, thereby enabling the RTG to consolidate the achievement of its universal health coverage by more systematically addressing the social determinants of health while at the same time enhancing Thailand’s role in global health. Many national partners indicated clear expectations of WHO’s strategic contribution in this respect. In their opinion, intellectual capital broadly refers to WHO’s leading role in technical health expertise, while social capital refers more to WHO’s reputation, influence, authority, name and trust. More widely, Thailand and other countries in similar situations are facing issues that require tailored approaches and support from their respective regional office and headquarters.

Question 2: What is the contribution/added value of WHO toward addressing the country’s health needs and priorities?

Main achievements. Overall, during the period 2012 to 2016, the WCO in Thailand provided a valuable contribution in supporting the RTG’s national health sector plans. The CCS 2012-2016 created an enabling environment for various players in the Thai health sector to form partnerships around key health issues and this positive environment was strengthened for the CCS 2017-2021. Positive results were noted in the area of noncommunicable diseases, international trade and health, road safety, border and migrant health and communicable diseases. Community health and ageing were sunsetted as priorities while the disaster preparedness and response priority made limited progress over the course of the CCS.

Programme management challenges. The difficulty in measuring results against planned targets and assessing WHO’s contributions to the same are indications of a number of systemic challenges in planning and monitoring processes within WHO at both corporate and country levels. This weakens WHO’s capacity to demonstrate results and its contribution to health improvements in any given country. Furthermore, it appears that, over the course of the CCS 2012-2016, the WCO was not able to develop its own mechanism to monitor the effects of

Page 3: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

WHO Evaluation Office its contribution to the various objectives defined for each priority of the CCS when developing its country workplans.

Question 3: How did WHO achieve the results?

Key contributions of core functions. Technical support stands out as the key core function contributing to the WCO work in support of the RTG and the implementation of the CCS 2012-2016. This core function enabled research activities, facilitated the adaptation of norms, standards and guidelines and provided evidence to inform policy options for decision makers. The other core function which played a major role was WHO’s leadership and convening power, allowing Thailand to avail of international expertise, and contributing Thai health expertise abroad. The WCO contribution to monitoring of health trends seems to have been more limited but, in the future, this core function is expected to play a much bigger role.

Partnerships. With respect to the work of the WCO with partners, the major shift introduced with the CCS 2012-2016 has been critical. Bringing together various actors around key priorities understandably takes time and, despite the mixed results obtained so far, it is considered by all as the way forward, establishing firm foundations for the design of the CCS 2017-2021. The initial collaboration with non-health actors that was introduced in the CCS 2012-2016 has been confirmed in the CCS 2017-2021.

Funding remains a critical means for WHO’s catalytic engagement in the country. It ensures that certain priorities remain high on the agenda. Funding mechanisms will need to follow the strategic shift from small projects to priority areas initiated with the CCS 2012-2016 and confirmed in the CCS 2017-2021, and new approaches through pooled funding mechanisms are being considered.

Staffing. The WCO team composition and skills mix has evolved over time and been strengthened with a doubling of the number of international staff over the CCS 2012-2016 period. It is important to be able to match staff profiles and expertise with the priorities set out in the CCS. Considering the weaknesses in planning and monitoring observed during the CCS 2012-2016 and the expectations from national counterparts in this area, the WCO needs to ensure that it can very quickly mobilize adequate levels of expertise in this area.

Best practices and innovations. This evaluation highlighted a certain number of emerging good practices and innovations framing WHO’s engagement in Thailand. For instance, the approach taken in the

design of the CCS 2012-2016 and the lessons learned strengthened the design of the CCS 2017-2021, highlighting partnerships with national actors beyond the health sector and instituting a transparent and consultative priority setting process for the CCS 2017-2021. The fact that the RTG has increased its funding to become the main funding source for the CCS 2017-2021 also represents a major shift in its collaboration with WHO.

Recommendations

On the basis of the above analysis, the evaluation would like to make the following recommendations:

Recommendation 1: The Head of the WHO Country Office and the WHO Country Office team to contribute actively to Country Cooperation Strategy governance activities and to engage with other national partners to support implementation of Country Cooperation Strategy priorities and activities, in particular in the area of programme management and monitoring. Recommendation 2: The WHO Secretariat to ensure that the WHO Country Office has the capacity to implement its workplans beyond the Country Cooperation Strategy priorities and activities, including through appropriate funding mechanisms and staffing of the Office. Recommendation 3: The WHO Country Office to build on a Theory of Change for the period 2017-2021 to better link the Country Cooperation Strategy 2017-2021 with the entire planned country-level results and deliverables and with the Country Office staff and activity workplans during operational planning for Programme budgets 2018-2019 and 2020-2021 Recommendation 4:. The WHO Country Office and the Royal Thai Government to strengthen inclusion of the gender and other social determinants of health dimension(s), as relevant, in the implementation of the Country Cooperation Strategy and other Country Office activities Recommendation 5: The WHO Secretariat to review the evolution of the Country Office’s contribution to, and relationship with, the Royal Thai Government over the recent Country Cooperation Strategy cycles, in order to consider the lessons learned, innovation and best practices for Country Office interaction with, and contribution to, other upper-middle-income countries and emerging economies. Contacts

For further information please contact the evaluation office at the following address: [email protected]

Page 4: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

Country Office Evaluation

- Thailand -

(Volume 1: Evaluation Report)

August 2017

WHO Evaluation Office

Page 5: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

Acknowledgments

The evaluation team would like to thank the Head of the WHO Country Office in Thailand and his team for their help in organizing the evaluation team visit in Thailand and facilitating interviews with country partners. Their insights and perspectives on WHO’s role and contributions in-country were invaluable.

We would also like to thank all the representatives from the Royal Thai Government, UN agencies, bilateral agencies and academics who gave generously of their time to inform this evaluation.

Page 6: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

Table of contents

Executive summary .................................................................................................................................. i

1. Introduction .................................................................................................................................... 1

1.1 Evaluation features ................................................................................................................. 1

1.2 Methodology ........................................................................................................................... 2

1.3 Country context ...................................................................................................................... 3

1.4 WHO activities in Thailand ...................................................................................................... 5

2. Findings ............................................................................................................................................... 8

2.1 Relevance of WHO’s strategic choices .......................................................................................... 8

2.2 WHO’s contribution and added value (effectiveness) ............................................................... 13

2.3 How did WHO achieve the results? (Elements of efficiency) ..................................................... 18

3. Conclusions ....................................................................................................................................... 24

4. Recommendations ............................................................................................................................ 27

The following annexes are available in Volume 2:

Annex 1: Terms of reference

Annex 2: Evaluation methodology and evaluation matrix

Annex 3: WHO’s main planning instruments and associated challenges

Annex 4: Evaluation observations for each priority and main activities of CCS 2012-2016 and

CCS 2017-2021

Annex 5: List of people met

Annex 6: Bibliography

Page 7: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

Acronyms

AMR antimicrobial resistance

ASEAN Association of Southeast Asian Nations

CCS

CDC

Country Cooperation Strategy

Centers for Disease Control and Prevention

COE country office evaluation

EQ evaluation question

GPW Global Programme of Work

HQ WHO headquarters

IHR International Health Regulations

IOM International Organization for Migration

MDG Millennium Development Goal

MOPH Ministry of Public Health

MTSP Medium-Term Strategic Plan 2008-2013

NCD noncommunicable diseases

NESDP National Economic and Social Development Plans

NHDP National Health Development Plan

NIEM

NPO

ODA

PB

National Institute for Emergency Medicine

National professional officer

Official Development Assistance

Programme budget

RO Regional Office (WHO Regional Office for South-East Asia)

RTG Royal Thai Government

SDG Sustainable Development Goal

THE

TOC

total health expenditure

Theory of Change

TOR

UNAIDS

UNDP

UNFPA

UNICEF

Terms of Reference

Joint United Nations Programme on HIV/AIDS

United Nations Development Programme

United Nations Population Fund

United Nations Children’s Fund

Page 8: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

UNPAF

USAID

United Nations Partnership Framework

United States Agency for International Development

WCO

WHA

WHO country office

World Health Assembly

WHO World Health Organization

Page 9: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

i

Executive summary

Evaluation features

Country office evaluations (COE) were included in the Organization-wide evaluation workplan for 2016-2017, approved by the Executive Board in January 2016. They encompass the entirety of WHO activities during a specific period. The COEs aim to provide findings, recommendations and lessons that can be used in the design of new strategies and programmes in-country.

The main purpose of this evaluation was to identify and document best practices and innovations of WHO in Thailand on the basis of its achievements over the period 2012-2016. These included not only results achieved by the WHO Country Office (WCO) but also contributions at the regional and global levels to the country programme of work. Its main objectives were to:

a. Demonstrate achievements against the objectives formulated in the Country Cooperation Strategy (CCS) and other relevant strategic instruments; and corresponding expected results developed in the WCO biennial workplans, while pointing out the challenges and opportunities for improvement.

b. Support the WCO and partners to operationalize the various priorities of the next CCS (and the relevant strategic instruments) based on independent evidence of past successes, challenges and lessons learnt.

c. Identify best practices emerging from the unique relationship between the Royal Thai Government (RTG) and WHO. These can then usefully inform the development of future country, regional and global support through a systematic approach to organizational learning.

The main expected use for this evaluation is to support the WCO as it operationalizes the new CCS 2017-2021 and develops its next biennial work plan. It should also assist the Regional Office for South-East Asia (RO) and WHO headquarters (HQ) when deciding how best to support to the WCO. Finally, over the medium-term, it will contribute to build a body of evidence around possible systemic issues to be addressed corporately.

Guided by the WHO Evaluation Practice Handbook, the evaluation was based on a rigorous and transparent methodology that served the dual objectives of accountability and learning. The methodology ensured impartiality and lack of bias by relying on a cross-section of information sources (from various stakeholder groups) and using mixed methodological approaches (e.g. quantitative and qualitative data) to ensure triangulation of information through a variety of means. The evaluation was conducted between January and June 2017 by a core team of four members.

Relevance of WHO’s strategic choices

The priorities identified in both the CCS 2012-2016 and the CCS 2017-2021 were strategic to address Thailand’s major health needs and were coherent with government and partners’ priorities expressed in the United Nations Partnership Framework (UNPAF). Overall, the CCS 2012-2016 introduced a major shift from a fragmented approach through many small projects to a much stronger focus around five priorities and three main activities. This shift has been further strengthened in the design of the CCS 2017-2021 which includes only five main strategic priorities. This prioritization process for the CCS 2017-2021 was strengthened based on experience gained during the CCS 2012-2016 as follows: i) it included a very large number of stakeholders, reflecting the multiplicity of actors in the health sector in the country, and ii) it benefitted from a transparent consultation and a priority setting based on predefined selection criteria known to all relevant partners. This led to: i) some issues such as noncommunicable diseases (NCD) or border and migrant

Page 10: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

ii

health (WCO focus areas) becoming a priority for the Ministry of Public Health (MOPH) in the CCS; ii) the sunsetting of certain activities as appropriate; and iii) introduction of new ones such as global health diplomacy, reflecting a RTG priority. The CCS provides the strategic framework for WHO’s work in and with Thailand. However, the priorities and activities therein do not necessarily cover the totality of the WCO’s contribution to health in Thailand. For example, communicable diseases are no longer a CCS priority, but Thailand is still one of the 30 high burden countries for tuberculosis. Staff in the WCO continue to provide support to Thailand as necessary in this and other technical areas. There is a discrepancy between the WHO programme and funding structure, to which WCO workplans must conform, and the priorities elaborated in the CCS, making it challenging for the WCO to develop its workplans in line with the CCS priorities.

Gender. A gender analysis was not done in the CCS 2012-2016 and, though gender was given substantial attention in the CCS 2017-2021 analysis, it did not lead to the priorities retained paying explicit attention to the gender issues identified in the analysis.

WHO’s intellectual and social capital. The CCS 2017-2021 provides a unique opportunity for both the RTG and WHO to engage in a strategic partnership of a new kind where funding is no longer the main commodity but the means by which both partners contribute their respective added value. Building on its well-established and recognized intellectual capital, WHO now has to strengthen its positioning in terms of social capital and branding, thereby enabling the RTG to consolidate the achievement of its universal health coverage by more systematically addressing the social determinants of health while at the same time enhancing Thailand’s role in global health. Many national partners indicated clear expectations of WHO’s strategic contribution in this respect. In their opinion, intellectual capital broadly refers to WHO’s leading role in technical health expertise, while social capital refers more to WHO’s reputation, influence, authority, name and trust. More widely, Thailand and other countries in similar situations are facing issues that require tailored approaches and support from their respective regional office and HQ. Resource mobilization in such countries might require a specific strategy and the new financing mechanism being developed for the CCS 2017-2021 in Thailand may offer lessons for other countries with a similar WHO presence.

WHO’s contribution and added value

Main achievements. Overall, during the period 2012 to 2016, the WCO in Thailand provided a valuable contribution in supporting the RTG’s national health sector plans. The CCS 2012-2016 created an enabling environment for various players in the Thai health sector to form partnerships around key health issues and this positive environment was strengthened for the CCS 2017-2021, based on the experience gained from the CCS 2012-2016. Results have been documented for all four main expected outcomes identified in the Theory of Change (TOC). Positive results were noted in the area of NCD, international trade and health, road safety, border and migrant health and communicable diseases. Community health and ageing were sunsetted as priorities while the disaster preparedness and response priority made limited progress over the course of the CCS.

Programme management challenges. The difficulty in measuring results against planned targets and assessing WHO’s contributions to the same are indications of a number of systemic challenges in planning and monitoring processes within WHO at both corporate and country levels. This weakens WHO’s capacity to demonstrate results and contribution to health improvements in any given country. Furthermore, it appears that, over the course of the CCS 2012-2016, the WCO was not able to develop its own mechanism to monitor the effects of its contribution to the various objectives defined for each priority of the CCS when developing its country workplans.

Processes to achieve the results

Key contributions of core functions. Technical support stands out as the key core function contributing to the WCO work in support of the RTG and the implementation of the CCS 2012-2016.

Page 11: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

iii

This core function enabled research activities, facilitated the adaptation of norms, standards and guidelines and provided evidence to inform policy options for decision makers. The other core function which played a major role was WHO’s leadership and convening power, allowing Thailand to avail of international expertise, and contributing Thai health expertise abroad. These functions form the foundation of WHO’s intellectual and social capital. The WCO contribution to monitoring of health trends seems to have been more limited but, in the future, this core function is expected to play a much bigger role, both in the monitoring of CCS implementation and to support the country to monitor its progress towards the health-related Sustainable Development Goals (SDGs).

Partnerships. With respect to the work of the WCO with partners, the major shift introduced with the CCS 2012-2016 has been critical. Bringing together various actors around key priorities understandably takes time and, despite the mixed results obtained so far, it is considered by all as the way forward, establishing firm foundations for the design of the CCS 2017-2021. The initial collaboration with non-health actors that was introduced in the CCS 2012-2016 has been confirmed in the CCS 2017-2021.

Funding remains a critical means for WHO’s catalytic engagement in the country. It ensures that certain priorities remain high on the agenda, as has been the case with border and migrant health and with road safety. Funding mechanisms will need to follow the strategic shift from small projects to priority areas initiated with the CCS 2012-2016 and confirmed in the CCS 2017-2021, and new approaches through pooled funding mechanisms are being considered. Such mechanisms require even stronger attention to planning and monitoring as indicators at outcome level need to be identified and their achievements documented in order to release funding instalments at specific times of the CCS 2017-2021 implementation.

Staffing. The WCO team composition and skills mix has evolved over time and been strengthened with a doubling of the number of international staff over the CCS 2012-2016 period. It is important to be able to match staff profiles and expertise with the priorities set out in the CCS. The increase in the number of international technical professionals in the WCO is a welcome initiative, and there is also a need for appropriately skilled national professional officers (NPOs) in support of technical issues but also to facilitate discussions when language barriers are an issue. This can also be supplemented with locally sourced translation services. The rationale for organizing teams around programme categories, rather than around the CCS priorities plus communicable diseases, remains a challenge for the WCO. Considering the weaknesses in planning and monitoring observed during the CCS 2012-2016 and the expectations from national counterparts in this area, the WCO needs to ensure that it can very quickly mobilize adequate levels of expertise in this area, either through the support of HQ or RO colleagues, or through short-term experts.

Best practices and innovations. This evaluation highlighted a certain number of emerging good practices and innovations framing WHO’s engagement in Thailand. Indeed the WCO’s role has clearly evolved during the period evaluated and is continuing to do so. For instance, the approach taken in the design of the CCS 2012-2016 and the lessons learned strengthened the design of the CCS 2017-2021, highlighting partnerships with national actors beyond the health sector and instituting a transparent and consultative priority setting process for the CCS 2017-2021. The fact that the RTG has increased its funding to become the main funding source for the CCS 2017-2021 also represents a major shift in its collaboration with WHO. Finally, the new funding mechanism being explored could also be used, if proven effective, in the future in other similar countries.

Page 12: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

iv

Recommendations

On the basis of the above analysis, the evaluation would like to make the following recommendations:

A. The Head of the WHO Country Office and the WHO Country Office team to contribute actively to Country Cooperation Strategy governance activities and to engage with other national partners to support implementation of Country Cooperation Strategy priorities and activities, in particular in the area of programme management and monitoring.

i. Review the Country Cooperation Strategy workplans for each priority and define targets (qualitative or quantitative) for both the expected outcome and output levels and clarify expected WHO contribution in a measurable manner.

ii. Ensure adequate technical capacity for planning and monitoring Country Cooperation Strategy implementation.

B. The WHO Secretariat to ensure that the WHO Country Office has the capacity to implement its workplans beyond the Country Cooperation Strategy priorities and activities, including through appropriate funding mechanisms and staffing of the Office.

i. Ensure that new Country Cooperation Strategy priorities such as antimicrobial resistance are adequately covered with financial and human resources

ii. Ensure that language is never a barrier for the active engagement of the WHO Country Office with national partners.

iii. Headquarters and the Regional Office to support the WHO Country Office in the review and consideration of the Royal Thai Government’s request to support the implementation of the Country Cooperation Strategy 2017-2021 through the national pooled funding mechanism, and explore the possibility of linking a pooled funding mechanism with indicators of achievement.

C. The WHO Country Office to build on a Theory of Change for the period 2017-2021 to better link the Country Cooperation Strategy 2017-2021 with the entire planned country-level results and deliverables and with the Country Office staff and activity workplans during operational planning for Programme budgets 2018-2019 and 2020-2021.

i. Develop a Theory of Change for 2017-2021 to frame more specifically the pathway for change (it should include all Country Office activities, not only those of the Country Cooperation Strategy).

ii. Clarify for each relevant corporate output the targets relevant for Thailand in the current biennium and for each biennium thereafter.

iii. Set up an internal monitoring framework to measure WHO’s progress towards targets over the Country Cooperation Strategy implementation period.

D. The WHO Country Office and the Royal Thai Government to strengthen inclusion of the gender and other social determinants of health dimension(s), as relevant, in the implementation of the Country Cooperation Strategy and other Country Office activities.

i. Review programmes of work of each Country Cooperation Strategy priority with a gender lens, possibly with the support of the Regional Office or of headquarters, and amend as necessary to ensure the gender dimension is appropriately taken into consideration.

E. The WHO Secretariat (Department of Country Cooperation and Collaboration with the UN System and the Country Support Unit network) to review the evolution of the Country Office’s contribution to, and relationship with, the Royal Thai Government over the recent Country Cooperation Strategy cycles, in order to consider the lessons learned, innovation and best practices for Country Office interaction with, and contribution to, other upper-middle-income countries and emerging economies.

Page 13: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

v

i. Reflect further on the implications of the expectations of counterparts in terms of social capital, in particular with other Country Offices active in upper-middle-income countries. WHO should deploy experts with profiles and experience matching the Country Cooperation Strategy priorities.

ii. Develop a strategic note framing WHO’s engagement in upper-middle-income countries from the intellectual and social capital perspective.

iii. Document the prioritization process followed for the Country Cooperation Strategy 2017-2021 and share within the Organization.

Page 14: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

1

1. Introduction

1. COEs were included in the Organization-wide evaluation workplan for 2016-2017, approved by the Executive Board in January 2016. The workplan clarifies that COEs “will focus on the outcomes/results achieved by the country office, as well as contributions through global and regional inputs in the country. In addition these evaluations aim to analyse the effectiveness of WHO programmes and initiatives in the country and assess their strategic relevance within the national context”. They encompass the entirety of WHO activities during a specific period. The COEs aim to provide findings, recommendations and lessons that can be used in the design of new strategies and programmes in-country.

1.1 Evaluation features

2. Purpose. This COE was the first of this type undertaken by the WHO Evaluation Office. Its main purpose was to identify and document best practices and innovations of WHO in Thailand on the basis of its achievements. These include not only results of the WCO but also contributions at the regional and global levels to the country programme of work. As with all evaluations, this COE meets accountability and learning objectives endorsed by the Executive Board of the World Health Organization. It will be publicly available and reported on through the annual Evaluation Report.

3. Objectives. This evaluation built on the results of previous evaluative work to:

a. Demonstrate achievements against the objectives formulated in the CCS and other relevant strategic instruments; and corresponding expected results developed in the WCO biennial work plans, while pointing out the challenges and opportunities for improvement.

b. Support the WCO and partners to operationalize the various priorities of the next CCS (and the relevant strategic instruments) based on independent evidence of past successes, challenges and lessons learnt.

c. Identify best practices emerging from the unique relationship between the RTG and WHO. These can then usefully inform the development of future country, regional and global support through a systematic approach to organizational learning.

4. Expected use. The main expected use for this evaluation is to support the WCO as it operationalizes the new CCS 2017-2021 and develops its next biennial work plan. It should also assist the RO and HQ when deciding how best to support to the WCO. Finally, over the medium-term, it will contribute to build a body of evidence around possible systemic issues to be addressed corporately, such as the development of models of WCOs work/presence in upper-middle-income countries.

5. Scope. The evaluation covered the period 2012-2016 and included all contributions from the WCO in Thailand, the RO and HQ over the same period. It focused on WHO’s contribution to the objectives and the expected results defined in the CCS and the biennial country work plans as a whole rather than individual activities which have taken place during the period evaluated.

6. The CCS document for the period 2012-2016 and the new one for the period 2017-2021 (drawn up during 2016) served as the references to frame the evaluation scope but were not the only ones. All other strategic contributions made by WHO wee also included.

7. Evaluation questions. All COEs address the 3 main evaluation questions (EQ) identified below. The sub-questions are then tailored according to country specificities and detailed in an evaluation matrix (see Annex 2).

Page 15: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

2

EQ1 - Were the strategic choices made in the CCS (and other relevant strategic instruments) the right ones to address Thailand’s health needs and coherent with government and partners’ priorities? (relevance). This question assesses the strategic choices made by WHO at the CCS design stage and its flexibility to adapt to changes in context. This question assesses both the CCS 2012-2016 and the new CCS 2017-2021 design.

EQ2 - What is the contribution/added value of WHO towards addressing the country’s health needs and priorities? (Effectiveness/elements of impact/progress towards sustainability). To address this question, the evaluation used the results per programme area already presented in the CCS 2012-2016 mid-term review and CCS final evaluation and focused on the best practices and innovations observed.

EQ3 – How did WHO achieve the results? (efficiency) In this area the evaluation sub-questions cover the contribution of the core functions, the partnerships and allocation of resources (financial and staffing) to deliver the expected results and, for each, seek to identify best practices and innovations.

1.2 Methodology

8. Guided by the WHO evaluation practice handbook, the evaluation was based on a rigorous and transparent methodology to address the evaluation questions in a way that serves the dual objectives of accountability and learning. The methodology (summarized in Figure 1 below and developed further in Annex 2) ensured impartiality and lack of bias by relying on a cross-section of information sources (from various stakeholder groups) and using mixed methodological approaches (e.g. quantitative and qualitative data) to ensure triangulation of information through a variety of means.

Figure 1: Methodological approach

9. The evaluation was conducted between January and June 2017 by a core team of four members. The WHO Evaluation Office formed a team led by Dr Elil Renganathan (DG Representative for Evaluation and Organizational Learning) and supported by Anne-Claire Luzot (Chief Evaluation

Page 16: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

3

Officer), Dr Mohamed Jama and Kathryn Tyson (consultants). During the inception phase, the team reconstructed the TOC (see Figure 2) framing WHO’s engagement in-country, which was then validated by the Head of the WCO and the WCO team during the field mission. The TOC is aligned with the one validated by WHO in the context of the evaluation of WHO’s presence in countries.1 Using the TOC, the team developed an evaluation matrix, unpacking for each evaluation question of the Terms of Reference (TOR) specific indicators/measures for assessing each sub-question, the data collection method and data sources used. The evaluation mainly used existing data collected by WHO and partners during the timeframe evaluated. Therefore during the data collection phase, the team, after a comprehensive document review, conducted a one-week mission in-country and a large number of interviews (list available in Annex 5) with WHO colleagues at the three levels of the Organization as well as with all main partners in-country. All the data were then analysed to produce the present report.

10. The evaluation ensured that gender, equity and human rights issues were addressed to the extent possible and through several means. A number of sub-questions within the evaluation matrix were gender sensitive with appropriate related indicators. The document review paid specific attention to how these issues were addressed at planning, implementation, monitoring and evaluation stages of WHO contributions. Finally, these dimensions have been reflected in the interviews.

11. The evaluation encountered a few other relevant issues already identified to some extent in the CCS mid-term review and in the CCS final evaluation and further elaborated in Annex 2:

Though there are broad linkages between the CCSs and other WHO corporate planning and reporting tools, these are not clear enough to identify outputs and outcomes specific to the CCSs within the WCO work plans.

In the absence of a clear TOC or of a logical or result framework, the corporate outcomes and outputs defined in the Programme budget (PB) are not systematically translated at country level with corresponding benchmarks and quantified targets.

Considering that the expected contribution of WHO to national programmes prioritized in the CCSs is not systematically identified at the planning stage, it was challenging to establish the extent to which activities undertaken contribute to the achievement of objectives defined in national programmes, plans or strategies.

1.3 Country context2

12. Despite having undergone much social and political unrest during the last decade, with the Coups d’Etat in 2006 and 2014 and very severe floods in 2011, Thailand has enjoyed sustained economic growth and social development during the last several decades thanks to, among other things, the implementation of national policies and strategies articulated in successive National Economic and Social Development Plans (NESDP) and, in the case of the health sector, the National Health Development Plan (NHDP) which is a subset of the national development plan. The country has made remarkable progress and is, since 2011,3 an upper-middle-income country in the high development category with a human development index of 0.726 (for both men and women) in 2014.4

13. Though Thailand has achieved many of the global Millennium Development Goals (MDGs), some challenges remain, especially at sub-national level, and the Government has expressed strong commitment to achieving the SDGs. Poverty and inequality have declined substantially over the past

1See WHO, 2015, Evaluation of WHO’s presence in countries.

2 The information in the section is mainly summarized from the WHO Country Cooperation Strategy Thailand 2017-2021.

3 http://www.worldbank.org/en/news/press-release/2011/08/02/thailand-now-upper-middle-income-economy (accessed

20 March 2017). 4 http://hdr.undp.org/sites/all/themes/hdr_theme/country-notes/THA.pdf (accessed 20 March 2017).

Page 17: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

4

decades. However, 11% of the population still remains below the poverty line and the poorest 10% of the population accessed only 1% of the country total income in 2013.5 Thailand promotes gender equality, including for career development, jobs and income, but women remain significantly underrepresented in public decision-making roles even though the national women’s development plan (2012-2016) has set ambitious goals to address this issue.

14. Thailand has shown concrete achievements in health over the last 25 years (see Table 1). Improvement in the health of the Thai population, as measured by health indicators such as maternal mortality ratio per 100 000 live births and infant mortality rates per 1 000 live births, have seen dramatic decline while life expectancy has risen to 79 years for females and 72 years for males.

15. NCDs (71% of deaths in 2014)6 and road traffic injuries are well-recognized challenges. The health of migrants (about 4 million migrants in the country)7, though recognized as an issue, does not figure prominently in the national health agenda. Antimicrobial resistance (AMR) is an emerging problem receiving national attention, while other issues, such as tuberculosis, malaria, and HIV/AIDS, climate change and adolescent health, require continued attention.

16. Universal health coverage for Thai citizens (migrant population not fully covered yet) was achieved in 2002 and since then public expenditure on health has increased consistently to represent 86% of total health expenditure in 2014.

Table 1: Selected population and health indicators

Source: CCS 2017-2021

17. Thailand has taken solid steps to strengthen its health system through a dedicated policy framework, increased public funding for health and improved performance of health services delivery. Being an upper-middle-income country, Thailand’s health system relies mainly on domestic funds; donor or development partners represent less than 0.5% of total health expenditures, mainly focused on technical support in few highly specialized areas.

5 http://www.worldbank.org/en/country/thailand/overview (accessed 20 March 2017).

6 http://www.who.int/nmh/countries/tha_en.pdf

7 National Health Commission of Thailand, Royal Thai Government 2012.

8 Global Health Observatory http://apps.who.int/gho/data/node.country.country-THA

Indicators 1990 2000 Latest available statistics

Total population (x 1 000) 54 548 62 056 67 959 (2015)

Population < 15 years (%) 17.7 (2016)

Population > 60 years (%) 7.4 9.2 16.5 (2016)

Population in urban areas (%) 19 35 48 (2016)

Life expectancy Female Life expectancy Male

68.8 63.5

75 70

78.6 (2016) 71.8 (2016)

Fertility rate (births per woman) 2.14 1.82 1.6 (2016)

Contraceptive prevalence rate (%) 79.3 (2012)

Infant mortality /1 000 live births 35 25 6.4 (2013)

Maternal mortality /100 000 live births 44.5 (2003) 20 (2015)8

Deliveries attended by health staff (%) 90.8 99.6 (2012)

Total health expenditure (THE) as proportion of Gross Domestic Product

3.5% (1994) 3.4% 6.5% (2014)

Public expenditure as a proportion of THE 45% (1994) 56% 86% (2014)

THE per capita (USD) 86 (1994) 67 256 (2012)

Page 18: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

5

18. There are 17 independent public health agencies9 outside the MOPH that wield considerable influence in both shaping the health policy options of the country and the subsequent priority setting which has a huge impact on the health of the Thai people. These agencies are financed by public funds.

1.4 WHO activities in Thailand

19. WHO has been present in Thailand since its inception and, since 2002, the cooperation between WHO and the RTG has been framed through a series of CCSs. In the preparation of the CCS for 2012-2016, it was decided to move away from the very fragmented approach in the CCS 2008-2011, and focus on 5 key health priorities where WHO could add value to the national efforts. The CCS identified additional collaborative programmes in the area of normative functions, a selected number of communicable diseases and Thailand’s role in health beyond borders.

20. The CCS 2012-16 includes four clusters of activities: i) five priority areas;10 ii) normative functions; iii) major public health challenges and unfinished agendas; and iv) support to Thailand’s role in health beyond its border. In addition, the WCO is engaged in activities outside the CCS framework.

21. Table 2 below clarifies the main areas of cooperation of both the CCS 2012-2016 and 2017-2021. Reference is also made to the CCS 2008-2011 to provide a longer-term perspective. It shows clear continuity in some priority areas, such as NCD and international trade and health, while others, such as global health diplomacy and AMR, have become priorities more recently. It also shows that some priorities of the CCS 2012-2016 have been discontinued, such as disaster preparedness and response and ageing.

Table 2: WCO main activities across 3 CCS periods

Main activities 2008-2011 2012-2016 2017-2021

Community health system Included in CCS CCS Priority until 2013

Multisectoral networking for NCD control

Included in CCS CCS Priority

CCS Priority

Disaster preparedness and response

CCS Priority

International trade and health CCS Priority CCS sub-programme of priority on global health diplomacy

Road safety CCS Priority CCS Priority

Ageing CCS priority included in 2013-2014 when priority on community health system was sunsetted

Border and migrant health Included in CCS CCS activity prioritized when priority on community health system was sunsetted

CCS Priority

International health regulations (IHR)

CCS activity Other WCO activity

Communicable diseases Included in CCS CCS activity Other WCO activity

Global health diplomacy CCS Activity CCS Priority

AMR Other WCO activity CCS Priority

9 WHO, 2012, Country Cooperation Strategy for Thailand 2012-2016.

10 Annex 3 provides a short summary of the CCS priorities as described for both the CCS 2012-2016 and the CCS 2017-2021.

Page 19: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

6

22. The TOC11 (Figure 2) clarifies WHO’s contribution to the national health objectives and goals in terms of health outcomes and potentially the health impact of its collaborative programmes with the Government of Thailand, as defined in the CCS 2012-2016 and the biennial work plans. It encompasses contributions from all levels of the Organization and all strategic contribution areas of WHO in the country. The TOC was reconstructed by the evaluation team during the inception phase, then validated by the Head of the WCO and WCO team during the field mission.

Figure 2: Theory of Change – WHO contributions in Thailand 2012-2016

23. The diagram below indicates the level of investment in the various activities over the period 2012-2016. The highest financial investment went to border and migrant health followed by NCDs and communicable diseases (even though not a CCS 2012-2016 priority).

24. The overall WCO expenditure (activities and staff costs) 12 for the period 2012-2016 amounted to US$ 20,7 million. The main sources of funding13 over the period were assessed contributions (67.3%), the European Commission (8.3%), the Bloomberg Family Foundation (2.7%), the RTG (1.9%), the United States Agency for International Development - USAID (1.7%), and the United States Centers for Disease Control and Prevention - US CDC (1.4%). Both US CDC and USAID stopped their funding in 2016, while the RTG contributions started in 2014.

11

Theory of Change is a description and illustration of how and why a desired change is expected to happen in a particular context. It is focused in particular on mapping out or “filling in” what has been described as the “missing middle” between what a program or change initiative does (its activities or interventions) and how these lead to desired goals being achieved (for further details see http://www.theoryofchange.org/what-is-theory-of-change/) 12

Data on expenditures and funding were extracted from the WHO Global Management System. 13

Data on funding sources were extracted from the WHO Global Management System.

Page 20: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

7

Figure 3: Estimated14 proportion of WCO expenditures 2012-2016 for main activities (activity & staff costs combined)

Source: WHO Global Management System (data extracted in 2017)

25. The CCS 2012-2016 was the object of a CCS mid-term review in 201415 and of a CCS final evaluation 16 in 2016, the main conclusions and recommendations of which are summarized in the table below.

Table 3: Main recommendations of evaluative exercises related the CCS 2012-2016

Mid-term review

Internal review process with partners prior to next biennium plans

Continuing evaluation of the priority programmes

More active involvement of the MOPH in priority programmes

Application of the priority programme approach

Flexible approach to CCS by WCO thereby accommodating new emerging priorities within the CCS framework

CCS final evaluation

Clear development process to select new priority programmes for the next CCS and careful identification of lead agencies to manage them

Multisectoral committees to be continued

Importance of partnerships beyond MOPH

Lighter management process for the CCS needed

Technical assistance to be identified early on in the process

Staff turnover is an issue to be considered if use of subcommittees is to be pursued

Considering the social determinants of health, multisectoral work is indispensable and justifies the added complexity in programme implementation

26. The RO and HQ also have direct relationships with Thai entities, for instance WHO collaborating centres (currently 34 in Thailand). These include, among others, Government entities such as departments within the MOPH, academia and civil society organizations. These relationships lead to activities outside of the WCO work plans, usually undertaken with minimum involvement and knowledge of the WCO.

14

Estimated by the evaluation (see section 2.2 first sub-section for further explanation). 15

WHO, 2014, ‘Mid-term review of the WHO Country Cooperation Strategy -Thailand 2012-2016’. 16

WHO, 2016, ‘Final Evaluation of the WHO country Cooperation Strategy Thailand 2012-2016’.

Page 21: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

8

2. Findings

27. The findings of the evaluation are presented following the three main evaluation questions and sub-questions identified in the TOR (see Annex 1 for the full list). More detailed observations per CCS priority are available in Annex 4.

2.1 Relevance of WHO’s strategic choices

Relevance of WHO’s strategic choices in view of Thailand’s population health needs

28. Both the 2012-2016 and 2017-2021 CCSs include, as per the corporate guidelines, an overview of the health situation and trends in the country. A comprehensive consultation process with all stakeholders and a thorough review and analysis of the social, economic and health indicators contained in the background documents were conducted. This process has allowed a better understanding of the health challenges and needs of the Thai population. The CCS 2017-2021 is also informed by a comprehensive health needs assessment undertaken in 2016, to enlighten the selection of its priorities.17 According to this needs assessment and based on the MDG report for Thailand,18 it is clear that, while the country made significant progress towards achieving the MDGs, disparities across regions and social groups remain, requiring further effort within the SDG framework. Overall, reference to gender remains largely limited to sex disaggregated data, which is a first step but the CCS guidance calls for further gender analysis as well as analysis of health equity and human rights.

29. NCD, included in the CCS since 2008, is a critical health issue in Thailand as NCD deaths accounted for 71% of deaths in Thailand in 2014.19 According to WHO’s 2015 global report on road safety, Thailand has the second highest incidence of road traffic fatalities in the world and both the CCS 2012-2016 and the CCS 2017-2021 identified road safety as a priority. AMR is becoming very problematic worldwide and is now a priority of the CCS 2017-2021.

30. In addition to refugees from Myanmar, Thailand is the primary host country for low-skilled workers from three neighbouring countries. While initially not a priority for the RTG, due to significant concern among some partners, including WCO, border and migrant health was part of the strategic agenda of the 2008-2011 CCS, then part of the CCS 2012-2016, first as an activity, then a priority when the community health programme was sunsetted. It is again a priority in the CCS 2017-2021.

31. Despite positive trends in terms of reducing malaria incidence and the elimination of mother-to-child transmission of HIV and syphilis, Thailand remains one of the 30 high burden countries for tuberculosis.20 Communicable diseases, which were a priority in the CCS 2008-2011, became an activity in the CCS 2012-2016 and are not included in the CCS 2017-2021. Some parts of MOPH consider that Thailand can now take care of this issue by itself while others, including key partners outside Government, consider it important to continue benefitting from WHO’s technical support. The WCO indicated a clear continued commitment to providing high level international technical expertise particularly in the area of multi-drug resistant tuberculosis.

17

WHO, 2016, ‘Needs assessment for the selection of priorities for the Thailand-WHO Country Cooperation Strategy 2017-2021’ draft for comments and discussion. 18

Office of the National Economics and Social Development Board, 2015, ‘Millennium Development Goals Thailand 2015’. 19

WHO, 2014, ‘Noncommunicable disease country profiles’. 20

WHO, 2016, ‘Global tuberculosis report – 2016’.

Page 22: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

9

Box 1 - Good practice - definition of CCS 2017-2021

priorities based on predefined criteria:

- Contribute to the NESDP strategies

- Address public health issues identified by the MOPH and

the UNPAF

- Contribute to achieving targets of WHO leadership and

flagship priorities

- Contribute to targets of SDG 3

- Support achievement of current CCS

Coherence with the national health priorities and the relevant MDG and SDG targets

32. The CCS 2012-2016 was developed after comprehensive consultations with the MOPH, 17 public health agencies, line ministries, academia, nongovernmental organizations, civil society organizations and relevant UN agencies. As the CCS 2012-2016 was developed at the same time as the NHDP, WHO closely coordinated with the national team responsible for its formulation to ensure that it informed the CCS 2012-2016 document adequately.

33. This approach was further strengthened for the CCS 2017-2021 as follows:

The objective was defined from the onset to have at most five or six priorities included in the CCS, building on the CCS 2012-2016 experience

75 stakeholders from inside and outside the MOPH were consulted

38 comprehensive proposals on potential priorities to be included in the CCS 2017-2021 were reviewed on the basis of predefined criteria.

34. Consequently, and as also shown in Annex 3, the priorities of both CCSs are well aligned with MDG and SDG targets, the NHDPs and with other national policies or strategies as relevant. For instance, the border and migrant health component was well aligned with the MOPH Border Health Development Master Plan 2012-2016, the priority on NCDs is aligned with the Thai Healthy Lifestyle Strategic Plan 2011-2020, AMR with the national strategic plan on AMR and global health diplomacy with the national global health strategic framework issued by the MOPH and the Ministry of Foreign Affairs.

Coherence with the UNPAF

35. All UN agencies have the obligation to coordinate with the UN-wide country programmes for Thailand under the umbrella of the UNPAF 2012-2016,21 which has a very broadly defined result framework to accommodate all participating agencies. In this regard, WHO identified three relevant outcomes where joint action and synergy was envisaged: climate change; capacity building and promotion of Thailand as a global partner; and the area of social reform with particular focus on health equity. It should also be noted that the CCS 2012-2016 priorities are recognized as part of the UNPAF alongside those of other agencies.

36. Effective collaboration within the UNPAF is partially hampered by the different accountability and governance structures and mandates of each UN agency, which require them to report back to their respective governing bodies.

37. UN Partners report that strategic engagement of WHO within the UNPAF framework and thematic working groups is somewhat limited. There is a perception that WHO remains too focussed on health issues. Also, the geographic location of the WCO in the MOPH and away from the other UN agencies contributes to limited participation in the various UNPAF working groups and a perception on the part of the other agencies of the very specialized role of WHO in health.

38. However, partners also clearly recognize WHO’s technical leadership and convening power, for instance in the area of border and migrant health and communicable diseases where effective collaboration with IOM, UNAIDS, UNFPA and UNICEF was reported.

21

United Nations Country Team in Thailand, 2011, ‘Partnership Framework (UNPAF) for Thailand 2012 -2016’.

Page 23: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

10

Box 2 - Good practice – Validation matrix aligning CCS strategic priorities with national, WHO, United Nations and global priorities In the CCS 2012-2016, each priority was associated with

several strategic objectives of the MTSP (see further details

in Annex 4). The CCS 2017-2021, as suggested in the CCS

2016 Guide, includes a validation matrix which establishes

linkages between each CCS priority and a specific GPW

outcome and also with the national SDG targets, the

NHDP, the UNPAF outcomes and the Regional Flagship

areas.

39. With the UNPAF 2017-2021 there is a need for a strong strategic and policy level positioning of the UN agencies in the country which can be a challenge for agencies when positioned mostly at technical level. There is a clear call from partners for WHO to engage more substantively and strategically in support of the SDGs.

Coherence with the General Programme of Work

40. The CCS 2012-2016 coincided with the last 2 years of the 11th General Programme of Work (GPW).22 The five priorities in the CCS were aligned with the strategic objectives articulated in the 11th GPW and explicit linkages were made in the document between each priority and several strategic objectives of the Medium-Term Strategic Plan (MTSP). 23 Since the 12th GPW24 took on board most of the priorities identified in the previous GPW, there was a good alignment between the CCS 2012-2016 and the 12th GPW. Similarly the CCS 2017-2021 priorities are explicitly linked to specific outcomes of the 12th GPW. These priorities also conformed with WHO international commitments as expressed in World Health Assembly (WHA) resolutions.

Support to gender equality and the empowerment of women

41. When it comes to gender there is a clear shift between the two CCS documents. While gender equality and empowerment of women were only addressed in a limited manner in the CCS 2012-2016 document, they were given substantial attention in the analysis of health-related issues in CCS 2017-2021. However, there is as yet no clear advocacy for these issues in the strategic agenda implementation.

WHO’s evolving approach

42. The health sector in Thailand is characterized by a large number of actors from both the public and the private sector. Several other public health agencies with autonomous or semi-autonomous status operate side by side with the MOPH, as well as 34 WHO collaborating centres and 35 other institutions considered as centres of excellence.25

43. The CCS 2008-2011 implementation was highly fragmented with many different small projects across the MOPH. This changed with the CCS 2012-2016 and the identification of five priorities from 21 concept papers initially submitted. Seventeen major public health agencies voted and identified four priorities. WHO added the fifth one, on road safety. The MOPH led one of the priorities, three health agencies and one WHO centre of excellence were identified to lead the others. A subcommittee was created to manage each priority as well as a steering committee to govern the entire CCS 2012-2016. This represented another shift from the CCS 2008-2012 to foster multi-agency collaboration. It implied new governance mechanisms which worked with varying degrees of success across the priorities. For instance, the international trade and health programme lead agency developed solid workplans and the subcommittee widened the influence of the programme beyond MOPH. On the other hand, the community health programme was stopped after

22

WHO, 2005, ‘11th

General Programme of Work 2006-2015’. 23

WHO, 2007, ‘Medium-Term Strategic Plan 2008-2013. 24

WHO, 2013, ‘12th

General Programme of Work 2014-2019’. 25

See CCS 2012-2016.

Page 24: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

11

the lead agency withdrew from the programme. The choice of the lead agency is critically important, needing strong leadership and convening capacity as well as a highly competent programme manager to lead the programme implementation. The CCS 2012-2016 final evaluation made a certain number of recommendations in that regard which have largely been taken on board in the design of the CCS 2017-2021.

44. The prioritization process was further strengthened for the CCS 2017-2021. Following a comprehensive consultation process, five priorities plus one sub-programme were selected on the basis of predefined criteria (see Box 1) out of 38 proposals. The lead agencies for the CCS 2017-2021 include the MOPH, health agencies and one WHO collaborating centre and more than 66 ministries, agencies and organizations will be participating directly in its implementation. This time around the CCS reached out to other ministries, making it a truly multisectoral approach from the onset. The implementation of the CCS 2017-2021 will therefore require very strong governance and financing26 mechanisms reflecting increasing country ownership.

45. Considering the constantly growing strength of the health sector in Thailand, WHO’s role is becoming gradually more that of a catalyst, for instance, bringing together traditional health actors with other non-health actors on the social determinants of health, who had not been included in the previous CCSs.

46. WHO was also instrumental in highlighting, in the CCS 2017-2021, health-related issues which might not have originally been MOPH priorities, such as road safety or border and migrant health; and in ensuring that health concerns that are not prioritized by the MOPH as a priority for WHO support within the CCS continue to benefit from WHO technical support (e.g. tuberculosis).

Strategic positioning of WHO when it comes to…

47. … WHO’s comparative advantage. This was one of the criteria used to prioritize programmes within the CCS 2017-2021 (see Box 1). WHO is highly valued for :

the access it provides national partners to international regional or corporate health technical expertise;

the exposure it gives Thai health experts to regional and international health issues;

its “brand” name and the steer it can provide to national initiatives;

its convening capacity, especially for health-related issues requiring collaboration and partnership with non-health actors in-country.

48. These are often presented as intellectual and social capital by the RTG (see para 52 below).

49. The above elements are much more important to the RTG partners than the funding that WHO can bring into the country. The funding from the RTG will increase from less than 2% in the CCS 2012-2016 to about 70% in the CCS 2017-2021. In this context WHO is expected to play a catalytic and proactive role and support :

the governance of the programmes as they require engagement with, and coordination of, numerous partners in various sectors

the CCS 2017-2021 at a policy and strategic level and not just at the technical level as in the past.

50. … Positioning health priorities in the national agenda. As mentioned earlier, border and migrant health as well as road safety are recognized as two areas which have consistently been prioritized by WHO when designing the CCSs. WHO also remains committed to support the unfinished agenda related to communicable diseases. However, the evaluation did not find any strong evidence of WHO’s proactive engagement supporting gender equality and empowerment of

26

Financing mechanism discussed in section 2.3 with funding

Page 25: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

12

women.

51. Partnership between WHO and the RTG. Historically, the MOPH was the only WHO counterpart. With the emergence of numerous health agencies and in order to reach out to non-traditional health actors, WHO is increasingly playing a convening and coordination role between the various national partners in country. Partnership is a key element of CCS 2012-2016 and this is confirmed in the CCS 2017-2021.

52. ... WHO’s contribution in terms of intellectual and social capital. Many national partners indicated clear expectations with regard to WHO’s strategic contribution in this respect. In their opinion, intellectual capital broadly refers to WHO’s leading role in health technical expertise, while social capital refers more to WHO’s reputation, influence, authority, name and trust. The CCS 2012-2016 benefitted from WHO’s contribution in terms of intellectual capital and the CCS 2017-2021 design depended to some extent on WHO’s social capital to facilitate the transparent consultation and selection of priorities process among a very large group of national partners from health sector and beyond.

Summary of key findings Overall, both CCSs are based on critical health issues for the country, especially in the area of

NCDs and road safety. The CCS 2017-2021 prioritizes AMR, a new major global health issue, and border and migrant health. Though no longer a CCS priority, a remaining area of concern is tuberculosis, considering Thailand is one of the 30 remaining high burden countries.

The priorities of both CCSs are well aligned with MDG and SDG targets, the NHDPs and with other national policies or strategies, as relevant. Both CCSs benefitted from a design process ensuring alignment with national health priorities through extensive consultations. The CCS 2017-2021 selection of priorities was based on transparent criteria.

The CCS is an innovative model mobilizing both the financial resources and social intellectual capital of WHO and Thailand’s key health agencies to support the implementation of national health priorities. Its basic principles are aligned with Paris Declaration on Aid Effectiveness, based on single program development management and reporting.

The CCS 2012-2016 is coherent with the UNPAF 2012-2016 which provides a large umbrella framework for all UN agencies. Technical collaboration with other UN agencies is perceived positively by partners, recognizing WHO’s specialized role. But there are also strong expectations that WHO will position itself at a more strategic and policy level within the UN country team with the start of the UNPAF 2017-2021.

Both CCSs were aligned with the prevailing GPWs at the time of their design and included explicit linkages with specific corporate objectives and outcomes.

Reference to gender remains limited to sex-disaggregated data for some indicators. Indeed, gender equality and empowerment of women issues were not addressed in the CCS 2012-2016, and only included in the analytical part of the CCS 2017-2021.

Since 2012 a strategic shift occurred to change the CCSs from very fragmented support to numerous projects within MOPH, to a clearly prioritized catalytic approach developed through a comprehensive and transparent consultative process.

The choice of the lead agency is critically important, needing strong leadership and convening capacity as well as a highly competent programme manager to lead the programme implementation. The implementation of the CCS 2017-2021 will require very strong governance and financing mechanisms.

WHO also pushed for some issues such as road safety and border and migrant health to be consistently included in the CCSs, though initially not necessarily a priority for the RTG.

WHO’s comparative advantage, namely, in technical areas and as convening power, is well recognized. However, the RTG also expects WHO to play a catalytic role at policy and strategic multisectoral levels.

Page 26: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

13

Partnerships were at the core of the CCS 2012-2016 and are reconfirmed in the CCS 2017-2021. The focus of the CCS 2012-2016 was mainly on advancing coordination and networking among

the many health partners, both national and international, with the aim of building the delivery capacity of the public health agencies and other partners to the Thai population to improve health outcome. The CCS 2017-2021 takes the WHO collaborative programme to the next stage where the CCS articulates clear deliverables under each of the five priorities. Measurable indicators are still under development.

There are strong expectations from the RTG when it comes to WHO’s contribution in terms of intellectual and social capital. It means WHO needs to remain on the cutting edge technically and consolidate its emerging social capital role.

2.2 WHO’s contribution and added value (effectiveness)

Articulation of the WCO biennial work plans with the focus areas as defined in the CCS 2012-

2016

53. As shown in Figure 4, the CCS 2012-2016 timeline overlapped with the MTSP and the 12th GPW timelines. While the CCSs for Thailand are for a period of five years, the workplans are organized per biennium, which coincided with the start of CCS 2012-2016, but CCS 2017-2021 starts in the middle of a PB and GPW. Though unavoidable, as CCSs can have different durations depending on national and UN in-country planning time frames, this makes direct alignment of workplans and reporting with the CCS challenging. As mentioned earlier, the CCS timeframe is aligned with the UNPAF timeframe.

Figure 4: Timeframes of key planning instruments at the different levels of the Organization

54. As currently available in the WHO Global Management System, the workplans at country level are all explicitly connected with the corporate outputs as defined in the PB. The 2012-2013 workplan is connected to the MTSP while the subsequent workplans are connected to the 12th GPW outcomes and outputs. The CCS document however links each priority to several strategic objectives of the MTSP. Therefore, as explained in Annex 3, there is a missing link between workplans drafted at country level and the strategic priorities established in the CCS. The WCO has not explicitly translated the PB corporate outcome and output targets as measurable outcomes and outputs at country level nor has it developed any clear results framework (or logical framework or impact pathway or theory of change) for each of the priorities of the CCS, identifying its expected contribution to each priority for the CCS period with corresponding baselines and targets.

Page 27: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

14

55. It was only by manually mapping the tasks included in the workplans against the CCS 2012-2016 priorities that the evaluation was able to assess which tasks and expenditures could be allocated to each CCS 2012-2016 priority. This mapping was used to inform both estimates on planned costs and expenditures and the analysis of results presented below.

56. According to this mapping, the evaluation estimated that CCS 2012-2016 priorities and activities represented more than two thirds of the WCO activity expenditures over the CCS period. The rest of the activity expenditures are mostly related to strategic activities of the WCO and the running of the office.

Main results achieved27

CCS 2012-2016 priorities

57. Community health system. Unanimously ranked top priority during the planning process of the CCS 2012-2016, it was sunsetted following governance issues.28 The CCS mid-term review reported a certain number of activities completed but no major results achieved during the period of implementation of about 18 months.

58. Multisectoral networking for NCD control. NCD is a continued priority for both the RTG and WHO. Progress was noted both in the CCS mid-term review and in the CCS final evaluation and confirmed through interviews, such as: the inclusion of four risk factors and diseases in the integrated national NCD plan in order to have it aligned with the global NCD plan approved by the WHA in 2013; the development of the first national NCD guidelines; and the formalization, in 2016, of the NCD alliance of Thailand (a network of academia, NGOs and professional bodies). However, the CCS final evaluation also found that the main objective of building a multisectoral network for NCDs to facilitate the implementation of the Thai Healthy Lifestyle Strategic Plan was not achieved. WHO’s research mainly supported national policy development on tobacco control. Overall, the results achieved during the CCS 2012-2016 are perceived by most stakeholders as good foundations for the CCS 2017-2021.

59. Disaster preparedness and response. There was a shift in the objectives identified in the CCS during the planning process from establishing a network on disaster health emergency management, to mainstreaming disaster risk reduction in the health sector. According to the CCS mid-term review, WHO contributed mostly to building a momentum in engaging MOPH, the National Institute for Emergency Medicine (NIEM) and other stakeholders in enhanced focus and work on disaster preparedness and response. Finally the MOPH developed a disaster response plan for people with disabilities with the support of WHO, placing Thailand among the first countries in the world to have such a plan.

60. International trade and health. This was a new priority of the CCS 2012-2016. According to the CCS mid-term review, the initial major outcomes are the collaborative engagement of health and non-health government officials and policy makers and the enhancement of capacities of all partner organizations related to the interface between international trade and health. The CCS final evaluation confirmed that the programme had achieved most of its objectives while recognizing that impact on trade agreements could not be documented considering the natural opacity of trade negotiation processes. WHO technical and policy contributions in the promulgation of laws and

27

Considering the limitations identified in previous section, the evaluation assessed progress for each of the four main groups of outcomes as presented in the TOC (Figure 2) but was not able measure them against planned targets as they were not identified in a measurable manner. 28

According to the CCS mid-term review, the MOPH issued the Ministerial order 272/2526, dated 10 February 2013, which effectively cancelled the steering committee of the RTG–WHO CCS, and established the new executive committee to oversee the CCS. The Health Systems Research Institute, the lead agency for the community health programme, was not included in this committee and it later notified WHO that it would like to terminate its contract signed with WHO, effective 1 July 2013.

Page 28: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

15

regulatory frameworks in the area of tobacco control, alcohol abuse and access to medicines were found useful.

61. Road safety. At the time of the CCS mid-term review, progress was still far from satisfactory. The initial focus of the programme was on campaigns which were neither countrywide, visible nor continuous. The CCS final evaluation confirmed that the stated objectives were not achieved but nevertheless recognized a certain number of achievements. Indeed the second part of the CCS implementation saw a productive period of legislative activities which were instrumental towards stronger road safety laws and regulations. WHO commissioned Thailand’s Road Safety Institutional and Legal Assessment which revealed the need for various legislative improvements. WHO also successfully advocated for the establishment of the Working Group to Review Road Safety Legislation under the national road safety directing centre. This group submitted for Cabinet approval a set of road safety legislative amendments aimed for implementation by the end of 2016.

62. Border and migrant health. WHO has been active in this area since the early 1980s. Included as an activity in the CCS 2012-2016, it became a priority along with ageing, as community health was sunsetted. The CCS final evaluation found that planned activities were achieved and that WHO’s advocacy in various relevant committees and task forces had been critically important. These included WHO support to establish and scale up border health information centres for migrants in a province and helping to maintain a health information system operating in camps along the Thai-Myanmar border.

63. Ageing. This is an issue of increasing importance for Thailand as it is going through a demographic transition. It was introduced mid-course in the CCS but activities remained marginal around literature reviews. The activity was sunsetted at the end of the CCS 2012-2016.

Dissemination of norms, standards and guidelines and generating knowledge

64. The WCO has been active across the CCS priorities to ensure adaptation of norms and standards at national level across relevant areas of work. It has also produced studies and research in various areas relevant to the health agenda in the country, thus contributing to build the evidence base to inform decision-making processes.

Major public health challenges

65. Communicable diseases were a priority of the CCS 2008-2011 and the country has registered notable progress in this area especially in HIV/AIDS control and reduction of incidence of malaria. However, malaria and tuberculosis require continued attention and, though no longer a CCS priority, the WCO continued to support the country in these areas. WHO is the lead agency for the national malaria programme reviews and provided technical and policy advice on the development of national plans reflecting the global and regional strategies on malaria elimination (i.e the national strategic plan for malaria elimination (2017-2026). Following the joint international monitoring mission review conducted in 2013, the WCO and MOPH decided to continue prioritization of tuberculosis control under the auspices of the CCS 2012-2016 and the WCO provided continued technical support. Although the CCS 2017-2021 does not include tuberculosis as a priority and considering Thailand is still a tuberculosis high burden country, the WCO remains committed to continue to provide support in this area.

66. AMR, though not referred to in the CCS 2012-2016, was already supported by the WCO and is now a priority of the CCS 2017-2021. The WCO contributed to the development of the draft national strategic plan which will serve as the basis for engagement for the forthcoming period.

Strengthened role of Thailand as a bilateral and multilateral partner in health

67. Over the last few years, Thailand has played an increasingly important role in health development beyond its borders. Among other things, the WCO regularly provided opportunities for Thai expertise to contribute regionally and internationally through participation in relevant

Page 29: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

16

conferences and workshops. For instance, Thailand is now in a position to offer some support to neighbouring countries in terms of vaccine and drug supply in response to outbreaks in these countries. These elements laid the foundations for the CCS 2017-2021 priority on global health diplomacy.

In brief

68. The CCS moved from a high number of small projects to a set of priorities and a small number of additional activities for the period 2012-2016. Though confronted by a number of challenges in planning and clarity on the expected contributions of WHO and in governance, notable progress and results have been consistently reported, as summarised in Table 4, from document reviews, WCO self-assessments and interviews conducted with partners.

Table 4: Indication of progress towards results for main CCS 2012-2016 priorities and activities29

CCS priorities and main activities Indications of

Good progress Some progress Limited progress

Community health

Multisectoral networking for NCD control

Disaster preparedness and response

International trade and health

Road safety

Border and migrant health

Ageing

Communicable diseases

Key to ratings - Good progress: when document reviews, interviews and WCO reporting provide converging information towards results even though nuances were also provided and there were indications that further work was needed; Some progress: when information included elements of progress combined with clear issues related to implementation; limited progress: when information recognized implementation but also sunsetting of the priority or of the activity.

Added value of regional and headquarters contributions to achievement of results

69. Regional and headquarters contributions to the WCO. The main added value recognized across the board related to the overall access to regional or international expertise in specific technical areas as well as the possibility for Thai health expertise to contribute regionally and globally through WHO’s convening power. The RO and HQ both provided support to the WCO at the time of the CCS design. Access to timely and relevant international health expertise from the RO and HQ or from outside WHO was found critically important for an appropriate implementation of the CCS, for instance, in the areas of international trade and health, nutrition, reduction in salt intake and tobacco control. It is also recognized that the need for technical contributions from RO and HQ are highly dependent on the level of expertise already present in the WCO. While all partners acknowledge and appreciate the technical contribution received from the RO and HQ, it was, however, difficult to quantify it as it was not possible to capture the relevant expenditure data (covering technical visits and staff time expended) from these offices, thus making it difficult to assess the totality of WHO input to the country programme.

70. WHO norms and standards. These regularly served as a framework/guide for implementation activities in-country (see Annex 4). Corporate initiatives can also benefit countries, such as the road safety priority which benefited from funding and technical support as part of the Bloomberg Initiative managed by HQ. The Regional Director’s strategic vision and 7 flagship priorities for South-East Asia were also found useful to embed CCS priorities within a larger framework.

29

Detailed information available in Annex 4. Information for other areas was too limited to be useful, and is neither included in Annex 4 nor Table 4 above.

Page 30: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

17

71. Regional collaboration across the South-East Asia and Western Pacific Regions of WHO. Thailand and Myanmar are the only countries in the Mekong sub-region that are part of the South-East Asia Region of WHO, the rest being members of the Western Pacific Region of WHO. The evaluation noted that there is a need to better harmonize the contributions from the two regional offices in the Mekong sub-region. The Thailand WCO also needs to play a significant role in the Association of Southeast Asian Nations (ASEAN), to supplement regional collaboration between the two WHO Regions which, while providing strategic regional perspectives, takes time away from national priorities.

72. WCO contribution to the RO and to HQ. Thailand being a regional hub for many other UN agencies, the WCO is regularly required to represent the RO in various regional meetings which is on the one hand very enriching for the WCO but also time consuming.30 The WCO is also regularly expected to provide data to the RO and HQ without these requests necessarily being coordinated in terms of timing or methodology, sometimes putting heavy pressure on a relatively small team.

Contribution of WHO results to long-term changes in health status

73. It is largely recognized that the WHO long-term presence in-country and active role for a long period of time has certainly contributed to major results in communicable disease control. WHO has also been active since 2004 in the area of border and migrant health and has consistently ensured attention to this area as part of the CCS 2012-2016 priorities or activities. Finally, in the area of road safety and NCD, new legislation directly benefitted from WHO advocacy efforts and technical support.

National ownership of the results and capacities developed

74. Thailand has made substantial progress in health. Its positioning internationally in the area of health is a strong indication of capacities developed through WHO contributions over time. For example, in the area of immunization and vaccine development, the programme previously supported by WHO has now become self-reliant with 100% government funding.

75. Interviews largely confirmed that the comprehensive consultations between more than 75 stakeholders from within and outside the MOPH and the bottom-up approach, with 38 proposals submitted from the national partners for inclusion in the CCS 2017-2021, constituted an indication of strengthened national ownership, building on that already initiated during the design of the CCS 2012-2016.

76. While RTG funding for the CCS 2012-2016 amounted to only 1.9 % of overall funding, it is now expected to become the main source of funding of the CCS 2017-2021, showing a very clear sign of national ownership of the CCS priorities.

Summary of key findings The country office workplans do not have an explicit link with the strategic priorities established

in the CCS 2012-2016. The WCO has not explicitly translated the PB corporate outcome and output targets at country level nor has it developed any clear results framework for each of the priorities of the CCS, identifying its expected contributions to each priority for the CCS 2012-2016 with corresponding baselines and targets.

Good progress has been observed for three of the main CCS 2012-2016 priorities; some progress made in one area; and two priorities have been sunsetted. Good progress can be reported in the area of communicable diseases, even though no longer a CCS 2012-2016 priority.

30

The WCO has at least 2 administrative staff who work almost full-time on regional programmes while part of the country office workplan.

Page 31: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

18

Box 3 - Good practice – WCO’s convening role beyond the health sector. The WCO brought together a very large number of actors of

the health sector who had not worked together before to

partner around CCS 2012-2016 priorities. Beyond this it also

played key role in bringing together the MOPH and other

ministries dealing with the economic and social

determinants of health, especially in the areas of road

safety and international trade and health.

The main added value of RO and HQ is the provision of access to high-level technical expertise regionally and internationally as well as the regional and global exposure it provides for Thai health expertise.

WHO long-term contributions to health changes in the country were observed mostly in the area of communicable diseases and prioritization of border and migrant health issues in the national agenda

Increasing ownership by the RTG of the CCS is especially noticeable in the design and funding of the CCS 2017-2021

2.3 How did WHO achieve the results? (Elements of efficiency)

Contribution of the core functions

77. WHO core functions are substantially interconnected: to advance the research agenda or develop policy options requires technical support. Document reviews and interviews all show very strong linkages between these three core functions and leadership. Nevertheless an attempt has been made in Annex 4 to clarify to the extent possible the contributions of each core function to the CCS 2012-2016 priorities and main activities. These are briefly summarized below.

78. Technical support. Most technical support under the CCS 2012-2016 came from two main sources of expertise: i) internally within WHO, mostly from the WCO, but also from the other levels of the Organization; and ii) from WHO’s ability to access highly relevant regional and global technical expertise. This happened across the board and remains a strong expectation from the RTG when it speaks of WHO’s “ intellectual capital”.

79. Leadership. Considering the high level of fragmentation of the health sector in Thailand, all stakeholders and documents reviewed recognized the key role played by WHO to convene and facilitate dialogue among partners, both within the health sector between the MOPH and the various health agencies and also with sectors dealing with the economic and social determinants of health, especially in the areas of road safety and international trade and health. That said, several stakeholders found that WHO could better use its reputation and technical clout to influence national polices as well as the role of health partners. In addition, there are strong expectations from the RTG that in future WHO’s convening power will provide a platform to showcase Thai public health experience to the wider world.

80. Norms and standards developed by WHO corporately have been adapted/translated in various national guidelines, as relevant. When working on international health regulation in the past there was an overall lack of public confidence in the MOPH, which was successfully bolstered by the support of WHO’s normative authority.

81. Research and contribution of evidence to inform decision-making by the MOPH have played a key role across the CCS 2012-2016 agenda. Research took place to inform traditional health-related issues in the area of communicable diseases as well as newer areas of work such as international trade and health. This core function remains critically important even in a country such as Thailand with its substantial progress in health and its own international health experts.

82. Policy options. Technical support and research have regularly been provided and undertaken to support policy options in the five priorities in the CCS and other health issues. This has

Page 32: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

19

happened especially in the area of road safety but also in NCD for instance where the WCO contributed towards the promulgation of tobacco laws/regulations and processes to effectively implement the sin tax to strengthen government efforts to control tobacco use.

83. Monitoring health trends appears to be the least important core function at country level. According to the report of the taskforce on the roles and functions at the three levels of WHO, this refers to WHO’s leadership to monitor and evaluate national policies and programmes as well as to support the collection, analysis, dissemination and use of data for monitoring the national health situation. Key national partners have expressed strong expectations for the WCO to lead on setting up a strong monitoring and evaluation mechanism for the CCS 2017-2021.

Table 5: Level of contributions of the core functions to the results of the

2012-2016 period

Priorities - activities

Technical support

Leadership Norms and standards

Research Policy options

Monitoring health trends

Community health system

+ +

Multisectoral networking for NCD control

+ +++ + ++ +++

Disaster preparedness and response

++ + + +

International trade and health

+++ +++ ++

Road safety +++ +++ + +++

Border and migrant health

+++ ++ + + + +

Ageing + +

Communicable diseases

+++ ++ ++ + + +

Note: rating relates to information available (for further details see Annex 4) on the contribution of core functions and this information here is reflected in terms of: +++ substantial contribution, ++ some contribution and + limited contribution. The intent is not to be exhaustive but reflect where emphasis has been laid during the 2012-2016 period.

Contribution of strategic partnerships to the results achieved

84. Three main types of partnership have played a key role in WCO activities between 2012 and 2016.

85. National partnerships. The CCS final evaluation undertook a systematic analysis of the partnerships, the results of which have been largely confirmed through the evaluation team’s interviews. At the time of CCS 2012-2016 design, the option was taken to have the lead agency for each CCS priority outside the MOPH in order to foster inter-agency collaboration from the onset, while the MOPH would keep the overall oversight of the CCS through a steering committee. This represented a major shift from the CCS 2008-2011 and produced mixed results. In some cases, for instance in international trade and health, the lead agency had experience in dealing with multiple agencies and produced collaboratively a solid work plan, organized regular and well documented meetings and overall made good progress over the course of the CCS 2012-2016. On the other hand, the lead agency for the community health priority was side-lined following internal political changes and finally withdrew mid-course of the CCS implementation. At a time of a major governance shift taken with the CCS 2012-2016, the numerous internal changes in leadership of the WCO during this

Page 33: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

20

Box 4 - Towards good practice – learning from governance challenges of the CCS 2012-2016

- Importance of solid work plans at the onset of the CCS

agreed by all relevant partners

- The work plan to clarify expected contributions of all

partners for each expected result or objective,

including expected contributions from WHO.

- Lead agency to have credibility among the partners

and be used to leading inter-agency processes

- Selection of a full-time competent programme

manager

- Proactive role to be played by all partners

- Appropriate funding mechanism to be identified.

period (three heads of WCO and one officer in charge for almost a year) represented an additional constraining factor to the smooth implementation of the CCS.

86. While some partnerships have been less fruitful than others, this was overall recognized as being the way forward for the CCS 2017-2021, where partners are in even greater number, thus intensifying the importance of learning from the governance successes and failures of the CCS 2012-2016 to move forward.

87. Partnerships with national WHO collaborating centres. In 2017, Thailand counted 34 active WHO collaborating centres, some of them playing a key role as the lead agency for the road safety priority or as a key partner in other activities such as AMR.

88. Partnerships with UN agencies and other partners in-country. The WCO and UN agencies reported useful collaboration on specific CCS 2012-2016 priorities, among other things, in the area of border and migrant health where there is a long-standing positive partnership with the IOM regional office. Partnerships with UNICEF and UNDP were mentioned as useful to move forward the agenda on nutrition and tobacco control. Partnership with US CDC mainly focused on communicable diseases and border health.

Funding

89. According to the CCS 2012-2016, as WHO is not a funding agency, its main contribution would be to exert its technical power, its “social credit” and its neutral convening power. It would, however, use its modest financial support in a catalytic way to leverage additional funds (when needed) from donors to fully support selected areas of work. The CCS 2012-2016 does not include any information about the size or magnitude of expected funding through its duration or across the various elements of the CCS. It is therefore not possible to assess whether budgets were appropriate or funding levels timely.

90. Figure 5 below compares, for the two biennia completed during the CCS 2012-2016, the level of expenditure compared with the planned costs for the main activities undertaken and shows clearly that expenditure rates have consistently improved, reflecting very understandably a slower expenditure rate at the start of the CCS as the partnerships and workplans were being developed.

Page 34: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

21

Box 5 - Towards good practice – Piloting of a new CCS financing mechanism WHO and Thai authorities have agreed in principle to a unique funding mechanism for the CCS 2017-2021 which should reduce transaction costs and include the following key elements:

- Un-earmarked funding from all funders, including

WHO, is pooled annually for each of the five

priority areas

- All funds are provided against a single annual

plan for each priority area

- A single technical and financial report is

produced for each priority area.

Figure 5: proportion of expenditures versus planned costs for PB 2012-2013 and PB 2014-2015

Source: GSM

91. The overall proportion of expenditures versus planned costs increased from 79% during the 2012-2013 biennium to 89% during the 2014-2015 biennium. Considering limited resources available to the WCO, close monitoring facilitates full expenditure at the end of each biennium. That said, the data available do not shed light on the flow or the timing of the funding which might well have been intermittent or released towards the end of the biennium and might therefore partially explain the expenditure rates. Staff represented 62% of the overall costs.

92. The WHO financial contribution to the national programmes was modest though recognized as critically important to leverage support or to launch new initiatives. Funding came from both assessed contributions and voluntary grants for specific initiatives. These played a major role in funding activities in the areas of border and migrant health as well as road safety.

93. Increased transparency in budget planning has been observed for the CCS 2017-2021 which has identified ahead of its finalization an overall financial requirement of US$ 13,7 million. This CCS will be mainly funded by the RTG (increasing over time to become about two-thirds of the planned budget while WHO will fund only up to 30% of it). This is meant to achieve two complementary objectives: i) to increase country ownership and engagement by having national agencies active in the health sector contribute to jointly set national priorities; and ii) to encourage WHO to not just be a financial partner but also to leverage its intellectual and social capital to advance goals in the priority areas set out in the CCS.

94. Early in 2017, however, WHO funding for the first year of the CCS 2017-2021 was challenged by resource constraints at the regional level. It remains to be seen whether or not this has any significant impact on the “leveraging” power of the catalytic funding.

95. Finally, the overall financing mechanism used for the CCS 2012-2016 was found cumbersome and far from the Paris Declaration Pinciples. Many national partners are actively seeking another approach to facilitate a more appropriate and predictable funding of the CCS 2017-2021. Box 5 highlights the main characteristics of a possible new financing mechanism. It will be important to identify clear

0% 20% 40% 60% 80% 100% 120%

Community Health

Noncommunicable Diseases

Disaster Management

International Trade & Health

Road Safety

Ageing

Border & Migrant Health

International Health Regulation

Communicable Diseases

Total

2012-13

2014-15

Page 35: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

22

triggers (baselines and targets) to release the funds in order to ensure that WHO fulfils its obligation with regard to Member States in terms of accountability and transparency of the use of funding.

Staffing

96. The composition of the WCO evolved considerably over the CCS 2012-2016 period with a very strong international staff presence mostly at P4 and P5 level. The team has also been confronted with various changes (and vacancy) in leadership over the 2012-2016 period as well as a high staff turnover which, according to the CCS final evaluation and interviews, affected levels of implementation.

Table 6: Profile of WCO staff in 2011 and 2017

Organigram Q3 2011 Organigram Q1 2017

International professional officers 4 8

National professional officers 7 4

Source: WCO organigrams

97. The CCS final evaluation assessed positively the capacity of the WCO to dedicate national professional officers to some of the CCS 2012-2016 priorities. Also confirmed during the interviews, the knowledge of Thai is critically important to ensure active engagement of the WCO in technical meetings. The increase in the number of international technical professionals in the WCO is a welcome initiative, and there is also a need for appropriately skilled NPOs, not only in support of technical issues but also to facilitate discussions when language is an issue (this can also be supplemented with locally sourced translation services).

98. The WCO requested a review of individual staff functions and team structure at the end of 201631 to ensure an appropriate staffing to implement the CCS 2017-2021. The WCO has now developed an organigram around the six programme categories identified in the 12th GPW and has identified a focal point per CCS priority.

99. The implementation of the CCS 2012-2016 faced issues of governance but also of weaknesses in monitoring (see below). Strong expectations in this regard on the part of national partners confirmed this CCS final evaluation finding. However, at the moment there does not seem to be any profile in the organigram with professional capacities in programme management to advise on planning issues and develop a comprehensive, strong monitoring and evaluation framework for the entire CCS 2017-2021. This might be a missed opportunity at a critical point in WHO’s engagement in the country.

Monitoring mechanisms

100. Each component of the CCS 2012-2016 included a workplan which, according to the CCS final evaluation, was more or less solid depending on the CCS priorities. For instance, in international trade and health, the lead agency developed a solid work plan. However, in road safety, the workplan did not provide clearly defined activities that identified responsible agencies, indicators and budget lines.

101. The varied quality of workplans directly impacted on the monitoring frameworks. While a good quality workplan does not guarantee an operational monitoring framework, it is a necessary precondition to actually develop one. The CCS final evaluation reviewed all monitoring frameworks of the CCS and did not identify one that was working appropriately in delivering regular data on progress towards targets. This was confirmed during the interviews when most national partners confirmed the need for stronger monitoring of the CCS 2017-2021 implementation.

31

Burkholder, Dr Brent, 2016, ‘Review of individual staff functions and team structure at WHO country office’.

Page 36: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

23

102. The CCS 2017-2021 now includes a monitoring and evaluation sub-committee and there are strong expectations on the part of the MOPH to see WHO play a key role here.

103. At the WCO level, the monitoring of WHO’s contribution and progress towards targets has also not taken place in a systematic manner. Reporting through the corporate management tool currently does not constitute an adequate monitoring mechanism to support programme management, mainly because it does not clarify the targets to be achieved nor does it report at output or outcome level but rather at the level of tasks.

Use of the CCS 2012-2016 to inform WHO country workplans, budget allocations and staffing

104. The staffing review conducted in 2016 clearly refers to the CCS priorities in order to identify technical staffing requirements to implement the CCS 2017-2021. On the other hand, no evidence of use of the CCS 2012-2016 could be established by our evaluation to inform the WCO workplans or the budget allocations during the 2012-2016 period.

Summary of key findings: The core function which contributed the most to progress in the CCS 2012-2016 is technical

assistance, used to inform adaptation of research findings, norms, standards and guidelines, as well as to bring evidence to the table to inform policy options. The other critical core function during the course of the CCS is the leadership function and the convening power of WHO to bring regional and international expertise in-country through conferences, workshops etc. and to engage Thai expertise to support regional and international health conferences and workshops. Monitoring of national health programmes, on the other hand, did not receive adequate attention.

The major shift introduced with the CCS 2012-2016 of building multi-agency partnerships and putting health agencies as lead of CCS priorities produced mixed results and contributed to important lessons for the design and implementation of the CCS 2017-2021.

WHO is no longer perceived as a funding agency but as one that can play a catalytic role using its modest financial support to leverage additional funds (when needed) to fully support selected areas of work. It also keeps the ability to fund support in areas not prioritized in the CCS. Its funding mechanisms were however found to be rather cumbersome for the amounts at stake and alternative funding mechanisms are envisaged for the CCS 2017-2021.

The staffing profile and structure evolved during the CCS 2012-2016 period. The team is becoming more international and high-level than it was five years ago there is also a need for appropriately skilled NPOs, not only in support of technical issues but also to facilitate discussions when language is an issue (this can also be supplemented with locally sourced translation services). Finally, a high-level international profile on planning and monitoring is of critical importance.

Planning of the CCS 2012-2016 was of varying quality across the priorities; and monitoring has been overwhelmingly found weak in all areas of the CCS. Furthermore, the WCO has not developed any mechanism to monitor its own contribution to the CCS against clear targets.

Evidence of use of the CCS was found for staffing purposes but not for budgeting or work planning.

Page 37: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

24

3. Conclusions

105. Based on the findings presented in the previous section, the following conclusions are articulated around the three main evaluation questions (EQs) all of which inform the recommendations presented in Section 4.

106. Relevance of the strategic choices. The priorities identified in both the CCS 2012-2016 and the CCS 2017-2021 were strategic to address Thailand’s major health needs and were coherent with government and partners’ priorities expressed in the UNPAF. They were also coherent with WHO’s GPW as per the assumptions made in the theory of change (see Figure 2) in terms of health needs and alignment.

107. Overall the CCS 2012-2016 introduced a major shift from a fragmented approach through many small projects to a much stronger focus around five priorities and three main activities. This shift has been further strengthened in the design of the CCS 2017-2021 which includes only five main strategic priorities.

108. The prioritization process for the CCS 2017-2021 was strengthened based on experience gained during the CCS 2012-2016 as follows: i) it included a very large number of stakeholders, reflecting the multiplicity of actors in the health sector in the country, and ii) it benefitted from a transparent consultation and priority setting based on predefined selection criteria known to all relevant partners. This led to: i) some issues such as NCD or border and migrant health (WCO focus areas) becoming a priority for the MOPH in the CCS; ii) the sunsetting of certain activities, as appropriate; and iii) introduction of new ones such as global health diplomacy, reflecting a RTG priority.

109. The CCS provides the strategic framework for WHO’s work in and with Thailand. However, the priorities and activities therein do not necessarily cover the totality of the WCO’s contribution to health in Thailand. For example, communicable diseases are no longer a CCS priority, but Thailand is still one of the 30 high burden countries for tuberculosis. Staff in the WCO continue to provide support to Thailand as necessary in this and other technical areas.

110. There is a discrepancy between the WHO programme and funding structure, to which WCO workplans must conform, and the priorities elaborated in the CCS, making it challenging for the WCO to develop its workplans in line with the CCS priorities.

111. Gender. A gender analysis was not done in the CCS 2012-2016 and, though gender was given substantial attention in the CCS 2017-2021 analysis, it did not lead to the priorities retained paying explicit attention to the gender issues identified in the analysis.

112. WHO’s intellectual and social capital. The new CCS provides a unique opportunity for both the RTG and WHO to engage in a strategic partnership of a new kind where funding is no longer the main commodity but the means by which both partners contribute their respective added value. Building on its well-established and recognized intellectual capital, WHO now has to strengthen its positioning in terms of social capital and branding, thereby enabling the RTG to consolidate the achievement of its universal health coverage by more systematically addressing the social determinants of health while enhancing at the same time Thailand’s role in global health. Many national partners indicated clear expectations with regard to WHO’s strategic contribution in this respect. In their opinion, intellectual capital broadly refers to WHO’s leading role in the provision of technical health expertise, while social capital refers more to WHO’s reputation, influence, authority, name and trust.

113. More widely, Thailand and other countries in similar situations are facing issues that require tailored approaches and support from their respective regional office and HQ. Resource mobilization in such countries might require a specific strategy and the new financing mechanism being

Page 38: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

25

developed for the CCS 2017-2021 in Thailand may offer lessons for other countries with a similar WHO presence.

114. Main achievements. Overall, during the period 2012 to 2016, the WCO in Thailand provided a valuable contribution in supporting the RTG’s national health sector plans. The CCS 2012-2016 created an enabling environment for various players in the Thai health sector to form partnerships around key health issues and this positive environment was strengthened for the CCS 2017-2021, based on the experience gained from the CCS 2012-2016. Results have been documented for all four main expected outcomes identified in the TOC. Positive results were noted in the area of NCD, international trade and health, road safety, border and migrant health and communicable diseases. Community health and ageing were sunsetted as priorities while the disaster preparedness and response priority made limited progress over the course of the CCS.

115. Programme management challenges. The difficulty in measuring results against planned targets and assessing WHO’s contributions to the same are indications of a number of systemic challenges in planning and monitoring processes within WHO at both corporate and country levels. This weakens WHO’s capacity to demonstrate results and contribution to health improvements in any given country. Furthermore it appears that, over the course of the CCS 2012-2016, the WCO was not able to develop its own mechanism to monitor the effects of its contribution to the various objectives defined for each priority of the CCS when developing its country workplans.

116. Key contributions of core functions. Technical support stands out as the key core function contributing to the WCO work in support of the RTG and the implementation of the CCS 2012-2016. This core function enabled research activities, facilitated the adaptation of norms, standards and guidelines and provided evidence to inform policy options for decision-makers. The other core function which played a major role was WHO’s leadership and convening power, allowing Thailand to avail of international expertise, and contributing Thai health expertise abroad. These functions form the foundation of WHO’s intellectual and social capital.

117. The WCO contribution to monitoring of health trends seems to have been more limited but, in the future, this core function is expected to play a much bigger role, both in the monitoring of CCS implementation and in supporting the country to monitor its progress towards the health-related SDGs.

118. Partnerships. With respect to the work of the WCO with partners, the major shift introduced with the CCS 2012-2016 has been critical. Bringing together various actors around key priorities understandably takes time and, despite the mixed results obtained so far, it is considered by all as the way forward, establishing firm foundations for the design of the CCS 2017-2021. The initial collaboration with non-health actors that was introduced in the CCS 2012-2016 has been confirmed in the CCS 2017-2021.

119. Funding remains a critical means for WHO’s catalytic engagement in the country. It ensures that certain priorities remain high on the agenda, as has been the case with border and migrant health and with road safety.

120. Funding mechanisms will need to follow the strategic shift from small projects to priority areas initiated with the CCS 2012-2016 and confirmed in the CCS 2017-2021, and new approaches through pooled funding mechanisms are being considered. Such mechanisms require even stronger attention to planning and monitoring as indicators at outcome level need to be identified and their achievements documented in order for funding instalments to be released at specific times of the CCS 2017-2021 implementation.

121. Staffing. The WCO team composition and skills mix has evolved over time and been strengthened with a doubling of the number of international staff over the CCS 2012-2016 period. It is important to be able to match staff profiles and expertise with the priorities set out in the CCS. The increase in the number of international technical professionals in the WCO is a welcome

Page 39: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

26

initiative, and there is also a need for appropriately skilled NPOs in support of technical issues but also to facilitate discussions when language barriers are an issue. This can also be supplemented with locally sourced translation services. The rationale for organizing teams around programme categories rather than around the CCS priorities plus communicable diseases remains a challenge for the WCO. Considering the weaknesses in planning and monitoring observed during the CCS 2012-2016 and the expectations on the part of national counterparts in this area, the WCO needs to ensure that it can very quickly mobilize adequate levels of expertise in this area, either through the support of HQ or RO colleagues, or through short-term experts.

122. Best practices and innovations. This evaluation highlighted a certain number of emerging good practices and innovations framing WHO’s engagement in Thailand. Indeed the WCO’s role has clearly evolved during the period evaluated and is continuing to do so. For instance, the approach taken in the design of the CCS 2012-2016 and the lessons learned strengthened the design of the CCS 2017-2021, highlighting partnerships with national actors beyond the health sector and instituting a transparent and consultative priority setting process for the CCS 2017-2021. The fact that the RTG has increased its funding to become the main funding source for the CCS 2017-2021 also represents a major shift in its collaboration with WHO. Finally, the new funding mechanism being explored could also be used, if proven effective, in the future in other similar countries.

Page 40: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

27

4. Recommendations

123. Based on the findings and conclusions elaborated in the previous sections, the evaluation would like to make the following recommendations:

A. The Head of the WHO Country Office and the WHO Country Office team to contribute actively to Country Cooperation Strategy governance activities and to engage with other national partners to support implementation of Country Cooperation Strategy priorities and activities, in particular in the area of programme management and monitoring.

i. Review the Country Cooperation Strategy workplans for each priority and define targets (qualitative or quantitative) for both the expected outcome and output levels and clarify expected WHO contribution in a measurable manner.

ii. Ensure adequate technical capacity for planning and monitoring Country Cooperation Strategy implementation.

B. The WHO Secretariat to ensure that the WHO Country Office has the capacity to implement its workplans beyond the Country Cooperation Strategy priorities and activities, including through appropriate funding mechanisms and staffing of the Office.

i. Ensure that new Country Cooperation Strategy priorities such as antimicrobial resistance are adequately covered with financial and human resources

ii. Ensure that language is never a barrier for the active engagement of the WHO Country Office with national partners.

iii. Headquarters and the Regional Office to support the WHO Country Office in the review and consideration of the Royal Thai Government’s request to support the implementation of the Country Cooperation Strategy 2017-2021 through the national pooled funding mechanism, and explore the possibility of linking a pooled funding mechanism with indicators of achievement.

C. The WHO Country Office to build on a Theory of Change for the period 2017-2021 to better link the Country Cooperation Strategy 2017-2021 with the entire planned country-level results and deliverables and with the Country Office staff and activity workplans during operational planning for Programme budgets 2018-2019 and 2020-2021.

i. Develop a Theory of Change for 2017-2021 to frame more specifically the pathway for change (it should include all Country Office activities, not only those of the Country Cooperation Strategy).

ii. Clarify for each relevant corporate output the targets relevant for Thailand in the current biennium and for each biennium thereafter.

iii. Set up an internal monitoring framework to measure WHO’s progress towards targets over the Country Cooperation Strategy implementation period.

D. The WHO Country Office and the Royal Thai Government to strengthen inclusion of the gender and other social determinants of health dimension(s), as relevant, in the implementation of the Country Cooperation Strategy and other Country Office activities.

i. Review programmes of work of each Country Cooperation Strategy priority with a gender lens, possibly with the support of the Regional Office or of headquarters, and amend as necessary to ensure the gender dimension is appropriately taken into consideration.

E. The WHO Secretariat (Department of Country Cooperation and Collaboration with the UN System and the Country Support Unit network) to review the evolution of the Country Office’s contribution to, and relationship with, the Royal Thai Government over the recent Country Cooperation Strategy cycles, in order to consider the lessons learned, innovation and best practices for Country Office interaction with, and contribution to, other upper-middle-income countries and emerging economies.

Page 41: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

28

i. Reflect further on the implications of the expectations of counterparts in terms of social capital, in particular with other Country Offices active in upper-middle-income countries. WHO should deploy experts with profiles and experience matching the Country Cooperation Strategy priorities.

ii. Develop a strategic note framing WHO’s engagement in upper-middle-income countries from the intellectual and social capital perspective.

iii. Document the prioritization process followed for the Country Cooperation Strategy 2017-2021 and share within the Organization.

Page 42: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

Country Office Evaluation

- Thailand -

(Volume 2: Report Annexes)

August 2017

WHO Evaluation Office

Page 43: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

Table of contents

Annex 1: Terms of Reference ............................................................................................................... 1

Annex 2: Evaluation methodology and evaluation matrix ................................................................. 9

Annex 3: WHO’s main planning instruments and associated challenges ...................................... 18

Annex 4: Evaluation observations for each priority and main activities of CCS 2012-2016 and

CCS 2017-2021 .................................................................................................................................... 21

Annex 5: List of people met ................................................................................................................ 36

Annex 6: Bibliography ......................................................................................................................... 38

Page 44: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

1

Annex 1: Terms of Reference

I. Introduction

1. Country Office Evaluations (COE) are part of the Organization-wide evaluation workplan approved by the Executive Board in January 2016. The workplan clarifies that COEs “will focus on the outcomes/results achieved by the country office, as well as contributions through global and regional inputs in the country. In addition these evaluations aim to analyse the effectiveness of WHO programmes and initiatives in the country and assess their strategic relevance within the national context”. They encompass the entirety of WHO activities during a specific period. The COEs provide lessons that can be used in the design of new strategies and programmes in-country.

2. The Thailand COE is the first country office evaluation undertaken by WHO Evaluation Office. The evaluation will cover period of the Country Cooperation Strategy (CCS) 2012-2016.

II. Country context

3. Thailand is an upper-middle-income country since 20101 whose human development index increased from 0.692 (medium human development group) in 2012 to 0.723 (high human development group) in 2014. However, its annual Gross Domestic Product growth rate of 7.5 percent in 2012 decreased to 2.8 percent in 2015.4 Poverty is mostly a rural phenomenon with varying incidence across the regions. The Northeast is the poorest region and is home to almost half the country’s poor. Similarly significant progress towards MDG achievement hides persistent disparities among regions and social groups.5 In addition, while there has been significant progress on the major communicable diseases, morbidity, mortality and disability due to noncommunicable diseases have continued to rise.

4. The 11th National Economic and Social Development plan (NESDP) 2012-2016 emphasizes equity issues which are translated in the vision of the 11th National Health Development Plan covering the same period. This plan includes five strategies as follows:

1) Strengthen partners for health promotion and self-reliance in health with Thai wisdom

2) Further develop systems for monitoring, warning and management of disasters, accidents and health threats

3) Focus on health promotion, disease prevention and consumer protection in health for Thais to be physically, mentally, socially and spiritually healthy

4) Strengthen health-care systems with quality and standards at all levels in response to health needs of all age groups and improve seamless referral systems

5) Create national mechanisms for enhancing the efficiency of health-care system governance and resources management systems

5. As reported in the CCS 2012-2016, a unique feature of the health sector in Thailand is that in addition to the Ministry of Public Health there are other key public health agencies operating side by side with the Ministry, such as the Health Systems Research Institute, the Thai Health

1 http://data.worldbank.org/country/thailand?view=chart (downloaded 7 December 2016).

2 http://hdr.undp.org/sites/default/files/reports/14/hdr2013_en_complete.pdf (downloaded 7 December 2016).

3 http://hdr.undp.org/sites/default/files/2015_human_development_report.pdf (downloaded 7 December 2016).

4 http://data.worldbank.org/country/thailand?view=chart (downloaded 7 December 2016).

5 WHO, 2016,” Needs assessment for the selection of priorities for the Thailand-WHO Country Cooperation Strategy”.

Page 45: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

2

Promotion Foundation and the National Health Commission. In addition, there are 34 WHO collaborating centres. The civil society is also a powerful actor in health development.

6. In 2012, the Official Development Assistance (ODA) for health commitment represented 7% of overall ODA commitments.6 According to the OECD,7 in 2014, ODA mainly came from the United States, followed by France and Japan to a much lesser extent as well as from WHO, the Asian Development Bank and UNAIDS.

7. While a recipient of ODA, Thailand is also providing ODA to neighbouring countries and making major contributions to global health development. Thailand is financially supporting WHO activities in country. Partnership in health is therefore a key strategy for health development in the country.

8. The United Nations have developed a partnership framework (UNPAF) for 2012-2016 aligned with the development strategies identified in the 11th NESDP, underlining the importance of a two-way partnership of knowledge and experience sharing.

III. WHO activities in Thailand

9. WHO is present in Thailand since 1949. The health sector landscape considerably changed over the recent past: many new actors (both at national and international level) have emerged and partnerships multiplied.

10. Table 1 below identifies briefly the main areas of activities undertaken in the WHO Country Office (WCO) and corresponding levels of investment.

Table 1: CCS and non-CCS implementation (HR & activity combined in US$) 3 biennia as of 8 June 2016

Programs 2012-13 2014-15 2016-17 Total %

CCS priorities 1 Community Health 1123171

1123171 5.94%

2 Noncommunicable Diseases 190439 780524 169435 1140398 6.03%

3 Disaster Management 2644885 78782 150586 2874253 15.21%

4 International Trade and Health 546468 319882 93991 960341 5.08%

5 Road Safety 112911 493031 245120 851062 4.50%

Other CCS activities

Ageing

67677 13022 80699 0.43%

Border and Migrant Health 501724 1403488 160558 2065770 10.93%

Antimicrobial Resistance

49798 49798 0.26%

Other activities

International Health Regulations 491208 138720 95773 725701 3.84%

Tuberculosis Control 563541 275806 90937 930284 4.92%

Non CCS activities 627272 1430863 360042 2418177 12.79%

WHO Country Office 2671862 2500473 509940 5682275 30.06%

Source: Final evaluation of WHO CCS Thailand 2012-2016

11. The CCS 2012-2016 was WHO’s key instrument to guide its collaboration with the Royal Thai Government, in support of the national health agenda as formulated in the 11th National Health

6 See WHO, 2014, “From whom to whom? ODA for health fourth edition 2002-2012”.

7 http://stats.oecd.org/Index.aspx?QueryId=58193 (downloaded 8 December 2016).

Page 46: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

3

Development Plan 2012-2016. It provides the strategic direction for WHO’s contribution in-country. The WHO activities are all run from Bangkok.

12. The strategic agenda of WHO as described in the CCS identifies four clusters of activities: 1) five priority areas; 2) normative functions; 3) major public health challenges and unfinished agendas; and 4) support for Thailand’s role in health beyond its borders.

13. Over the years, a large network of WHO collaborating centres has evolved and is being constantly updated. As of 8 December 2016 there were 348 of them in the country.

14. The WCO conducted a mid-term review9 of the CCS in 2014 and its final evaluation mid-2016.10 It mainly concluded that the CCS was well aligned with the country’s health priorities. It has oriented most of WHO’s resources towards the priority programmes. Most of the activities have been implemented and have contributed to the stated objectives. The method of working through lead agencies and multisectoral committees has been a partial success and holds promise for the future. Its main recommendations can be summarized as follows:

a. have a clear development process for the next CCS b. have clear criteria for lead agency selection c. continue to foster multisectoral work but perhaps involve the MOPH more d. recognise that multisectoral work requires specific technical skills e. explore lighter management processes f. move the oversight committee towards more sustainable funding over time g. slow down the rate of turnover of key personnel h. continue pushing multisectoral working methods in spite of obstacles.

15. In 2016, the WCO also conducted a needs assessment for the selection of priorities for the next CCS as well as a functional review of its office. All these elements informed the CCS 2017-2021, about to be finalized with the Royal Thai Government.

IV. Objectives and scope of the COE

16. The main purpose of the COE is to identify and document best practices and innovations of WHO in Thailand on the basis of its achievements. These include not only results of the WCO but also contributions from the regional and global levels in-country.

17. As all evaluations, this COE meets accountability and learning objectives. It will be publicly available and reported on through the annual Evaluation Report. This evaluation will build on the results of previous evaluative work to:

a. Demonstrate achievements against the objectives formulated in the CCS (and other relevant strategic instruments) and corresponding expected results developed in the WCO biennial workplans, while pointing out the challenges and opportunities for improvement.

b. Support the WCO and partners to operationalize the various priorities of the next CCS (and other relevant strategic instruments) based on independent evidence of past successes, challenges and lessons learnt.

c. Identify best practices emerging from the unique relationship between the Royal Thai Government and WHO as exemplified during the current CCS. These can then usefully inform the development of future country, regional and global support through a systematic approach to organizational learning.

8 http://apps.who.int/whocc/List.aspx?cc_code=THA (downloaded 8 December 2016).

9 http://apps.who.int/iris/handle/10665/161132 (downloaded 8 December 2016).

10 WHO, 2016, Final Evaluation of WHO Country Cooperation Strategy Thailand 2012-2016.

Page 47: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

4

18. The evaluation will cover all activities undertaken by WHO (WCO, regional office and headquarters) in Thailand as framed in the CCS 2012-2016 and other strategic documents covering activities not part of the CCS that took place over that period of time. In addition it will also include the development process of the new CCS 2017-2021 which will be finalized by the time the evaluation starts.

V. Stakeholders and users of the evaluation

19. Table 2 shows the role and interest of the main evaluation stakeholders and expected users of the evaluation.

Table 2: preliminary stakeholders’ analysis

Internal stakeholders Role and interest in the evaluation

WCO Thailand As lead for the development and implementation of the CCS, the CO is the main stakeholder of the evaluation because it has an interest in enhancing accountability of WHO in-country as well learning from evaluation results for future programming

WHO Regional Office for South East Asia

As a contributor to the development of the CCS, the Regional Office has a direct stake in the evaluation in ensuring that WHO’s contribution in-country is relevant, coherent, effective and efficient. The evaluation findings and best practices in Thailand will be directly useful to inform other WCOs in the Region as well as regional approaches in health.

Headquarters management

The results of the evaluation should be of interest as headquarters management is in charge of coordination of the CCSs and strategic analysis of its content and implementation and is responsible for promoting application of best practices in support of regional and country technical cooperation.

Executive Board The Executive Board has a direct interest in being informed about the added value of WHO’s contributions in countries and will be kept abreast of best practices as well as challenges through the annual evaluation report.

External stakeholders

All individuals in Thailand

WHO’s action in Thailand has to ensure that it benefits all population groups, prioritize the most vulnerable and does not leave anyone behind The evaluation will look at the way WHO addresses equity and ensures that all population groups are considered in the various policies and programmes.

Royal Thai Government

As a donor and recipient of WHO’s action, it has an interest in ensuring that the partnership with WHO and the future programming under the new CCS is the most relevant, effective and efficient. Considering its engagement in international health development, it also has an interest in seeing its best practices independently assessed and disseminated. In addition to the Ministry of Public Health, there are a large number of public health partners in-country who all have an interest in the evaluation.

UN Country Team WHO contributes to several outcomes of the UNPAF alongside other UN agencies. There is therefore an interest for the UNCT (and UNAIDS in particular) to be informed about WHO’s achievements and be aware of the Royal Thai Government’s best practices in the health sector.

Donors In addition to Thailand, other donors such as the United States, Australia

Page 48: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

5

Canada and philanthropic foundations, have an interest in knowing whether their contributions have been spent effectively and efficiently and if WHO’s work contributes to their own strategies and programmes.

VI. Evaluation questions

20. All COEs address the 3 main EQs identified below. The sub-questions are then tailored according to the country’s specificities and detailed in an evaluation matrix to be developed during the inception phase by the evaluation team. Sub-questions have been tailored taking into account the timing of this COE and the available evaluative information. Good practices and lessons learned will be identified across the findings.

EQ1 - Were the strategic choices made in the CCS (and other relevant strategic instruments) the right ones to address Thailand’s health needs and coherent with government and partners’ priorities? (relevance)

21. This question assesses the strategic choices made by WHO at the CCS design stage and its flexibility to adapt to changes in context. This question will assess both the CCS 2012-2016 and the new CCS 2017-2021 design which will be finalized by the time the evaluation starts. When addressing each evaluation sub-question the evaluation team will build on past evaluative information and seek to identify best practices in the design process of the new CCS. The evaluation sub-questions focus on the following elements:

1.1 Are the CCSs based on a comprehensive health diagnostic of the entire population and on Thailand’s health needs?

1.2 Are the CCSs coherent with the national health development plans, any other relevant national health strategies and the MDG and SDG targets relevant to Thailand?

1.3 Are the CCSs coherent with the UNPAF? And are the key partners clear about WHO’s role in Thailand?

1.4 Are the CCSs coherent with the General Programme of Work and aligned with WHO’s international commitments?

1.5 Has WHO learned from experience and changed its approach in view of evolving contexts (needs, priorities, etc.) between both CCSs but also during the course of the CCS 2012-2016?

1.6 Are the CCSs strategically positioned when it comes to: i. Clear identification of WHO’s comparative advantage and clear strategy to

maximise it and make a difference? ii. Capacity of WHO to position health priorities (based on needs analysis) in

the national agenda and in those of the numerous national partners in the health sector?

iii. Specificities of the partnership between WHO and the Royal Thai Government especially in view of the numerous actors involved in the national health sector? And has this positioning evolved between the two CCSs? If so how?

iv. Reflecting the contribution of WHO in terms of intellectual and social capital.

EQ2 - What is the contribution/added value of WHO towards addressing the country’s health needs and priorities? (effectiveness/elements of impact/progress towards sustainability)

22. To address this question the evaluation team will build on the analysis of results per programme area already presented in the CCS 2012-2016 evaluation conducted earlier this year and will focus on the best practices and innovations observed for the following:

Page 49: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

6

2.1 To what extent were the country biennial work plans (operational during the evaluation period) articulated with the focus areas as defined in the CCS (and other relevant strategic instruments) document (or as amended during course of implementation)?

2.2 What were the main results achieved for each outcome, output and deliverable for WCOs as defined in the country biennial work plans, especially in terms of intellectual and social capital ?

2.3 What has been the added value of regional and headquarters contributions to the achievement of results in-country?

2.4 What has been the contribution of WHO results to long-term changes in health status in-country?

2.5 Is there a national ownership of the results and capacities developed?

EQ3 – How did WHO achieve the results? (efficiency)

23. In this area the evaluation sub-questions will mainly cover the contribution of the core functions, the partnerships and allocation of resources (financial and staffing) to deliver the expected results and, for each, will seek to identify best practices and innovations.

3.1 What were the key core functions most used to achieve the results? 3.2 How did the strategic partnerships contribute to the results achieved? 3.3 How did the funding levels and their timeliness affect the results achieved? 3.4 Was the staffing adequate in view of the objectives to be achieved? 3.5 What were the monitoring mechanisms to inform CCS implementation and progress

towards targets? 3.6 To what extent has the CCS been used to inform WHO country work plans, budget

allocations and staffing?

VII. Methodology

24. Guided by the WHO Evaluation Practice Handbook, the evaluation will be based on a rigorous and transparent methodology to address the evaluation questions in a way that serves the dual objectives of accountability and learning.

25. During the inception phase the evaluation team will design the methodology which will entail the following:

Adapt the theory of change developed for the evaluation of WHO’s presence in countries. The theory of change to frame the COE Thailand will: i) describe the relationship between the CCS strategic priorities, the focus areas and the activities and budgets as envisaged in the biennial work plans; ii) clarify the linkages with the General Programme of Work and programme budgets, and iii) identify the main assumptions underlying it.

Develop and apply an evaluation matrix11 geared towards addressing the key evaluation questions taking into account the data availability challenges, the budget and timing constraints.

Adhere to WHO cross-cutting strategies on gender, equity and human rights and include to the extent possible disaggregated data and information.

11

An Evaluation Matrix is an organizing tool to help plan for the conduct of an evaluation. It is prepared by the evaluation team during the inception phase of the evaluation, and is then used throughout the data collection, analysis and report writing phases. The Evaluation Matrix forms the main analytical framework for the evaluation. It reflects the key evaluation questions and sub-questions to be answered and helps the team consider the most appropriate and feasible method to collect data for answering each question. It guides analysis and ensures that all data collected is analysed, triangulated and used to answer the evaluation questions and make conclusions and recommendations.

Page 50: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

7

Follow the principles set forth in the WHO Evaluation Practice Handbook and the United Nations Evaluation Group (UNEG) norms and standards for evaluations and ethical guidelines.

26. The methodology should demonstrate impartiality and lack of bias by relying on a cross-section of information sources (from various stakeholder groups) and using a mixed methodological approach to ensure triangulation of information through a variety of means.

27. The COE will rely mostly on the following data collection methods:

a. Documents review will include analysis of key strategic documents, such as the general programmes of work, the programme budgets, the WCO workplan and budget, the CCS (and other relevant strategic instruments), the UNDAF, relevant national policies, strategies and other relevant documentation.

b. Stakeholder interviews. Interviews will be conducted with external and internal stakeholders at global, regional and country levels of the Organization. External stakeholders for this evaluation are: ministry of health officials and officials of other relevant governmental institutions; healthcare professional associations and other relevant professional bodies; relevant research institutes, agencies and academia; health care provider institutions; NGOs and civil society; UN Agencies and other relevant multilateral organizations; donor agencies; and other relevant partners.

c. Mission in-country. Following the document reviews and some stakeholder interviews, the country visit will be the opportunity for the evaluation team to develop an in-depth understanding of the perspectives of the various stakeholders around the evaluation questions and collect additional secondary data, in particular from external stakeholders. Depending on the need, the mission might include field visits.

28. Stakeholder consultation. In addition to acting as key informants during the evaluation process, both internal and external stakeholders will be consulted at the drafting stages of the terms of reference, inception note and evaluation report and will have the opportunity to provide comments.

29. Limitation. No major primary quantitative data collection is envisaged to inform this evaluation. The evaluation team will mainly use data (after having assessed their reliability) collected by WHO and partners during the timeframe evaluated.

VIII. Phases and deliverables

30. The evaluation is structured around five phases summarized in Table 3 below.

Table 3: Summary tentative timeline – key evaluation milestones

Main phases Timeline Tasks and deliverables

1. Preparation December 2016 January 2017

Draft and final TOR Evaluation team contracted

2. Inception Jan –Feb 2017 Desk review of existing literature, HQ and RO Briefings Draft and final inception note

3. Data collection and analysis

Feb- March 2017 Key interviews with HQ and RO WHO staff Country visit Aide memoire of key findings (PPT)

4. Reporting March 2017 Draft and final evaluation report

5. Management April 2017 Management response to the evaluation

Page 51: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

8

response and dissemination

recommendations Dissemination of evaluation results

31. Preparation. These TOR are prepared following the WHO Evaluation Practice Handbook. The final version of the TOR takes into consideration results of consultations with key internal and external stakeholders.

Final TOR

32. The inception phase will start with a first review of key documents and briefings with HR, RO and WCO. During the inception phase, the evaluation team will assess the various logical/results frameworks and their underlying Theory of Change. The inception note will close this phase. Its draft will be shared with key internal stakeholders (HQ, RO and CO levels) for their feedback.

Inception note. It will be prepared following the Evaluation Office template and will focus on methodological and planning elements. It will present, taking into account the various logical/results frameworks and evaluation questions, a detailed evaluation framework and the evaluation matrix. Data collection tools and approaches will be clearly identified in the evaluation matrix.

33. Data collection and analysis. This phase will include additional document review, key stakeholder interviews at HQ and RO levels and a country visit. The mission will start a briefing to the WCO and key partners and end with a debriefing with the same group.

Aide memoire of key findings to be prepared at the end of the country visit to be used to support the debriefing with the stakeholders.

34. Reporting. This phase is dedicated to the in-depth analysis of the results of the data and documents analysis and of the data collected through the field work. The results of this analysis will be presented in the evaluation report. The draft evaluation report will be shared with key internal and external stakeholders for comments.

Evaluation Report will be prepared in accordance with the WHO Evaluation Practice Handbook; it will provide an assessment of the results according to the evaluation questions identified above. It will include conclusions based on the evidence generated in the findings and draw actionable recommendations.

35. To be noted: Submission of revised versions of any of the deliverables by the evaluation team will be accompanied by a feedback on each comment provided. This feedback will succinctly summarize if and how comments were addressed and, if they were not, it will justify why.

36. Management response and dissemination of results. The management response will be prepared by the WCO and posted on the internet once finalized alongside the evaluation report. Dissemination of evaluation results and contribution to organizational learning will be ensured at all levels of the Organization as appropriate.

IX. Evaluation team

37. The evaluation team will include two senior consultants and the Chief Evaluation Officer and will be led by the DG Representative for Evaluation and Organizational Learning. Together they bring the relevant expertise in terms of expertise in evaluation, health and WHO’s governance mechanisms.

Page 52: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

9

Annex 2: Evaluation methodology and evaluation matrix

This annex summarizes the approach adopted in this COE and the main methods and tools employed. It draws on the inception note.

Guided by the WHO Evaluation Practice Handbook, the overall methodological approach adopted by the evaluation team is summarized in Figure 1. This shows the sequencing and interrelationship of activities under each of the three main phases of the evaluation process. Concretely, the evaluation was conducted between January and May 2017 by a core team of four members.

Figure 1: Methodological approach

Inception phase

a. Theory of change underlying WHO’s contribution in Thailand

In the absence of an explicit logic model or theory of change (TOC) to frame the contributions of WHO in Thailand over the evaluation period, the evaluation team reconstructed a TOC that clarifies WHO’s contribution to the national health objectives and goals in terms of health outcomes and potentially the health impact of its collaborative programmes with the Government of Thailand, as defined in the CCS and the biennial work plans.

The TOC aims to encompass contributions from all levels of the Organization and all strategic contribution areas of WHO in the country. The TOC is aligned with that validated by WHO in the

Page 53: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

10

context of the evaluation of WHO’s presence in countries12 and was validated by the WR and WCO team during the field mission.

Figure 1: Theory of Change (TOC) – WHO contribution in Thailand 2012-201613

b. Evaluation matrix

Using the TOC, the evaluation team developed an evaluation matrix which defines specific indicators/measures for assessing each sub-question and indicates what data collection method and data sources will be used to inform each of these. The evaluation matrix is available at the end of this Annex.

c. Inception note

The inception note was prepared following the Evaluation Office template and focused on methodological and planning elements of the evaluation. It presented, taking into account the various logical/results frameworks and the evaluation questions, a detailed evaluation framework and the evaluation matrix. Data collection tools and approaches were clearly identified in the evaluation matrix. It was shared with the WCO prior to the mission for their comments.

Data collection phase

The evaluation team used a pragmatic mixed-methods approach in addressing the evaluation questions. The evaluation matrix details for each sub-question the main data collection methods. To this end, different instruments have been employed and evidence from different sources triangulated.

12

See WHO, 2015, Evaluation of WHO’s presence in Countries. 13

The main framing document of the theory of change is the CCS 2012-2016.

Page 54: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

11

a. Documents review

The evaluation matrix identified key documents that were reviewed prior to the mission. Relevant information has been extracted to address the corresponding sub-questions. A preliminary review of documents available had shown limitations in terms of data availability as some of the sub-questions do not easily lend themselves to quantitative assessment. This reinforced the case for combining careful review of different data sources.

b. Stakeholder interviews

These have been the main form of primary data collection. The evaluation team conducted a large number of interviews (list available in Annex 5) with WHO colleagues at the three levels of the Organization as well as with all main partners in-country. Care was taken to ensure that the interviewees felt comfortable to express their opinions. The evaluation used a combination of individual and group interviews across the different activities. In practice, individual interviews were usually the most useful in providing detailed information and opinions. Group interviews, on the other hand, provided helpful insights into retrospectively understanding the processes of decision-making (which have often not been systematically recorded) as well as the implementation processes (where participants identified what elements fed into decisions, and how the implementation process took place over time). By default all interviews have been treated as confidential by the evaluation team.

c. Country mission

Planned after the document review, it took place in February 2017 and was the opportunity for the evaluation to complement the information gathered through stakeholder interviews. The mission started with a briefing with the WCO. An in-country feedback session was organized at the end of the mission with the WCO.

d. Data analysis

The evaluation team triangulated all information collected and compiled information in an evaluation grid structured by evaluation question (EQ), sub-question and indicator. Evaluation findings were then drawn only after a thorough cross-checking and triangulation of all information related to each EQ. This ensured that answers to EQs were based on solid and cross-checked evidence. The evaluation team identified a certain number of challenges to address some of the evaluation questions, which are described below.

Reporting

On the basis of the cross-checked evaluation findings, the team formulated answers to the evaluation questions. These answers informed the drafting of the conclusions. These included, to the extent possible, lessons learned and best practices identified in the course of the evaluation to further strengthen the current CCS.

Finally, the evaluation team provided practical, operational recommendations for future adjustments and actions. Each recommendation is based on the answers to evaluation questions and overall conclusions, which in turn will be linked to evaluation findings per evaluation question and ultimately to the data collected.

Gender, equity and human rights

The evaluation ensured that gender, equity and human rights issues were addressed to the extent possible and through several means. A number of sub-questions within the evaluation matrix are gender sensitive with appropriate related indicators. The document review paid specific attention to how these issues were addressed at planning, implementation, monitoring and evaluation stages of WHO contributions. Finally, these dimensions have been reflected in the interviews.

Page 55: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

12

Limitations of the evaluation

The evaluation encountered a few other relevant issues already identified to some extent in the mid-term review and in the CCS final evaluation:

Though there are broad linkages between the CCS and other WHO corporate planning and reporting tools, these are not clear enough to identify outputs and outcomes specific to the CCS within the WCO work plans.

In the absence of a clear theory of change or of a logical or result framework, the corporate outcomes and outputs defined in the programme budget are not systematically translated at country level with corresponding benchmarks and quantified targets.

Considering that WHO’s expected contribution to national programmes prioritized in the CCS is not systematically identified at the planning stage, it was challenging to establish the extent to which activities undertaken contribute to the achievement of objectives defined in national programmes, plans or strategies.

No major primary quantitative data collection was undertaken to inform this evaluation. The evaluation team mainly used existing data collected by WHO and partners during the timeframe evaluated.

Considering the limitations identified above, the evaluation team could only assess progress for each of the main outcome groups identified in the TOC but was not able to measure them against planned targets as they were not identified in a measurable manner.

Page 56: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

13

Evaluation matrix

Evaluation sub-questions Indicator/measure Main source of information

EQ1 - Were the strategic choices made in the CCS (and other relevant strategic instruments) the right ones to address Thailand’s health needs and coherent with government and partners’ priorities? (relevance)

1.1 Are the CCSs based on a comprehensive health diagnostic of the entire population and on Thailand’s health needs?

- Availability in both CCSs of a comprehensive health diagnostic inclusive of gender-related issues and covering all populations (minorities, migrants) living in Thailand - Changes in health issues/challenges between the two CCSs

Documents review - WHO MTR

14

- WHO Global Health Observatory data - UNICEF MICS 2012 - WB indicators for Thailand - Needs assessment for the selection of priorities

for the CCS 2017-2021

1.2 Are the CCSs coherent with the national health development plans, any other relevant national health strategies and the MDG and SDG targets relevant to Thailand?

Level of alignment of health priorities identified in both CCSs with: - Priorities of the health development plans - MDG targets in Thailand - SDG targets in Thailand

Documents review - WHO MTR - 11

th National Health Development plan (2012-

2016) - MDG indicators (latest national report is 2009) - SDG indicators - Needs assessment for the selection of priorities

for the CCS 2017-2021

1.3 Are the CCSs coherent with the UNPAF? Are the key partners clear about WHO’s role in Thailand?

- Level of alignment of both CCSs with the UNPAFs Document review - UNPAF 2012-2016 KII : UNDP - WCO

- Level of clarity among partners about the role of WHO in Thailand

Documents review - MTR and final evaluation KII - WCO - Government - MOPH - Natational health institutions? (CCS pages 23-25) - WB? - UNDP, UNICEF, UNFPA, UNWOMEN - Main donors to WHO? - Civil society?

14

MTR = Mid-term review of the WHO CCS Thailand 2012-2016 , WHO 2014.

Page 57: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

14

Evaluation sub-questions Indicator/measure Main source of information

1.4 Are the CCSs coherent with the General Programme of Work and aligned with WHO’s international commitments?

Level of coherence between the CCSs and: - MTSP - GPW

Documents review - MTR and final evaluation - MTSP - 11

th and 12

th GPWs

And do they support good governance, gender equality and the empowerment of women?

Availability of explicit reference in both CCS docs to: - good governance - gender equality and empowerment of women

Document review - CCSs KII. - WCO

1.5 Has WHO learned from experience and changed its approach in view of evolving contexts (needs, priorities, etc.) between both CCSs but also during the course of the CCS 2012-2016?

- Changes of orientation in the implementation of the CCS 2012-2016 and rationale for these changes

- Differences between both CCSs based on: o Changes in health needs o Changes in RTG priorities o Changes in WHO regional/global priorities?

Document review - CCSs - MTR - Final evaluation KII. - WCO - RO

1.6 Are the CCSs strategically positioned when it comes to:

- Indication of best practice in terms of strategic positioning Documents review - Both CCSs - Any relevant WCO documents - MTR and final evaluation KII. - WCO - Government - MOPH - National health institutions? (CCS pages 23-25) - WB? - UNDP, UNICEF, UNFPA, UNWOMEN - Main donors to WHO? - Civil society?

1.6.1 Clear identification of WHO’s comparative advantage and clear strategy to maximise it and make a difference?

- Explicit elements of WHO’s comparative advantage identified in both CCSs

- Explicit strategy to value the comparative advantages identified

1.6.2 Capacity of WHO to position health priorities (based on needs analysis) in the national agenda and in those of the numerous national partners in the health sector?

- Clear linkages between CCS priorities and most important health needs in the country as identified in the health diagnostic (see 1.1)

- Indication of role played by WHO in the development of the national health agenda

- Indication of role played by WHO in development of main national partners in the health sector

1.6.3 Specificities of the partnership between WHO and the Royal Thai Government especially in view of the numerous actors involved in the national health sector? And has this positioning evolved between the two CCSs? If so how?

- Indication of partnership elements in both CCSs - indication of evolution between both CCSs - Reasons for change in partners - Reasons for evolution within continuing partners

Page 58: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

15

Evaluation sub-questions Indicator/measure Main source of information

1.6.4 Reflecting the contribution of WHO in terms of intellectual and social capital

- indication of WHO’s role in intellectual and social capital

EQ2 - What is the contribution/added value of WHO towards addressing the country’s health needs and priorities? (effectiveness/elements of impact/progress towards sustainability)

2.1 To what extent were the country biennial work plans (operational during the evaluation period) articulated with the focus areas as defined in the CCS (and other relevant strategic instruments) document (or as amended during course of implementation)?

- availability of explicit linkages between the work plans and the focus areas described in the CCS 2012-2016

- Weight (and trend) of activities in work plans not included in the CCS and rationale for their inclusion in the work plans

Documents review: - Biennial work plans - Others? KII: - WCO management and various programme

managers/lead?

2.2 What were the main results achieved for each outcome, output and deliverable for WCOs as defined in the country biennial work plans, especially in terms of intellectual and social capital ?

- Level of achievement for each CCS priorities and other key activities within and outside the CCS

- Identification of best practices especially in terms of intellectual and social capital

Documents review: - Previous evaluations - Other relevant documents KII: - WCO management and various programme

managers / lead ? - Main partners for programmes with key

achievements identified in the MTR and final eval

2.3 What has been the added value of regional and headquarters contributions to the achievement of results in-country?

- Indication of HQ and/or RO contributions to CCS development (both)

- Indication of HQ and/or RO contributions to specific activities in Thailand

- Indication of participation of Thai partners in regional or global initiatives/capacity development opportunities directly linked to CCS priorities

- Identified best practices

KII - WCO - RO? - HQ? - Partners?

2.4 What has been the contribution of WHO results to long-term changes in health status in-country?

- Indication of long-term WHO engagement in selected areas of work

- Perception of stakeholders on WHO’s role with regard to changes in these areas

- Identified best practices

Document review - CCS 2008-2012 ? KII - WCO - Government - MOPH - National health institutions? (CCS pages 23-25) - WB?

Page 59: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

16

Evaluation sub-questions Indicator/measure Main source of information

- UNDP, UNICEF, UNFPA, UNWOMEN - Main donors to WHO? - Civil society?

2.5 Is there a national ownership of the results and capacities developed?

- Indication of key areas of national capacities developed - Indication of changed practices among partners following

WHO support and capacity development activities - Indication of continued activities by national partners

following end of WHO support - Identified best practices

Document reviews - MTR and final evaluation KII - WCO - Government - MOPH - National health institutions? (CCS pages 23-25) - WB? - UNDP, UNICEF, UNFPA, UNWOMEN - Main donors to WHO? - Civil society?

EQ3 – How did WHO achieve the results? (efficiency)

3.1 What were the key core functions most used to achieve the results?

- Reference to core functions supporting achievement of results in MTR and final evaluation

- Linkages between activities in programme budgets and core functions

- Perception of stakeholders about WHO functions most used - Identified best practices

Document reviews - MTR and final evaluation - Programme budgets KII - WCO - Government - MOPH - National health institutions? (CCS pages 23-25) - WB? - UNDP, UNICEF, UNFPA, UNWOMEN - Main donors to WHO? - Civil society?

3.2 How did the strategic partnerships contribute to the results achieved?

- Reference to the strategic partnerships identified in the CCs, in the MTR and final evaluation

- Indication of their contributions to the results - Perception of strategic partners about the contribution of

the partnerships to the achievements

Document reviews - MTR and final evaluation KII - WCO - Government - MOPH - National health institutions? (CCS pages 23-25) - WB? - UNDP, UNICEF, UNFPA, UNWOMEN - Main donors to WHO? - Civil society?

Page 60: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

17

Evaluation sub-questions Indicator/measure Main source of information

3.3 How did the funding levels and their timeliness affect the results achieved?

- Level of funding compared with budget planned for CCS and other activities

- Timing of funding over the CCS period - Main funding mechanisms used

Document review - Funding data KII - WCO - RO? - HQ? - Main donors to WHO?

3.4 Was the staffing adequate in view of the objectives to be achieved?

- Level and number of staff available for CCS implementation and other activities

- Perception of stakeholders of staffing situation

Document review - Staffing data KII - WCO - RO? - HQ? - Main national partners

3.5 What were the monitoring mechanisms to inform CCS implementation and progress towards targets?

- Availability of monitoring mechanisms - Availability of monitoring reports on progress towards

targets - Identified best practices

Document review - Monitoring reports KII - WCO - RO? - HQ? - Main national partners

3.6 To what extent has the CCS been used to inform WHO country work plans, budget allocations and staffing?

- Availability of explicit linkages between CCS and work plans, budget allocations and staffing

- Weight of the CCS versus other activities undertaken by WCO

Document review - Work plans, budgets KII - WCO - RO? - HQ?

Page 61: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

18

Annex 3: WHO’s main planning instruments and associated

challenges

This Annex presents briefly the main planning instruments WHO has developed to frame its action at

the various levels of the Organization and main implications for country office evaluations.

Figure 1: timeframes of key planning instruments at the different levels of the Organization

The WHO high-level strategic planning document is the General Programme of Work (GPW). It sets

priorities and provides an overall direction for a perennial period. The current 12th GPW

encompasses six years (2014–2019) and defines 6 categories as high-level domains for technical

cooperation and normative work (e.g. communicable diseases, health systems). These categories are

divided into individual programme areas (e.g. malaria, nutrition) and provide a programmatic and

budget structure for the work of WHO. Through a results chain, the GPW furthermore explains how

WHO’s work will be organized over the specific timeframe and how the work of the Organization will

contribute to the achievement of a set of intended outcomes and impacts.15 Hence, the GPW is the

high-level strategic vision for the work of the entire Organization.

Another high-level strategic planning document was the medium-term strategic plan (MTSP), a one-

off format for the time period 2008-2013. It was introduced to update the agenda of the (Eleventh)

GPW at the time, which was laid out for a long period of ten years. The MTSP identified specific

health impacts for 13 strategic objectives, including indicators and targets to be achieved over its six-

year period.16 This approach was then similarly adopted in the subsequent 12th GPW.

At country level, the main strategic planning document to guide WHO’s work is the Country

Cooperation Strategy (CCS). It is a medium-term strategic vision for its technical cooperation in and

with a given Member State, responding to the country’s specific needs and the national targets

under the Sustainable Development Goals17 (WHO 2016: 3-5). The CCS therefore identifies a set of

15

WHO (2014). Twelfth General Programme of Work 2014-2019. Not merely the absence of disease. World Health Organization, Geneva. http://apps.who.int/iris/bitstream/10665/112792/1/GPW_2014-2019_eng.pdf 16

WHO (2012). WHO Reform: Meeting of Member States on programmes and priority setting, World Health Organization, Geneva. http://www.who.int/dg/reform/consultation/WHO_Reform_1_en.pdf?ua=1 17

WHO (2016). WHO Country Cooperation Strategy. Guide 2016. World Health Organization, Geneva. http://www.who.int/country-cooperation/publications/ccs-formulation-guide-2016/en/

Page 62: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

19

priorities, each of which is further broken down into individual focus areas. These focus areas are

linked to the MTSP strategic objectives or GPW outcomes (depending on when they were designed)

and thereby establish a link between the strategic planning at country and corporate level. The time

frame of the CCS is flexible to be aligned with national and United Nations planning cycles and to

accommodate changing circumstances (e.g. emergencies, humanitarian crises or post-conflict

situations).

The strategic priorities and desired results in the GPW find their operational expression for a

particular biennium in WHO’s Programme budget (PB), which puts in concrete terms how intended

outcomes and impacts shall be achieved. The PB is structured by programme area, each one with a

set of outputs defining what the Secretariat will be accountable for delivering during the respective

biennium.18

The PB then serves as the biennial guidance document for the development of workplans. Each

workplan consists of a set of products and services, with associated activities and related costs but

these are not related to the CCS in any explicit way. In WHO’s internal planning system, all products,

services and associated activities are considered as tasks.19 Each task is explicitly linked to one

output in the programme budget at corporate level, which means the task supports its expected

achievement. The workplans ultimately break down the desired results of WHO’s strategic planning

into sets of corresponding tasks. Workplans are developed and implemented by budget centres,

which are generally organizational units.

Some challenges

The elaborations above show that planning at WHO is based on various documents, which are

connected through cross-references at different organizational levels. WHO’s planning framework

intends an explicit interaction between the strategic plans at country (CCS) and corporate level

(GPW/PB). Concretely, CCS priorities and focus areas provide country-level input into the PB bottom-

up planning process and thus into the identification of corporate priorities and budget allocations.

On the other hand, the PB priorities in turn inform new CCS agendas if they are outdated and about

to be renewed.20 However, the concrete processes of the mutual interaction between the CCS and

the PB are not clear. As shown above, all workplans and their respective tasks must relate to

outputs in the PB, regardless of the organizational level at which they are being developed and

implemented. This implies that the PB is directly influencing activities at country level (insofar as

they must at least be linked to it). It is however not clear how or to what extent the worldwide

heterogeneous CCS agendas inform the biennial PB planning process.

As shown in Figure 1, all planning documents have a different timeframe. This can cause

programmatic divergences between the different levels insofar as perennial planning documents,

once drafted and adopted, cannot take into account upcoming strategic shifts being introduced on

another level. Figure 1 visualizes the various planning cycles and timeframes while using the country-

level example of the CCS Thailand.

There is presently a missing link between workplans drafted at country level and the strategic

priorities established in the CCS. WHO's organization-wide planning system is designed in such a

18

WHO (2014). Twelfth General Programme of Work 2014-2019. Not merely the absence of disease. World Health Organization, Geneva. http://apps.who.int/iris/bitstream/10665/112792/1/GPW_2014-2019_eng.pdf 19

WHO (2015). Programme Management. Glossary of Terms. Unpublished internal document. World Health Organization, Geneva 20

WHO (2016). WHO Country Cooperation Strategy. Guide 2016. World Health Organization, Geneva. http://www.who.int/country-cooperation/publications/ccs-formulation-guide-2016/en/

Page 63: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

20

way that all workplans and their respective tasks relate to outputs in the PB (see left side in Figure

2). The programmatic structure in this process are the categories that represent the high-level

domains for WHO‘s work (e.g. communicable diseases). These categories may be, but are often not,

congruent with CCS priorities. Instead, each CCS is supposed to explicitly specify how its various

focus areas are connected to one or more outcomes in the GPW, thus providing another link

between the country and corporate level (see right side in Figure 2). However, this does not allow

drawing conclusions regarding the link between workplans and the agenda of a specific CCS.

Hence, there is no documented traceability how individual tasks in the workplans at country level

are supposed to support CCS priorities or their focus areas. This also means that there is no

systematic way to assign financial figures to CCS priorities.

Finally, there is no systematic monitoring and reporting against results at country level. Indeed, the

tasks included in the workplans are not framed together against a specific objective or expected

outcome expressing the expected contribution of WHO in-country over a period of time in a specific

area of engagement. Nor are there any indicators associated with these except for expenditures and

self-reporting under the form of a narrative.

Figure 2: Relation between strategic and operational planning on country level

Page 64: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

21

Annex 4: Evaluation observations for each priority and main

activities of CCS 2012-2016 and CCS 2017-2021

This annex covers the five CCS priorities plus communicable diseases which are the areas where it was possible to collect information in the most systematic manner. The added value of this annex is to summarize systematically the priority and activity-specific observations to address some of the EQs in the main report. For each of these, two tables have been developed:

The first table clarifies their inclusion in the following three CCSs: 2008-2011, 2012-2016 and 2017-2021, to show trends over time in various areas of engagement.

The second table summarizes the main observations mapped against sub-evaluation questions defined in the evaluation matrix in Annex 2 (column 1). Column 2 summarizes the main findings from the CCS mid-term review (CCS MTR) and CCS final evaluation (CCS FE). Source of text included is always clarified. The third column summarizes key information collected by the evaluation through other documents reviewed, the WCO self-assessment and elements from the interviews specific to the particular topic.

Community health system

CCS 2008-2011 CCS 2012-2016 CCS 2017-2021

Included Priority until 2013. Not included

Objective: - Empower and strengthen the subdistrict health system so that the community health system will be more effective and responsive to the health needs of the population.

Main focus area:

Building up and strengthening the community health system

Approach: - Develop and advocate for national policies on strengthening community health systems;

- Support the decentralization policy; - Strengthen primary care; - Support social movements to gain

support and public recognition for community health systems; and

- Support development of new tools and social innovations.

Lead agencies: National Health Security Office (NHSO) Health Systems Research Institute (HSRI)

Source: CCS 2012-2016 Summary of key observations per sub-evaluation question

EQ sub-questions Key CCS mid-term review (CCS MTR) and CCS final evaluation (CCS FE) observations

Key COE observations (documents & interviews)

1.2 Coherence with national health plans (NHDP), strategies, etc.

Primary health care at the core of Thai health policies since 1978 (CCS MTR )

- Coherence with Strategy 4 of the NHDP aiming at strengthening health care systems, ensuring thorough and equitable access

1.4 Coherence with GPW - Strengthening health systems is prioritized across most strategic objectives of the MTSP and the priorities of the 12th GPW.

1.6.3 Partnerships - The MOPH changed the membership of the CCS steering committee in 2013 excluding the HSRI which consequently notified WHO that it would like to

Page 65: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

22

terminate its contract signed with WHO effective July 2013. (CCS MTR)

2.2 Main results achieved - The CCS MTR identified 4 strategic areas of work with very variable results in each of them.

- Various activities undertaken but no results reported

2.5 National ownership of results

- Planning stage: Participative consultation and planning process unanimously ranked the community health programme area at the top (CCS MTR)

3.1 core functions Among others

- Technical support - Initially WHO provided regular technical assistance (extensive peer review process, participation in steering committee meetings). But following staffing change in the office that the technical assistance reduced (CCS MTR).

- Leadership - WHO convening power helped to integrate the visions and strategies of existing autonomous health agencies and build up a partnership. (CCS MTR)

3.2 Partnerships - Huge implementation challenges in pooling funds and integrating efforts from all parties to implement the programme (CCS MTR)

3.4 Staffing - Initially WHO had a national professional officer participating regularly in the programme management team meetings but afterwards it did not play a significant role following staffing change in the office (CCS MTR)

- National programme manager might not have all the required knowledge and skills. (CCS MTR)

3.5 Monitoring - Over time WHO contribution seemingly changed towards financial accountability and fund management and away from technical advice. (CCS MTR)

Multisectoral networking for NCD control

CCS 2008-2011 CCS 2012-2016 CCS 2017-2021

Included Priority Priority

Objective:

- Promote collaboration, partnership and integration among various sectors to tackle NCDs, including health-related and non-health related sectors in Thailand.

- Strengthen national policies, plans and interventions for prevention and control of five main NCDs: cardiovascular diseases, diabetes, cancers, chronic respiratory diseases and hypertension.

Impact: Thailand on track to achieve the nine national and global NCD targets

Main focus area:

Building up networks for implementing integrated NCD control.

- Tobacco control - Early detection,

prevention and control of cardiovascular disease (hypertension and diabetes)

- Reduce childhood obesity

Approach:

- Promote collaboration and partnership among agencies;

CCS deliverables - NCD coordination

Page 66: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

23

Source: CCS 2012-2016 and CCS 2017-2021

Summary of key observations per sub-evaluation question EQ sub-question Key CCS MTR and final evaluation

observations Key COE observations (documents &

interviews)

1.1 Priority based on population health needs

- Behavioural factors in the Thai population facilitate the increase in NCDs and injuries

21

- In 2014 NCD accounted for 71% of total deaths in Thailand

22

1.2 Coherence with national health plans (NHDP), strategies, etc.

- The RTG recognizes the importance of both health and non-health sectors to meet the challenges of NCD prevention and control as translated in the Thai Healthy Lifestyle strategic plan 2011-2020 (CCS MTR)

- Coherent with NHDP strategy 2

1.3 Coherence with UNPAF

- For the period 2017-2021 NCD indicators to be integrated in the new UNPAF result matrix.

1.4 Coherence with GPW - In 2012, NCD was placed more firmly in the global agenda by the UN 66

th

General Assembly asking WHO and Member States to prevent and control NCDs. (CCS FE)

- Aligned with the 11th GPW (SO 3, 6 and 7) - Aligned with MTSP strategic objective 3 to

prevent and reduce disease, disability and premature death from chronic noncommunicable conditions, mental disorders, violence and injuries and visual impairment

- Aligned with the 12th GPW leadership priority addressing the challenge of noncommunicable diseases and mental health, violence and injuries and disabilities.

1.5 WHO adaptation capacity to evolving context

- While the CCS 2012-2016 was focused on partnerships and networking to set the foundations, the CCS 2017-2021 is clearly focused on 3 diseases and has identified clear deliverables.

1.6.1 WHO comparative advantage

- WHO presence and visibility highlighted to stakeholders, including high-level management, the importance of the programme at the global level. (CCS MTR)

- Following the experience of the past 2 CCSs, the proposal for NCD for the CCS 2017-2021, considers that WHO will add value, among others, in the following areas: advocacy convening power as a neutral broker, promotion of Thai expertise abroad through RO and HQ, etc.

1.6.3 Partnerships - Positive partnership and complementarity observed with CDC on NCD

- Focus on supporting NCDNET (network): amalgamation of multiple partners to achieve the goals of the Thai healthy life style policy

- An important determinant of the success of NCD control is coordination and collaboration

21

MOPH, 2007, ‘Survey report of behavioural factors of non-communicable disease and injuries in Thailand’. 22

http://www.who.int/nmh/countries/tha_en.pdf (consulted 30 March 2017).

- Networking integration and cooperation through established mechanisms to strengthen policy, social communication and capacity building; and

- Establish the linkage and collaboration with regional and global levels of NCD networks.

mechanisms strengthened and streamlined

- New knowledge generated, disseminated and used for policy development and programme improvement

- NCD surveillance system harmonized and rationalized

Lead agencies:

Thai Health Promotion Foundation Thai Health Promotion Foundation

Page 67: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

24

between the public health agencies which remains a key challenge for the RTG.

2.2 Main results achieved - Successful collaboration efforts in forming Salt Net which aimed to reduce salt consumption. Otherwise, majority of the work had been concentrated on organizing meetings and forums which have merits in building working relationships with the network members. (CCS MTR)

- Initiated a forum for academics, a policy advocacy training course was developed and delivered several times. Capacities of junior policy researchers have been strengthened and good documents on the NCD situation in Thailand were produced. (CCS FE)

- However, the major expectation was to have this NCDs network as a mechanism to implement the Thai Healthy Lifestyle strategic plan. It is clear that the network did not fulfil this purpose (CCS FE)

- Thai NCD alliance, a network of agencies working on NCD, was established in April 2016.

- Four risk factors and diseases were included in the integrated national NCD plan to align it with the global NCD plan.

- MOPH developed the first national NCD guidelines.

2.3 Regional and HQ contributions

- Technical support provided by HQ and SEARO particularly in the areas of nutrition, salt reduction, tobacco control

- Use of advocacy materials developed by HQ and SEARO

2.4 Contribution to long term changes

- The Thai Cabinet approved: o a new Tobacco Product Control

Act which, among others, aims at limiting where tobacco products can be sold and consumed

o the control of marketing of infant and young children food act.

2.5 National ownership of results

- The lack of continued commitment from high-level management in the MOPH affected deeply the advancement of the Thai Healthy Lifestyle strategic plan. This impacted the national resources available and involvement of national partners to achieve the objectives (CCS MTR)

- For the new CCS, NCD planning is done jointly between national partners and WHO with wider and stronger engagement of stakeholders than before.

3.1 core functions Among others Among others

- Technical support - WHO active involvement in drafting the strategic plan and providing guidance and technical inputs to the secretariat team of this programme (CCS MTR)

- Translation of HQ guidance in national language

- Leadership - WHO presence and visibility highlighted to stakeholders, including high-level management, the importance of the programme at the global level. (CCS MTR)

- WHO convened first meeting on government partnership to tackle NCD in Thailand to align activities of NCD bureau at MOPH and NCD net.

- WHO Director-General explicit support to MOPH on Thai Healthy Lifestyle strategic plan

- Norms & standards - Four risk factors and diseases were included in the integrated national NCD plan to align it with the global NCD plan.

- Development of national physical activity guidelines taking into account best practices and WHO recommendations

- Research - Produced good NCD research and documents and generated knowledge

- Generation of evidence on tobacco use among the youth

Page 68: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

25

on the topic (CCS FE)

- Policy options - Creation of a public health agency dealing with tobacco control.

- Promulgation of tobacco laws and regulations and introduction of sin tax as a control measure

- Monitoring health trends

- Supported data collection analysis and reporting for the global youth tobacco survey and the global school health survey

3.2 Partnerships - The NCD network did not function as a mechanism to strengthen the implementation of the Thai Healthy Lifestyle strategic plan. (CCS FE)

- NCD programme management moved to MOPH with the new CCS

- For the new CCS, planning for NCD is done jointly between national partners and WHO with wider and stronger engagement of stakeholders than before.

3.4 Staffing - Challenges in establishing a common understanding of the roles, time commitments, qualification and capacity of staff of the network secretariat (CCS MTR)

- WHO recruited a recognized retired Thai senior official as programme manager for the first 2 years of implementation which helped connect WHO and the partners as long as he was there. (CCS FE)

- This priority is supported by both an international and a national staff position since 2016.

Disaster preparedness and response

Source: CCS 2012-2016

CCS 2008-2011 CCS 2012-2016 CCS 2017-2021

Not included Priority Not included

Objective:

- Establish coordination and collaboration mechanisms in the Disaster Health Emergency Management System among various national and international agencies;

- Further support the development of the Disaster Health Emergency Management System to be effectively and efficiently integrated and linked with relevant agencies at all levels in institutional, legislative frameworks, policies, SOPs, contingency plans and capacity building;

- Engage various sectors systematically to establish mechanisms for disaster prevention, preparedness, response, recovery and rehabilitation.

Main focus area:

- Strengthening national capacity and coordination in disaster management, particularly in the health area.

Approach:

- Establish a well-functioning agency network for maximum coordination, cooperation and collaboration in disaster health emergency management;

- Strengthen human resource capacity and resource planning;

- Establish a management structure and disaster response plan.

Lead agencies:

NIEM, WHO

Page 69: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

26

Summary of key observations per sub-evaluation question EQ sub-question Key CCS MTR and final evaluation

observations Key COE observations (documents &

interviews)

1.2 Coherence with national health plans (NHDP), strategies, etc.

- After the 2004 Tsunami, the RTG recognized the need to strengthen disaster risk management. Also, following the 2011 worst floods in half a century, the Prime Minister declared disasters and emergencies priority areas. There is overall a strong commitment and mandate from the RTG, MOPH and WHO to develop a more effective disaster preparedness and management system within the health sector. (CCS MTR)

- Coherence with the 11th

NHDP strategy for further development of systems for monitoring, warning and management of disasters, accidents and health threats

1.3 Coherence with UNPAF

- This priority is coherent with the UNPAF expected outcome on mainstreaming of climate change adaptation by the key line ministries into their sectoral and provincial plans, policies and budgets (this outcome includes both WHO and NIEM as key partners).

1.4 Coherence with GPW - In 2011, the WHA passed resolution WHA64.10 to strengthen all-hazards health emergency and disaster risk management programmes as part of national and subnational health systems. (CCS MTR)

2.2 Main results achieved - Development of a framework for health sector management for Thailand with the objective of bringing all ongoing and planned activities together within one systematic strategic framework (CCS MTR)

- WHO contributed mostly to build a momentum in engaging MOPH, the NIEM and other stakeholders in enhance focus and work on disaster preparedness and response

- Adaptation and piloting of the hospital safety index

- Development of the public health emergency operation centre

- MOPH developed a disaster response plan for people with disabilities with the support of WHO, placing Thailand among the first countries in the world with such a specific plan.

2.3 Regional and HQ contributions

- RO contribution to the assessment of national emergency preparedness in Thailand using the regional WHO benchmarks (CCS MTR)

2.5 National ownership of results

- According to NIEM, WHO funding through the direct financial contribution mechanism implies less strict rules on use of the funds compared to other mechanisms of financial support from WHO (CCS MTR)

3.1 core functions

- Technical support - WO provided a technical officer, much appreciated by the key partners, which created a momentum in engaging various national stakeholders in enhanced focus and work on disaster preparedness and response (CCS MTR)

- Leadership - WHO used its convening power to call events at national or regional level (international conference on the implementation of the health aspects of the Sendai framework for disaster risk reduction. (CCS FE)

- Norms & standards - Adaptation and piloting of the hospital safety index (CCS FE)

- Research - Emphasis on studies and research (CCS

Page 70: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

27

MTR) - WCO has played a recognized role in

knowledge sharing, including in research (e.g. Technical Officer cooperating with Mahidol University), and there were repeated requests for WHO to further develop its work in knowledge management (CCS FE)

- WHO played a significant advocacy role in the DM programme. WHO was the initiator of the Global Hospital Safety initiative (CCS FE)

3.2 Partnerships - Establishment of the Bureau of Public Health Emergency Response within MOPH showed increased priority by MOPH but also created some ambiguity among stakeholders in understanding the different roles and responsibilities of all partners (CCS MTR)

- Concern over the bicephalic management of the programme which caused delays and misunderstandings. (CCS FE)

- Issues or roles not resolved across the CCS duration

3.3 Funding - Pooled funding from WHO and various national agencies has created national ownership and is perceived as more sustainable compared to the traditional manner in which WHO supported programme implementation (CCS MTR)

3.5 Monitoring - Further reporting according to the indicators and set targets in the agreed monitoring framework would be an advantage. Limited documentation on potential outcomes and impact of the programme. Request from MOPH to WHO to enhance skills in designing general monitoring and evaluation frameworks for the programme. (CCS MTR)

International trade and health

CCS 2008-2011 CCS 2012-2016 CCS 2017-2021

Not included Priority Global Health Diplomacy sub-programme

Objective:

- Build individual and institutional capacities and generate evidence to support coherent policy decisions on international trade and health for positive health outcomes of the population.

Impact: - Evidence-based and

participatory policy decisions and trade negotiation process towards coherent trade and health policies for positive health outcomes

Main focus area:

Build national capacity in trade and health negotiation

Approach:

- Knowledge generation - Capacity building - Network strengthening

CCS deliverables - Concrete and timely evidence

to support international trade policy decisions and preparedness

- International trade and health information clearing house accessible by the networks and general public

- Strengthened capacities for

Page 71: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

28

Source: CCS 2012-2016 and CCS 2017-2021

Summary of key observations per sub-evaluation question EQ sub-question Key CCS MTR and final evaluation

observations Key COE observations (documents &

interviews)

1.2 Coherence with national health plans (NHDP), strategies,. etc.

- Consistent with the national economic development plan (CCS MTR)

1.4 Coherence with GPW - Various WHA resolutions have urged Member States to consider linkages of trade-related policies with health policies. (CCS MTR)

- Though not a priority, issues related to trade are brought up regularly in the GPW.

1.5 WHO adaptation capacity to evolving context

- Thailand is one of the few countries where international trade and health is one of the CCS priorities

1.6.1 WHO comparative advantage

- Following the experience of the CCS 2012-2016, the proposal for international trade and health for the CCS 2017-2021, considers that WHO will add value, among others, in the following areas: extensive technical knowledge, access to international expertise, guidance, access to experience from other countries.

- International trade and health issues are not very well known in Thailand. According to the RTG, without the CCS progress in this area could not have been made. The credibility of WHO on this issue has been pivotal to enable work of the health sector with the ministry of commerce.

2.2 Main results achieved - Major outcomes are the collaborative engagement of health and non-health government officials and policy makers and the enhancement of capacities of all partner organizations related to the interface between international trade and health (CCS MTR)

- Increased knowledge and evidence available on topics surrounding trade negotiations and health. (CCS FE)

- Availability of evidence to inform trade negotiations and political decisions

- WHO support for the first national conference on international trade and health

- Recognition by Ministry of Commerce and Ministry of Foreign Affairs that their knowledge on health issues and global health agreements is limited

-

2.3 Regional and HQ contributions

- RO contributions of expertise have been critically important to RTG as the WCO did not have the relevant expertise.

2.5 National ownership of results

- Trade negotiations are almost exclusively a Thai undertaking but contribution of WHO considered useful. (CCS FE)

3.1 core functions

- Technical support - Mobilisation of individual experts (CCS MTR)

- Mobilisation of expertise and provision of technical support to generate evidence proved to be critically important.

knowledge generation and policy advocacy

- Strong networks and collaboration with partners and stakeholders to enable better knowledge generation and participatory trade negotiation process where health is of concern

Lead agencies:

International Health Policy Programme International Health Policy Programme

Page 72: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

29

- Leadership - Recognized leadership of WHO in coordinating other stakeholders (outside health) in programme implementation and establishing linkages with international experts and organizations (CCS MTR)

- Research - Importance of generating evidence especially around tobacco control, alcohol and access to medicines where there are powerful national and international interests lobbying against public health interests (CCS FE)

3.4 Staffing - The WCO contributed to this issue but was not able to meet all the expectations in terms of technical expertise. According to the RTG, it would be important for the WCO to ensure availability of expertise in areas of other determinants of health ( economics, trade, etc.)

Road safety

CCS 2008-2011 CCS 2012-2016 CCS 2017-2021

Not included Priority Priority

Objective:

- Establish international coordination and knowledge sharing on strengthening of Thailand’s road safety network, particularly in relation to motorcycle safety;

- Substantially reduce the rate of motorcycle-related injuries and death.

Impact: - Reduced morbidity and

mortality from road traffic injuries

Main focus area:

- Improve national road safety programme effectiveness through multisectoral and international collaboration.

- Strengthen road safety management and coordination

- Improve national traffic data system

- Improve legislation and enforcement

Approach:

- Identify a lead agency in government to guide the national road traffic safety effort and a mechanism for intersectoral action;

- Undertake an assessment of the problem in terms of its magnitude, policy, and institutional settings;

- Strengthen the national master plan on road safety on aspects of behavioural and legislative strategies and actions and; allocate the needed human and financial resources;

- Implement specific actions to prevent road traffic crashes, minimize injuries and their consequences, and evaluate the impact of these actions as they relate to motorcyclists;

- Maintain high-quality, real-time information on road traffic accidents in order to accurately monitor levels and trends; and

- Support the development of national capacity and international cooperation.

CCS deliverables - Effective coordination and

management through reorganization of the Road Safety Directing Center into a robust government agency capable of leading road safety action in Thailand toward Vision Zero implementation

- Excellence in road safety data integration with timely analytics supporting evidence based investments in road safety action. The quality of data will be improved to the degree that WHO will not need to estimate the fatality rate for the next Global Report on Road Safety in 2019 and will use data submitted by Thailand.

- Road safety legislation meets international best practice for all risk factors and improves enforcement leading to improved behaviours, reduction in crashes and reduced fatalities.

Page 73: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

30

Source: CCS 2012-2016 and CCS 2017-2021

Summary of key observations per sub-evaluation question EQ sub-question Key CCS MTR and final evaluation

observations Key COE observations (documents &

interviews)

1.1 Priority based on population health needs

- Road traffic injuries is a an important public health problem.

- The WHO’s 2015 global report on Road Safety ranked Thailand 2nd in the world as regards mortality rates for road accidents: 24,000 annual deaths, while RTG estimate was about 14,000. This led to in-depth joint analysis of death certificates, police reports and insurance data....and a revised national estimate of 21,000 deaths/yr.

1.2 Coherence with national health plans (NHDP), strategies, etc.

- Coherence with the 11th

NHDP strategy for further development of systems for monitoring, warning and management of disasters, accidents and health threats.

1.4 Coherence with GPW - Coherence with GPW as road safety is a clear component of noncommunicable disease priorities

1.6.2 WHO positioning of health priorities

- WHO pushed to have road safety included in the CCS and also proved to be a strong advocate for road safety in the country.

2.2 Main results achieved - At the time of the CCS MTR, progress was still far from satisfactory mainly because: coordination was an issue for implementation; no funding was available for joint implementation of overall activities; the lead agency has no legal authorities and powers to implement programmes, regulations and activities; and the lead agency was not accepted by all partners. (CCS MTR)

- Initial focus of the programme was predominantly on campaigns but these were not countrywide, visible and continuous but occurred more during festival times. (CCS MTR)

- Although the stated objectives were not achieved, the programme completed a set of planned and useful activities. (CCS FE)

- Despite a slow start, a productive period of legislative activities, starting in late 2015 through 2016, has been instrumental in putting Thailand on the way towards stronger road safety laws and regulations. In December 2016, amendments to driver licensing regulations were adopted and announced in the Royal Gazette, effective 1 January 2017.

- WHO commissioned Thailand’s Road Safety Institutional and Legal Assessment with the financial support from the Bloomberg Initiative. The assessment revealed legislative improvement needs for speed, drink driving enforcement, helmets, seatbelts and child restraints. WHO also successfully advocated for the establishment of the Working Group to Review Road Safety Legislation under the national Road Safety Directing Center. The Working Group submitted to the Cabinet for approval a set of road safety legislative amendments with an aim that they are effective and enforced by the end of 2016.

2.3 Regional and HQ contributions

- The Bloomberg funding is managed by HQ and Thailand is one of the countries benefitting from the second phase of this initiative.

- WHO support, both through the WCO and RO/HQ, has been critical in influencing national policy, including through direct contact with the minister to promote road safety.

2.4 Contribution to long term changes

- Major overhaul, most significant in the last 40 years of the legislation related to road safety, directly attributable to WHO advocacy efforts and technical support

Lead agencies:

Thai Health Thai Health

Page 74: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

31

3.1 core functions

- Technical support - Translation (into Thai) and dissemination of major technical documents has also been a component of project coordination to make these materials accessible to the Thai audience and use them to provide needed technical support

- Leadership - WCO leveraged various opportunities – a National Road Safety Seminar in December 2015, a national road safety conference following the 2nd Global High Level Conference on Road Safety in Brasilia in 2016, and the formation of the Embassy Friends of Road Safety to advocate for improvements in road safety including strengthening laws, regulations and enforcement.

- Research - WHO commissioned Thailand’s Road Safety Institutional and Legal Assessment.

- Policy options - WHO support, both through the WCO and RO/HQ has been critical in influencing national policy, including through direct contact with the minister to promote road safety.

- Monitoring health trends

3.2 Partnerships - Road safety was undertaken as an independent and joint activity by several stakeholders without significant coordination, with each organization pursuing its own independent agenda until these stakeholders were brought together under the CCS. The preparation of a proposal on road safety meant that road safety would now be addressed through a coordinated programme of work. (CCS MTR)

- Explicit role of a WHO collaborating centre in the implementation of this programme. It mainly played a role in advocacy and capacity building.

- Need for further clarity on role and responsibilities of all members of the partnerships. With nearly 15 ministries and departments involved in road safety, this represents a major challenge. (CCS MTR)

- A new stakeholder assessment was completed in December 2015 to provide better understanding of key road safety players and their roles, help guide the strengthening of existing coordination mechanisms, and help to define strategies to push the improvements forward.

3.3 Funding - WHO was able to use the Thailand road safety programme to mobilise resources from the Bloomberg Initiative.

3.4 Staffing - WHO should ensure that one technical staff member is assigned to the WCO work with national counterparts throughout the term of the CCS.

Page 75: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

32

Border and migrant health programme

Source: CCS 2012-2016 and CCS 2017-2021

Summary of key observations per sub-evaluation question EQ sub-question Key CCS MTR and final evaluation

observations Key COE observations (documents &

interviews)

1.1 Priority based on population health needs

- Thailand is the primary host country for low-skilled workers (about 2.7 million) from three neighbouring countries: Cambodia, Laos and Myanmar. In addition, there are refugees living in camps and Myanmar displaced persons all living in the four border provinces (about 428,000 people). (CCS MTR)

1.2 Coherence with national health plans (NHDP), strategies, etc.

- Although migrant-related public health challenges and concerns are well documented and recognized, policies to better attend to migrant health lag behind this recognition and are incoherent. (CCS MTR)

- A major development was the launching of the MOPH Border Health Development Master Plan 2012-2016 aligned with the NHDP. (CCS MTR)

1.3 Coherence with UNPAF

- Migrant issues explicit in the UNPAF and regular collaboration with UNICEF, UNFPA and IOM observed

1.6.2 WHO positioning of health priorities

- WHO played a critical role to position migrant health in the CCS though it is not considered a major issue by the MOPH in terms of burden of disease.

2.2 Main results achieved - As part of the programme, border health units were established and staffed in the targeted provinces; guidelines and monitoring and evaluation framework were developed; capacity building was provided in all 31 provinces. Migrant

CCS 2008-2011 CCS 2012-2016 CCS 2017-2021

Included Part of unfinished agenda and became a priority after ending of community health component

Priority

Objective:

Ensuring equitable access to health services among migrants and mobile populations

Impact: Improved health service delivery and health status of migrants in Thailand

Approach:

- Conducting a systematic literature review on migrants health

- Encourage and promote collaboration among partners involved in developing health system for migrants

CCS deliverables - Timely strategic information is

generated to guide policy decisions related to the health security of border and migrant populations.

- Clear administrative structure established to respond to the health needs of border and migrant population at national and sub-national levels

- Increased health and insurance coverage among migrant and vulnerable populations

- Migrant friendly health services promoted

Lead agencies: Bureau of Policy and Strategy International Health Policy Program

Page 76: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

33

health information centres are operational in the four border provinces and at central level, establishing a foundation for data on border and migrant health. In depth analysis of financing mechanisms for migrant health care was carried out. (CCS MTR)

- All planned studies were undertaken but these represent only a small part of activities undertaken and to which WHO provided support.

2.3 Regional and HQ contributions

- Access to knowledge and expertise of other parts of the Organization and similarly WHO Thailand shared experience with other countries and regions (CCS MTR)

- WR Mekong group provides a venue to work on migrant health at a sub-regional level covering countries in SEARO and others in WPRO (which is sometimes perceived as a challenge by some external stakeholders) .

- Direct support to country office from either the Regional or HQ level has been very limited.

2.5 National ownership of results

- Migrant health is gaining importance in the RTG’s agenda. (CCS FE)

- Increasing engagement of RTG with policies and broader support for migrant health, including increasing resource allocation over time

3.1 core functions

- Technical support - WHO support to the Border Health Development Master Plan 2012-2016 (CCS MTR)

- Technical assistance in and out of refugee or displaced persons encampments (CCS FE)

- WHO provided technical advice mostly on communicable diseases which are at the centre of the work on migrant health.

- Leadership - Neutral space to bring together various stakeholder groups and to ensure health messages are included in multisectoral debates (CCS MTR)

- Convening power of WHO to call all stakeholders for migrant health-related national, sub-regional or regional events (CCS FE)

- 2015 meeting of the ASEAN included an agenda item related to migrant health

- Norms & standards - Provision of guidelines and protocols for NGOs assisting migrants (CCS FE)

- Research - Evidence-based advice (CCS MTR)

- Policy options - Support for the development of the border health development master plan (2012-2016)

- Monitoring health trends

- WHO support for the development of information systems (CCS MTR)

3.2 Partnerships - Ability of WHO to connect partners with relevant persons in the government is pertinent. (CCS MTR)

- Initially WHO was not included in the programme management board

3.3 Funding - Programme 80% funded by the EU and 20% by WHO (CCS MTR)

3.4 Staffing - 1 international, 1 national and 1 support staff in WCO (CCS MTR)

Page 77: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

34

Other CCS activity: communicable disease control

Source: CCS 2012-2016 and CCS 2017-2021

Summary of key observations per sub-evaluation question EQ sub-question Key CCS MTR and final evaluation

observations Key COE observations (documents &

interviews)

1.1 Priority based on population health needs

- Although incidence of mortality from many communicable diseases has decreased, some diseases of this group continue to pose major challenges to health development in the country (CCS MTR)

1.2 Coherence with national health plans (NHDP), strategies, etc.

- Strong ownership by MOPH of communicable disease control and therefore not considered a priority for WHO involvement through the CCS

1.6.2 WHO positioning of health priorities

- WHO contribution has been perceived as bringing the success in communicable disease control. These are no longer leading causes of mortality in the general population. This however resulted in communicable disease control becoming a lower priority in the CCS. But WHO support has continued.

- WHO continues to prioritize malaria and tuberculosis though not a priority of the CCS.

1.6.3 Partnerships - The department of disease control is hosting 2 WHO collaborating centres active in this area.

2.2 Main results achieved - A major collaboration between WHO and the RTG has been on supporting disease control programmes. The collaboration has been perceived as bringing the success in disease control in the country. Communicable diseases are no longer leading causes of mortality in the general population. The success has, to some degree, had a negative effect in that communicable disease control has become a low priority for collaboration, and hence it is not in the list of priority programmes in the current CCS. Nevertheless, support from WHO on disease control has continued. (CCS MTR)

- As per global polio end-game strategy, technical advice and support in-country provided by WHO

- Technical advice and support in the development of national plans reflecting the global/regional strategies on malaria elimination provided by WHO

- Lead agency for the national malaria program review (MPR) 2011 and 2015

- National Operational Plan to End AIDS supported by WHO

- Development of the National Strategic Plan for Tuberculosis Control supported by WHO.

- Completion of surveillance and epidemiological assessment of tuberculosis in Thailand supported by WHO

- Support to the Bureau of AIDS Tuberculosis and STIs for the development of guidelines on National Anti-Retroviral Treatment and Elimination of Congenital Syphilis. These are now completed.

- Support for completion of National Tuberculosis Prevalence Survey, and

CCS 2008-2011 CCS 2012-2016 CCS 2017-2021

Included Part of the major public health challenges and the unfinished agenda

Not included

Main focus areas:

- TB control - HIV prevention and care - Malaria control

Page 78: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

35

coordination of revised epidemiological estimates for the burden of tuberculosis in Thailand.

2.3 Regional and HQ contributions

- Contribution of WCO to global reporting processes (Universal Access, UNGASS (2015 and 2016), Global Tuberculosis Report (2015), SEARO Tuberculosis Report (2015) and SEARO HIV Report (2015)

- HQ and regional office have been most effective at providing the “WHO brand”. WHO has credibility in the country and regional and global WHO experts have influence.

2.4 Contribution to long term changes

- Elimination of mother-to-child transmission of HIV and syphilis, only the 2nd country after Cuba to do so

3.1 core functions

- Technical support - Technical assistance has been provided in almost all communicable disease areas. Most support is in the form of technical advice and participation by experts from the WCO and the RO (CCS MTR)

- WCO instrumental in advocating and providing technical guidance in the updating of the national malaria treatment guidelines.

- Leadership - WHO also promotes intercountry collaboration, which brings together national health authorities and disease control personnel from various countries in the Region, to exchange information on the disease situation and explore intercountry and cross-border collaboration. (CCS MTR)

- Norms & standards - Guideline development and consensus generation on the treatment regimen for extensively drug resistant to tuberculosis.

- Research - Research and reviews to inform policy decision-making in selected areas such HIV treatment

- Policy options - Factilitate upstream policy implementation through high level consultations

Page 79: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

36

Annex 5: List of people met

WHO Country Office Kritsiam Arayawongchai National Professional Officer Richard Brown Programme Officer Sushera Bunluesin National Professional Officer Daniel Kertesz Country Representative Aree Moungsookjareaoun National Professional Officer Renu Marg Medical Officer Mukta Sharma Programme Manager Liviu Vedrasco Technical Officer Isabelle Walhin Administrative Officer WHO Regional Office Yonas Tegen Planning Officer and former WHO Representative in Thailand Arun Bhadra Thapa Director Programme Management WHO Headquarter Shambhu Acharya Director, Country Cooperation & Collaboration with UN System Georgia Galazoula Planning Officer, Planning Resource Coordination and Performance

Monitoring Malgorzata Grzemska Coordinator, Technical Support Coordination Imre Hollo Director, Planning Resource Coordination and Performance

Monitoring Etienne Krug Director, Management NCDs, Disability, Violence and Injury

Prevention Evelyn Murphy Technical Officer, Unintentional Injury Prevention Bernard Tomas Planning Officer, Planning Resource Coordination and Performance

Monitoring Marianna Trias Public Health Officer, Country Cooperation & Collaboration with UN

System Rui Vaz Coordinator, Country Cooperation & Collaboration with UN System National partners Dr Bundit Sornpaisarn Director, Thai Health Promotion Foundation Dr Chutima Akaleephan Researcher, International Health Policy Programme Foundation Dr Kumnuan Ungchusak Former Senior Expert, Department of Disease Control, MOPH Dr Nakorn Premsri Director, Principal Recipient Administrative Office, Department of

Disease Control, MOPH Dr Nithima Sumpradit Pharmacist, Professional Level, Bureau of Drug Control, Thai Food

and Drug Administration (FDA), MOPH Dr Orapan Srisookwatana Deputy Secretary General, National Health Commission Office Dr Pathom Sawanpanyalert Senior Expert in Health Promotion (Public Health Physician), MOPH Dr Phalin Kamolwat Director, Bureau of Tuberculosis, Department of Disease Control,

MOPH Dr Phumin Silapunt Deputy-Secretary General, National Institute of Emergency

Medicine Dr Phusit Prakongsai Director, Bureau of International Health, MOPH Dr Siriwan Pitayarangsarit Director, International Health Policy Programme Foundation

Page 80: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

37

Dr Somsak Akksilp Deputy Permanent Secretary, Office of the Permanent Secretary, MOPH

Dr Suchada Chaivooth Director HIV/Aids and Tuberculosis Program, National Health Security Office

Mr Suksunt Jittimanee Chief of Strategy and Evaluation, Bureau of Tuberculosis, Department of Disease Control, MOPH

Dr Supakit Sirilak Inspector General, Office of the Inspector General, MOPH Dr Supamit Chunsuttiwat Former Senior Expert, Department of Disease Control, MOPH Dr Supattra Srivanichakorn Director, Center for Policy & Strategy Development for NCDs,

Senior Expert and Chief NCDs Planning and Strategy Office, Department of Disease Control, Ministry of Public Health

Dr Supreda Adulyanon Chief Executive Officer, Thai Health Promotion Foundation Dr Suriya Wongkongkatep Former Deputy Secretary, MOPH Dr Suwit Wibulpolprasert Vice Chair, International Health Policy Programme Foundation Dr Thanapong Jinvong Manager, Road Safety Group Thailand Dr Wittaya Wongkongkatep Chair, Road Safety Policy Foundation Dr Wiwat Rojanapithayakorn Director, Centre for Health Policy and Management, Faculty of

Medicine, Mahidol University International partners John MacArthur Director US CDC Collaboration Nenette Motus IOM, Regional Director for Asia and the Pacific Tatiana Shoumilina Country Director UNAIDS Luc Stevens UN resident coordinator & Representative UNDP Dr Wassana Im-em Assistant Representative, UNFPA

Page 81: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

38

Annex 6: Bibliography

Burkholder, Dr Brent (2016). Review of individual staff functions and team structure at WHO country office (internal document) Human Development Report 2016. Human Development for Everyone, Briefing note for countries on the 2016 Human Development Report, Thailand, http://hdr.undp.org/sites/all/themes/hdr_theme/country-notes/THA.pdf (accessed 20 March 2017). Ministry of Public Health, Thailand (2009). Survey results of behavioural risk factors of non-communicable disease and injuries, 2007 http://www.searo.who.int/entity/noncommunicable_diseases/data/tha_brfss_2007_v_thailand_eng.pdf?ua=1 Ministry of Public Health, Thailand (2012). The 11

th National Health Development Plan under the National

Economic and Social Development Plan, 2012-2016, http://www.nationalplanningcycles.org/sites/default/files/country_docs/Thailand/11ththailandnational_health_development_plan.pdf National Economic and Social Development Board, Thailand (2011). The Eleventh National Economic and Social Development Plan (2012-2016), http://portal.mrcmekong.org/assets/documents/Thai-Law/11th-National-Economic-and-Social-Development-Plan-2012-2016.pdf National Economic and Social Development Board, Thailand (2015). Millennium Development Goals 2015, MDGs Thailand 2015, National Health Commission of Thailand, Royal Thai Government (2012). Thailand, http://www.who.int/nmh/countries/tha_en.pdf (accessed 20 March 2017).

Thailand now an upper middle income country, http://www.worldbank.org/en/news/press-release/2011/08/02/thailand-now-upper-middle-income-economy (accessed 20 March 2017). Thailand overview, http://www.worldbank.org/en/country/thailand/overview (accessed 20 March 2017). United Nations Country Team in Thailand (2011). United Nations Partnership Framework for Thailand 2012 -2016, http://www.th.undp.org/content/thailand/en/home/library/other-publications/united_nations_partnership_framework_thailand_2012_2016.html What is a theory of change, http://www.theoryofchange.org/what-is-theory-of-change/ (accessed 30 March 2017). WHO (2005) Eleventh General Programme of Work 2006-2015. Engaging for Health. World Health Organization, Geneva, http://apps.who.int/iris/bitstream/10665/112792/1/GPW_2014-2019_eng.pdf. WHO (2007) Medium-Term Strategic Plan 2008-2013. World Health Organization, Geneva, http://apps.who.int/gb/archive/e/e_amtsp.html. WHO (2012). WHO Reform: Meeting of Member States on programmes and priority setting, World Health Organization, Geneva, http://www.who.int/dg/reform/consultation/WHO_Reform_1_en.pdf?ua=1. WHO (2014). Noncommunicable disease country profiles, World Health Organization, Geneva, http://www.who.int/nmh/publications/ncd-profiles-2014/en/. WHO (2014). Twelfth General Programme of Work 2014-2019. Not merely the absence of disease. World Health Organization, Geneva, http://apps.who.int/iris/bitstream/10665/112792/1/GPW_2014-2019_eng.pdf.

Page 82: WHO Evaluation Office - BIHMOPH of CO Evaluation 2017.pdf · WHO Evaluation Office Country Office Evaluation - Thailand Evaluation brief 2017 Context Country office evaluations were

Pre-publication version, August 2017

39

WHO (2015). Programme Management. Glossary of Terms. Unpublished internal document. World Health Organization, Geneva. WHO (2015) Evaluation of WHO’s presence in countries. WHO Evaluation Office, World Health Organization, Geneva, http://www.who.int/about/finances-accountability/evaluation/prepublication-country-presence-evaluation.pdf?ua=1. WHO (2016) Final Evaluation of the WHO country Cooperation Strategy Thailand 2012-2016 WHO (2016). Global tuberculosis report 2016. World Health Organization, Geneva, http://apps.who.int/iris/bitstream/10665/250441/1/9789241565394-eng.pdf?ua=1. WHO (2016). Needs assessment for the selection of priorities for the Thailand-WHO Country Cooperation Strategy 2017-2021 - draft for comments and discussion. WHO (2016). WHO Country Cooperation Strategy. Guide 2016. World Health Organization, Geneva, http://www.who.int/country-cooperation/publications/ccs-formulation-guide-2016/en/. WHO Country Office for Thailand (2007). WHO Country Cooperation Strategy Thailand 2008-2011, http://apps.who.int/iris/handle/10665/161144. WHO Country Office for Thailand (2011). WHO Country Cooperation Strategy Thailand 2012-2016, http://apps.who.int/iris/handle/10665/161157.

WHO Country Office for Thailand (2014). Mid-term review of the WHO Country Cooperation Strategy - Thailand 2012-2016, http://www.searo.who.int/thailand/areas/ccs-midterm-review-2012-2016-tha.pdf?ua=1. WHO Country Office for Thailand (2016). WHO Country Cooperation Strategy Thailand 2017-2021.