Conflict, Security & Justice Global Indicators Final Report

Development of a Basket of Conflict, Security and Justice Indicators

Eric Scheye, Independent Consultant Diana Chigas, CDA Collaborative Learning Projects

May 29, 2009

i

Table of Contents

I. Introduction II. HMG Evaluation Needs

III. Conflict, Justice and Security Indicators:

Summary and Explanation A. Uses and Limitations of the Indicators B. Categories of Conflict, Justice and Security Indicators

B.I. Differences between conflict and justice and security indicators B.II. The need for reformulation of conflict indicator categories

B.III. Framework for conflict, justice and security indicators C. Criteria for Selection of Indicators

IV. Strengths and Weaknesses of the Indicators

A. Undertracking of Important Factors B. Misleading Indicators

V. Strengths and Weaknesses of the Data VI. Matrix of Conflict, Justice and Security Indicators

A. Assumptions Underlying the Choice of Indicators B. Using the Indicators C. Matrix of Indicators

A. Conflict Intensity B. Institutions/Mechanisms that Handle Grievances

B.I Capacity of Government to Deal with Grievances B.II Government Legitimacy and Accountability B.III Human Rights and Civil Liberties

C. Community Initiatives for Peace D. Military and Militarisation E. Justice

E.I Criminal Justice Capacity and Effectiveness E.II Rule of Law and Administration of Justice E.III Access to Justice E.IV Corruption

F. Security Sector F.I Prisons F.II Policing F.III Oversight of Justice and Security Sector

G. Personal Safety and Security

ii

Table of Contents, cont. VII. Conclusion and Recommendations

VIII. Appendices

A. Example: Application of Indicators to Burundi and Nepal, 2006-2008 B. List of Data Sources Used in the Matrix C. Matrix of Countries Covered by Indices/Data Sources D. List of Interviewees

1

Report:

Proposed Basket of Conflict, Security and Justice Indicators Eric Scheye and Diana Chigas

May 29, 2008

I. Introduction

HMG and DFID acknowledge their need “to be better at monitoring, evaluating, demonstrating and communicating the impact and value of its conflict, security and justice activities and programming”1 The need to measure more systematically the effect of programming resides at a number of different levels: global, regional, country, and programmatic. Each of these levels, however, targets a different audience and addresses different needs, which often are disparate and do not necessarily coincide or overlap. Consequently, it is not possible to construct a single, comprehensive set of measures for evaluation that answer all of HMG and DFID’s needs and are grounded upon “a short list of indicators that can be backed up by credible, [existing] datasets.”2 As a result, this report concentrates only on those conflict, justice, and security indicators that are “measurable at country level”3 so as “to help staff monitor trends”4 and the overall direction in which a country is moving.5

More specifically, this report recommends baskets of indicators that are derived from the “most commonly available global and regional datasets,” which reflect a selected number of conflict, justice, and security categories, including, but not limited to:

• conflict and armed violence; • levels of exclusion, participation, representation; • criminal justice; • access to justice; • oversight of the security sector; • qualitative perceptions of safety and security; and • violence against women.6

In developing the baskets of indicators, the authors interviewed a number of DFID and HMG staff concerning their needs for and experience with indicators. The purpose of the interviews

1 Terms of Reference, 30 January 2009, p. 1. 2 Terms of Reference, 30 January 2009, p. 1. 3 Terms of Reference, 30 January 2009, p. 2. 4 Terms of Reference, 30 January 2009, p. 1. 5 The recommended indicators do not directly measure state fragility, though it may be possible to extrapolate from the indicators and reach certain conclusions as to a country’s fragility. 6 Terms of Reference, 30 January 2009, pp. 2-3.

2

was to understand better the target audiences for the indicators, as well as needed uses and abilities to interpret indicators. We also examined a range of existing datasets to develop the baskets of indicators to determine the validity, timeliness, and accuracy of the data available. Throughout the report, the baskets of indicators have been constructed by extracting selected measures from pre-existing datasets. The resulting baskets, therefore, are composed of discrete indicators, which when combined provide multiple perspectives on a particular category. The original intent was to assemble baskets that would measure outcome, process, financial considerations, perceptions, and activity. That objective, however, could not be attained because of an overall paucity of data from existing databases. In a number of cases, individual indicators are, at best, proxies to measure the stated phenomenon, which is noted when appropriate in the comments section of the category/indicator matrix.

II. HMG Evaluation Needs A small number of DFID, MOD and FCO staff, both in headquarters and in the field, were interviewed to elicit their needs, experience with indicators, and ability to interpret them. The interviews revealed a variety of views regarding the organization’s priority needs for conflict, justice and security indicators. Depending upon the responsibility and perspective of the interviewee, the stated need was global, regional, country, and/or programmatic. It is safe to conclude that there is no consensus on either HMG or DFID’s priorities. In the opinion of the authors it is also unlikely that such a consensus is attainable or desirable. It may be most prudent to develop different evaluation products to address the wishes and needs of the varying audiences. Most interviewees emphasized the need to “measure impact,” though how impact was defined varied depending upon the stated purpose. The purposes ranged from, inter alia:

• programmatic outcomes for country offices; • sustainability of programmes; • submissions to thematic or regional pools of funding; • demonstration of impact on the ground to the British public; • establishment of baseline against which normal programming trends in fragile or

conflict situations can be ascertained; • measurements of mandatory DFID processes such as PSAs and CGAs; to • compilation of reports for parliamentary oversight.

A significant group of interviewees identified the need for country offices to measure progress and monitor trends at a higher level than at the programme or sectoral level. In addition, these interviewees noted that once programmes are designed, and, in particular, once goals are well-

3

defined, it is not veryexceedingly difficult to specify indicators to help evaluate their effectiveness. Moreover, as one person noted, the existing datasets do not have enough “granularity” to be useful at a country programme level. Those who identified the country office level as a priority also stated a desire to have a menu of indicators, from which they could choose in the design and implementation of actual programmes and which could give country offices an idea of what a good indicator would look like. There was general recognition that a single set of indicators cannot meet everyone’s needs. Choices will have to be made concerning the level and uses of these indicators. It was suggested that those choices could be reached, reflecting efforts of other donors and organisations, such as the Oslo Democracy Centre/UNDP effort to provide guidance on indicators of fragility and conflict and DFID/HMG’s own initiative to develop indicators for accountability, transparency, fragility, etc. All agreed that it would be wise to avoid a proliferation of indicators and important to build consensus with other organisations. III. Matrix of Conflict, Justice and Security Indicators: A Summary and

Explanation The following section outlines an approach to the development of baskets of indicators for conflict, justice and security, summarizes the categories and indicators propose, and identifies their most appropriate uses. There are also a number of challenges to the usefulness and robustness of the recommended baskets of indicators as measures of effectiveness. Attempts have been made to address these challenges in the criteria and process for selecting the discrete indicators, in the groupings of indicators into baskets, and in the variety of perspectives and methodologies included for measurement.

A. Uses and Limitations of the Indicators In the conflict and justice and security, Tthe range, level and nature of indicators provided in existing datasets suggest that they may be appropriate as high level indicators to monitor broad trends and the direction in which movementing is occurring, such as deaths in conflicts, fractionalization, overcrowding of prisons, days a court case requires, etc. Valid uses of the indicators, therefore, would include:

• To assess broad trends and directions of a country and region; and • To assess the impact of DFID and HMG strategies, policies and assistance on the

situation (peace, security, justice) “writ large.”

4

Caution should be exercised in using the indicators for the following:

• Country comparisons. There is limited utility in comparing these gross numbers across countries and regions, given the incommensurability of the data and the requisite need to interpret the data. Individuals may be displaced because of conflict or typhoons and a gross statistic cannot discriminate between causes. Prisons may be overcrowded due to a police service’s misguided arrest regime, an inadequate bail system, and/or an overloaded prosecutorial agency. Moreover, many of the datasets rank countries one against another. This may have some ‘quick glance’ usefulness to determine which countries are the most violent, conflictual, and fragile, but rankings are not prudent methods of understanding the country context and the direction in which a country is progressing or deteriorating. The advancement of a country up a ranking list may indicate an improvement in that country’s situation. It is equally likely, however, that it silently bemoans the deterioration of the situation in the country or countries that had been ahead of it the list the year(s) before.

• Programme or project evaluation. Existing datasets are not useful for use at the programme or project level, for a number of reasons;

i. The indicators are highly aggregated, especially in the conflict area. Many different

indicators are combined into one score, and because there is no consensus both within the field or across countries on which observable traits combine for measuring a concept, use of the indicators at the programmatic level is problematic.

ii. There is also aggregation with regard to ethnic group, sex/gender, race, etc., which makes discrimination and interpretation challenging, at best.

iii. There are significant gaps in data and thematic areas covered, as explained later in this report. These include: the activities of non-state actors, private security actors and civil society and community-based actors and local government. Many of these activities receive significant donor support, and thus could not be covered by the indicators. Even for programmes focused on governance or government capacity to provide justice and security, the lack of data in these areas makes it difficult to understand the impact of those activities on justice, security or peace “writ large.”

iv. Public perception surveys exist, but their coverage is scanty and spotty. There is little consistency in the countries covered or the questions probed. Timeliness is an addition challenge. As a result, the existing datasets over-rely on expert opinion, and have little data that could help measure impacts of programmes on the intended beneficiaries.

v. Proxy indicators are often used, making it difficult to assess the relevance and validity of the indicators in specific contexts. They may be valid in some contexts, but not in others.

5

vi. The indicators used in existing datasets do not permit attribution of progress to specific programmes or projects, as they measure the elements of the justice, security or conflict system, but not causes of improvements.

vii. The time delays in availability of data may make the existing datasets less useful for programmes of short duration, lasting two to three years.

viii. The degree of contextualisation needed to determine relevance and significance of the indicators makes these baskets less useful at the programmatic or project level.

This does not mean that the baskets of indicators presented here are irrelevant for programmes or projects. They can be used to assist in monitoring and evaluation of programme and project impacts, but only as supplemental evidence and then only when carefully done. When deployed to evaluate a programme, these baskets of indicators must be bolstered by specific country-level information that is directly related to the initiative under examination. A country-wide pre-trial detention number may hold little relevance, for example, unless reinforced by statistics that trace, over a number of years, the length of days in jail for those who have not yet been sentenced. The baskets need to be understood and used in light of guidance outlined in the OECD DAC Guidelines for Evaluation of Conflict Prevention and Peacebuilding Activities (working draft):7

• Conflict and context analysis. Conflict/context analyses do not need to be of the depth required by the SCA. An analysis of the specific drivers of conflict, as well as those affecting justice and security in the country, will help country offices and programme staff identify which indicators are most relevant and whether other indicators are needed or data gathered. For example, in Georgia, an analysis of the conflicts in that region would suggest that indicators related to fairness of elections or governance effectiveness would not be very relevant to assessing impact on conflict. By contrast, levels of grievances would be particularly useful. Conflict and context analysis would also assist in the determining attribution of impacts to DFID/HMG programmes by permitting the identification of various factors that may contribute to escalation or mitigation of conflict.

• Examination of theories of change. The OECD DAC Guidelines recommend that

evaluations of conflict prevention and peacebuilding activities identify and test the theory(ies) of change and implementation of the intervention—the the programme’s assumptions and logic about how and why it will work to change the conflict or situation and how the program activities will lead to the desired outcomes and impacts. This is important for selection of the appropriate indicators that will allow for testing of that theory in the particular context, as well as for consideration of whether any progress (or

7 OECD DAC Network on Conflict, Peace and Development Co-operation and the DAC Network on Development Evaluation, Guidance on Evaluating Conflict Prevention and Peacebuilding Activities (Working Document for Application Period) (Paris: OECD, 2007).

6

lack thereof) can reasonably attributed to DFID/HMG’s programmes. For example, a capacity-building programme for the courts in a post-conflict country may be based on the theory that improved rule of law will reduce tensions by building a minority’s confidence in the government and by providing a viable avenue for addressing grievances. The programme would want not only to examine indicators of criminal justice effectiveness and access to justice, with a focus on minority perceptions, but to look at indicators related to grievances and legitimacy of the state, also disaggregated by demographic group. This would likely require additional information related an examination of the kinds of cases being resolved in the courts (i.e. are they addressing issues at the heart of grievances that drive conflict?).

• As country indicators and not as a means to evaluate programme outputs and outcomes. As mentioned above, the indicators in the baskets outlined here do not directly refer to programme outputs or outcomes. They are country level indicators. In the example above, the programme would need to establish and monitor indicators related to the quality and outcomes of the capacity-building (e.g., are the beneficiaries gaining and using knowledge and skills; have behaviors changed; are new structures and processes being put into place and utilized; etc.). The next step would be to assess whether those changes instigated by the programme have contributed to improvements in the overall situation. The indicators supported by the datasets can be helpful to the latter question, but the diversity of contexts, needs and programmes make it impossible to establish relevant programme indicators.

The indicators presented below and in the Matrix should also be used and interpreted with care in light of the aforementioned problems and weaknesses inherent in the existing datasets. They therefore should be used with caution as guides for policy.

B. Categories for Conflict, Justice and Security Indicators The terms of reference proposed four categories of conflict prevention and peacebuilding indicators, in addition to measurement of “death/injury rates” as a result of conflict and armed violence:

• formal peacebuilding structures (including elections, parliament, human rights, media); • informal peacebuilding structures (traditional mechanisms, peace committees, mediation

capability, etc.); • qualitative peacebuilding (attitudes/ perceptions between groups, of state/non-state

structures, etc.); and • social cohesion. In the justice and security field.

7

For justice and security, the TORs requested indicators in: • rule of law; • security sector; and • personal safety and security.

The full framework and indicators for conflict, justice and security follows in Section VI.C. of this report. This section provides an explanation of the framework of categories of indicators and the criteria for selecting indicators and grouping them into baskets. (i) Differences between conflict and justice and security indicators. Differences in the fields, respectively, of conflict and peacebuilding and justice and security necessitate different approaches to the development of indicators. In the justice and security field, a limited number of indicators can be used to gauge the overall “health” of the system. The characteristics of a functioning system and the impacts on the population are relatively well-defined in number, especially if conceived as measuring processes across the individual institutions, agencies, and organizations of the sector, though how the provision of justice and security is ‘experienced’ by citizens and residents cannot be overlooked. Indicators relating activities and resources (by courts, police, prisons) to results (objective and perceived) can reveal salient features of the system. For example, the rate of conviction per arrest may highlight relevant information as to the effectiveness and efficiency of the criminal justice system, especially if correlated to various public perception data. While the indicators may not disclose the causes of any strengths and weaknesses or may hide possible systemic ethnic discrimination if not disaggregated, they still provide a set of standards for tracking basic progress. Interpretation is still required as to what specifically the collected data means, but the indicator is a good starting point, will raise pertinent questions, and is highly suggestive of what a country office should explore further .

In the conflict field, by contrast, there is no generalized theory of conflict. There is also little consensus on a standard set of variables and drivers of conflict. In any particular context, different variables can indicate strength or weakness. For example, successfully holding free and fair elections are often viewed as improvement in a conflict situation and a step toward building or strengthening democracy. However, in transitions to democracy, elections can, in some cases, exacerbate conflict. Similarly, government provision of basic services to the population and the strength of the economy are often associated with reduced conflict. In a situation in which the public expects little from its government, however, the lack of effective service delivery by the government may bear little to no relation to conflict. In other words, the indicators must be directly tied to an analysis of the conflict analysis to ensure their relevance. It is also likely that the indicators will change depending on what factors are driving the particular conflict.

The conflict indicators proposed here are designed to track elements of a transition to peace, not necessarily the presence (or absence) of structures and processes characteristic of a “healthy” system, such as elections, checks and balances, government services, GDP per capita, etc. By

8

contrast, the justice and security indicators track the “health” of the system and can be used throughout the programme or policy cycle – from needs assessment, implementation monitoring through to mid-term and end of programme evaluation. Under any and all circumstances, conflict and country analyses are critical for assessment of the characteristics of a “healthy” transition to peace. Different conflicts have different causes, different drivers, and different dynamics. Failure to undertake a conflict analysis may result in the selection of indicators that are not relevant to the conflict. Similarly, it is good practice once indicators are selected, to test the indicators at the time of conducting the baseline to confirm their validity and reliability and to ensure there are not other more valid indicators.

(ii) The need for reformulation of conflict indicator categories. The authors have reformulated the categories and the indicators for conflict from those suggested in the TORs to ensure that issues of impact of interventions can be tracked. The authors deemed this necessary for several reasons. First, several of the indicators/categories in the TORs refer to structures or outcomes, rather than impact. In other words, they do not indicate whether and how these elements reduce conflict, and, therefore, may not be as good indicators of reduction of conflict as they are of the establishment of mechanisms that we assume are good regulators of conflict. Second, many of the indices from which conflict indicators can be drawn do not make conceptual distinctions between statebuilding, peacebuilding, governance and development. Clearly all of these phenomena are related, and activities in all domains—socio-economic, governance, justice and security, and reconciliation and culture8—are needed. They are, however, not all the same. State weakness is not the same as conflict, nor its only cause, even when it may be a contributor to its escalation. Similarly, conflict can be seen as a result, a symptom or a cause of fragility.9

The authors, therefore, have reformulated the categories of conflict indicators to reflect the CDA Criteria of Effectiveness developed in the Reflecting on Peace Practice Project of CDA Collaborative Learning Projects. The Criteria summarize what the Reflecting on Peace Practice Project found to be effective intermediate-level benchmarks for assessing impact on “Peace Writ Large,”

Consequently, the categories of indicators of fragility cannot be equated with those of conflict.

10

8 See OECD DAC Network on Conflict, Peace and Development Co-operation and the DAC Network on Development Evaluation, Guidance on Evaluating Conflict Prevention and Peacebuilding Activities (Working Document for Application Period) (Paris: OECD, 2007), at 18.

or the broader societal peace:

9 Fabra-Mata, Javier & Ziaja, S. 2009 (forthcoming). Users’ Guide on Sources Measuring Fragility and Conflict (Draft 1.0) Oslo: UNDP and German Development Institute. 10 The CoE emerged from an intensive four-year process of collaborative learning and reflection about what makes peace process effective vis-à-vis “Peace Writ Large,” or societal-level peace. The process involved 26 case studies of projects, programmes and groups of programmes and over thirty consultations and feedback workshops with over a thousand practitioners, policy makers, donors and academics to analyse the cases in comparative perspective, identify issues and provide feedback and new experience on tentative conclusions. The initial findings are presented

9

1. Institutions that handle grievances underlying conflict are created or reformed; 2. Beneficiaries and communities develop their own peace initiatives; 3. People (citizens) increasingly resist violence and provocations to violence; 4. People’s security and their sense of security are increased; 5. Inter-group relations, reflected in, for example, changes in group attitudes, public

opinion, social norms and public behaviours, improve meaningfully.

(iii) Framework for Conflict, Justice and Security Indicators. We believe the reformulated categories preserve the essence of the elements suggested in the TORs. Following is an explanation of the reformulated categories and types of indicators. As mentioned above, the conflict indicators are organized around the RPP Criteria of Effectiveness. The justice and security indicators are organized largely by sector or area of activity: criminal justice, civil and administrative justice, access to justice criminal and civil justice, security sector (prisons, policing, oversight), and personal safety. A number of indicators are useful for assessment of effectiveness in peacebuilding and justice and security development. Comments are included in the full matrix of indicators in section regarding how to use the indicators as conflict or justice and security indicators. A. Conflict intensity: Includes deaths, but also other consequences of conflict, including

displacement, which may be source of conflict and instability, as well as non-physical conflict intensity

B. Institutions, mechanisms and process that can and do address grievances, inequalities,

injustices, etc. that cause and fuel conflict.

B.I. Capacity of government to deal with grievances. These indicators measure: • government “effectiveness” (can the government can develop and implement

policies that serve the needs of the population?); • leadership culture and process (how do the values, relationships and interactions

of leadership fuel or mitigate conflict?). It should be noted that these are proxies, and look at capacity, not results. In a fragilel state, there is likely to be little capacity. There also may be limited expectations of service provision from such a state. Consequently, a “low” score on effectiveness may not signify a worsening of conflict. Similarly, a “high” score on capacity does not necessarily mean that conflict is improved.

in Mary Anderson and Lara Olson, Confronting War: Critical Lessons for Peace Practitioners (Cambridge, MA: CDA, 2003).

10

B.II. Grievances. This category measures the results of government policies and practices. It also looks at the degree to which group grievances exist that may fuel conflict, including data, to the extent possible, on actual inequalities and perceptions of unfair treatment and inequality.

B.III. Legitimacy and accountability of government. This category also relates to the government’s capacity to deal with grievances, focusing on the relationship between citizens and the state to provide a proxy for the availability of non-violent means to address conflict. It includes measures of:

• elections to assess the representativeness and legitimacy of the government; and

• perceptions of the legitimacy of the state. This latter variable is a proxy for the degree to which people are likely to look to or accept the state to handle grievances and promote their well-being. Combined with the indicators under B.I., these measures could be an assessment of a state’s fragility.

B.IV. Human rights and civil liberties. These are indicators of the availability of peaceful

means to deal with grievances. They are also basic indicators of rule of law, one of the ways in which conflict and justice and security measurements can be somewhat knitted together. While B.III measures structural (elections) and perceptual (legitimacy of the state) aspects of this element, the degree of respect/repression of human rights and civil liberties assesses government behavior.

C. Community/Popular initiatives for peace. These indicators are intended to measure the extent to which there are local capacities for peace in the non-governmental sphere and to what extent do communities, citizens groups and others develop their own initiatives for peace?

D. Resistance to violence or provocations to violence. Military capacity and societal

militarization indicators look at the degree of militarization of society at governmental level, but are best considered to be proxies of violence and the tendency to utilise violent means. Because it is difficult to obtain data on non-state armed groups, we have used the accessibility of small arms and the number of deaths by small arms as proxies for the level of activity of non-state actors.

E. Rule of Law and Access to Justice

E.I. Criminal Justice Capacity and Effectiveness. Capacity and effectiveness are

measured from three perspectives:

11

• government activity and results in criminal justice, including data on whether the system is capable of dealing with the level of crime present;

• popular perceptions of effectiveness of criminal justice; and • resources (financial) dedicated to criminal justice, as a process indicator of

government commitment and action.

E.II. Civil and Administrative Justice. It is insufficient to examine rule of law with reference only to the criminal justice system. Consequently, this section looks at a set of wider issues revolving around judicial independence, perceptions of fairness, impartiality of decisions and judgments, and the police operating under civilian control. It should be noted that there is a dearth of reliable indicators that reach beyond the criminal justice system.

E.III. Access to Justice: State and Non-State Systems. Access to justice indicators include:

• capacity (does the government have the capacity to provide adequate access?); • access (is access being provided, or are costs, government restrictions of rights or

provision of services undermining access?); • perception (popular perceptions and experience of frequency with which state and

non-state systems are accessed); and • women’s access and capacity.

E.IV. Corruption. It is necessary to measure corruption as a phenomenon that can impede

the delivery of justice and security. At the same time, corruption can be a cause of conflict, as a grievance, and one of its many by-products. There are many ways of measuring corruption as a perception, an activity, and as a sign of governmental ineffectiveness.

F. Security Sector

F.I. Prisons. Measurements on prison capacity are ascertainable, but they do not indicate

the treatment received by those who have been incarcerated. Pre-trial detention is a challenging issue, but there is no method of directing assessing and pinpointing its causes or ways of measuring that will suggest how to reduce the severity of the challenge.

F.II. Policing. As with criminal and civil and administrative justice, the basket of indicators provided is intended to provide three perspectives on policing: a) capacity; b) behavior and effectiveness (homicides, extrajudicial killings), as well as perceptions of the police; and c) financial resources/commitment.

12

F.III. Oversight of Justice and security sector. Appropriate oversight of the actions and

institutions of the sector is a key component of democratization of the justice and security system. It is a means by which good and effective governance can be identified. It is, however, difficult to evaluate given existing datasets. The institutionalization of corruption is one proxy by which to measure oversight, given the premise that embedded corruption circumscribes the government’s ability to conduct adequate oversight. Similarly, appropriate oversight may be a means by which conflict can be addressed before its onset or thereafter.

G. Personal safety and security. Measures perceptions of the effectiveness of justice and

security sectors broadly, and of efforts to mitigate conflict. If the data could be disaggregated by conflict group, these would serve as more precise conflict indicators, as, in their current form, they do not distinguish between sources of insecurity nor which populations feel more or less secure. Most appear not to do so, except perhaps in the case of the perceptions surveys, where at least the Afrobarometer asks respondents about their identity/group, and thus in theory disaggregation by ethnicity, religion or group (as well as gender, etc.) would be possible. An adequate proxy for increase in security/sense of security related to the conflict might be obtained by looking at conflict intensity, state behavior violating physical integrity and perceptions of security, in particular concerning fears of political violence, fears of reprisals for political involvement.

C. Criteria for Selection of Indicators Within the categories, indicators were selected based on a number of criteria. Some were identified in the TORs, including:

• Geographical coverage, i.e. data is available for as many countries as possible; • Time coverage and update, i.e. data is available over time and is regularly updated; • Validity and reliability, i.e. the indicator measures what it is supposed to measure as

much as possible, and is replicable; • Sources of measurements that are sensitive to short-term change (to the extent possible);

and • Consistency with other international indicators, to the extent possible and valid.

The primary criterion for constructing the baskets was to provide a multi-dimensional picture of each phenomenon or category. The objective of these indicators is to move beyond tracking discrete institutional activities and evaluate the process and “the impact of the system as a whole

13

on public safety or the rule of law.”11

For conflict, justice and security, this implies that at least three distinct elements – activity/process, outcome, and beneficiary perception -- need to be taken into account to determine the impact of a project or programme. Consequently, clusters or baskets of indicators are required. The issue is not only what each individual indicator may be, but how the basket as a whole functions and is balanced. As much as the existing datasets permitted, the baskets were constructed based on the following considerations:

• To measure structure/capacity (e.g., legal frameworks, rights, human resources relative to the scale of the problem, etc.;

• To measure process (efforts to achieve outcomes, e.g. budgets dedicated to criminal

justice, military expenditure as % of GDP, etc.); • To measure outcome (changes in the conditions in the country with respect to conflict,

justice or security).

• To balance more objective (e.g. # homicides) and subjective (perceptions of insecurity) measures of the phenomenon. For conflict indicators, data on perceptions of people inside the country is especially important as a measure of impact, in that it reflects how changes in institutional, structural or political factors are viewed and provide information on whether these changes may lead to less conflict behavior.

• To minimize explicit or implicit adoption of a particular ideology or theory of change for conflict indicators.

IV. Strengths and Weaknesses of the Indicators

The existing datasets of conflict, justice and security indicators have several limitations that should be taken into account as they are used. Any DFID follow-up on these indicators should take their limitations into account as well.

A. Undertracking of important factors Several important categories of indicators are not tracked sufficiently or at all by existing datasets. This significantly affects the usefulness of the existing datasets for measurement of conflict, justice and security. The gaps include:

11 Todd Foglesong and Christopher Stone, Measuring the Contribution of Criminal Justice Systems to the Control of Crime and Violence: Lessons from Jamaica and the Dominican Republic. John F. Kennedy School of Government - Harvard University, April 2007, p. 5.

14

• Non-state capacities, processes and mechanisms for conflict transformation, justice and security. The conflict indices and justice and security indicators focus primarily on state mechanisms. The activities of non-state actors are rarely measured in existing datasets. There is little data on civil society initiatives for peace and equally little information on non-state mechanisms for provision of justice and security. Similarly, there is little to no measurement of private security actors, which suggests that another significant slice of the security arena escapes evaluation. This profoundly affects the ability to measure justice and security, given that a lack of state-provided access to justice is not equivalent to an absence of justice. Similarly, given the amount of service delivery, conflict resolution and peacebuilding work that is done by traditional structures and other non-governmental actors, and the amount of financial and other support given to them (youth, women, CSO and NGOs, media, etc.), this lack of data significantly undermines the ability to understand conflict and conflict resolution and the impact of assistance on conflict, in a country. What information that does exist on these phenomena is found in anthropological, sociological and other more qualitative research, is not readily measurable (or available), and, therefore, cannot be converted into a form consistent with the datasets. It is a significant missing element that would bear on measures of conflict, justice and security.

• Perceptions. Most of the datasets rely on expert perceptions of a particular phenomena –

from issues of inclusion (how inclusive is the governance system) to perceptions of the electoral process to questions of trust and confidence between groups. There are a few existing sources that try to gauge public perceptions, such as the Gallup World Poll, Afrobarometer, Arab Barometer, among others. However, these instruments often do not ask questions that are entirely relevant (valid), or, as in the case of the Afrobarometer, they do not ask the same questions repeatedly over time. Data regarding perceptions of a representative sampling of sub-groups is also scarce because what survey information that does exist may not consistently disaggregate the data. As a result, there is an over-reliance on expert opinion and under-reliance on public perceptions in the existing datasets. To the extent expert opinions are flawed, biased, or reflect a particular world view, the data may not be accurate. Greater measurement of public perceptions would provide direct data on perceptions of actors in and beneficiaries of conflict, justice and security policies and programmes, and permit more accurate assessment of impacts.

• Inter-group relations. Some indicators regarding factionalization (among leadership) and the existence of inter-group tensions (expert opinions) do exist and are useful. However, little data concerning inter-group trust, perceptions, or perceptions is collected systematically. The indicators focus more on generalized trust (do you trust other people?). Moreover, information about how different policies and actions affect differing groups differentially (i.e. data about horizontal inequalities) is not collected

15

systematically. This could be gathered to a large extent by disaggregating data on other indicators (e.g., perceptions of justice and security, conviction rates, inequalities of income, etc.) by ethnic/conflict group. Doing so may be quite sensitive for partner governments in post-conflict situations. However, without robust data on the state of inter-group relations and the ways in which policies affect groups differentially (horizontal inequalities), assessment of impacts of policies, availability of justice and security for vulnerable groups, and programmes addressing conflict will be difficult to evaluate.

• Conflict-specific dimensions of justice and security. Conflict indices have integrated existing human rights, civil liberties, justice and security indicators that were developed to track progress in these sectors to measure peace and conflict. The same is true for socio-economic indicators.12

However, these indicators, by themselves, do not necessarily measure conflict impacts. While the justice and security indicators available (and provided in the matrix below) are applicable to justice and security development, their lack of disaggregating the data makes them imprecise for the conflict impacts of justice and security. For example, as mentioned earlier, an improvement in the rate of convictions per arrest may suggest an improvement in the effectiveness and efficiency of the criminal justice system, yet if the majority of those arrested and convicted are from one ethnic group, this “improvement” may be the source of grievance and conflict. The sectoral indicators can measure underlying structural-institutional capacities for conflict resolution and peace, but without disaggregation by demographic group, or other further information about how they impact conflict actors, they can provide misleading information about conflict impacts.

B. Misleading Indicators. Several aspects of existing datasets can provide misleading information about trends in conflict, justice and security in a country. This is, in part, a consequence of the fact that in most of the categories, the indicators are proxies, rather than direct measures. A number of patterns emerged regarding ways in which the indicators in existing datasets can be misleading: • Ideological/theoretical or normative biases. Most of the peace and conflict datasets are based

on well-defined (although not necessarily explicit) theories of change, or assumptions about the mechanisms by which peace and stability will come about.13

12 Despite contributions of human rights, civil liberties, justice and security to peacebuilding, many of the indices actually are dominated by socio-economic and other governance indicators See F. Barton and K. von Hippel, “Early Warning? A Review of Conflict Prediction Models and Systems” (Washington, DC: Center for Strategic and International Studies, 2008).

In cases where the theory of

13 See Cheyanne Church and Julie Shouldice, The Evaluation of Conflict Resolution Interventions, Part II: Emerging Practice and Theory (Ulster: INCORE, 2003), at 33, for a fuller discussion of theories of conflict, peacebuilding and change. See also DAC Guidance on Evaluation of Conflict Prevention and Peacebuilding at 35

16

change is quite strong (e.g., Index of State Weakness in the Developing World, which defines fragility as the cause of conflict), the indicators potentially measure whether the elements of the theory of change are present, but may not reflect what is occurring in a country with respect to conflict. This is why we chose not to adopt any one of the peace, conflict and fragility indices available, but rather included indicators from several of these indices.14

A similar normative or theoretical bias can also result from the selection of indicators. For example, a country peacebuilding strategy that focused on improving economic and social policy capability or government service delivery might choose indicators related to government effectiveness. This selection of indicators might track whether the government overall is becoming more effective in service delivery, but would not track whether the implicit theory of change – that improved service delivery will reduce conflict – is valid in that context. In this case, additional indicators related for example to group grievances or the legitimacy of the state would be useful to add to mitigate the theoretical bias.

• Comparisons between countries. The TORs indicate cross-country comparability as a

criterion for indicator selections. A number of problems with cross-country comparison for conflict and justice and security indicators have been discussed earlier in this report. Most of the peace and conflict indices compare and rank countries relative to each other. However, the comparison, especially in light of with the inclusion of highly developed democratic countries such as Norway or Sweden included in the dataset, skew ratings and are of limited use. Moreover, comparisons potentially introduce a strong normative bias by defining a country as an underperformer because it does not have the institutional and social characteristics of the developed democracies.15

Finally, as already noted, comparisons between countries may mislead the user if a country rises in rankings not due to its own progress, but rather because of the decline of countries previously ranked above it.

In the justice and security sectors, where statistical information is more prevalent, cross-country comparisons are dangerous. For example, European data suggests that the ratio of prosecutors/100,000 persons in Switzerland is 5.5; in Sweden, 9.9; and in the Ukraine, 21.0. Similarly, the total judicial budget/per capita in Switzerland is .28; in Sweden, .23; and in the Ukraine is .49. Although these statistics may raise important questions about the individual countries’ judicial systems (when combined with a raft of additional data), in and of themselves they provide little valid insight into actual judicial performance and, for

for a discussion of uses of theory of change and implementation theories that elaborate the logic and assumptions of how programmes make a contribution to the larger peace. 14 A useful analysis of the different indices, and their strengths and weaknesses as indices, is being developed by UNDP (the Oslo Governance Center) and the German Development Institute, in a User’s Guide on Sources Measuring Fragility and Conflict. A first draft has been made available for comment. It will be a useful supplement and interpretive guide to these indicators. 15 See Fabra-Mata & Ziaja (2009 forthcoming.)., User’s Guide on Sources Measuring Fragility and Conflict (1st draft) (2009) for more discussion of this point.

17

comparative purposes, are misleading. Cross-country comparability is particularly difficult with respect to criminal justice data, given wide differences in police accessibility, known variability in what constitutes a criminal act, inconsistent legal definitions in how crimes are recorded, and vast discrepancies in the rates at which crime is reported. It may be more possible to compare victimization rates across countries, but even these measures remain susceptible to some of the aforementioned challenges.

For justice and security development, the establishment of broad guidelines and standards could, at best, be used as generic goals for countries might be more useful. For instance, the UN has suggested a ratio of approximately one police officer/450-500 persons. While that statistic may be a rough and perhaps idealized goalpost, its applicability to any one country would need significant modification to account for differences in geography, communication and transportation systems, population demographics, education, GDP, urbanization, etc. The same can be said for similar justice and security guidelines, such as:

• personnel costs comprising 70-80% of total police budgets (US); • patrol units comprising 60-65% of total police budgets (US); • training costs comprising 5% of total police budgets (US); • personnel costs of judge and court staff comprising 65% of total judicial budgets

(Europe); • court buildings and premises capital costs comprising 15% of total judicial budgets

(Europe); and • court fees contributing 10% to total judicial budgets (Europe).

The most suitable indicators for cross-country comparison may be those based upon perceptions and institutional/systemic processes, each indicator coinciding with the process and outcome of an activity as it is experienced by the intended beneficiaries. For these reasons, the baskets of indicators chosen for justice and security development try to incorporate as often as possible such data.

• Use of Proxy Indicators: Over-inclusiveness and Misleading Conclusions. The difficulty in

direct observation of improvements of conflict, justice and security (especially conflict) has meant that many indicators in existing datasets and in the proposed baskets are proxies.16

16 For example, leadership acceptance of democratic institutions is a proxy indicator for legitimacy of democracy and for the potential of elections to support peaceful transition rather than conflict. The GNI Coefficient/Uneven development is a proxy for concentration of wealth and horizontal inequalities. In the rule of law, the number of days required to resolve a dispute is a proxy for possibility of corruption, and attorney/court/enforcement costs a proxy for access to state justice systems.

In addition, many of the datasets identify conditions that may provide a good environment for conflict management and prevention and justice and security development, e.g., high GDP per capita, economic performance, governance according to democratic structures and

18

processes, improvement of government services, reduction in poverty, etc. However, while these factors may correlate highly with peace, they are not necessarily indicators of improved conflict prevention or peacebuilding. Furthermore, it cannot be assumed that conflict has been reduced if there have been general improvements in the economic, governance and justice/security indicators, even if these are important structural factors. In a number of instances—including elections, increased GDP, increased openness of political competition, etc.—improvements may also bring about increased conflict in the short- to medium-term, which could undermine long-term development and governance. Similarly, some of the indicators may indeed be good warnings of a worsening situation, but not of an improving situation. For example, deteriorating provision of public services, included in the Failed States Index, may indeed be an indicator of conflict and state failure. However, the reverse may not indicate improvement of the conflict situation. Caution should be exercised in interpreting baskets of indicators.

The justice and security indicators, as other sectoral indicators, are important supplements to the more general conflict indicators provided here. They can help country offices and headquarters hone in on particular dimensions of the peacebuilding process. However, the indicators for justice and security development, similar to those for the socio-economic dimensions of conflict, are not necessarily “conflict-sensitive.” In other words, one cannot measure the impact of justice and security on conflict using these indicators alone. Indicators on human rights and civil liberties also lack this conflict sensitivity. For example, if conviction rates in a particular country increase, this may be interpreted as improvement in the justice system. If, on the other hand, a significantly greater number of one group is convicted than another, this reputed general improvement may have negative repercussions for conflict management. Similarly, numbers of homicides may go down, indicating an improvement in personal security of the population. However, if one group remains disproportionately affected by homicides (or other crimes), then an improvement in this indicator could not be interpreted as an improvement in conflict. Disaggregation of crime, justice and security indicators by social/ethnic/religious group, therefore, increases the validity of these indicators as conflict indicators. Disaggregation also increases the utility for determining how to improve justice and security service delivery. For both conflict and justice and security development, the broader picture can be obtained by including perceptions of vulnerable groups about their treatment by their government, group inequalities and other potential grievances.

• Aggregation. A number of datasets, such as the World Governance Indicators, and a number

of the peace and conflict indices, aggregate indicators from a number of different sources. Aggregate indicators have a number of advantages. They may be more accurate in that they provide more precise evaluations of a situation than individual indicators. However, for the aggregate governance indicators, the margin of error appears to be large, so that small

19

changes in the indicators over time may not be statistically significant. Therefore, it is important to balance the use of aggregate and individual indicators, as has been attempted.

• A General Caution Regarding Interpretation. For all the indicators, whether conflict, justice

or security, caution needs to be exercised in determining what conclusions should be drawn from movements in any one indicator, a series of indicators, and/or in any category of indicators. This is particularly important if these indicators are to be used in country offices for programming purposes, which, as already discussed, is not recommended unless supplemented by a contextual analysis and other data. The indicators have been blended into baskets for a number of reasons, only one of which is to tackle a category from multiple perspectives. Equally importantly, broad conclusions should not be derived from the movement of any single indicator. For instance, an increase in trust in the police is, frequently, associated with an increase in crime rates as more victims of alleged crimes may lodge police reports. This rise in reported crime may not occur, however, if the reason the police have not been alerted to alleged crimes is due to the public’s generalized belief that the police are ineffective. Similarly, pre-trial detention statistics are notoriously difficult to interpret. A reduction in detention numbers, which may at first blush be considered a progressive movement, but may, upon further research, indicate that police have changed their arresting policy. Raw movements in re-trial numbers, therefore, are not be able to shed light upon whether that change was for the good. Reductions in pre-trial detentions may also be the result of increased bond provisions, which, in turn, if not coordinated with the local neighborhoods, may trigger various forms of extra-judicial killings and lynching. The dangers of misinterpretations of movements in indicators are equally great in the conflict arena. For example, a decrease in inter-ethnic physical violence may reflect better security, an improvement in the conflict situation, or that fewer acts of violence are sufficient to sustain. The latter conclusion was reached in a study of violence in Kosovo prior to the 2004 riots, when intimidation tactics had shifted from personal assaults to destruction of property intended to persuade minorities to leave. The same study also found that minorities’ increased movement outside their enclaves, which was interpreted by most as a sign of increased feelings of security on the part of minorities and improved inter-group relationships, was, in fact, a reflection of minorities’ confidence in peacekeepers and their increased knowledge of where they would be protected as they moved.17

Interpretations of movements in the indicators should be tested and verified against other indicators and based on a rigorous analysis of the conflict, justice and security situation.

17 D. Chigas et al., What Difference Has Peacebuilding Made in Kosovo? (Cambridge, MA and Prishtine/Pristina: CDA and CARE, 2006).

20

V. Strengths and Weaknesses of the Data

A number of issues regarding the types of data and measurement of indicators from the datasets should be highlighted:

1. Availability of data. It should be noted that for many of the datasets, information is not readily available and thus may be of limited short- to medium-term use in guiding country programmes. While some datasets are updated yearly, many are not, and often it takes up to a year for the collected data to become publicly available. This implies a significant time lag in the availability of data for assessment, severely limiting the utility of indicators for monitoring programmes and adjusting them to events.

2. Consistency. For perceptions data in particular – such as the Afrobarometer, the Arab

Barometer, and the Asian Barometer – there is also an issue of consistency. The same questions are not asked in each country or each round of a particular country’s surveys. Given the paucity of perceptions data generally, this is of particular concern.

VI. Matrix of Conflict, Justice and Security Indicators

The matrix presents baskets of indicators designed to assess country trends in conflict, justice and security so that DFID/HMG can better evaluate the impact and value of its strategies and assistance. The indicators were developed using existing datasets. The source of the data and scale for each indicator is contained in the second column of the matrix. More information, including geographic and time coverage, methodology and sources of the data for the indicators, is available in the Appendices B and C. The matrix includes brief comments on appropriate and inappropriate uses and on strengths and weaknesses of most indicators as well.

A. Assumptions underlying the choice of indicators The conflict, justice and security indicators were developed based on several assumptions and principles, in order to enhance their accuracy and usefulness. • No single theory or perspective of conflict. Most of the peace and conflict datasets are based

on well-defined, although not necessarily explicit, theories of change, peace and stability In cases where the theory of change is quite strong (e.g., Index of State Weakness in the Developing World, which defines fragility as the cause of conflict), the indicators measure whether the elements of the theory of change are present. Such indices and indicators, however, may not reflect what is occurring in a country with respect to conflict. Consequently, the baskets of indicators do not adopt or rely on any single index, but rather include indicators from several of these indices, in order to avoid implicit or explicit adoption of a particular theory of peace or conflict.

21

• Distinction of fragility and conflict. Underlying the choice of indicators is a distinction

between concepts of state fragility and conflict. While fragility and conflict are clearly related, they are not the same thing. There are many definitions of fragility, but the OECD’s characterization may best capture the essence of the concept -- state structures lacking political will and/or capacity to provide the basic functions needed to reduce poverty, stimulate development, and safeguard security and the human rights of their populations (OECD 2007). Conflict, by contrast, refers to real or perceived incompatibilities between needs, values and interests, primarily at the societal level, which have the potential to become or have become violent. In this sense, fragility can be the result, the cause, or a symptom of conflict. Therefore, some indicators related to fragility are included as indicators. However, they cannot be used alone as indicators of conflict or potential conflict.

• The importance of viewing phenomena from variety of perspectives. The baskets of indicators

represent a variety of perspectives on an issue, such as criminal justice, government effectiveness or grievances. They include indicators on structure, dynamics and outcome. They attempt to balance objective (# homicides) and subjective (perceptions of insecurity) measures. This ought to provide a more accurate picture of the phenomenon and underlines the fact that no one measure is adequate. It should also be noted that many indicators are proxies for the phenomenon they are intended to measure and this is highlighted in the uses and comments columns.

B. Using the Indicators

As noted above, the matrix concentrates on conflict, justice and security indicators that are measurable at country level to help staff monitor trends and the overall direction in which a country is moving. The indicators should be used in light of the cautions outlined in section III.A. above (Uses and Limitations of the Indicators). For purposes of interpretation, it is useful to presume a relationship between and among the indicators. It is partially for this reason that the baskets of indicators have been developed and organized to minimize the time required to collect data. At the same time, DFID and HMG’s strategy or assistance programme may focus on a particular sector, set of issues, or dimension of conflict, justice or security. A country programme may provide budget support and wish to assess whether the resources contributed to progress. In this example, use of the entire set of indicators would make sense. Care also needs to be exercised to avoid normative or theoretical bias that can result from the selection of too narrow a set of indicators. For example, a country peacebuilding strategy that focused on improving economic and social policy capability or government service delivery might choose indicators related to government effectiveness. This selection of indicators might

22

help to track whether the government is becoming more effective in service delivery. It would not track whether that improved service delivery reduces conflict and promotes peacebuilding. Indicators related to group grievances or the legitimacy of the state would be useful to add to mitigate the theoretical assumption that improved service delivery reduces conflict. Similarly, if a country wishes to assess trends in justice and security development as they relate to conflict, it would insufficient to focus only on justice indicators. If data disaggregated by group were not available, conflict indicators on grievances, inclusion, perceptions of unfair treatment, etc. would need to be examined. In this case, the combination of justice and conflict indicators would not directly assess a reduction in conflict due to improvements in justice. Rather, they could provide a proxy for determining whether general improvements in criminal justice provision also ameliorate the conflict situation.

(i) Selecting “baskets”: some criteria

a. Use a range of indicators. Too many may be burdensome and reduce focus; too few will provide insufficient or misleading picture of what is going on.

b. Include measurements of different aspects and different perspectives. Given the paucity

of perceptual indicators and data, perception surveys should be included wherever possible. They provide an important check of whether actions, policies and strategies are having desired impacts and whether outsider observations or measures correspond to people’s perceptions.

c. Avoid overreliance on one kind of source. The indicators rely on a number of sources: expert/narrative reports, events/numbers, surveys, media analysis, among others. Although it may not be possible to diversify sources, overreliance on one source will skew the analysis and introduce potential biases.

(ii) Interpreting the indicators

For all the indicators, as has been previously discussed, caution needs to be exercised in determining what conclusions should be drawn from movements in any one or several indicators, or in any one category of indicators. A few simple questions can serve as a guide to interpreting the indicators and deriving a coherent analysis of trends.

a. What is the story the movement of indicators tells? The story should be about trends and progress in the country under examination. It is not about comparisons with other countries or about the state of the country at a particular point in time. Deciphering the direction in which a country is moving may be difficult, particularly where indicators

23

have large margins of error; small changes in any one given year are statistically insignificant; and time delays or dataset coverage makes timely data unavailable. Understand what the indicator can tell you and what it cannot. For example,

government effectiveness can portray what the government may be capable of doing. It does not reveal what the government actually does. GINI provides information about the distribution of wealth, but does not disclose the relevance of that distribution to conflict. The homicide rate bespeaks of the number of people killed, but does not indicate whether lowering the murder rate is the priority need voiced by the local population.

Be careful about attributing meaning to what the indicator says. Can be misleading.

b. Is the story accurate and relevant? Relevance should refer to the analysis of the country

context, conflict and/or justice and security situation. Accuracy takes account of the degree to which the indicators faithfully represent the phenomenon they are supposed to measure.

c. What are gaps and weaknesses in the story that the indicators tell? Inevitably there will

be gaps in the story indicators tell, particularly given the lack of reliable data on non-state justice and security mechanisms and the service provided by private security companies. Other gaps may exist because of poor perception data. Still others may arise due to the normative biases of the indicators. Still others may become apparent because data is aggregated and analysis cannot be conducted into the services provided to different groups, which is crucial for a conflict analysis. The issue is to be aware of these gaps and seek ways to address them. It will be important to analysis what the absence of data implies and how the story may change if data were to become available.

d. Are there contradictions in the story that the indicators tell? Under each and every

circumstance, the contradiction(s) requires explanation. Does the contradiction reveal a lack of internal consistency within a category of indicators or between categories? Does the contraction disclose a lack of consistency between individual indicators? Or is the apparent contradiction disguising an issue for which more analysis is needed? For example, when government effectiveness improves, grievances may also increase. Launching a programme to address violence against women will, very likely, increase the reported incidents of such violence. When trust in the police strengthens, complaints about alleged police misbehavior and malfeasance may spike up. An initially more accessible state justice system could lower the satisfaction the citizenry have in the performance of the state justice system because the demand for justice outstrips its supply.

24

e. What is the minimum story that needs to be told? In justice and security development, the minimum story should outline the priority needs of the citizens and residents of the country and, if possible, their perceptions of the services delivered to them and by whom those services are delivered. In conflict, the minimum story should consider conflict intensity, leadership culture/process and grievances. The other factors are clearly relevant, but focus on socio-economic, governance, electoral or security factors will vary depending interest and relevance; they should be interpreted, however, in conjunction with the three “minimum story” elements.

f. What does the story leave out? Existing datasets have significant gaps in important kinds

of indicators. Specifically, perceptions data is often missing or there are large gaps in coverage. Country offices can fairly easily obtain this data through surveys or, if resources are an issue, staff (DFID or others) can go out to listen and ask questions to people, making sure to travel outside the capital. In addition, data on non-state mechanisms for conflict resolution, peacebuilding and provision of justice and security is missing, as is data on private security. These too would need to be collected in-country. Finally, the indicators that exist for socio-economic dimensions of peacebuilding, for governance, development and for justice and security, are largely not conflict sensitive. It is difficult to understand the conflict impact of improvements in justice systems, for example, from the indicators and the data that is currently available, or from the indicators of personal security. Measures of social capital (“trust”) also usually refer to generalized trust rather than amongst conflict groups. To the extent that disaggregated data can be disaggregated by group, this can provide conflict-sensitivity. A much less valid alternative is to interpret that data alongside indicators of grievances and conflict intensity.

UNDP, in its User’s Guide for governance indicators, advises that indicators should be among the first questions asked; they should not the last. For additional questions on indicators, there are several useful resources that can be consulted, in addition to DFID’s own, including:

• UN User’s Guide for Governance Indicators • Forthcoming User’s Guide on Fragility and Conflict Indices • DFID Guidance on Governance, and forthcoming on Voice and Accountability

25

Matrix of Indicators

A. Conflict Intensity Indicator Source Use Cautions

1. # deaths from armed conflict

Uppsala Conflict Data Project

Direct measure of level of civil war-type violence. Uppsala defines armed conflict as “contested incompatibility that concerns government and/or territory where the use of armed force between two parties, of which at least one is the government of a state, results in at least 25 battle-related deaths in one calendar year.”

There is a difference between deaths attributed to conflict and fatalities from other forms of violence, mainly criminal. The number of deaths from non-conflict causes far exceeds deaths from war. Consequently, very violent societies may be low on this indicator.

2. # displaced as % of population

UNHCR http://www.unhcr.org/ statistics/45c063a82.html

This should supplement # of deaths in order to capture the severity of the consequences and the level of fighting.

Persons displaced may have had to leave their homes for reasons other than conflict, such as hurricanes, drought, typhoons, etc.

3. Territory affected by conflict

Political Instability Task Force

Measures the scope of conflict. If a conflict is confined to a particular geographical area of the country, its severity may be considered to be lower, even if its intensity in that area is high.

4. Seriousness of ethnic, religious, social conflicts

Bertelsmann Transformation Index (BTI) Question 13.3 http://www.bertelsmann-transformation-index.de/ fileadmin/pdf/Anlagen_BTI_2008/ Alternate: World Governance Indicators, “Political Stability and Absence of Violence,” http://info.worldbank.org/ governance/wgi/index.asp

Serious conflict may exist without deaths. This indicator captures the severity of polarization, mobilization of people, and the potential for violence. The WGI “Political Stability and Absence of Violence” reflects a much broader set of stability concerns – from the possibility of military coup to political assassination and terrorism – and is primarily concerned with the stability of the government. BTI is a more narrow measurement of the intensity of social cleavages. Choice of the Bertelsmann vs. WGI indicator should be made based on the context. In countries where political, criminal or terrorism-based violence are of primary concern, WGI may be more appropriate, as well as if short–term stability concerns are paramount.

BTI is a less reliable indicator, in that experts’ assessment of seriousness or “irreconcilable” cleavages may differ. However, it is a more valid measure of the seriousness of polarization. WGI is less up-to-date than BTI. Additionally, while the WGI includes the Bertelsmann measure, it may be too broad to measure conflict intensity. The WGI could be useful to as a cross-check and verification. If the two measures are not similar, then further analysis may be warranted.

26

Indicator Source Use Cautions

5. Neighborhood War

Peace and Conflict Instability Ledger, University of Maryland

Not a direct indicator of conflict intensity in a country. It represents an exacerbating factor that can prolong and intensify conflict, as well as compound conflict consequences (e.g., displaced).

Most useful in regions in which there are ongoing and/or simmering conflicts, such as in the Great Lakes region. Can also provide data regarding risks of flows of refugees triggering movements in underlying societal cleavages.

B. Institutions/Mechanisms/Processes that Handle Grievances B.I. Capacity of Government to Deal with Grievances The capacity includes capacity to develop and implement policies that address grievances Effectiveness

Indicator Source Use Cautions

1. Governance Effectiveness World Governance Indicators http://info.worldbank.org/ governance/wgi/index.asp

This measures the overall quality of public service provision, bureaucracy, competence and independence of the civil service, inter alia. It reflects the quality of the “inputs” for the government to be able to produce and implement policies that would be needed to address (or prevent) grievances.

Aggregate indicator that includes many dimensions of effectiveness. Increases or decreases in the indicator will not provide sufficient information from which to draw concrete conclusions about the impact of interventions. In addition, small changes are likely to be statistically insignificant, unless viewed over a number of years.

2. Efficiency of revenue mobilisation

World Bank Country Policy and Institutional Assessment (CPIA) Criterion 14

Measures tax policy and tax administration. A proxy for the availability of state resources to implement policies and provide services that might address grievances.

While tax mobilization is a common measure of state capacity, it is also problematic and should be relied upon with caution. There is little reliable tax data in fragile states and the validity of this measure outside the developed countries is limited. States can rely on alternative sources of revenue (e.g., natural resources or other national industries).

27

Indicator Source Use Cautions

3. Extent to which government can coordinate conflicting objectives into coherent policy

Bertelsmann 15.2 http://www.bertelsmann-transformation-index.de/ fileadmin/pdf/Anlagen_BTI_2008/ Detailed_Results_BTI2008.xls

The capacity of those in government to formulate policies and develop consensus. It implies as well the extent to which the government is or is not affected by factionalization and/or polarization, which are important elements of a country’s capacity to handle grievances.

A low capacity here may reflect stalemate over how to deal with grievances or an inability to put into place policies and mechanisms to address problems.

4. Access to improved water sources

Alternatives: Human Development Index Primary school enrollment (% gross) Youth unemployment

UNICEF/WHO Global Water and Sanitation Report; World Development Indicators Human Development Index: UNDP Human Development Reports http://hdr.undp.org/en/ statistics/data/ Primary school enrollment and youth unemployment: World Development Indicators, WDI Online.

We have chosen access to improved water because it is a key government service, one which requires concerted effort and planning, important for human development (not just economic growth) and often destroyed during conflict. If this could be disaggregated by geographic region, it would be helpful to assess inequalities as well. Updated data may be difficult to obtain on this indicator. Consequently, the composite human development index, or primary school enrollment might serve as alternative proxies. Outside assistance may contribute to increases in these indicators, so they are not fully valid as indicators of government effectiveness. Where youth unemployment data is available, employment would be a useful indicator, in light of evidence

Many socio-economic indicators have been proposed and are used in the conflict indices. Infant mortality has been preferred by several that have found a correlation with conflict. However, improvements in infant mortality may result from outside intervention, even during conflict. Education would be an alternative indicator (mean years of schooling, or primary school enrolment), but, if there is significant displacement, education may be provided by churches, non-governmental organizations or in refugee camps. In both cases, there may be a correlation with conflict, not because of issues of government effectiveness, but because conflict affects the government’s ability to provide services and also affects access by humanitarian and development actors to provide the service. These measures could be used as indicators of conflict, but caution should be exercised in attributing

28

that youth unemployment enhances risks of conflict.

improvements to an improved conflict situation. These indicators should be viewed in conjunction with deterioration of public services (E.II.9), which has more consistent and recent scoring.

Leadership culture/ process This is an aspect of capacity, but has been separated out because of its importance to tracking conflict. Additionally, leadership is an oft-forgotten component of development policy, which needs to be more systematically taken into account. These indicators look at the degree to which the values and interactions of leaders/elites in governance: a) may or may not escalate political conflict; and b) may or may not support the government’s ability to acknowledge and deal effectively with grievances that may underlie conflict. Three aspects of leadership are assessed: degree of polarization/factionalization, degree of commitment or belief of leadership in democratic processes (a proxy for likelihood of abiding by democratic rules), and degree of inclusiveness of leadership/governance.

Indicator Source Use Cautions/Comments

5. Factionalized Elites Failed States Index/Fund for Peace (1.11) http://www.fundforpeace.org/web/ index.php?option=com_content&task =view&id=99&Itemid=146

Measures fragmentation of elites along group lines and degree of polarizing behavior of elites and leadership. A direct indicator of elite relationships amongst elite groups. It is also a proxy for a power struggle within the elite, polarization, and/or manipulation by elites, all of which has the potential to escalate conflict.

6. To what extent can political leadership manage political cleavages?

Bertelsmann Question 16.3

Relevant where there are deep societal cleavages along ethnic, class, regional or religious lines. Measures the capacity of leadership to bridge these cleavages. If B.I.5 is high, indicator is redundant; use if factionalization is not high.

Although relevant and valid as one measure of bridging social capital, its reliability is problematic. Terms such as “depolarization” and “consensus across the dividing lines” can be interpreted differently.

7. Regime/governance inclusion (Political legitimacy score)

State Failure Index, Center for Systemic Peace, George Mason University based on Polity IV

Measures inclusiveness of the governance process -- the degree to which minority groups (social, ethnic or religious) are included in governance.

A composite drawing on 4 indicators: Factionalism (Polity IV), Ethnic group political discrimination (Discrimination 2007), Political salience of elite ethnicity (Elite Leadership

29

characteristics), polity fragmentation. As an aggregate, it has all the problems with such types of indicators.

8. Extent to which democratic institutions are accepted or supported by relevant actors

Bertelsmann Question 4.2

Use in conjunction with indicators on elections and governance. Provides some triangulation for the elections’ capacity to support peaceful transition rather than greater conflict. Intended to measure the strength of “veto powers” or spoilers of democracy, it can also provide data on the degree of key actor “buy-in” to democratic processes.

Concepts of “relevant actors” are vague and may affect reliability of indicator.

Grievances These indicators attempt to look at the effects of government policies and practices on the grievances that can cause conflict. Several indicators of potential inequalities that may be sources of grievance and evidence of unwillingness to address them are included here. These should be seen in conjunction with indicators measuring perceptions of unfairness and grievance. In conflict situations, it is the perception of grievance that matters more than the objective measure of the grievance itself. The the nature and severity of grievances, and their potential to lead to violence, will naturally differ country to country; consequently, these indicators should be selected and viewed in connection with an analysis of conflict, if they are to be relied upon in a country setting. If grievances are worsening, it is likely that the government does not have mechanisms in place to deal with them.

Indicator Source Use Cautions/Comments

9. Group grievances

Failed States Index, Fund for Peace (I.3)

Mostly measures behavior vis-à-vis communal groups that could lead to grievances around which these groups could mobilize, including atrocities against groups, political exclusion, public scapegoating, “hate” radio, etc.

Because FSI relies on media material, data related to institutionalized political exclusion, atrocities or persecution may be under- or overweighted. The potential measurement errors should be balanced by the relevance of the indicator, one of the few in existing datasets, measuring this dimension of conflict.

10. Minority access to education, commerce, professions, etc.

Minorities at Risk, University of Maryland http://www.cidcm.umd.edu/ mar/data.asp

This and the following two refer to specific minority rights that are often areas around which grievances are mobilized. For use if relevant to the context.

They may or may not be relevant to a particular country situation. Caution should be exercised in reading the data, as the data is coded by minority group; where there are minority-led governments, the “majority” group should be considered.

30

Indicator Source Use Cautions

11. Restrictions on religion Minorities at Risk, University of Maryland

12. Restrictions on use of language or language instruction

Minorities at Risk, University of Maryland

13. GINI Coefficient

World Development Indicators Should be viewed in conjunction with uneven development (B.14) or group grievances (B.9) to provide a picture of the effects of inequalities of wealth on conflict.

14. Uneven Development Failed States Index/Fund for Peace (1.5)

A measure of horizontal inequalities that could fuel conflict.

Combined with the GINI coefficient could provide a proxy for the degree to which concentration of wealth is group-based and may impact on group relations.

15. How often has your group been treated unfairly by the government?

Afrobarometer Ghana 2008 survey, Question 8218

Experiential measure of sense of exclusion/oppression. Complements more “objective” measures of inequalities and group grievances.

If used in conjunction with human rights and civil liberties indicators, this public perception data can provide some indication of whether human rights/civil liberties restrictions may be a source of group-based grievance.

16. Your group has same, worse, or better economic conditions than others

Afrobarometer Ghana 2008 survey, Question 11

Experiential/perceptual measure of “horizontal inequalities.” Complements more “objective” measures of inequalities.

Used in conjunction with more standard development indicators (GNP per capita, e.g.), this public perception data can provide perspective on whether economic growth is likely to promote or mitigate conflict.

18 All Afrobarometer sources refer to the 2008 Ghana survey, unless otherwise noted.

31

B.II. Governance Legitimacy and Accountability Elections free and fair Is the government representative, does it reflect the voice of the people, and is it legitimate in the eyes of the population? The better the electoral process and the fairer it is in the eyes of the population, the more likely it is, theoretically, to have popular confidence to discuss and develop adequate solutions to problems. Strong caution should be exercised in relying only on data related to elections, as free and fair elections can also exacerbate societal and political cleavages. This indicator should be used, at a minimum, alongside indicators of political cleavage and governance inclusion.

Indicator Source Use Cautions/Comments

17. Electoral Process and Pluralism

Economist Intelligence Unit Democracy Index after 2006 Freedom House Freedom in the World, Political Rights A (Electoral Process) and B (Political Pluralism and Participation) http://freedomhouse.org/ template.cfm?page=15 (Tables and Charts)

Measures how free and fair elections are, at the national and municipal levels, as well as whether citizens can vote fairly. Should be triangulated with perceptual data from B.II.18. EIU Democracy Index was created in 2006. Freedom House has earlier data and permits separate examination of electoral process-related trends and trends in political participation and pluralism; EIU combines both.

EIU index is scored on 1-10 based on expert assessment of questions cross-checked with the World Values survey when feasible.

18. Confidence in honesty of elections Alternative for Africa/Arab: “Fairness of last national election?”

Gallup World Poll Afrobarometer Ghana 2008 survey, Question 71, Arab Barometer Q 211

Measures popular perception of the fairness of elections. For African and Arab countries covered, Barometer questions are more specific and recent. These surveys should be substituted when available.

Current year’s data available; historical data available with subscription.

32

Legitimacy of the State Perceptions of the legitimacy of the state are an important supplement to data and perceptions of elections. Important as it measures the extent to which people are likely to look to the state and have confidence in the state to handle grievances and promote their well-being.

Indicator Source Use Cautions

19. Delegitimization of state

Failed States Index/Fund for Peace (I.7) Alternative: State Fragility Index, Political legitimacy and Social legitimacy scores

Direct indicator of popular confidence in the government. Proxy for the capacity of government to respond constructively to grievances and popular confidence in the government’s ability to deal with grievances. Looks at the degree of delegitimization of the state, including elite corruption, resistance of ruling elites to political representation, loss of popular confidence in state institutions (e.g., election boy-cotts, public demonstrations, inability of state to collect taxes, etc.) and linkage of crime syndicates to ruling elites.

Delegitimization of the state does not necessarily lead to conflict, as the “revolutions” in Georgia and Ukraine suggest. It may, however, reflect background conditions that, in combination with other factors, could result in conflict, such as in Timor-Leste and the 2006 crisis.

20. Relevant groups agree about citizenship and accept the nation state as legitimate?

Bertelsmann 1.2 Relevant in the event of possible separatist conflict, where the unity of the nation state is in question. Also can be used in conjunction with group grievances, as a component of the indicator looks at the extent of discrimination against groups.

“De facto” exclusion of groups from political citizenship may be susceptible to differing interpretations by experts and, consequently, the indicator may not be entirely reliable. The fact that the indicator covers two important concepts – support for the nation state and discrimination – makes it very relevant, but the inclusion of two related by separate concepts makes it imprecise.

33

Indicator Source Use Cautions

21. Public support for government restrictions on civil liberties a. Agree or disagree with statement: "Government should be able to ban any organization that goes against its policies." And the reverse statement

b. Agree or disagree with statement: "Government should be able to close newspapers that print stories it does not like." And the reverse statement

c. Agree or disagree with statement: "Government should not allow the expression of political views that are fundamentally different from the views of the majority." And the reverse statement

Afrobarometer Ghana 2008 survey, Question 19 Afrobarometer Ghana 2008 survey, Question 20 Afrobarometer Ghana 2008 survey, Question 21

Use in conjunction with indicators on human rights and civil liberties and/or deligitimization of the state. Provides interpretive data on the significance for conflict of human rights and civil liberties restrictions and delegitimization of the state: is there public support for repressive actions by government? If so, the repressive actions or corruption in the government may not be a source of conflict and improvements in those indicators may not signify any impact on conflict.

Limited coverage, as only Afrobarometer includes these questions. To be an accurate interpretive measure, these indicators should be disaggregated by group. Afrobarometer does ask about group identity in its surveys. Although the published survey results are not disaggregated, disaggregation should be possible.

22. Control of Corruption World Governance Indicators Corruption reflects on the legitimacy of the state, but there is no direct relation to conflict. If there is serious concentration of wealth, and factionalization of elites (B.I.5. or 7), this indicator may be relevant to assessing the degree to which capture of state resources is a driver of conflict.

34

B.III. Human rights & Civil liberties These are indicators of rule of law, as well as a proxy for the potential for the extent to which peaceful mechanisms for handling grievances exist. We have specifically excluded physical issues here; these have been placed under perception of safety and security.

Indicator Source Use Cautions/Comments

23. Civil liberties EIU Democracy Index Freedom House Freedom in the World D (freedom of expression/ belief), E (associational/ organizational rights), and G (personal autonomy) for greater disaggregation and/or data prior to 2006.

Mostly freedom of media-related and freedom of expression data, but also includes questions about mechanisms to redress grievances, security, and whether property rights are protected.

This may be over inclusive as a conflict indicator, and should be interpreted in conjunction with other indicators of horizontal (group-based) socio-economic, political inequality, and/or grievances.

24. Suspension or Arbitrary Application of the Rule of Law and Widespread Violation of Human Rights

Failed States Index, Fund for Peace (9)

This is an aggregate of four descriptive sentences, ranging from emergence of authoritarian/ dictatorial rule to rising number of political prisoners.

Alternative is State Repression Index from Political Terror Index; FSI is more comprehensive.

25. Allowable advocacy activities CIVICUS Civil Society Index 2.5.2 Looks at extent to which CSOs are free to engage in advocacy and criticize government. A proxy for the availability of non-violent means for civil society actors to have voice and address grievances. In combination with data under C. below, provides an impression of the feasibility and existence of civil society initiatives to promote peace.

This indicator does not provide any data on existing local capacities for peace, only the existence of space for action by local, non-state actors. It does not include traditional mechanisms or processes for conflict resolution, which may, in some contexts, be significant. It is also very limited in its coverage of countries.

35

C. Community Initiatives For Peace Community/Civil society initiatives for peace This looks at civil society initiatives for peace: to what extent are there local capacities for peace? In general, there is very little data on local, non-governmental initiatives or capacities for peace that is systematic and covers a wide geographical area. The existing datasets provide very little, the indicators below have very limited geographic coverage, and do not address the range of issues for which indicators are needed.

Indicator Source Use Cautions/Comments

1. Civil society actions to promote non-violence and peace

CIVICUS 3.4.2 Measures extent of civil society activity and advocacy for peace/non-violence.

Note there are similar indicators for gender equity and democracy. Alternative or additional indicators would be CIVICUS 3.3.2 on how much civil society promotes tolerance at the societal level. Qualitative data, unlike most of the other indicators in conflict, justice and security.

2. Activity and success of civil society in influencing public policy in human rights

CIVICUS 4.1.1 Approximates potential influence and effectiveness of civil society on peace issues, premised on the assumption that human rights, like peace, is a sensitive issue.

36

D. Capacity/Willingness to Resist or to Escalate Violence Military and societal militarization The indicators in this category reflect opportunity to fight/military feasibility of war and justice and security development

Indicator Source Use Cautions/Comments 1. Military expenses as

% of GDP The Military Balance, Stockholm International Policy Research Institute

Expenses, numbers of armed forces, and security officers/police give an indication of the resources dedicated and available to the government to engage in conflict. In light of the difficulty in determining the resources available to non-state actors, these are used as proxies for the opportunity and willingness to fight as it is assumed that a government will increase its capability to respond to threats of armed violence.

Many security expenditures -- military, intelligence, police -- may be off-budget. Consequently, this may not be an accurate reflection of expenses.

2. # armed forces per 100,000 people

The Military Balance, SIPRI

A proxy for resources available to pursue conflict. In conjunction with D.3 and D. 1, assesses the level of resources available for conflict.

To the extent that international and/or regional (i.e. cross-border) conflict is an issue in a region, this may not be a good indicator of resources available for internal conflict. Caution should be exercised in interpreting trends in D.1-D.3. Upward trends may reflect intensifying conflict, but may also reflect greater outside threats to security, as well as threats from crime.

3. # security officers and police per 100,000

10th Crime Trends Survey, UN Office on Drug and Crime

Similar to D.2, it is a proxy for resources available to pursue conflict.

Have been unable to obtain definition of “security officers” from UNODC.

4. Ease of access to small arms

Global Peace Index (collected by EIU)

The availability of small arms is a proxy for the ability of non-state groups to engage in violence and conflict. Ease of access to small arms, however, also pertains to violence that can be committed that has nothing to do with conflict.

This is an expert assessment and, as a proxy, cannot be considered reliable. However, as there are no other measures of capacity of non-state groups to engage in violence and conflict, this is a default measure.

37

Indicator Source Use Cautions

5. Intentional homicide committed by firearm, per 100,000

UNODC 2.6 Measure of level of violence. More valid for security development than as a measure of conflict, given the distinction between violence pertaining to conflict and other, usually criminal, activities.

Have been unable to obtain a meaningful definition of “intentional” from UNODC. Even if a consistent definition were to have been obtained, there will be little consistency across countries in how “intentional” is determined by the responsible authorities. Nevertheless, it is the best measure available.

6. Non-violence within the civil society arena

CIVICUS 3.4.1 Measures use of nonviolent means by civil society to express their interests in the public sphere.

Does not fully measure the propensity to use violent/non-violence means in the non-governmental sphere, as it is limited to civil society organizations. However, in the absence of other data, it can provide an indication, which should be interpreted in light of other contextual knowledge. The limited coverage of the data, however, further erodes the indicator’s utility.

38

E. Justice E.I. Criminal justice capacity and effectiveness The indicators in this category reflect the overall structure, capacity, and effectiveness of the criminal justice system and its component parts. Activities and results

Indicator Source Use Cautions

1. Conviction rate (total # of persons brought before the court/# total number of persons convicted)

UNODC 9.1 This is a process indicator, measuring the effectiveness of the overall criminal justice system, from police through prosecution and courts. It may be the best single non-perception statistic, although interpreting what the data means will require additional analysis and investigation.

Almost all UNODC statistics can be broken down by age, gender, type of crime. The organization’s indicators, however, cannot be broken down by group. Consequently, these indicators are useful as the starting points, but would need further specification if they are to be valuable for conflict analysis. UNODC statistics are compiled from reports filed by governments with the UNODC. There statistics, therefore, are as reliable as the original government database, which may be questionable. Coverage of UNODC’s statistics varies too according to if a government files a report.

2. # of recorded crimes per 100,000 people

UNODC 2.0 Is the standard measure of crime rates around the world, though admittedly a notoriously unreliable one.

Reported crime is a dubious indicator, given that a preponderance of crime goes unreported. At the very least, reported crime rates need to be balanced by victimization studies. It is preferable if reported crime rates are also interpreted in light of trust/confidence in the police and indicators that assess the effectiveness of the entire criminal justice process, such as E.1.

39

Indicator Source Use Cautions

3. # convictions/# police UNODC Q11/UNODC 1.1

Measures the efficiency of the police. A low score should lead to questions about the overall management and police effectiveness. A low score may also be indicative of a poor relation between the police and the prosecutorial service. Conversely, a high score may suggest that the police are addressing only low level crimes, one for which obtaining a conviction is relatively easy.

Cannot be used without reference to other indicators, namely E.1 and E.4. Also best used in conjunction with police satisfaction and victimization surveys.

4. # convictions/# prosecutors UNODC Q11/UNODC 5.0

Measures the efficiency of the prosecutors. All these efficiency statistics can and should be used together.

As with E.3., this indicator needs to be placed within the context of the overall criminal justice system and cannot be used as a stand-alone measure.

5. # prosecutions/# prosecutors UNODC Q7/UNODC 5

Measures the workload of the prosecutors and may be indicative of their capacity, particularly suggestive of their being underresourced. Needs to be analyzed alongside E.4.

Care should be used with this indicator because it is possible that prosecutors have work rules that determine the number of cases they handle.

6. # convictions/# prosecutions UNODC Q11 / UNODC Q7

This is another measure of the effectiveness of the criminal justice system.

The significance of this number can be interpreted only in conjunction with the other criminal justice indicators, in particular E.I.1. E.!. 3, 4, and 6 are complementary.

7. # prosecutions/# judges UNODCQ7/ UNODCQ8

Measures the workload of judges. To be used in conjunction with E.5.

Criminal justice is only one type of case heard by the judges. This capacity statistic, therefore, is best used when the judicial system conducts its work in specialized courts.

8. Average # months of criminal trials

UNODC Q 9.2 Measures the efficiency of the judicial system. If the average # of months appears to be excessive, it would be a leading indicator from which to probe further.

There should be no assumption that the speed with which a criminal trial is dispatched implies that it is more or less fair and equitable. Adherence to rule of law principles cannot be measured by length of a trial. Therefore, it must be used in relation to the other indicators.

40

Perceptions

Indicator Source Use Cautions

9. How well or badly would you say the current government is handling the following matters, or haven’t you heard enough to say?... F. Reducing Crime (1 = very badly… 4 = very well)

Afrobarometer 57F

Perception question useful to reveal the attitude of the population to government policy. It may be suggestive of the population’s confidence in the government and criminal justice system.

It suffers from the same weaknesses as all other indicators derived from the Afrobarometer – consistency, timeliness, etc.

10. Who do you think has primary responsibility for managing each of the following tasks - national government? local government? traditional leaders? Members of community? None of them? Don't know.

Afrobarometer 58 E and H

Perception question meant to begin to assess the relationship between state-provided and non-state provided justice and security. It is only a proxy because there may be a discrepancy between who is believed to have primary responsibility and to whom individuals go when they have a dispute.

There are very few indicators that address the range of non-state justice and security issues. Hesitancy should be used in its interpretation with respect to the distinctions between local government, traditional leaders, and members of the community, as there may be significant overlap between and among these levels.

11. In your opinion, how often… do officials who commit crimes go unpunished?

Afrobarometer 45 E and D

Measure of perceived impunity of government officials and is one measure of adherence to rule of law principles.

Perceived impunity may be an important indicator for conflict if it were susceptible to disaggregation.

12. In your opinion, how often… do ordinary people who break the law go unpunished?

Afrobarometer 45.E

Measure of perceived impunity of the citizenry and is one measure of adherence to rule of law principles.

Should be used in conjunction with E.10 to evaluate the perceived impunity of government vis-à-vis citizenry.

41

E.II. Rule of Law: civil and administrative justice

Indicator Source Use Cautions

1. # days to resolve dispute World Bank Doing Business

Though an increase in number can suggest an increased possibility of corruption, this indicator is, at best, a proxy. An increase in the number may also be suggestive of overall government capacity. World Bank Doing Business are perception surveys of 181 economies and are administered to local business leaders, lawyers, professionals involved in regulatory activities, and government officials. Because the surveys have been conducted since 2003 and are administered annually, they are among the most reliable and consistent perception surveys.

In general, World Bank Doing Business surveys are not meant to be comprehensive measurements of corruption, governance, or adherence to rule of law. Nevertheless, because they are administered to selected segments of the population, they are suggestive and should be used in conjunction with other indicators to provide a more rounded assessment. It should also be noted that the segment of the population to which these surveys are administered is a vital group, even if it is not vulnerable or marginal one.

2. # steps to claim, obtain, enforce contracts

World Bank Doing Business

As with E.11.1, an increase in number can suggest increased possibility of corruption. This indicator, however, has the added value of being a proxy for adherence to rule of law, given that one of its components is the enforceability of contracts.

This indicator is adherence to the rule of law from the perspective of those engaged in regulation and business in the country’s most populace city.

3. To what extent does an independent judiciary exist?

Bertelsmann 3.2 An independent judiciary is one of the principal criteria of rule of law. Bertelsmann uses country experts to write and evaluate so that this indicator may not be as useful as perception surveys.

It is important to note that judicial independence is only one principle among many by which rule of law is evaluated. There is discussion among justice development practitioners whether judicial independence is the judicial paramount principle, but it remains an open question.

42

Indicator Source Use Cautions

4. Court system fair/impartial? Law and regulations predictable and consistent

World Bank Enterprises Survey

Perception from business leaders, which, albeit a limited constituency, is an important one.

5. Should the President always obey the law?

Afrobarometer 37 This is a perception question whether it is believed that the President is beholden to the law. In this sense, it is a proxy for popular belief in the rule of law, one of major principles of which is that no one is above the law. Again, however, the difficulty is the phrasing of the question as “should” cannot be equated to “does.”

Should be used in relation to questions of impunity above.

6. How much do you trust… Courts of law?

Afrobarometer (49H), alternatively, see Gallup World Poll for Asia

In addition to being about “trust” per se, it is also presumed that “trust in the courts” implies a belief that the decisions rendered by the courts are fair, impartial, and independent. Hence, this is also a proxy indicator.

If it could be broken down by group (which for Afrobarometer may be possible), could give an indication of group grievance/perceptions of exclusion.

7. Does the rule of law prevail in civil and criminal matters? Are police under direct civilian control?

Freedom House, Freedom in the World F2

Freedom House indicators are based upon the opinion of country experts. This indicator is an aggregate of seven separate questions, ranging from the political independence of prosecutors, detainees access to legal counsel, to law enforcement officials independent from non-state actors (criminal enterprises, influential players and others).

This indicator may be best used in relation to E.II.3, as both are administered by country experts. Whether the police are under civilian control is a primary principle of democratic governance.

8. Do laws, policies, and practices guarantee equal treatment of various segments of the population?

Freedom House. Freedom in the World F4

This indicator is an aggregate of six separate questions, ranging from different groups’ ability to exercise their human rights to whether violence against such groups widespread.

This is one of the indicators that bridges justice and security measures and conflict indicator.

43

In addition to measuring an important criteria of rule of law (equality before the law), this indicator is a proxy for exclusion, possibly factionalization. In conjunction with other indicators, may help to understand conflict dimension and impacts of rule of law.

E.III. Access to Justice: state and non-state systems

Capacity

Indicator Source Use Cautions

1. # police per 100,000 UNODC 1.1 (Rate) Standard measure of police capacity used around the world.

The UN has promulgated that a rough standard would be roughly in the range of 1 police officer per 450 persons. However, the standard does not take into account education levels; difficulty of terrain roads, and communication; equipment/logistics possessed by the police; poverty rates, and age distribution of the population, all of which will affect the needed police manpower.

2. # prosecutors per 100,000 UNODC 5.0 (Rate) Measure of prosecutorial capacity. Should be used in conjunction with other capacity and efficiency indicators. The capacity of the prosecutorial service is suggestive of whether a state-provided court system can meet a country’s short- to intermediate-term needs or whether there ought to be donor support directed to non-state/local justice networks.

As with other capacity indicators, this does not measure activities or outcomes. It may be suggestive, however, of whether the prosecutorial service is understaffed. It can be eliminated, if process data is ascertainable – E. 1, 4, 5, and 6.

44

Indicator Source Use Cautions

3. # lawyers per 100,000 Law Schools / Bar Associations

The number of lawyers in a country is an important indicator of whether a state-provided court system can meet a country’s short- to intermediate-term needs or whether there ought to be donor support directed to non-state/local justice networks.

There is no dataset that contains this statistic, but it ought to be readily obtainable by consulting the national Bar Association or through the country’s law faculties. Caution must be used in analyzing the gross number because there is no certainty that each and every lawyer practices the law.

4. # hours to pay taxes, # payments per year

World Bank Doing Business This is a proxy indicator under the assumption that if both increase, businesses will pay fewer taxes and funding for state-provided justice will be limited, compelling people to resort to non-state system.

To be used with caution and mainly as a supplemental or supporting indicator.

Access

Indicator Source Use Cautions

5. Attorney, court, enforcement costs as % of claim value

World Bank Doing Business

The high cost to obtain access to state system will compel people to resort to non-state system.

This is not a direct measure and should be used with caution, but, once again, it is an attempt to uncover the relationship between state and non-state justice.

6. Government repression of groups

Minorities at Risk If government engages in repression, equitable access will, by definition, be limited. This indicator is an aggregate of three questions on repression.

This is not a perception survey, but is scored by graduate students and reviewed by experts.

7. Deterioration of public services

Failed States Index/Fund for Peace (8)

This indicator is a proxy, for it is presumed that if public services deteriorate justice and security services do as well. Also an indicator of government effectiveness and grievances; can be used in conjunction with B.I.1, B.I.4 and B.I.14 to gauge conflict potential of socio-economic indicators.

The indicator is estimated based upon readings of open source media from the country under examination.

45

Women’s access & capacity

Indicator Source Use Cautions

8. # women police/total police UNODC 1.2/UNODC 1.1

This is an estimate of gender equity in the police. However, as a measure of representation, this indicator does not record equity in power and authority.

Access to justice for women is not limited to their employment representation, but no other data is available.

9. # women prosecutors/total prosecutors

UNODC Q5.1/UNODC Q5.0

Same as above. Save as above.

10. # women judges/total judges UNODC Q8.1/UNODC Q8.0

Save as above. Save as above.

11. # women prison staff/total prison staff

UNODC (14.1+14.4)/UNODC (14.3+14.0)

Save as above. Should be comparable to # women prisoners/total prisoners (available from UNODC or King’s College).

12. Women as % of legislators, senior officials, managers

IMD World Competitive Yearbook 2.5.10

This indicator is a survey of business executives, which is thereafter supplemented by statistical data, when available. Though the question is one of representation (gross number), it is suggestive of access and equality.

This source is extremely limited (only 55 countries), but it is the only dataset found that estimates % of women in positions of authority and influence.

Perception

Indicator Source Use Cautions

13. Trust in courts vs. trust in traditional leaders (how much do you trust each?)

Afrobarometer (Compare 49H vs. 49I)

All below are perception surveys, measuring to whom people turn to for justice.

The comparison of trust in courts vs. traditional leaders is suggestive of which mechanism the citizenry turns toward to address their disputes. Should be used in conjunction with other perception data to disclose discrepancies, if any.

14. Use of wasta (clout)(During the past five years, have you ever used wasta to achieve something personal, family-related, or a neighborhood problem?)

Arab Barometer 228 This is a proxy question and not a direct one

15. Reference to traditional leaders (asks to list the types of wasta – choice 4 is

Arab Barometer 227

46

traditional leaders)

16. Go to court to resolve disputes? (If you were to have a dispute with another citizen, would you try to resolve it in a court or in another government institution?)

Arab Barometer 228

17. Who would you go to resolve a dispute? Arab Barometer 229

18. Women are/should be subject to traditional laws19

Afro Barometer 23.B This question may or may not be in each country survey and the # of the question depends on which survey it is.

E.IV. Corruption

Indicator Source Use Cautions

1. Do you think the government is doing enough to fight corruption or not?

Gallup World Poll This is not a perception of corruption, but of the government response to it.

May be best used in connection with other impunity indicators. If there are variations between and among them, this would be an issue for further investigation.

2. Corruption perceptions Transparency Int’l - Corruption Perceptions Index

This is a country’s overall score, which is derived from thirteen separate sources, each of which is a different measure of corruption.

3. Registering property - # of procedures, # of days

World Bank Doing Business

The longer it takes and the more complicated it is, the greater the opportunity for corruption and the more likely corruption may occur (especially because of the involvement of property).

As this is a proxy, its use must be judicious.

19 In the 2005 survey, respondents were asked: “In our country, women should have equal rights and receive the same treatment as men do.” B. “Women have always been subject to traditional laws and customs, and should remain so.” Respondents could agree with A, B, or neither.

47

Indicator Source Use Cautions

4. Gifts given to receive public services

World Bank Enterprise Survey

Includes selected questions about electricity, construction, inspections, % government contracts given away, % total annual sales for business, import license, operating license.

This survey is completed by business leaders and is an estimate of their perception of a country’s governance structure and system and the degree to which government services have been effectively privatized.

F. Security Sector F.I. Prisons

Indicator Source Use Cautions

1. # prisoners/# beds King’s College This is a measure of how overcrowded the prison system is.

Overcrowding is a primary way of evaluating the performance of a prison system.

2. # juveniles/# beds for juveniles King’s College/ UNODC Measure of how overcrowding affects juveniles.

3. # pre-trial detention prisoners prior to disposition of case/# convicted prisoners

UNODC/King’s College This is a gross number recording the depth of the pre-trial detention challenge. More useful data would be the average number of months in prison an incarcerated person languishes prior to initial disposition of his/her case, but that data is unavailable.

As a gross number, it does not disclose the causes or reasons why there is a pre-trial detention problem.

4. # persons with AIDS, mental illness, TB/# prisoners

UNODC Q (18.2+18.3+18.4)/ UNODC 15.0

Health is a prime issue in prison and a prime determinant of prison quality. It would be expected that overcrowding and health would, in most circumstances, be closely correlated.

The problem with the indicator is that that data is unavailable on other illnesses, or on how much is pre-existing conditions, and therefore not the result of prison mismanagement.

48

F.II. Policing

Capacity

Indicator Source Use Cautions

1. # police per 100,000 UNODC Q 1.1 (Rate)

Behavior & Effectiveness

Indicator Source Use Cautions

2. # of intentional homicides completed per 100,000 people

UNODC 2.3 (rate) This is the standard measure of homicide.

There is no consistent or workable definition of “intentional,” and none provided by UNODC.

3. In the last 12 months, were you assaulted, mugged, property or money stolen?

Gallup World Poll This is the best victimization proxy available.

Victimization surveys are crucial to balance crime rate data and give a depth of understanding to perception surveys that measure fear and insecurity. Victimization surveys are also important means by which to evaluate questionnaires that look at public satisfaction in police performance.

4. How much trust do you have in… the Police

Arab (201.4), Asian (13) and Afro-barometers (49G)

Trust in the police is a crucial variable and one that ought to be measured, if at all possible.

If (5)-(8) could be broken down by group and income, it would give good indication of the feelings of security among conflict parties. It should be noted that in the first few years after the cessation of a conflict and with the rebuilding of the national police, public trust and confidence in the police often rises. This increase, however, may be more aspirational than real and, often, trust in the police declines sharply thereafter.

49

Indicator Source Use Cautions

5. Get help from police when needed? 20

Afro Barometer 71 (2005)

If police are not forthcoming with help or assistance, then individuals might resort to the non-state system.

In many fragile countries, individuals and groups do not appeal, first and foremost, to the police for help. Instead, they opt to consult their local leaders. In some countries, the police are not accessible, so that it would be inconceivable to request help from the police.

6. Difficulty in getting help from the police?21

Arab Barometer 213

Same as above.

7. Extrajudicial killings CIRI Index Important in itself. Killings may be by police or other government bodies. When the death is caused by other actors, the overall issue remains one that still concerns policing.

A proxy by which to evaluate the effectiveness of policing. There may be a correlation between extrajudicial killings and the entire criminal justice system, but the relationship is not contained within the CIRI Index’s raw score. It will have to be ferreted out.

8. Criminalization/ delegitimization of the state

Failed States Index, Fund for Peace (I.7)

Broader than policing, but does raise policing issues. Looks at acts of resistance to government (boycott elections, resist taxes, protests, etc.)

The extent to which a state is criminalized and/or has privatized the provision of its services is a proxy for the effectiveness of the criminal justice system.

20 This question was asked in the 2005 survey: “Based on your experience, how difficult is it to obtain the following services? Or do you never try to get these services from government? C: help from the police when you need it. (4=very easy, 1=very difficult). 21 The question asked: “Based on your experience, how difficult is it to obtain the following administrative or social services from the government?” One option was 4.” Get help from the police when you need it.”

50

F.III. Oversight of justice and security sector

Indicator Source Use Cautions

1. Security sector operating as a state within a state

Failed States Index, Fund for Peace (I.10)

Looks at emergence of elite guards that operate with impunity, state-sponsored private militias, army within the army.

2. Corruption mechanisms institutionalized

Bertelsmann 15.3 This is a proxy for oversight, given the assumption that institutionalization of corruption implies the absence of effective oversight.

Should be used in conjunction with E.VII.1. This is presumed that they will be correlated. If there is a discrepancy, that would be grounds for further investigation.

G. Personal Safety and Security Perception of personal safety and security In conjunction with indicators from A (conflict intensity), intentional homicide and ease of access to small arms (D. 4, 5; F.II.3), recorded crimes (E.I.2), and policing, these can give an idea of safety and security. Without disaggregation by demographic group, the indicators cannot provide information about conflict-related vs. non-conflict-related security concerns.

Indicator Source Use Cautions

1. State behavior violating physical integrity

Political Terror Scale This is a broader measure than extrajudicial killings.

If these indicators could be disaggregated by group, they would be more helpful as conflict indicators.

2. Feel safe walking alone at night in your community?

Gallup World Poll A perception question measuring insecurity.

Gender and age disaggregation available. Caution must be used, however, in that the term ‘community’ is undefined. One could, for example, feel safe walking within one’s own community, but not in the neighboring one.

3. Experienced fear, stress, worry a lot of the day yesterday?

Gallup World Poll This indicator measures security writ large. It is not specifically measuring physical security.

This indicator borders on human security and should be treated with care.

4. Over the past year, how often, if ever, have you or anyone in your family… B. had something stolen from your house, C. been physically attacked?

Afrobarometer 9B, C This indicator is a victimization question.

51

Indicator Source Use Cautions

5. In this country, how often do people have to be careful of what they say about politics?

Afrobarometer 47 A proxy for psychological security and for conflict-related security fears.

6. In your opinion, what are the most important problems facing this country that the government should address? (one choice is political violence)

Afrobarometer 56 Provides a sense of how important issues of political violence are to people.

7. # reported rapes per 100,000 women

UNODC Violence against women is an underreported crime and rape is a proxy question.

There is a lack of data on violence against women.

52

VII. Conclusion and Recommendations The existing datasets provide a number of indicators that can be useful in monitoring and evaluating the impacts of conflict, justice and security activities at the country level. While not perfect, they provide a means of monitoring and assessing the general direction of a country, and thus can provide a supplemental perspective on assessing the impact of DFID/HMG strategies and activities. They can help answer a number of questions relevant to assessment of impact, including: How has the situation changed over time? Which changes in structures, attitudes, behaviours, relationships or practices (of how many people/classified according to horizontal divisions) can be ascertained? 22

These indicators are not a comprehensive treatment of indicators. Nor do they provide a complete guide for assessment of impact. For more complete guidance on the selection and interpretation of indicators, it is advisable to refer to good practices in selecting indicators, the UNDP User’s Guide for Governance Indicators, and the forthcoming Users’ Guide on Sources Measuring Fragility and Conflict. In addition, the OECD/DAC Guidance on Evaluation of Conflict Prevention and Peacebuilding Activities lays out a useful framework within which these indicators may be used to measure the impact of DFID/HMG activities and strategies. The OECD/DAC guidance recommends several preconditions that help staff and policy makers prepare for, support and learn from evaluations.

Summary: Key steps for policy makers, programme planners and managers 23

Establish relevant, clear and measureable objectives State an explicit theory of change and programme logic Complete and monitor a conflict analysis Develop and monitor relevant indicators Focus on strategy and policy coherence Conduct systematic, rigorous evaluation

Within this framework, there are a number of things DFID/HMG might do to improve the availability of useful and robust conflict, justice and security indicators.

1. Invest in local capacity to manage performance of the “system,” in particular peacebuilding, justice and security. Donor effectiveness may be best measured by assessing the performance of the institutions, agencies, and organizations programmes are designed to support. To the extent that the sustainability of any intervention for justice and security development or conflict prevention/peacebuilding depends on local ownership, it is important that justice and security sector institutions, as well as institutions related to peacebuilding, improve their own systems for assessing performance. Moreover, development and monitoring of indicators that are relevant

22 OECD DAC Guidance on Evaluating Conflict Prevention and Peacebuilding Activities (Working Draft), p. . 23 OECD DAC Guidance on Evaluating Conflict Prevention and Peacebuilding Activities (Working Draft), p. 20.

53

and appropriate to the specific context could supplement and correct for the gaps and deficiencies in global-level indicators covered in existing datasets and would allow for more accurate assessment of the impact of interventions, policies and actions. It is, therefore, recommended that a percentage of assistance to a country be earmarked for the development performance management capacity, including appropriate indicators and data collection methods and systems.

2. Development of indicators and datasets for non-state mechanisms and processes for justice,

provision of security and peacebuilding. This should be an area for significant investment—either at country level or globally. Similarly, there is need to design and support methods of measuring the performance of private security companies, whose business is booming around the world, but particularly in fragile countries.

3. Strengthening of indicators and data on perceptions. The Afro- and other barometers can provide valuable data on public perceptions of conflict, justice and security. However, the questions are not entirely consistent, the frequency of collection of data not sufficient, and the coverage also relatively limited. Gallup World Poll has more extensive coverage, but few of the questions are relevant to the elements of conflict, justice and security that are useful to monitor in order to assess impact. We would recommend that DFID/HMG invest in Afrobarometer (and other barometers) to further systematize and expand the collection of data. In addition, at relatively low cost for DFID/HMG, questions might be proposed or added to the barometers or to the Gallup polls to gather missing information about perceptions.

4. Disaggregation by group (ethnic/religious/social/ideological). In order for sectoral indicators to become more conflict-sensitive and provide data on conflict impact, disaggregation by relevant conflict group would be useful. Some perceptions surveys – such as the Afrobarometer – as about respondents’ identity, and thus might easily be able to provide disaggregated data. In some post-conflict situations, such as in Kosovo, information on crime, justice and security has been disaggregated by ethnicity, and has provided valuable information on distributional impacts of policies and actions. In other situations, this may be sensitive, but would be useful for DFID/HMG to take up with host governments.

Appendix A

Example: Application of Indicators to Burundi and Nepal, 2006-2008

A. Conflict Intensity

Indicator Source/Scale Nepal Burundi Comments

1. # deaths from armed conflict

Uppsala Conflict Data Project Absolute number

2006 2007 2008 2006 2007 2008

Burundi: There is a postive reduction in most indicators, reflecting the momentum in the peace process and the negotiations with the last remaining rebel group. The conflict here would no longer be defined as an “armed conflict” by UCDP, and one sees also a significant reduction in “total population of concern” (includes IDPs and refugees), especially in the IDP population. Tensions have decreased significantly, but still are not low.

Nepal: Nepal has also seen a significant reduction in conflict intensity – particularly in deaths (it too would no longer be classified as “armed conflict”). The stable level of displaced includes returns of about 50,000 IDPs who are counted in the “total population of concern” because UNHCR is following them. Nonetheless, the level of tension, although decreasing, remains extremely high. The World Governance Indicators also suggest high levels of political instability and violence (a rating of -2.13 in 2007, from -2.09 in 2006 – a trend within the margin of error).

457 0 0-24 137 0-24 missing

2. # and increase in IDPs/refugees

UNHCR (in 000) http://www.unhcr.org/ statistics/45c063a82.html

105 105.5 464 423

3. Territory affected by conflict

Political Instability Task Force 0 = best, 4 = worst

4 Ended ended ended ended ended

4. Seriousness of ethnic, religious, social conflicts

Bertelsmann Transformation Index (BTI) Question 13.3 1 = best, 10= worst http://www.bertelsmann-transformation-index.de/fileadmin/pdf/Anlagen_BTI_2008/ Detailed_Results_BTI2008.xls

Alternate: World Governance Indicators, “Political Stability and Absence of Violence.” http://info.worldbank.org/governance/wgi/index.asp

9 Unpub-lished

8 10 Unpub-lished

5

5. Neighborhood War - # neighboring states experiencing armed conflict

Peace and Conflict Instability Ledger, University of Maryland red = high risk (2 or more) yellow = medium (1) green = low risk (0)

medium risk

Unpub-lished

n/a Unpub-lished

B. Institutions/Mechanisms/Processes that Handle Grievances B.I. Capacity of Government to Deal with Grievances The capacity includes capacity to develop and implement policies that address grievances Effectiveness

Indicator Source/Scale Nepal Burundi Comments

1. Governance Effectiveness

World Governance Indicators 2.5= best -2.5= worst

2006 2007 2008 2006 2007 2008

In both countries, government effectiveness is rated as low, and has not changed in the 2006-2008 time period. One would not, however, be able to detect small changes in an aggregate indicator such as WGI’s governance effectiveness, and, unfortunately, data is unavailable on access to improved water sources. We turned to HDI and primary school enrollment to gather some information about provision of services. The only indicator with up-to-date data is primary school enrollment, which increased in Burundi and remained the same in Nepal. It might be noted that infant mortality (also available through the World Development Indicators) decreased slightly in both.

The general picture thus appears to be one of slight improvement, but with significant challenges, particularly in light of the slightly deteriorating score on B.I.3 (government coordinating conflicting objectives) in Nepal, and the low, even if improving, score for Burundi.

-0.82 -0.81 Unpub-lished

-1.26 -1.34 Unpub-lished

2. Efficiency of revenue mobilisation

World Bank Country Policy and Institutional Assessment (CPIA) Criterion 14 1=low, 6=best

3.5 3.5 Unpub-lished

3 3 Unpub-lished

3. Extent to which government can coordinate conflicting objectives into coherent policy

Bertelsmann 15.2 1 = worst 10= best

5 Unpub-lished

4 2 Unpub-lished

3

4. Access to improved water sources

Human Development Index

Primary school enrollment

World Development Indicators

89

.530

126

Unpub-lished

126

Unpub-lished

124

71

.382

103

Unpub-lished

114

Unpub-lished

Unpub-lished

Leadership culture/ process

Indicator Source/Scale Nepal Burundi Comments

5. To what extent can political leadership manage political cleavages?

Bertelsmann Question 16.3 10 = best 1= worst

2006 2007 2008 2006 2007 2008

In both countries, the ability of the leadership to manage the political cleavages has improved, which is to be expected given the progress in the peace processes in both Nepal and Burundi during this period. The increase in both countries in the acceptance of democratic institutions is also a positive sign of commitment to the peace process.

However, the high scores on “factionalized elites” – which is a much more short-term-sensitive indicator of political polarization, would suggest that the upward trend is very vulnerable. There have been improvements in Nepal, but Burundian politics remains quite factionalized despite progress in the peace process.

3 Unpub-lished

6 4 Unpub-lished

7

6. Factionalized Elites Failed States Index/Fund for Peace (1.11) 0=best, 10=worst

9 8.5 8.3 7.8 7.5 7.8

7. Extent to which democratic institutions are accepted or supported by relevant actors

Bertelsmann Question 4.2 10 = best 1= worst

2 Unpub-lished

4 2 Unpub-lished

5

8. Regime/governance inclusion (Political legitimacy score)

State Failure Index, Center for Systemic Peace, George Mason University based on Polity IV 0 = best (stable) 3 = worse (fragile)

2 0

Grievances

Indicator Source/Scale Nepal Burundi Comments

9. Group grievances

Failed States Index, Fund for Peace (I.3) 0=best, 10=worst

2006 2007 2008 2006 2007 2008

Several of the indicators (B.I.11-12) are not relevant in these situations, and consequently can be ignored.

Burundi: There appears to be improvement in behavior that generates group grievances (B.I.9), but little movement on the socio-economic factors that may underlie group grievances. Viewed in conjunction with the indicators on leadership process and relationships, there appears to be less progress on dealing with underlying drivers of conflict. Regrettably, the Afrobarometer does not include Burundi in its latest polls; data on popular perceptions of the government and inter-group relations would be valuable here.

Nepal: Uneven development and group grievances are very high, and seem to have remained stable or become slightly worse, despite the peace agreement and a reduction in conflict-related violence. This may merely reflect the challenges of implementing the peace accord – both the difficulties encountered in implementation, and the time it will take to address some of the underlying issues.

9.2 8.9 9 7 6.7 6.7

10. Minority access to education, commerce, professions, etc.

Minorities at Risk, University of Maryland, Column AF 0=no restrictions 4=worst restrictions

n/a Unpub-lished

Unpub-lished

1 Unpub-lished

Unpub-lished

11. Restrictions on religion

Minorities at Risk, University of Maryland 0=no restrictions 3=sharply restricted

na Unpub-lished

Unpub-lished

0 Unpub-lished

Unpub-lished

12. Restrictions on use of language or language instruction

Minorities at Risk, University of Maryland 0=no restrictions 3=sharply restricted

na Unpub-lished

Unpub-lished

0 Unpub-lished

Unpub-lished

13. GINI Coefficient World Development Indicators

Unpub-lished

Unpub-lished

Unpub-lished

33 Unpub-lished

Unpub-lished

14. Uneven Development Failed States Index/Fund for Peace (1.5) 0=best, 10=worst

8.5 9.2 9.2 8.8 8.8 8.8

15. How often has your group been treated unfairly by the government?

Afro Barometer Ghana 2008 survey, Question 82

Inap-plicable

Inap-plicable

Inap-plicable

na na na

16. Your group has same, worse, or better economic conditions than others

Afrobarometer

Ghana 2008 survey, Question 11

Inap-plicable

Inap-plicable

Inap-plicable

na na na

B.II. Governance Legitimacy and Accountability

Elections free and fair

Indicator Source/Scale Nepal Burundi Comments

17. Electoral Process and Pluralism

Economist Intelligence Unit Democracy Index 0=worst 10=best Freedom House (FH) 0 = smallest degree of rights

2006 2007 2008 2006 2007 2008 Nepal has experienced significant improvement in this area; this is consistent with the history of the peace process. This is also reflected in popular perceptions of the fairness of elections. Burundi, having experienced a successful 2005 election that was a watershed, has remained stable. While this is a positive sign of the development of democratic processes, the high degree of factionalization (leadership culture and process) suggest that the electoral process could as easily fuel as mitigate conflict.

EIU: .08

FH: A – 1 (of 12) B – 6 (of 16)

EIU: 1.33 FH: A – 4 B – 9

EIU: 4.42

FH: A – 9 B – 11

EIU: 4.42 FH: A – 9 B – 9

18. Perceptions of honesty of elections

Gallup, alternatives are Afrobarometer #71 or Arab Barometer 211

25% 60% (Oct 2008)

56% (July 2008)

Legitimacy of State

Indicator Source/Scale Nepal Burundi Comments

19. Delegitimization of state Failed States Index/Fund for Peace (I.7) 0=best, 10=worst

2006 2007 2008 2006 2007 2008 While the state appears to be gaining legitimacy in Nepal, there is an interesting discrepancy between this indicator and B.I.6 and B.I.8. The mediocre score, which has remained unchanged since 2006, on B.I.20 suggests that the danger of group-based challenges to the state, and conflict, remain present. Requires further analysis.

9.2 8.5 8.3 7.2 7.1 7.1

20. Relevant groups agree about citizenship and accept the nation state as legitimate?

Bertelsmann 1.2 1 = worst 10= best

6 Unpub- lished

6 6 Unpub- lished

8

21. Public support for government restrictions on civil liberties

Afro Barometer Ghana 2008 survey, Question 19

Inap-plicable

Inap-plicable

Inap-plicable

na na na

22. Control of Corruption World Governance Indicators best=2.5 worst= -2.5

-0.67 -0.66 Unpub-lished

-1.12 -1.06 Unpub-lished

B.III. Human rights & Civil liberties These are indicators of rule of law, as well as a proxy for the potential for the extent to which peaceful mechanisms for handling grievances exist. We have specifically excluded physical issues here; these have been placed under perception of safety and security.

Indicator Source/Scale Nepal Burundi Comments

23. Civil liberties EIU Democracy Index After 2006 0=worst, 10=best

Freedom House (FH) Freedom in the World D, E, G 0 = worst/smallest

2006 2007 2008 2006 2007 2008

There has been an improvement Nepal, given the results of 2008. Based on the Freedom House scores, the biggest improvement has been in freedom of expression and belief. Human rights and civil liberties remain at concerning level in both Nepal and Burundi, and are not improving in Burundi.

EIU: 5.59 FH: D – 6 (of 16) E – 3 (of 12) G – 6 (of 16)

EIU: 5.59 FH: D – 7 E – 6 G – 10

EIU: 4.71 FH: D – 6 (of 16) E – 5 (of 12) G – 9 (of 16)

EIU: 4.71 FH: D – 6 E – 5 G – 8

24. Suspension of Rule of Law and Human Rights

Failed States Index, Fund for Peace 0=best, 10=worst

9.1 8.8 8.8 7.5 7.5 7.5

25. Allowable advocacy activities

CIVICUS Civil Society Index 2.5.2

2 Unpub-lished

Unpub-lished

na Unpub-lished

Unpub-lished

C. Community Initiatives For Peace Community/Civil society initiatives for peace This looks at civil society initiatives for peace: to what extent are there local capacities for peace?

Indicator Source/Scale Nepal Burundi Comments

1. Civil society actions to promote non-violence and peace

CIVICUS 3.4.2 2006 2007 2008 2006 2007 2008 This is an important category of indicators for which there is nearly no data. CIVICUS does not collect data regularly, and thus comparisons over time, even for those countries that are covered, is difficult.

3 Unpub-lished

Unpub-lished

na Unpub-lished

Unpub-lished

2. Activity and success of civil society in influencing public policy in human rights

CIVICUS 4.1.1 1 Unpub-lished

na Unpub-lished

Unpub-lished

D. Capacity/Willingness to Resist or to Escalate Violence Military and societal militarization The indicators in this category reflect opportunity to fight/military feasibility of war and justice and security development

Indicator Source/Scale Nepal Burundi Comments

1. Military expenses as % of GDP

The Military Balance, Stockholm International Policy Research Institute www.milexdata.sipri.org

2006 2007 2008 2006 2007 2008

Data for UNODC is available only to 2006. We were unable to obtain the Global Peace Index data for individual factors, but it should be available through the Economist Intelligence Unit or through Vision of Humanity, which sponsors the GPI. Based on the current data, militarization trends cannot be assessed; it may be useful to go back further in time.

2% 1% 5% 5%

2. # armed forces per 100,000 people

The Military Balance, SIPRI

2.298850575

3. # security officers and police per 100,000

10th Crime Trends Survey, UN Office on Drug and Crime

202.83 Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

4. Ease of access to small arms

Global Peace Index (collected by EIU)

1=very low, 5=very high

Unpub-lished

Unpub-lished

na Unpub-lished

na Unpub-lished

5. Intentional homicide committed by firearm, per 100,000

UNODC 2.6 0.3 Unpub-lished

Unpub-lished

na Unpubl-ished

Unpub-lished

6. Non-violence within the civil society arena

CIVICUS 3.4.1 2 Unpub-lished

Unpub-lished

na Unpub-lished

Unpub-lished

E. Criminal Justice E.I. Criminal justice capacity and effectiveness

Activities and results

Indicator Source/Scale Nepal Burundi Comments

1. Conviction rate UNODC 9.1 2006 2007 2008 2006 2007 2008

14.29

15.93 (2005)

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

2. # of recorded crimes per 100,000 people

UNODC 2.0 14.96 15.07 (2005)

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

3. # convictions/# police UNODC Q11 / UNODC 1.1

0.052

0.049 (2005)

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

4. # convictions/# prosecutors

UNODC Q11/UNODC 5.0

13.5

12.6 (2005)

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

5. # prosecutions/# prosecutors

UNODC Q7/UNODC 5 6.02

6.45 (2005)

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

6. # convictions/# prosecutions

UNODC Q11 / UNODC Q 7

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

7. # prosecutions/# judges UNODCQ7/UNODCQ8

1307/232

1400/195 (2005)

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

8. Average # months of criminal trials

UNODC Q 9.2 n/a Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Perceptions

Indicator Source/Scale Nepal Burundi Comments

9. Government reducing crime?

Afrobarometer 57F

2006 2007 2008 2006 2007 2008

na Inap-plicable

Inap-plicable

Inap-plicable

na na

10. Who do you think has primary responsibility for managing each of the following tasks - national gov? local gov? traditional leaders? Members of community? None of them? Don't know.

Afrobarometer 58 E and H

na Inap-plicable

Inap-plicable

Inap-plicable

na na

11. In your opinion, how often… do officials who commit crimes go unpunished?

Afrobarometer 45 E and D

12. In your opinion, how often… do ordinary people who break the law go unpunished?

Afrobarometer 45.E

na Inap-plicable

Inap-plicable

Inap-plicable

na na

E.II. Rule of Law: civil and administrative justice

Indicator Source/Scale Nepal Burundi Comments

1. # days to resolve dispute World Bank Doing Business

2006 2007 2008 2006 2007 2008

2. # steps to claim, obtain, enforce contracts

World Bank Doing Business

39 39 39 44 44 44

3. Independence of Judiciary

Bertelsmann 3.2 1 = best, 10= worst

3 Unpub-lished

3 4 Unpub-lished

5

4. Court system fair/impartial? Law and regulations predictable and consistent

World Bank Enterprises Survey

Unpub-lished

Unpub-lished

Unpub-lished

n/a Unpub-lished

Unpub-lished

5. Should the President always obey the law?

Afrobarometer 37 Inap-plicable

Inap-plicable

Inap-plicable

na na na

6. How much do you trust… Courts of law?

Afrobarometer (49H), alternatively, see Gallup World Poll for Asia

47% na na na

7. Rule of law/police under civilian control?

Freedom House, Freedom in the World F2

n/a n/a n/a n/a n/a n/a

8. Equal treatment of demographic groups?

Freedom House. Freedom in the World F4

n/a n/a n/a n/a n/a n/a

E.III. Access to Justice: state and non-state systems

Capacity

Indicator Source/Scale Nepal Burundi Comments

1. # police per 100,000 UNODC 1.1 (Rate)

2006 2007 2008 2006 2007 2008

202.83

206.96 (2005)

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

2. # prosecutors per 100,000

UNODC 5.0 (Rate)

0.79

0.80 (2005)

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

3. # lawyers per 100,000 Law Schools / Bar Associations

4. # hours to pay taxes, # payments per year

World Bank Doing Business

408 hrs 34 payments

408 hrs 34

payments

408 hrs 34

payments

140 hours 32 payments

140 hours 32 payments

140 hours 32 payments

Access

Indicator Source/Scale Nepal Burundi Comments

5. Attorney, court, enforcement costs as % of claim value

World Bank Doing Business

2006 2007 2008 2006 2007 2008 There appears to be a deterioration of public services in both countries. This can be viewed in conjunction with B.I.6 and B.I.8. The question arises is whether higher tensions contributed to the deterioration, or whether the deterioration has exacerbated tensions. Has the inability of the governments to respond triggered unhappiness with the leadership and government agencies?

26.8 26.8 26.8 38.6 38.6 38.6

6. Government repression Minorities at Risk 0=no repression; 5=violent repression and killing

n/a Unpub-lished

Unpub-lished

0 Unpub-lished

Unpub-lished

7. Deterioration of public services

Failed States Index/Fund for Peace (1.8) 0=best, 10=worst

6.2 6.6 7 8.5 8.9 9.0

Women’s access & capacity

Indicator Source/Scale Nepal Burundi Comments

8. # women police/total police

UNODC 1.2/UNODC 1.1

2006 2007 2008 2006 2007 2008

1418/56064 same in 2005

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

9. # women prosecutors/total prosecutors

UNODC Q5.1/UNODC Q5.0

2 women/ 217 total

prosecution personnel

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

10. # women judges/total judges

UNODC Q8.1/UNODC Q8.0

3/232 Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-ished

11. # women prison staff/total prison staff

UNODC (14.1+14.4)/UNODC (14.3+14.0)

61/658

61/657 (2005)

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

12. Women as % of legislators, senior officials, managers

IMD World Competitive Yearbook 2.5.10

Perception

Indicator Source/Scale Nepal Burundi Comments

13. Trust in courts vs. trust in traditional leaders (how do you trust each?)

Afrobarometer (Compare 49H vs. 49I)

2006 2007 2008 2006 2007 2008

Inap-plicable

Inap-plicable

Inap-plicable

na na na

14. Use of wasta (clout) (During the past five years, have you ever used wasta to achieve something personal, family-related, or a neighborhood problem?)

Arab Barometer 226

Inap-plicable

Inap-plicable

Inap-plicable

Inap-plicable

Inap-plicable

Inap-plicable

15. Reference to traditional leaders (list the types of wasta – choice 4 is traditional leaders)

Arab Barometer 227

Inap-plicable

Inap-plicable

Inap-plicable

Inap-plicable

Inap-plicable

Inap-plicable

16. Go to court to resolve disputes? (If you were to have a dispute with another citizen, would you try to resolve it in a court or in another government institution?)

Arab Barometer 228

Inap-plicable

Inap-plicable

Inap-plicable

Inap-plicable

Inap-plicable

Inap-plicable

17. Who would you go to to resolve a dispute?

Arab Barometer 229

Inap-plicable

Inap-plicable

Inap-plicable

Inap-plicable

Inap-plicable

Inap-plicable

18. Women are/should be subject to traditional laws1

Afro Barometer 23.B

Inap-plicable

Inap-plicable

Inap-plicable

na na na

1 In the 2005 survey, respondents were asked: “In our country, women should have equal rights and receive the same treatment as men do.” B. “Women have always been subject to traditional laws and customs, and should remain so.” Respondents could agree with A, B, or neither.

E.IV. Corruption

Indicator Source/Scale Nepal Burundi Comments

1. Government doing enough to fight corruption?

Gallup World Poll 2006 2007 2008 2006 2007 2008

Corruption has significantly increased in Burundi. Once again, this could be seen in relation to B.I.6, B.I. 8, and E.III.9. Is this an emerging overall trend?

2. Corruption perceptions Transparency Int’l - Corruption Perceptions Index 1=most corrupt 10=least corrupt

2.5 2.5 2.7 2.4 2.5 1.9

3. Registering property - # of procedures, # of days

World Bank Doing Business

3 steps 5 days

3 steps 5 days

3 steps 5 days

5 steps 94 days

5 steps 94 days

5 steps 94 days

4. Gifts given to receive public services

World Bank Enterprise Survey

Unpub-lished

Unpub-lished

Unpub-lished

n/a Unpub-lished

Unpub-lished

5. How many of the following people do you think are involved in corruption?

Afro Barometer 50 (ALL)

Inap-plicable

Inap-plicable

Inap-plicable

n/a n/a n/a

6. Municipal/national officials involved in corruption?

Asian Barometer 114-115

Inap-plicable

Inap-plicable

Inap-plicable

F. Security Sector

F.I. Prisons

Indicator Source/Scale Nepal Burundi Comments

1. # prisoners/# beds King’s College 2006 2007 2008 2006 2007 2008

0.986450989 Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

2. # juveniles/# beds for juveniles

King’s College/ UNODC

50 Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

3. # pre-trial detention prisoners prior to disposition of case/# convicted prisoners

UNODC/King’s College

n/a Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

4. # persons with AIDS, mental illness, TB/# prisoners

UNODC Q (18.2+18.3+18.4)/ UNODC 15.0

% of prisoners

with HIV: 0.6 % with mental

illness: 0.6

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

F.II. Policing

Capacity

Indicator Source/Scale Nepal Burundi Comments

1. # police per 100,000 UNODC Q 1.1 (Rate)

2006 2007 2008 2006 2007 2008

202.83 Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

2. # cases discontinued/# cases referred to prosecutors

UNODC 9.0-7.0 n/a Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Behavior & Effectiveness

Indicator Source/Scale Nepal Burundi Comments

3. # of intentional homicides completed per 100,000 people

UNODC 2.3 (rate) 2006 2007 2008 2006 2007 2008

The Nepalese state has improved. How does this relate to the other indicators that have suggested deterioration of public services, ability of the leadership to address policies, etc. The surface discrepancy may have to do with the end of the insurgency vis-à-vis government services. Needs

1.84 Unpub-lished

Unpubl-ished

Unpub-lished

Unpub-lished

Unpub-lished

4. In the last 12 months, were you assaulted, mugged, property or money stolen?

Gallup World Poll 9% theft 4%

assaulted

5. How much trust do you have in… the Police

Arab (201.4), Asian (13) and Afro-barometers (49G)

n/a n/a n/a

6. Get help from police when needed? 2

Afro Barometer 71

Inap-plicable

Inap-plicable

Inap-plicable

n/a n/a n/a

2 This question was asked in the 2005 survey: “Based on your experience, how difficult is it to obtain the following services? Or do you never try to get these services from government? C: help from the police when you need it. (4=very easy, 1=very difficult).

7. Difficulty in getting help from the police?3

Arab Barometer 213

Inap-plicable

Inap-plicable

Inap-plicable

Inap-plicable

Inap-plicable

Inap-plicable

more analysis.

8. Extrajudicial killings CIRI Index 0=worst, practiced frequently; 1= practiced occasionally; 2=best, not practiced at all

0 0 Unpubl-ished

0 1 Unpub-lished

9. Criminalization/ delegitimization of the state

Failed States Index, Fund for Peace (I.7) 0=best, 10=worst

9.2 8.5 8.3 7.2 7.1 7.1

F.III. Oversight of justice and security sector

Indicator Source/Scale Nepal Burundi Comments

1. Security sector operating as a state within a state

Failed States Index, Fund for Peace (I.10) 0=best, 10=worst

2006 2007 2008 2006 2007 2008

In both countries, this indicator would suggest significant improvement, but the meaning of that improvement cannot be ascertained from this one indicator alone. In the case of Nepal, this movement correlates well with E.VI.9.

9 8.3 8.5 7.3 6.8 6.8

2. Corruption mechanisms institutionalized

Bertelsmann 15.3 1 = best, 10= worst

4 Unpub-ished

4 2 Unpub-lished

4

3. General government transparency

IMD World Competitive Yearbook 2.3.12

3 The question asked: “ Based on your experience, how difficult is it to obtain the following administrative or social services from the government?” One options was 4.” Get help from the police when you need it.”

G. Personal Safety and Security

Perception of personal safety and security

Indicator Source/Scale Nepal Burundi Comments

1. State behavior violating physical integrity

Political Terror Scale 1=lowest level of terror; 5=highest level of political terror

2006 2007 2008 2006 2007 2008

It would appear that Nepal has improved on the political terror scale, which the next data point in the series (2008?) would be able to verify. However, the lack of data about perceptions of safety, and the time lag in obtaining data on homicides, small arms and other violence makes it difficult to capture any deterioration of public security such as that which has occurred in Nepal over the last two years. In light of the absence of data, these figures could be vied in connection with B.II.19 (delegitimization of the state), and A. 4 (seriousness of social conflicts) as a proxy, although that too is not very satisfactory.l

There appears to be additional correlation with E.VI.9 and E.VII.1.

4.5 3.5 Unpubl-ished

4 4 Unpub-lished

2. Feel safe walking alone at night in your community?

Gallup World Poll 52%

3. Experienced fear, stress, worry a lot of the day yesterday?

Gallup World Poll

4. Over the past year, how often, if ever, have you or anyone in your family… B. had something stolen from your house, C. been physically attacked?

Afrobarometer 9B, C

Inap-plicable

Inap-plicable

Inap-plicable

n/a n/a n/a

5. In this country, how often do people have to be careful of what they say about politics?

Afrobarometer 47 Inap-plicable

Inap-plicable

Inap-plicable

n/a n/a n/a

6. In your opinion, what are the most important problems facing this country that the government should address? (one choice is political violence)

Afrobarometer 56 Inap-plicable

Inap-plicable

Inap-plicable

n/a n/a n/a

7. # reported rapes per 100,000 women

UNODC 0.76 Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Unpub-lished

Appendix B: List of Data Sources

Producer Index Geographic

CoverageTime Coverage Methodology Data Source Comments

Datasets Used for Indicators

1 Afrobarometer Afrobarometer

20 sub-Saharan African countries(No survey in Ethiopia, Sudan, DRC, Sierra Leone)

Every 2-3 years; earliest poll was 1993, but earlier surveys had very few countries

[Survey] Survey

Coverage is an issue. And only a few countries are covered in each round.Questions are good, but not all countries have exactly the same Questions, so comparability is questionable.

2 Arab Barometer Arab Barometer

10 Arab countries (five completed), data not yet available for UAE, Yemen, Oman, Iraq, Lebanon, Bahrain. None on Egypt and Saudi Arabia

First survey in 2006, in five countries; next wave will add six countries

[Survey] SurveyOnly five countries so farAlso, questions may not be the same in every countries.

3 Asianbarometer Asianbarometer13 Asian countries and 5 South Asian countries

First wave (2001-2003) for 8 countries; second wave (2005-2007) for 18 countries; third wave being planned

[Survey] Survey

All major countries are coveredBut like the others, different countries may not have the same questions

4Bertelsmann Transformation Index

Bertelsmann Transformation Index (BTI)

125 transformation countries. Does not include US, Canada, Western Europe, Australia, Japan.

Released every two years; goes back to 2003

[Survey] + [Expert]Survey w/Bertelsmann questionnaire of country experts

Survey w/Bertelsmann questionnaire of country experts

5CIDCM, University of Maryland

Peace and Conflict Instability Ledger (PCIL)

160+ countries 2001, 2003, 2005, 2008

[Statistical Model]: Based results of studies that identified factors strongly correlated with major incidents of instability in the past six decades, including: 1) Recent instability [PITF], 2) country size dummy var., 3) regime consistency [Polity IV], 4) infant mortality, 5) economic openness [(X+M)/GDP], 6) Militarization, 7) Neighborhood War (Upssala, IPRI)

Correlates of War Project, Gleditsch 2002, PITF data, Polity IV data, Penn World Table, Uppsala Conflict Data, International Peace Research Institute, World Bank World Development Indicators

Appendix C: List of Data Sources 2

Producer Index Geographic

CoverageTime Coverage Methodology Data Source Comments

6CIRI Human Rights Data Project

CIRI Human Rights Index

195 countriesannually from 1981 - 2007

[Expert]: At least two expert coders for each country meet with the CIRI staff, then score the indicators based on information from the State Dept annual reports on human rights practices, and country reports from Amnesty International

State Department annual reports on human rights practices, and Amnesty International's country reports, with the latter treated as authoritative when there are conflicts between the two

Relies on subjective coding, and coders only read two sources, so subjectivity and bias are big issues.

7 CIVICUS Civil Society Index

56 countries in the latest version. Asia and Eastern Europe are best covered. Very little coverage in Africa, Middle East and Latin America; North America and Central Asia are excluded

Every 3-4 years. Pilot phase in 2001-2002; Phase 2 in process 2008-2010

[Survey]: mainly surveys of experts, organizations, population - but combined with media monitoring and existing quantitative data

Mostly surveys; but also media reports

8Econmist Intelligence Unit (EIU)

Democracy Index

167 countries (missing a few like Somalia and Western Sahara)

2006, 2008[Expert]: Survey of experts on 60 multiple choice questions

Expert surveys

10 Freedom HouseFreedom in the World

193 countries Annually, since 1972

[Expert]: Research analysts and academic advisers produce country reports and propose ratings, which are then reviewed in a series of regional and then global meetings

foreign and domestic news reports, academic analyses, nongovernmental organizations, think tanks, individual professional contacts, and visits to the region

Scoring by experts are subjective, and it's unclear if different sets of experts would score the same way; the system enables cross-country comparisons

11 Fund for PeaceFailed States Index (FSI)

177 countries Annually since 2005

[Expert/Software]12 Indicators, each assessed by feeding English, web-accessible information (news, reports) through a software that looks for "key phrases" associated with each indicator. The more coverage, the higher the score. Each indicator is scored from 1 to 10.

Meltwater News Service

Assumption is that if key phrases appear more times in the English, electronic media, then a given issue is more severe or important.

Appendix C: List of Data Sources 3

Producer Index Geographic

CoverageTime Coverage Methodology Data Source Comments

12 Gallup Gallup World Poll

150 countries

Does not yet include Bahrain, Oman, Cote d'Ivoire, Libya, and Turkmenistan

Since 2005, continuously updated

[Survey]- random telephone interviews in countries where phones are common, and face to face interviews where they are not

SurveyThe most comprehensive global survey available. Full results are only accessible by subscription

13GMU / Center for Systemic Peace

State Fragility Index & Related Datasets

162 countries

Appears to be updated annually, data goes back to 1946 where available

[Expert] Assessments by academics based on available literature

Bienen/van de Walle, Center for Systemic Peace, Gurr/Harff, Minority aft Risk, Political Terror Scale, UNDP Human Development Report, UNDP Structure of Trade, US Census Bureau Intl Database, Polity IV, World Bank WDI

Good time and country coverage Does not report margin of errorWeighting can be suspect

14GMU / Center for Systemic Peace

Polity IV dataset

All major independent states from 1800 to 2007 - 162 countries at this moment

1800-2007; appears to be updated every few years

[Expert] Assessments by academics based on available literature

Published materials, unspecified

15 IMDWorld Competitiveness Yearbook

55 countries

Missing: all of Africa outside S. Africa; most of Middle East, East Europe, Balkans, all of Central Asia, much of SE Asia.

Annual, since 1989[Survey] of business executives + looking at available public data

Surveys and other publicly available hard data

16 Intl Institute of Strategic Studies

Military Balance 170 countries Annually, since 2004 [Primary data] National governments; NATO; UN; OSCE; IMF

17 SIPRIMilitary Expenditure Database

171 countries Annually, since 1988 [Primary data]National governments; NATO; UN; OSCE; IMF, secondary sources that cite primary data

Appendix C: List of Data Sources 4

Producer Index Geographic

CoverageTime Coverage Methodology Data Source Comments

18Transparency International

Corruption Perception Index

180 countries Annually since 1995[Composite] Combination of 13 sources that measure corruption

Asian Development Bank, African Development Bank, Bertelsmann Transformation Index, Country Policy and Institutional Assessment, Economist Intelligence Unit, Freedom House, Global Insight and Merchant International Group; IMD, Political and Economic Risk Consultancy, and the World Economic Forum

19UN High Commissioner on Refugees

Refugee statistics 150+ countries Annually since 1994 [Primary data]

20UN Office on Drug and Crime

Survey of Crime Trends and Operations of Criminal Justice Systems

86 countries

Missing: almost all of Africa; Brazil, Russia, China, Indonesia, Pakistan, Iran, Saudi, UK, etc.

Now every two years, used to be every five years, since 1970

[Survey]Data provided by national governments to a survey, which can be viewed online

Relies on countries to respond, so coverage is spotty. Even when countries do respond, they may leave out important stats; and there may be definitional issues that the country office may need to h k

23University of Maryland

Minority at Risk Project

284 groups and 118 ethnopolitical organizations

1945 - present; updated every 3 years or so

[Expert/Student]: Variables are coded by trained grad and undergrad students using open source info, then reviewed by senior staff. Inter-coder reliability has not been tested. Coders are "encouraged" to carry out their own reliability and validity tests

Unspecified open source information

Subjective coding, and system for reviewing coding decisions is weak. Also, updated only once every 3 years.

24University of North Carolina

Political Terror Scale

183 countries 1976-2007

[Expert/Student]: Staff and students assign scores to countries by reading State Department annual reports on human rights practices, and Amnesty International country reports

State Department annual reports on human rights practices, and Amnesty International's country reports

Relies on subjective coding, and coders only read two sources, so subjectivity and bias are big issues.

25 Uppsala UniversityUppsala Conflict Data Program

Global - all conflicts that match their definition

Annually updated, data goes back to 1946

[Primary data] Uppsala/PRIO researchers read primary and secondary sources and count or estimate the numbers

Unspecified primary and secondary sources

Appendix C: List of Data Sources 5

Producer Index Geographic

CoverageTime Coverage Methodology Data Source Comments

26 Vision of HumanityGlobal Peace Index (GPI)

140 countries

Missing: Pacific Islands, Sierra Leone, Liberia, Eritrea, Pacific Islands, but most major countries covered.

Annually since 2007

[Expert] A combination of existing quantitative data and qualitative scores done by Economist Intelligence Unit analysts

Economist Intelligence Unit, Expert Estimates, Uppsala Conflict Data Program, International Peace Research Institute in Oslo, UN Office of Drugs and Crime (UNODC), International Institute of Strategic Studies (IISS)

27 World BankWorldwide Governance Indicators

212 countries and territories

Annually since 1996[Composite] Composite indicators taken from a huge number of sources

See website, too many to list, including many sources listed here

28 World BankWorld Development Indicators

153 economies and 56 other smaller economies

Annually, since 1960

29 World BankCountry Policy and Institutional Assessment

76 countries Annually[Expert]: World Bank country staff assign the scores, which then get reviewed by higher levels

Lacks transparency

30 World Bank Doing Business 181 economies Annually, since 2003

[Survey]: Surveys of local experts including lawyers, business consultants, accountants, government officials etc

Surveys

31 World Bank Enterprise Survey 110 countries Every three years, since 2002

[Survey] Surveys

32World Economic Forum

Global Competitiveness Report

134 countries Annual, since 1979[Survey] of business executives + looking at available public data

Survey, IMF, World Bank, UN Population Fund, Economist Intelligence Unit, World Health Organization, UNESCO, etc. - see report for complete sources

Source Source Link Latest data

Jam

aica

Guy

ana

Bur

undi

CA

R

Cd'

l

DR

C

Eth

iopi

a

Cha

na

Gui

nea

Gui

nea

Bis

sau

Ken

ya

Libe

ria

Mal

awi

Moz

ambi

que

Nig

eria

Rw

anda

Sen

egal

Sie

rra

leon

e

Som

alia

Sou

th A

fric

a

Sud

an

Tanz

ania

Uga

nda

Zam

bia

Zim

babw

e

Kos

ovo

Ser

bia

BiH

Geo

rgia

Arm

enia

Aze

rbai

jan

Kyr

gyzs

tan

Tajik

ista

n

Leba

non

OP

Ts

Yem

en

Afg

hani

stan

Pak

ista

n

Ban

glad

esh

Indi

a

Nep

al

Sri

Lan

ka

Bur

ma

Uppsala/UCDPhttp://www.pcr.uu.se/research/UCDP/data_and_publications/datasets.htm

2008 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

World Development Indicators

2009 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

PITF http://globalpolicy.gmu.edu/pitf/pitfdata.htm

Years Vary X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Bertelsmann

http://www.bertelsmann-transformation-index.de/fileadmin/pdf/Anlagen_BTI_2008/Detailed_Results_BTI2008.xls

2008 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

World Governance Indicators

2.5=best-2.5=worst 2008 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

World Bank CPIA Criterion

http://go.worldbank.org/S2THWI1X60 2008 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

UNICEF/WHO Global Water and Sanitation Report; World Development Indicators

Latest Data varies 2000-2007

X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Failed States Index/Fund for Peace

http://www.fundforpeace.org/web/index.php?option=com_content&task=view&id=99&Itemid=146

2009 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

State Failure Index, Center for Systemic Peace based on Polity

http://www.systemicpeace.org/polity/polity06.htm 2007 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Afrobarometer http://www.afrobarometer.org/questionnaires.html 2008 X X X X X X X X X X

EIU Democracy Index Freedom House

http://graphics.eiu.com/PDF/Democracy%20Index%202008.pdf

2008 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Gallup World Poll 2009 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Failed States Index, Fund for Peace

http://www.fundforpeace.org/web/index.php?option=com_content&task=view&id=99&Itemid=141

2009 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

CIVICUShttp://www.civicus.org/csi/phase-one/csi-country-reports

2003-2006 X X X X X X X X X X X

SIPRI, The Military Balance

http://www.sipri.org/databases/milex 2007 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

SIPRI, The Military Balance

http://www.sipri.org/databases/milex 2007 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

UNODC 2005-2006 X X X X X X X X X X X X X X

Global Peace Index (collected by EIU)

http://www.visionofhumanity.org/gpi/results/rankings/2008/

2009 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

World Bank Doing Business

http://www.doingbusiness.org/ X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

World Bank Enterprises Survey

http://www.enterprisesurveys.org/

Years vary X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Appendix C

Source Source Link Latest data

Jam

aica

Guy

ana

Bur

undi

CA

R

Cd'

l

DR

C

Eth

iopi

a

Cha

na

Gui

nea

Gui

nea

Bis

sau

Ken

ya

Libe

ria

Mal

awi

Moz

ambi

que

Nig

eria

Rw

anda

Sen

egal

Sie

rra

leon

e

Som

alia

Sou

th A

fric

a

Sud

an

Tanz

ania

Uga

nda

Zam

bia

Zim

babw

e

Kos

ovo

Ser

bia

BiH

Geo

rgia

Arm

enia

Aze

rbai

jan

Kyr

gyzs

tan

Tajik

ista

n

Leba

non

OP

Ts

Yem

en

Afg

hani

stan

Pak

ista

n

Ban

glad

esh

Indi

a

Nep

al

Sri

Lan

ka

Bur

ma

Freedom House, Freedom in the World

2008 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Minorities at Riskhttp://www.cidcm.umd.edu/mar/assessments.asp?regionId=99

2009 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Failed States Index/Fund for Peace

http://www.fundforpeace.org/web/index.php?option=com_content&task=view&id=99&Itemid=145

2009 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

IMD World Competitive Yearbook

http://www.imd.ch/research/publications/wcy/list_of_countries.cfm

2009 X

Arab Barometer http://www.arabbarometer.org/reports/countryreports/comparisonresutls06.html

2006 X

Transparency Int’l - Corruption Perceptions Index

http://www.transparency.org/news_room/in_focus/2008/cpi2008/cpi_2008_table

2008 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Asian Barometerhttp://www.asianbarometer.org/newenglish/surveys/SurveyResults.htm

Years Vary X X X X X

CIRI Index http://ciri.binghamton.edu/myciri/my_ciri.asp 2006 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Political Terror Scale

http://www.politicalterrorscale.org/ 2007 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Source Source Link Latest data

Jam

aica

Guy

ana

Bur

undi

CA

R

Cd'

l

DR

C

Eth

iopi

a

Cha

na

Gui

nea

Gui

nea

Bis

sau

Ken

ya

Libe

ria

Mal

awi

Moz

ambi

que

Nig

eria

Rw

anda

Sen

egal

Sie

rra

leon

e

Som

alia

Sou

th A

fric

a

Sud

an

Tanz

ania

Uga

nda

Zam

bia

Zim

babw

e

Kos

ovo

Ser

bia

BiH

Geo

rgia

Arm

enia

Aze

rbai

jan

Kyr

gyzs

tan

Tajik

ista

n

Leba

non

OP

Ts

Yem

en

Afg

hani

stan

Pak

ista

n

Ban

glad

esh

Indi

a

Nep

al

Sri

Lan

ka

Bur

ma

Uppsala/UCDPhttp://www.pcr.uu.se/research/UCDP/data_and_publications/datasets.htm

2008 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

World Development Indicators

2009 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

PITF http://globalpolicy.gmu.edu/pitf/pitfdata.htm

Years Vary X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Bertelsmann

http://www.bertelsmann-transformation-index.de/fileadmin/pdf/Anlagen_BTI_2008/Detailed_Results_BTI2008.xls

2008 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

World Governance Indicators

2.5=best-2.5=worst 2008 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

World Bank CPIA Criterion

http://go.worldbank.org/S2THWI1X60 2008 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

UNICEF/WHO Global Water and Sanitation Report; World Development Indicators

Latest Data varies 2000-2007

X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Failed States Index/Fund for Peace

http://www.fundforpeace.org/web/index.php?option=com_content&task=view&id=99&Itemid=146

2009 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

State Failure Index, Center for Systemic Peace based on Polity

http://www.systemicpeace.org/polity/polity06.htm 2007 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Afrobarometer http://www.afrobarometer.org/questionnaires.html 2008 X X X X X X X X X X

EIU Democracy Index Freedom House

http://graphics.eiu.com/PDF/Democracy%20Index%202008.pdf

2008 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Gallup World Poll 2009 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Failed States Index, Fund for Peace

http://www.fundforpeace.org/web/index.php?option=com_content&task=view&id=99&Itemid=141

2009 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

CIVICUShttp://www.civicus.org/csi/phase-one/csi-country-reports

2003-2006 X X X X X X X X X X X

SIPRI, The Military Balance

http://www.sipri.org/databases/milex 2007 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

SIPRI, The Military Balance

http://www.sipri.org/databases/milex 2007 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

UNODC 2005-2006 X X X X X X X X X X X X X X

Global Peace Index (collected by EIU)

http://www.visionofhumanity.org/gpi/results/rankings/2008/

2009 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

World Bank Doing Business

http://www.doingbusiness.org/ X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

World Bank Enterprises Survey

http://www.enterprisesurveys.org/

Years vary X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Appendix D

Source Source Link Latest data

Jam

aica

Guy

ana

Bur

undi

CA

R

Cd'

l

DR

C

Eth

iopi

a

Cha

na

Gui

nea

Gui

nea

Bis

sau

Ken

ya

Libe

ria

Mal

awi

Moz

ambi

que

Nig

eria

Rw

anda

Sen

egal

Sie

rra

leon

e

Som

alia

Sou

th A

fric

a

Sud

an

Tanz

ania

Uga

nda

Zam

bia

Zim

babw

e

Kos

ovo

Ser

bia

BiH

Geo

rgia

Arm

enia

Aze

rbai

jan

Kyr

gyzs

tan

Tajik

ista

n

Leba

non

OP

Ts

Yem

en

Afg

hani

stan

Pak

ista

n

Ban

glad

esh

Indi

a

Nep

al

Sri

Lan

ka

Bur

ma

Freedom House, Freedom in the World

2008 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Minorities at Riskhttp://www.cidcm.umd.edu/mar/assessments.asp?regionId=99

2009 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Failed States Index/Fund for Peace

http://www.fundforpeace.org/web/index.php?option=com_content&task=view&id=99&Itemid=145

2009 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

IMD World Competitive Yearbook

http://www.imd.ch/research/publications/wcy/list_of_countries.cfm

2009 X

Arab Barometer http://www.arabbarometer.org/reports/countryreports/comparisonresutls06.html

2006 X

Transparency Int’l - Corruption Perceptions Index

http://www.transparency.org/news_room/in_focus/2008/cpi2008/cpi_2008_table

2008 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Asian Barometerhttp://www.asianbarometer.org/newenglish/surveys/SurveyResults.htm

Years Vary X X X X X

CIRI Index http://ciri.binghamton.edu/myciri/my_ciri.asp 2006 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Political Terror Scale

http://www.politicalterrorscale.org/ 2007 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X