From policy lessons to policy actions: motivation to take evaluation seriously

public administration and development

Public Admin. Dev. 24, 305–320 (2004)

Published online in Wiley InterScience

(www.interscience.wiley.com) DOI: 10.1002/pad.325

FROM POLICY LESSONS TO POLICY ACTIONS:MOTIVATION TO TAKE EVALUATION SERIOUSLY

GUSTAVO GORDILLO1 AND KRISTER ANDERSSON2*1Assistant Director General, Food and Agriculture Organisation of the United Nations Regional Office for Latin America

and the Caribbean, Santiago, Chile2Research Fellow, Center for the Study of Institutions, Population, and Environmental Change, Indiana University, Indiana, USA

SUMMARY

The purpose of this article is to analyse the institutional aspects of creating effective systems for monitoring and evaluations(M&E) in government-led rural development efforts in developing countries. We argue that the ultimate challenge of creatingeffective M&E systems for public policy is not only related to the supply of information and the delivery of new knowledge topolicy makers, but more importantly to their demand for lessons learnt about the effects of earlier policies. The challenge, then,is for governments to construct institutional arrangements that support the transformation of policy lessons into policy actions.This article shows that the likelihood of this transformation is closely related to the capacity of institutions to deliver mechan-isms for downward accountability and processes for organisational learning. We add to earlier work in this area by consideringthe conditions under which such a transformation process is more or less likely, given the severe power and information asym-metries that characterise the institutional context of many developing countries’ national governments. We use the tools ofinstitutional analysis to examine the incentive structures of the actors involved in two concrete field settings, and assess to whatextent these actors are likely to take evaluations seriously. Copyright # 2004 John Wiley & Sons, Ltd.

INTRODUCTION

Second-generation reforms in the developing world were designed to change the way governments relate to peo-

ple, organisations and natural resources. More than a decade has passed since these reforms were first introduced

and an increasing body of knowledge is being accumulated on the effects of these reforms through empirical policy

research in different countries in the region (see for example Haggard and Kaufman, 1995; Barro, 1996; Deininger

and Squire, 1998; Stiglitz, 1998; Gordillo, 1999; Kaimowitz et al., 1999). The research on the impacts of recent

political reforms have in some cases led to modifications in countries’ national policies; although the link between

policy evaluation and policy actions is often quite weak or entirely missing (Patton, 1997; Biggs and Matsaert,

1999; Cracknell, 2001; Picciotto, 2003). In fact, in many developing countries, policy evaluations are published

and delivered to government decision-makers, but are still not acted upon. There are many potential reasons why

such inactivity might result, and the main purpose of this article is to discuss some of the institutional reasons. Why

do so many governments take policy evaluation so lightly? The hypothetical explanation explored in this article

emphasises the role of democratic institutions and their mechanisms for downward accountability.

The main argument of the article is that the transformation of policy lessons into policy actions is a process

associated with specific political incentives that may sway policy makers to take into account information from

policy evaluations. By considering political incentives, it is possible to construct an empirically testable hypothesis

that has the potential to explain why governments may decide to act on information from policy evaluation and,

perhaps more important, why they so often do not.

Copyright # 2004 John Wiley & Sons, Ltd.

*Correspondence to: K. Andersson, Center for the Study of Institutions, Population, and Environmental Change (CIPEC), Indiana University,Indiana, IN, 46408 USA. E-mail: [email protected]

The work on monitoring and evaluation (M&E) of public policy has so far mostly focused on collecting, pro-

cessing and analysing policy activities. Less emphasis has been placed on constructing institutional arrangements

that would induce a variety of actors, including both politicians and citizens, to demand the relevant information

for decision-making. This article addresses this shortcoming by discussing the institutional underpinnings that

make M&E programmes effective.

Monitoring and evaluation of public policy

Monitoring and evaluation activities, when carried out properly, generate essential information about public sector

performance. As Picciotto (1995) puts it, ‘Evaluation is for the public sector what accounting is for the private

sector’ (p. 2). As such, independent M&E of public policy results constitute a fundamental part of good govern-

ance, and the lack of citizen participation in such activities is a serious threat to the creation of transparent and

accountable public organisations. While many analysts have pointed to the need for developing mechanisms for

participatory and independent M&E of political reform processes, few have explicitly addressed the institutional

designs of M&E systems that are capable of buttressing incentives for better governance in developing countries

(e.g., Rossi and Freeman, 1993; World Bank, 1994; Chelimsky, 1995; Irwin et al., 1996; OECD, 1996; Babu and

Reidhead, 2000; but see also Cracknell, 2001; as well as Biggs and Smith, 2003).

WHY WOULD POLITICIANS CARE ABOUT EVALUATIONS?

One of the main purposes of M&E is to provide essential information to the citizenry so the public sector officials

can be held accountable for their actions. Accountability mechanisms, which are fundamental to good governance,

can be strengthened by providing organised citizen groups with the opportunity to participate in and share the

results of public sector evaluations. In fact, if citizen groups and media do not have ready access to such informa-

tion, citizens’ assessment of public policy performance will be limited to isolated, personal observations. Unfortu-

nately, in many countries, this is often the citizens’ only source of information about the performance of

governmental rural development programmes.

The problems of M&E systems in Latin America

Surprisingly few governments in Latin America are systematically documenting results on how the reforms affect

the quality of public service delivery related to rural development. Considering the central role of M&E pro-

grammes in achieving accountable governance of public goods and services, it is surprising that only one country

in the region—Colombia—has included explicitly institutional capacity-building for evaluation in its decentrali-

sation reform package (Shah, 1998, p. 84). Why have evaluation programmes not caught on as an instrument for

organisational policy learning in the region? Several potential reasons exist. In recent reviews by Brown (2001) and

Piciotto and Wiesner (1998), the following appear as candidate explanations:

1. Budget allocations are made on the basis of a government’s arbitrary assessment of the relevance of a sector

ministry’s activities as well as of the previous year’s spending rate. Performance is not used as common criteria

for resource allocations.

2. Evaluations are costly and poor country governments find evaluations to be a luxury.

3. Evaluations are mainly associated with donor practices and as such a traditional mechanism for control and

accounting rather than learning.

4. Some countries in the region have a centralistic and elitist political tradition, characterised by patronage rela-

tionships and a general lack of effective mechanisms for checks and balances within government.

While all of these explanations provide useful insights as to why a government would not be motivated to invest

in policy evaluations, and why some governments in developing countries have limited institutional capacity to

generate policy lessons through evaluations, we believe that the fourth point is of particular relevance because it

provides a plausible explanation of why so many politicians take evaluations so lightly. Yet in the evaluation and

public administration literature, this argument is the least discussed of the four. This article seeks to further develop

306 G. GORDILLO AND K. ANDERSSON

Copyright # 2004 John Wiley & Sons, Ltd. Public Admin. Dev. 24, 305–320 (2004)

the institutional argument of why some M&E systems are capable of generating better governance while many

others are not. We start our analysis of this question by exploring potential reasons from the literature as to

why democratic political institutions are underdeveloped in many parts of Latin America and the Caribbean.

Historical origins of macro-institutional performance

There is a growing body of literature in economic history that analyses the influence of institutional arrangements

on economic performance. The seminal study of North (1990) argued that the superior economic performance of

Western Europe, the United States and Canada as compared to countries in Latin America had to do with the legal

and political institutions associated with the Anglican tradition, which were more conducive for economic affairs

than the Spanish, French and Portuguese systems (North, 1990).

Other institutional scholars, drawing on North’s approach, have further developed the institutional argument of eco-

nomic development. In a systematic review of the literature on the institutional origins of the variation in economic

performance in the Americas, Sokoloff and Engerman (2000) conclude that recent empirical evidence suggests that

factor endowments, rather than national heritage, provide a stronger explanation to the differences in economic per-

formance in the New World. Their basic argument is that countries rich in natural resources lend themselves to econo-

mies of scale in their resource exploitation. As a result they become subject to pronounced concentrations of property

rights among relatively small and closed political elites who can use political means to ensure a continued flow of

benefits from the factor endowments that they control. The concentration of political power among clustered and

closed political elites largely prevented the creation of public goods and services, such as public education and primary

health programmes, which have proven to be essential underpinnings to sustained economic growth. The less skewed

distribution of factor endowments in the United States and Canada was more conducive for developing more equitable

political structures, which have proven to be more appropriate for providing the essential public goods and services

that support economic development (Sokoloff and Engerman, 2000).

Karl (2002), examining the origins of inequality in Latin America, argues in the same vein: ‘Exceptionally high

inequalities of wealth and income are the basis for exceptionally inequitable distributions of political power and

representation, even in the continent’s young democracies, and these power arrangements are subsequently unli-

kely to address the basic problem of high inequality. Instead, economic and social policy operates largely through

the exercise of private influence and the skewed functioning of politicized bureaucracies structured to favor large

economic groups. The unequal power distributions they both reflect and reproduce, in turn, help to secure eco-

nomic privileges, undermine competition and efficiency, encourage corruption, undermine productive growth

and, in the end, subvert democracy’1 (p. 5).

The observation that closed political elites have historically controlled government in many countries in the

region suggests that the lack of institutional mechanisms for accountability of government may explain why eva-

luations have not caught on as a management tool in public policy. Because of the widespread social and political

inequalities in Latin America, the institutions that could provide accountability are likely to emerge only progres-

sively. Effective institutions for accountability are not likely to materialise until a critical coalition of political actors

agrees to address the issue of social exclusion as a priority concern of the country’s economic and political agenda.2

In Latin America in general, the institutions that could ensure effective feedback loops to policy makers are

frequently not working. This underprovision of effective institutional arrangements for M&E systems is a major

1Karl (2002) puts it in a slightly different way: ‘This means that Latin American public policies cannot be understood as the product orequilibrium outcome of democratic voting among rational individuals, as models based on the ‘‘median voter’’ claim, because such modelsassume that the distribution of political power is relatively egalitarian. As scholars of comparative politics have repeatedly shown, politics inLatin America work only in limited ways through the democratic process, and then votes are often ‘‘delivered,’’ especially in rural areas’ (Karl,2002, p. 6).2The economic inequalities have implications for efforts of poverty alleviation. Karl (2002) states: ‘Such a skewed distributional profile meansthat the poor face greater barriers to escaping their deprivation because, first, extreme inequalities reduce growth and, second, the alleviation ofpoverty becomes less responsive to the growth that does occur. Thus although Latin America’s per-capita Gross Domestic Product grew byalmost 6 per cent in real terms between 1990 and 1995, the years of highest growth over the past two decades, the number of extremely andmoderately poor actually increased by 1.5 and 5 million people, respectively, over the same period. No other region in the world shows thispattern’ (Karl, 2002, p. 6).

FROM POLICY LESSONS TO POLICY ACTIONS 307

Copyright # 2004 John Wiley & Sons, Ltd. Public Admin. Dev. 24, 305–320 (2004)

constraint for natural resource governance and rural development. Crafting effective institutions for M&E is not a

straightforward process, however, and no general design will serve the needs of all countries in the region. The

institutional incentive analysis, presented in this article, provides a starting point for the design of the institutional

arrangements that have the capacity to generate both policy lessons and actions.

The challenge of transforming a club good to a public good

All M&E systems seek to produce information that can be used to assess the quality of results and achievements of

specific activities. In a society where it may not be in the ruling political elite’s self-interest to diffuse the information

from M&E programmes, since this may threaten their future political status and privileges, a closed political elite

group may be able to limit the dissemination of M&E results (Schneider and Heredia, 2003; Taut and Brauns, 2003).

If this is the case, the characteristics of the M&E information resemble those of a club good, as the number of users

who have access to the information is strictly limited by the powerful elite (high excludability), although one user’s

consumption of information does not leave less information available for other authorised users (low subtractability).

Figure 1 presents a framework for how a variety of different information goods and services may be classified

according to a particular good’s degree of subtractability and excludability. In the first step of our institutional

analysis, we use this framework to define the nature of M&E information in terms of its biophysical attributes.

These attributes help determine the particular problems that M&E information is particularly vulnerable to.

The main problem with M&E information is its high excludability, which hampers the wider use of evaluations

in the democratic process.

Good governance requires this limited-access club good to be transformed into an open-access public good

regime, in which the individuals who consider themselves as information owners cannot exclude other potential

users of the produced M&E information (low excludability). Transparency of information characterises the public

good regime of M&E information. One of the problems in many democracies is that where independent policy and

programmes exist, the institutions that are supposed to provide public access to the results are often weak, bad or

entirely missing (Ostrom et al., 2002).

Transforming the M&E information from a relatively restricted club good into a public good can be a compli-

cated process, as it not only requires a change in the formal rules that govern the public’s access to the M&E infor-

mation, but also a change in the actual behaviour of the political actors. The mere creation or modification of the

rules can be a complicated collective action process as it is sometimes difficult to motivate participants to invest the

time and effort necessary to create such rules (Olson, 1965; Ostrom, 1990; Bates, 1998).3 It can be very difficult to

Figure 1. Attributes of information goods and services and their corresponding regimes.

3Again inequality plays a major role as a deterrent for collective action; ‘but if the rich are politically favored through exceptional access topower, the poor most decidedly are not. In general, they are simply too weak economically and politically to demand pro-poor policies. Asinformal sectors have grown and family incomes within them have fallen, the permanent struggle to survive does not bode well for opportunitiesto organise collectively, especially for women. Nor are memories of the recent authoritarian past, where they were the greatest victims ofrepression, conducive to mobilization to protect their interests. When compared to the rich, the poor are significantly more reluctant to engage inpolitical action . . . . In surveys conducted in Chile, Mexico and Costa Rica, the lowest income sector would never sign a letter of protest (41%compared to 27% of the highest income sector) or attend a demonstration (52% compared to 37%)’ (Karl, 2002, p. 18).

308 G. GORDILLO AND K. ANDERSSON

Copyright # 2004 John Wiley & Sons, Ltd. Public Admin. Dev. 24, 305–320 (2004)

modify the rules concerning the property rights regime of M&E information, because the new rules that make

information from M&E publicly available imply a restriction of the power and influence of the ruling elites.

The sources of the difficulty to what we can call the ‘public good problem of M&E information’ include motivation

problems as well as power and information asymmetries. Each of these collective-action problems is associated

with its own set of institutional incentives that will likely affect the outcome of the efforts to create publicly avail-

able M&E information.

Motivation problems

Efforts to produce public goods and services are plagued by perverse incentives for actors to free-ride and to seek

rents, rather than to cooperate for the benefit of the group at large (Olson, 1982; Krueger, 1993; Tendler and

Freedheim, 1994; Weimer and Vining, 1999). For M&E systems specifically, the political elites are tempted to

influence the rule-making processes so the new set of rules will increase the flow of personal benefits from the

M&E systems in one way or another. Likewise, it may be in the interest of a member of a political opposition party

to have certain government programmes evaluated because a wider publication of such results is believed to benefit

the individual and opposition party politically. Even so, members of the government are likely to resist evaluations

that will attempt to hold them accountable for apparent poor performance. The threat of a possible negative review

is often enough for the politicians in power to either postpone the evaluation or try to control the selection of con-

sultants as well as the actual collection and publication of information.

The more dramatic the power asymmetries are among actors—that is, the more power the political elites pos-

sess relative to the power of citizens and their political opposition and other actors—the weaker the incentive is for

such an elite to transform established M&E programmes into an instrument for more public sector accountability.4

Power and information asymmetries

Closed political elites face strong incentives to try to maintain their privileged status as unaccountable public ser-

vants. The introduction of M&E systems is often seen as a threat to their political privileges (e.g. see Biggs and

Smith, 2003). In the cases where independent M&E programmes function, these elites will try to thwart the efforts

of evaluators and limit the diffusion of information related to the performance of public sector programmes in

which the elites have a stake (Picciotto, 2003). As long as these information asymmetries exist, it is difficult to

establish mechanisms for accountable governance, as the principal prerequisite for public organisation account-

ability is a transparent flow of information between citizens, politicians and public sector professionals. When such

institutional transparency is limited, closed political elites and their professional staff will have weak incentives to

improve their performance in serving citizens. Without institutions capable of producing public access to M&E

information, politicians are likely to continue protecting their exclusive access to their ‘clubs’ M&E information.

Again, a process of rulemaking and subsequent enforcement is required to create more transparent and accountable

governance, a collective action effort that is far from an automatic process.

It is not just developing countries that face institutional problems related to putting M&E systems to use for the

policy making process. In fact, a recent study in psychology explores the sources of reluctance to learn from eva-

luations and finds that this is a common problem in virtually all organisations because of the human propensity to

resist change (Taut and Brauns, 2003, p. 248). Taut and Brauns (2003) argue that stakeholders will undertake a

personal cost-benefit analysis and if they find that the costs exceed the expected benefits, ‘stakeholders will try to

prevent the anticipated negative outcome and may resist evaluation’ (p. 252). It follows, then, that individuals

whose power is threatened by the evaluation have an incentive to campaign against its findings and the effective

implementation of its recommendations.

4Thomas Hobbes defines power as, ‘the means to obtain some future apparent good’ (Hobbes, [1651] 1960, p. 56). Along the same lines, VincentOstrom states that ‘power rests in the capacities and potentials of individuals in taking actions in arranging present means in appropriate ways torealize future apparent goods’ (Ostrom, 1997, p. 92). Powerful political elites can use their influence and power to seek and seize rents frompublic resources and will also face incentives to manipulate M&E systems in their favour.

FROM POLICY LESSONS TO POLICY ACTIONS 309

Copyright # 2004 John Wiley & Sons, Ltd. Public Admin. Dev. 24, 305–320 (2004)

In organisations where independent evaluation is mandatory, powerful individuals are likely to try to influence

the evaluation process in a way that minimises the prospects for changes to the status quo. One way of doing so is

to restrict the dissemination and socialisation of evaluation results within the organisation. A review of the use of

evaluations in a Swedish government agency observes that, ‘the further away that you are from a project in terms of

power and involvement, the less information you receive on the evaluation’s recommendations and findings’

(Carlsson et al., 1999).

Another strategy by power elites who resist change may be to limit the participation of those actors who are

likely to voice negative opinions regarding the programme results. For policy makers it may be risky to have citi-

zens participate in evaluations. Evaluations that exclude citizens, the true beneficiaries of public goods and ser-

vices, do little to promote accountable decision-making, organisational learning and beneficiary ownership of such

goods and services (Lawrence, 1989; Martens et al., 2002; Ostrom et al., 2002).

Institutional incentives to support policy evaluation feedback loops

Several studies have noted that the utility problem of evaluations is often related to the decision makers’ lack of

political will to take evaluation results seriously (Patton, 1997; Biggs and Smith, 2003; Pawson, 2002; Picciotto,

2003). It is not always clear in the literature, however, what political will means or what affects it. To understand

why some policy makers are interested in using evaluation results as a tool for organisational change while others

are not, we need to unpack the concept of political will.

We conceptualise political will as the combination of institutional incentives that motivate politicians to carry

out a certain task. We propose that political leaders will promote increased institutional transparency and account-

ability only when they, as the ‘owners’ of the M&E information, have incentives to share this information with

actors outside their immediate sphere. In political regimes where such positive incentives are weak, M&E infor-

mation is often used as a mechanism to exert top-down control, monitor and restrain their agents. Since the owners

of the information have the ability to restrict the diffusion of this information, they face an incentive to exploit the

M&E system and use it to their own personal benefit, that is, removing personal foes, rewarding their friends.

There are various sources of institutional incentives that could potentially counteract such unproductive institu-

tional arrangements to motivate politicians to make the M&E results more widely publicised and utilised in

decision-making. Earlier work on institutional incentives for good governance has identified the following poten-

tial sources of positive incentives:

� Pressure from interest groups. Interest groups, whose support is important for political elites to stay in power,

may demand that M&E information is publicised and used to hold a variety of actors accountable for their deci-

sions and performance. Such use of M&E information may serve to strengthen the downward accountability of

the political elites (Andersson, 2003; Gibson and Lehoucq, 2003).

� Pressure from electorate. M&E information can be valuable for performance assessment of political candidates,

if such information is available. If the M&E information is not available for the reasons discussed above, orga-

nised parts of the electorate may demand that this information is put in the public domain by making it an elec-

tion issue. Pressure of this kind is most effective when channels of mass media are utilised and when

independent organisations, such as the United Nations or international nongovernment organisations (NGOs),

verify the legitimacy of the electorate’s claims (Ribot, 2001; Ostrom et al., 2002; Gordillo and Andersson,

2004).

� Pressure from the executive. When a government programme is implemented by organisations at subnational

levels, it is sometimes difficult for central government agencies to acquire information about the performance of

the implementing organisations. This is an example of the classical principal-agent problem, which has no clear-

cut solution, but if the executive branch of the central government is able to oblige the implementing organisa-

tion to publicise information locally, such information may at least be used by local citizens to assess perfor-

mance of their local governments (Tendler, 1995).

� Pressure from donors. International donors that are interested in promoting good governance may demand that

M&E activities constitute an integral part of donor-funded activities. Theoretically, such conditionality of donor

310 G. GORDILLO AND K. ANDERSSON

Copyright # 2004 John Wiley & Sons, Ltd. Public Admin. Dev. 24, 305–320 (2004)

support could produce positive incentives for increased institutional transparency in public administration. How-

ever, as recent studies on this topic have pointed out, conditionality has had limited impact on good governance

in most developing countries (White, 1999).5

� Potential financial gains. By showing willingness to take reform measures that are compatible with the goals of

good governance, developing country governments may attract international donor funding.

These incentives are derived from the institutional structures in which the political actors operate. Because insti-

tutional arrangements define the distribution of information, power and resources within a political system, these

arrangements also influence the strength of a politician’s incentives to take evaluations seriously. For instance,

politicians are more likely to act on evaluations when there are institutional constraints that effectively limit their

ability to use political power to protect themselves from a negative performance assessment. One of the strengths

of a polycentric governance system, as compared to a monocentric or centralised governance structure, is that the

political power is vested with a multitude of political actors at different levels of governance. The polycentric

structure therefore enhances the possibility for political actors at any given level to hold decision-makers at any

other governance level accountable for their decisions.

The institutional mechanisms for downward accountability seem particularly important for dealing with the

problem of concentration of power within a political elite. Several institutional innovations exist in different poli-

tical systems that enable citizens to hold decision-makers accountable. One interesting example of such a mechan-

ism is the local grand jury in the United States.6 In most U.S. counties, the county prosecutor can call a grand jury

to investigate any hint of corruption or other problems associated with the local activities of a governmental

agency. Although giving citizens the power to hold politicians and civil servants accountable is not a guarantee

that good governance will emerge, it does affect the incentive structure for decision-makers to consider evaluation

results—especially if concerned citizens voice their demand for policy action.

It should be noted that the institutional underpinnings that we focus on in this article take on primary importance

only after the issues of relevance and timing have been resolved. If the evaluation authors fail to produce reliable

findings or are not able to deliver these in a timely fashion to decision-makers, chances are that the policy lessons

will be ignored.

In summary, the transformation of policy lessons into policy actions is a political process that is neither auto-

matic nor self-enforced. One cannot expect that the mere existence of robust policy lessons is sufficient to generate

policy actions. Few governmental actors will disseminate and act upon evaluation results unless they think they

will somehow benefit from doing so. Recognising the existence of institutional incentives is a first step toward

understanding why some evaluations may produce better policies while others collect dust in the government

archives. In the next section, we use institutional analysis to identify the institutional incentives to take evaluation

seriously in two concrete rural development field settings.

DETECTING INSTITUTIONAL PROBLEMS FOR M&E

One of the primary bottlenecks in contemporary policy evaluation in Latin America is of political nature. Once the

M&E results are produced, they need to be assimilated by the political actors in order to lead to modifications in

policy. We argue that evaluation results will seldom speak for themselves. Experiences from the region show that

policy making is not self-corrective in nature and that politicians are likely to take evaluation seriously only when it

will benefit them politically or economically. This raises a serious question about what specific constraints exist to

motivate politicians to take evaluations seriously. The tools of institutional analysis, we suggest, can help identify

what those constraints are.

5The evidence on the effectiveness of using ex-ante and ex-post conditionality as a way of encouraging recipient governments to implementdemocratic reform is far from straightforward. In a comprehensive review of the Swedish government’s aid programme’s use of thesemechanisms, White (1999) concludes that ‘there is ample evidence . . . for the view that conditionality has failed, and of the reasons for thisfailure; but there are also cases where conditionality has influenced policies and reasons to believe why this may have been so’ (p. vi).6We owe this observation to Lin Ostrom, who also pointed out that it was actually the investigation by a local grand jury that led to theresignation of President Richard Nixon in August of 1974.

FROM POLICY LESSONS TO POLICY ACTIONS 311

Copyright # 2004 John Wiley & Sons, Ltd. Public Admin. Dev. 24, 305–320 (2004)

Using institutional analysis as a diagnostic tool for policy making

In order to come to grips with the possible incentive problems associated with taking M&E seriously, it is impor-

tant to carefully analyse what the causes behind such incentive problems are. A useful diagnostic tool for the detec-

tion of institutional incentives in political decision-making is the Institutional Analysis and Development (IAD)

framework (Figure 2), developed by colleagues at Indiana University (Ostrom and Ostrom, 1977; Kiser and

Ostrom, 1982; Oakerson, 1992). The IAD framework has been used for a large number of empirical studies includ-

ing those that analysed (1) the impact of metropolitan-area governance structures on urban service delivery, (2)

how institutional incentives affect the sustainability of international development cooperation projects in develop-

ing countries, (3) how diverse forms of organisation affect irrigation system performance and (4) how ecological

conditions combined with institutional structure affect land-use change dynamics (Ostrom, 1990; Gibson, 1999;

McGinnis, 1999, 2000; Andersson, 2003).

The IAD framework is appropriate for the study of institutional incentives for several reasons. First, it allows for

a multidisciplinary integration because it is not strictly an economist’s or political scientist’s framework, but con-

siders variables that are deemed important for institutional incentives by a wide variety of disciplines, from biology

to anthropology. Second, IAD presents a practical method for dealing with multiple levels of analysis. The out-

comes of human interaction depend on complex processes that take place on different levels of decision-making.

For instance, when individuals interact in repetitive settings that directly affect physical outcomes, they are func-

tioning in operational situations. The rules that affect the structure of an operational situation are themselves

designed and agreed upon in policy-making situations. The policy-making decisions themselves, in turn, are made

according to the rules made in constitutional situations and define who can participate in policy-making decisions

and what such decisions may or may not entail.

A third reason why the IAD framework is appropriate for analysing incentive problems is that it allows for

examining configural or interactive processes. Configural processes are those where the relationship between

two or more variables is dependent on at least one other variable and are therefore not additive in nature. This

is often the case in determining the effect of certain institutional incentives in political decision-making processes,

where the effect of one set of rules can be very difficult to separate from the combined effect of other contextual

Figure 2. A framework for institutional analysis.

312 G. GORDILLO AND K. ANDERSSON

Copyright # 2004 John Wiley & Sons, Ltd. Public Admin. Dev. 24, 305–320 (2004)

factors, such as culture, social standing and trust. The IAD framework provides a practical tool for analysing the

complexities of political decision-making (Ostrom et al., 2002). In the next section, the institutional conditions for

taking evaluations into account in policy making in two cases the IAD framework is used to examine.

RESULTS FROM CASE STUDIES

Community forestry concessions in Bolivia

The first case study illustrates how motivational problems may get in the way of addressing clear indications of

policy failures. The action arena in this case is Bolivia’s forestry sector. The example draws from field observations

and secondary literature describing various action situations involving interactions between several arena actors.

The action arena, its actors and action situations

Bolivia’s 1994 Law of Popular Participation channeled about 20 per cent of the national budget to municipal gov-

ernments and introduced community control over municipal governments by establishing municipal oversight

committees, in which elected community representatives participate. The law also gave municipal administrations

a broader mandate to get involved in a wide variety of areas associated with Natural Resource Management. In the

forestry sector, this mandate was further expanded through the passing of the 1996 Forestry Law, as this law gave

municipal governments an explicit role and funds to carry out forest-related activities. Under the new law and

accompanying regulations, municipal governments are expected to administer up to 20 per cent of public forests

as municipal forest reserves to be exploited by local community groups and to ensure that concession holders and

sawmills comply with forestry regulations. The new provisions of the law meant that local community groups, who

formerly had very limited formal rights to manage forests commercially, were given conditional usufruct rights to

potentially valuable forest resources.

The improved access to forest management rights has yet to materialise, however, as an insignificantly small

amount of municipal forest reserves have actually been approved by the central government.7 Figure 3 presents the

number of community forestry groups that have actually been formed and approved, and acquired some degree of

formal property rights.

The central forestry agency is charged with the formal responsibility of continuously monitoring the effects

of policy and land-use practice on the forest resource base (Government of Bolivia, 1996, art. 22. I. a–k). Since

its creation in 1996, the Superintendencia Forestal (SF) has systematically gathered and analysed information

pertaining to the allocation of formal forestry property rights across different forest user groups and municipal

Figure 3. Community groups with varying levels of property rights.

7Both the Superintendencia Forestal and the Ministry for Sustainable Development and Planning need to approve the proposed forest concessionand concession holder.

FROM POLICY LESSONS TO POLICY ACTIONS 313

Copyright # 2004 John Wiley & Sons, Ltd. Public Admin. Dev. 24, 305–320 (2004)

territories. This information, including the number and extent of formal property rights assigned to local commu-

nity groups, is published in the agency’s annual report.8 The information produced by the agency’s ongoing mon-

itoring activities in the forestry sector are widely distributed and are subject to public debate and scrutiny

(Contreras-Hermosilla and Vargas, 2001; Superintendencia Forestal, 2001). Hence, the facts and figures on how

the new forestry regime is performing are well known to politicians and interested citizens in society at large.

If the information on policy effects is indeed of good quality and politicians do get this information in a timely

fashion, why is there so little political action to correct the apparent policy failure of community forestry conces-

sions? The question helps us define the most relevant action situations. In this case the main action situation is the

interaction that takes place when forest users use the information from the M&E system to hold governmental

authorities accountable.

Exploring contextual explanations

Several studies have pointed to the plausible reasons for the limited progress in this area (see for example

Contreras-Hermosilla and Vargas, 2001; Pacheco, 2001; Andersson, 2003). These studies inform us that

politicians face a number of constraints with regard to assigning more community forestry concessions. First,

in many municipalities there is a general lack of undisputed public forest land that could be considered for com-

munity concessions (conversation with Olvis Camacho, Head of Superintendencia Forestal for Santa Cruz, 27

April 2001). Second, even if there is land available, it is a notably cumbersome and bureaucratic process to actually

acquire a concession. Contreras-Hermosilla and Vargas (2001) found that the application process for community

concessions may last as long as two years, as applicants must comply with a total of 26 different bureaucratic

requisites.9

The number of formal rules and requirements form a tough barrier to community forestry in Bolivia. In effect,

the necessity to comply with such rules gives politicians an effective filter that reduces the number of applications

that make it to their tables. Bureaucratic procedures become an excuse for not approving more concession applica-

tions. To date, NGOs and bilateral projects have provided some support in overcoming these obstacles, but central

government assistance has been weak (Pacheco and Kaimowitz, 1998; Contreras-Hermosilla and Vargas, 2001).

There are lessons to be learnt, however, from the exceptional 24 cases where some concession rights indeed

have been issued. The vast majority of the groups with granted concessions were granted rights only after they

had exerted considerable political pressure at high political levels (Camacho et al., 2001). In some cases, the poli-

tical urgency instigated by these groups’ political activism took precedence over meeting formal requirements, as

politicians under pressure from organised farmers’ groups reacted swiftly to establish municipal forest reserves,

even when the formal requirements were not met (Pacheco, 2001). The evidence would suggest that one of the

underlying reasons for the failure of community forestry concessions in Bolivia is the low motivation among policy

makers to meet the demands of rural populations. This finding is consistent with previous empirical sector studies

that argue that politicians have little to gain, both politically and financially, from attending to the rural people’s

needs in the forestry sector (Andersson, 2003; Gibson and Lehoucq, 2003). Actually, many politicians often have

more to lose than to gain from granting concessions to rural communities, because in a sector dominated by a small

number of very large forestry firms, controlled by an even smaller number of families, politicians are prone to be

reluctant to upset the powerful forestry-sector elites.

8To get an idea of how forestry policy enforcement might affect the condition of the country’s forest, SF also gathers and analyses informationon changing biological conditions. In 2001, with support from the AID-financed BOLFOR project, the SF published a study on the state offorest-cover change in the Bolivian Lowlands (Camacho et al., 2001). Although the study is careful about explicitly passing judgment onwhether the existing forestry policy is effective, the study does make several observations of how the current formal rules have been unable tocontain unauthorised deforestation in many parts of the country. The study has been lauded by both civil society and the academic community asan important contribution to our understanding of the links between policy and forest-cover change (Andersson, 2002; Pacheco, 2002).9Under the 1996 law, the process of assigning public forests to local community groups (Agrupaciones Sociales del Lugar) begins with themapping and classification of all public forests by the Ministry of Sustainable Development and the Environment (MDSMA). Once this is done,the MDSMA and the municipal governments must provide Bolivia’s Forest Superintendent with a list of areas suitable for municipal forestreserves. The municipal councils then recommend who should be allowed to exploit the forest reserves, the municipal oversight committeesapprove those recommendations and the Forest Superintendent formally assigns the forests to those groups.

314 G. GORDILLO AND K. ANDERSSON

Copyright # 2004 John Wiley & Sons, Ltd. Public Admin. Dev. 24, 305–320 (2004)

In conclusion, the evidence from Bolivia’s forestry sector would suggest that good quality information on for-

estry-sector performance, in and of itself, is insufficient to achieve positive political action for the benefit of com-

munity forestry concessions. At the same time, the case of Bolivia’s forestry sector shows the power of M&E

systems when local user groups learn how to use its results to hold politicians accountable for their policy actions

or lack thereof.

Family agriculture in Brazil10

During the Presidency of Fernando Collor de Melo, the Minister of Agriculture, Antonio Cabrera asked the Food

and Agriculture Organisation (FAO) of the United Nations to evaluate the government programme that established

new smallholder settlement areas following what might be called Brazil’s agrarian reform after democratic restora-

tion in 1985.11 The evaluation consisted of a random sample of 44 settlements in the country’s five regions. Ten

months later, the results were presented to the government, which published the report shortly thereafter. The

results were surprising. The evaluation came to the conclusion that the agrarian reform, despite its many problems

associated with its conception and implementation, showed positive socioeconomic results. The evaluation found

that, ‘average income for families in reform settlements nationwide rose from [1 to 1.5 times the] minimum wages

prior to the reform to [equal] 3.70 minimum wages after the reform,’ as shown in Figure 4. Furthermore, the study

showed that, ‘the income earned by a worker within a reform settlement was significantly higher than what a ‘‘day-

worker’’ normally earns’ (Guanziroli et al., 2001).

The action arena, its actors and action situations

As a result of the FAO-led evaluation, the federal government launched a programme of family agriculture. To

this end, a technical cooperation agreement was signed between the FAO and Brazil’s National Institute for

Agrarian Reform (INCRA). In 1992, the objective of the cooperation was to draw up new agrarian policy guide-

lines that would incorporate the land tenure issue in a set of measures to promote and strengthen family agriculture

in Brazil. The efforts aimed at incorporating the redistribution of land into a more integrative rural development

policy, which meant modernising the policy instruments to solve not only the land access issues but also the tech-

nical challenges of making family agriculture more economically viable. This new approach was formulated to

deal with one of the agrarian reform’s unfortunate consequences—the favouring of large landholders—which

made it very difficult for the newly established family farms to be competitive.

The centralised policy context, inherited from the military regime in the 1950s and 1960s, made it difficult

to develop a policy response that could effectively meet the needs of small family farms. Moreover, INCRA,

as a centralised, public implementing organisation, was very ineffective at the time and did not involve citizens

Figure 4. Sources of family income by region.

10This case study draws extensively from Guanziroli et al. (2001).11Some authors refer to two agrarian reform periods in postmilitary regimes (1985–1987 and 1994–2001).

FROM POLICY LESSONS TO POLICY ACTIONS 315

Copyright # 2004 John Wiley & Sons, Ltd. Public Admin. Dev. 24, 305–320 (2004)

in decision-making or planning. INCRA realised a radical change was needed and contracted a small team of rural

development specialists to write a policy paper on how the agency could create policy that would effectively sup-

port family agriculture.

Before taking action on the proposal, the government opened up a period of extensive public consultation

throughout the country. FAO and INCRA organised a series of seminars, which during a period of six months gath-

ered around 5,000 people representing social movements, universities, NGOs, research institutes and state and

municipal governments. Each seminar lasted three days and offered thematic sections for in-depth discussion

of the most contentious issues related to family agriculture and possible policy instruments to support it. The

debate allowed for the modification of several points in the proposal and the incorporation of new points that

had not been considered. The final proposal was presented to the government in May 1995. This proposal made

a diagnostic of the current state of family agriculture and presented alternative policy options for agricultural pol-

icy adjustments and identified possible institutional improvements.

From lessons to actions

Several of the proposals presented in 1995 were subsequently adopted by the federal government. Among the more

notable were: (1) the creation of an integrated credit programme targeting rural smallholder families (PRONAF);

(2) a new type of agricultural extension system that was capable of responding to the needs of a larger number of

smallholder farmers (LUMIAR); (3) the creation of new instruments for gaining access to land that did not depend

solely on the confiscation of unproductive land, but that allowed families to purchase more productive land by

mortgaging the acquired land or renting it under favourable terms (SOF); (4) the decentralisation of some of

the governance responsibilities over infrastructure development in settlements to the state and municipal govern-

ments; (5) improved institutions for financial and political accountability to beneficiaries of the land reform

(PROCERA) and (6) the creation of rural development councils at the municipal and state levels in which farmer

organisations, including the national landless movement, Movimento Sem Terra (MST) and the agricultural

workers’ confederation (CONTAG) among others, were invited to participate in decision-making. These positive

results raise the question as to what motivated the policy makers at the time to take the evaluation recommenda-

tions seriously.

Exploring contextual explanations

According to the consultants themselves, there were several factors that combined to make the conditions ripe for

the proposed policy reforms. First, the message was both dramatic and unequivocal. The fact the team’s recom-

mendations were based on hard scientific evidence, shown in a powerful fashion, that despite all the constraints that

hamper family farms, they are more efficient users of the land than the large landowners. Second, the policy recom-

mendation arrived in a timely fashion, as the newly installed government was already committed to reform the

sector, paying particular attention to smallholder farmers and the landless. The reason they were committed to

reform is likely to be related to the high costs of continued rural development policy failures and the strong support

the new proposal had gained through the broad consultations.

Third, past attempts to address the concentration of land had failed, and the costs of such policy failures were

becoming increasingly significant for the government in power. The land issue was becoming an election issue.

The MST and other national grassroots groups exerted increasing political pressure on the government to take on

the burning issue of land redistribution. Since social movements now had access to the information it became more

costly for the government to ignore the recommendations.

Fourth, the broad consultations that the team carried out before finalising the policy recommendations served

two crucial purposes: it made the evidence on which the recommendations were based known to all important

actors in the Brazilian agricultural sector, and it modified the text to make the policy recommendations socially

acceptable to all of those actors. In this sense, the policy recommendations gained considerable political weight

when they were eventually presented to the government.

Finally, FAO’s participation was important, not only for the technical expertise that the organisation contributed

on the policy issues concerned, but also because the highly contentious topic required the participation of a

316 G. GORDILLO AND K. ANDERSSON

Copyright # 2004 John Wiley & Sons, Ltd. Public Admin. Dev. 24, 305–320 (2004)

politically neutral organisation such as the United Nations agency. The credibility of any national research institute

as compared to that of the FAO, would have been easier to undermine by the actors, who have more to lose than to

gain from policy reforms.

DISCUSSION OF FINDINGS

Despite their significant differences, the Bolivian and Brazilian cases share some fundamental lessons regarding

political motivation to respond to evaluation studies. Both cases suggest that producing good quality evaluation

studies alone are not sufficient to motivate political action. To generate adequate and science-based policy

responses to evaluations, the key seems to lie in the existence of a set of institutional incentives that are compatible

with action and change. The cases illustrate that the origin of such incentives is closely related to the way in which

the evaluations are carried out. Confirming earlier findings by Cracknell (2001) and Auluck (2002), the role of

beneficiary populations in the evaluation design and implementation seems to be particularly important in creating

incentives that support action.

In both cases, the establishment of mechanisms for downward accountability was crucial in motivating politi-

cians to take the results of the studies seriously. Despite the enabling community forestry policy in Bolivia, it was

only the groups that applied organised political pressure that eventually got a positive response from the govern-

ment. Another shortcoming in Bolivia’s accountability mechanisms was the lack of specificity in the policy objec-

tives regarding community concessions. The government had not pronounced any goals regarding the number of

concessions that should be attained in a given period of time. This nonspecificity in goals detracted from the cred-

ibility of the government’s commitment to change, and consequently, local beneficiaries had little evidence to sug-

gest that the government had failed. In this case, it is uncertain whether increased beneficiary participation in the

evaluation could have made any difference. In Brazil, on the other hand, the social pressure, which had been brew-

ing on the contentious land issue for a long time before the FAO-supported evaluation study was carried out, paved

the way for government-led action.

These observations would suggest that policy evaluations are most effective when stakeholders use them as

instruments to hold politicians accountable. The dilemma for many Latin American societies, however, is that

farmer groups are traditionally marginalised by politicians, and are seldom invited to participate as legitimate sta-

keholders in the policy process. Both NGOs and intergovernment organisations, such as FAO, have important roles

to play, facilitating such participation. One of the most obvious places to start to promote evaluations as an instru-

ment for downward accountability is in the development projects supported by international cooperation agree-

ments. As suggested earlier by the World Bank (1999) and Martens et al. (2002), such projects should, as a rule,

have target populations and beneficiary groups participate not only in the evaluation exercises as passive intervie-

wees but also as the designers of the project planning and evaluation studies.

The broad-based consultation exercise that the Brazilian consultants carried out, in which the study results were

discussed among a variety of stakeholders, served for both disseminating the results to key groups in the sector and

providing a space to discuss viable policy recommendations. If the effectiveness of an evaluation depends on how

policy target groups use the evaluation, then the issue of how beneficiary groups learn about the existence and

content of the evaluation becomes a central question. The Brazilian case would suggest that the best way for a

beneficiary group to learn about an evaluation study and its results is to participate in it.

CONCLUSIONS

In the process of discussing the motivation of policy makers, this article has identified several factors that are likely

to affect the probability of M&E data to be considered in future policy decisions. The theoretical review and

empirical analysis of the cases from Bolivia and Brazil pointed to the importance of moving beyond the content

of information generated by the M&E to consider the aspects related to the motivation of politicians to respond to

evaluations. First, we noted that the likelihood for policy actions is related to the position of the authors responsible

for the policy evaluation relative to the policy decision makers to whom the evaluation is delivered. If the evalua-

tion is produced by a group of actors who are linked to the government’s constituents and important interest groups

FROM POLICY LESSONS TO POLICY ACTIONS 317

Copyright # 2004 John Wiley & Sons, Ltd. Public Admin. Dev. 24, 305–320 (2004)

on which the government depends, policy makers will have a stronger incentive to take such evaluation seriously.

Hence, policy actions are more likely to follow.

Such an outcome, however, hinges on well-functioning institutions for political accountability. Many countries

in the developing world lack robust mechanisms in this area, perhaps not so much in the formal institutional design

as in the actual implementation, especially when it comes to downward accountability. In several countries, asym-

metries of power have produced oligarchies, in which the governed seldom have access to information about the

activities of the governors. To achieve good governance, based on empirical knowledge of past performance of

public policies, both economic and political inequities need to be addressed. Moreover, the Latin American experi-

ence, as evidenced by the cases from Bolivia and Brazil, shows that civil society actors such as mass media and

grassroots NGOs as well as international technical organisations can play a key role in the strengthening of

accountability mechanisms in public policy. A concerted effort among these actors has the potential to connect

the feedback loops from policy evaluations to public policy actions.

The full potential of empirically based policy making is not likely to be realised until decision-makers change

their attitudes toward evaluations. For policy lessons to lead to policy actions, evaluations need to be viewed less as

a mechanism for control and more as a tool for learning, and public policy needs to be viewed less as final pre-

scriptions and more as experiments. When policy makers and public sector managers are able to create an envir-

onment propitious for learning, the information contained in evaluations is not only likely to be more reliable—the

evaluated will feel less fear of revealing accurate information when they believe it will not be used against them—

but it will also increase the possibilities of adjusting sub-optimal policy decisions from the past. Such attitudinal

change is unlikely to occur without successively strengthening the institutions for equitable political representation

and downward accountability.

ACKNOWLEDGEMENTS

The authors would like to thank Carlos Guanziroli for his help with documenting the case study from Brazil. We

also thank Lin Ostrom and three anonymous reviewers for their constructive critiques and Joanna Broderick for her

help with editing. Krister Andersson acknowledges financial support from the Center for the Study of Institutions,

Population and Environmental Change (CIPEC) at Indiana University (NSF grant SBR9521918). The views

expressed in this article are those of the authors alone and should not be attributed to the organisations with which

they are affiliated.

REFERENCES

Andersson K. 2003. What motivates municipal governments? Uncovering the institutional incentives for municipal governance of forestresources in Bolivia. Journal of Environment and Development 12(1): 5–27.

Andersson K. 2004. Who talks with whom? The role of repeated interactions in decentralized forest governance. World Development 32(2):233–250.

Auluck R. 2002. Benchmarking: a tool for facilitating organisational learning? Public Administration and Development 22(2): 109–122.Babu SC, Reidhead W. 2000. Poverty, food security, and nutrition in Central Asia: a case study of the Kyrgyz Republic. Food Policy 25(6): 647–

660.Barro R. 1996. Democracy and growth. Journal of Economic Growth 1(1): 1–27.Bates RH. 1998. Institutions as investments. In The Political Dimension of Economic Growth, Borner S, Paldham M (eds). Macmillan: New

York; 3–19.Biggs S, Matsaert H. 1999. An actor-oriented approach for strengthening research and development capabilities in natural resource systems.

Public Administration and Development 19(3): 231–262.Biggs S, Smith S. 2003. A paradox of learning in project cycle management and the role of organisational culture. World Development 31(10):

1743–1757.Brown D. 2001. Evaluacion de Polıticas y Programas en la Region del Caribe de Habla Inglesa: Problemas Conceptuales y Practicos. Comi-

sion Economica para AMÉrica Latina y el Caribe (CEPAL-ILPES): Santiago, Chile.Camacho O, Cordero W, Martinez I, Rojas D. 2001. Tasa de Deforestacion del Departamento de Santa Cruz, Bolivia, 1993–2000. La Super-

intendencia Forestal and Proyecto BOLFOR: Santa Cruz, Bolivia.Carlsson J, Eriksson-Baaz M, Fallenius AM, Lovgren E. 1999. Are Evaluations Useful? Cases from Swedish Development Cooperation. Sida:

Stockholm, Sweden.

318 G. GORDILLO AND K. ANDERSSON

Copyright # 2004 John Wiley & Sons, Ltd. Public Admin. Dev. 24, 305–320 (2004)

Chelimsky E. 1995. New dimensions in evaluation. In Evaluation and Development: Proceedings of the 1994 World Bank Conference, PicciottoR, Rist RC (eds). World Bank Operations Evaluation Department: Washington, DC.

Contreras-Hermosilla A, Vargas TM. 2001. Dimensiones Sociales, Ambientales y Economicas de las Reformas en la Polıtica Forestal de Boli-via. Proyecto de Manejo Forestal Sostenible and the Center for International Forestry Research: Santa Cruz, Bolivia.

Cracknell BE. 2001. Knowing is all: or is it? Some reflections on why the acquisition of knowledge, focusing particularly on evaluation activ-ities, does not always lead to action. Public Administration and Development 21(5): 371–379.

Deininger K, Squire L. 1998. New ways of looking at old issues. Journal of Development Economics 57(2): 259–287.Food and Agriculture Organisation of the United Nations (FAO). 1992. Agreement FAO/MARA-UNDP. Author: Sao Paulo, Brazil.Gibson C. 1999. Politicians and Poachers: The Political Economy of Wildlife Policy in Africa. Cambridge University Press: New York.Gibson C, Lehoucq F. 2003. The local politics of decentralized environmental policy in Guatemala. Journal of Environment and Development12(1): 28–49.

Gordillo G. 1999. La ansiedad por concluir: la debil institucionalidad de las reformas estructurales en America Latina. In El Mercado de Valores.Nacional Financiera: Mexico.

Gordillo G, Andersson K (eds). 2004. El Desempeno de los Servicios Publicos Agropecuarios y el Rol del Gobierno Local. Food and AgricultureOrganisation Regional Office for Latin America and the Caribbean: Santiago, Chile.

Government of Bolivia. 1996. Ley Forestal, 1700. Ministerio de Desarrollo Sostenible y de Planificacion: La Paz, Bolivia.Guanziroli C, Romeiro A, Buainain AM, Di Sabbato A, Bittencourt G. 2001. Agricultura Familiar e Reforma Agraria no Seculo XXI.

Garamond: Rio de Janeiro, Brazil.Haggard S, Kaufman R. 1995. The Political Economy of Democratic Transitions. Princeton University Press: Princeton, NJ.Hobbes T. [1651] 1960. Leviathan or the Matter, Forme and Power of a Commonwealth Ecclesiastical and Civil, Oakeshott M. (ed. and intro.).

Basil Blackwell: Oxford.Irwin T, Fortin Y, Troja S. 1996. Performance Management in Government. Organization for Economic Co-operation and Development

(OECD): Paris, France.Kaimowitz D, Thiele G, Pacheco P. 1999. The effects of structural adjustment on deforestation and forest degradation in lowland Bolivia. WorldDevelopment 27(3): 505–520.

Karl TL. 2002. The vicious cycle of inequality in Latin America. Working Paper 2002/177. Center for Advanced Study in the Social Sciences,Juan March Institute: Madrid, Spain.

Kiser LL, Ostrom E. 1982. The three worlds of action: a metatheoretical synthesis of institutional approaches. In Strategies of Political Inquiry,Ostrom E (ed.). Sage: Beverly Hills, CA.

Krueger A. 1993. Political Economy of Policy Reform in Developing Countries. MIT Press: Cambridge, MA.Lawrence JES. 1989. Engaging recipients in development evaluation: the stakeholder approach. Evaluation Review 13(3): 243–256.Martens B, Mummert U, Murrell P, Seabright P. 2002. The Institutional Economics of Foreign Aid. Cambridge University Press: New York.McGinnis M (ed.). 1999. Polycentric Governance and Local Public Economies. Readings from the Workshop in Political Theory and PolicyAnalysis. University of Michigan Press: Ann Arbor.

McGinnis M (ed.). 2000. Polycentric Games and Institutions: Readings from the Workshop in Political Theory and Policy Analysis. University ofMichigan Press: Ann Arbor.

North D. 1990. Institutions, Institutional Change and Economic Performance. Cambridge University Press: New York.Oakerson RJ. 1992. Analyzing the commons: a framework. In Making the Commons Work: Theory, Practice and Policy, Bromley DE, Feeney D,

McKean MA et al. (eds). Institute for Contemporary Studies: Oakland, CA; 41–59.Organization of Economic Co-operation and Development. 1996. Agricultural Policies in OECD Countries. Volume I—Monitoring and Eva-

luation. OECD: Paris, France.Olson M. 1965. The Logic of Collective Action: Public Goods and the Theory of Groups. Harvard University Press: Cambridge, MA.Olson M. 1982. The Rise and Decline of Nations: Economic Growth, Stagflation and Social Rigidities. Yale University Press: New Haven, CT.Ostrom E. 1990. Governing the Commons: The Evolution of Institutions for Collective Action. Cambridge University Press: New York.Ostrom E, Gibson C, Shivakumar S, Andersson K. 2002. Aid, Incentives and Sustainable Development: An Institutional Analysis of Interna-tional Development Cooperation. Swedish International Development Cooperation Agency: Stockholm.

Ostrom V. 1997. The Meaning of Democracy and the Vulnerabilities of Democracies: A Response to Tocqueville’s Challenge. University ofMichigan Press: Ann Arbor.

Ostrom V, Ostrom E. 1977. A theory for institutional analysis of common pool problems. In Managing the Commons, Hardin G, Baden JWH(eds). Freeman: San Francisco.

Pacheco P. 2001. The Contribution of Forestry to Poverty Reduction: The Case of Bolivia. Food and Agriculture Organisation of the UnitedNations: Rome.

Pacheco P. 2002. Contribuye la decentralizacion a mejorar la gestion foretal? Las lecciones aprendidas del caso boliviano. In I Reunion Nacionalde Investigacion Forestal. La Superintendencia Forestal and Proyecto BOLFOR: Santa Cruz, Bolivia.

Pacheco P, Kaimowitz D (eds). 1998. Municipios y Gestion Forestal en el Tropico Boliviano. Centro de Estudios para el Desarrollo Laboral yAgrario, Taller de Iniciativas en Estudios Rurales y Reforma Agraria, and the Center for International Forestry Research: La Paz, Bolivia.

Patton MQ. 1997. Utilization-focused Evaluation. Sage Publications: Thousand Oaks, CA.Pawson R. 2002. Evidence-based policy: the promise of Realist Synthesis. Evaluation 8(3): 340–358.Picciotto R. 1995. Putting Institutional Economics to Work: From Participation to Governance. The World Bank: Washington, DC.Picciotto R. 2003. The logic of mainstreaming: a development evaluation perspective. Evaluation 8(3): 322–339.Picciotto R, Wiesner E (eds). 1998. Evaluation and Development. Transaction Publishers: New Brunswick, NJ.Ribot JC. 2001. Local Actors, Powers and Accountability in African Decentralisations: A Review of Issues. United Nations Research Institute

for Social Development: Geneva, Switzerland.Rossi PH, Freeman HE. 1993. Evaluation: A Systematic Approach. 5th ed. Sage Publications, Inc.: Newbury Park, CA.

FROM POLICY LESSONS TO POLICY ACTIONS 319

Copyright # 2004 John Wiley & Sons, Ltd. Public Admin. Dev. 24, 305–320 (2004)

Schneider BR, Heredia B (eds). 2003. Reinventing the Leviathan: Politics of Administrative Reform in Developing Countries. Lynne RiennerPublications: Boulder, CO.

Shah A. 1998. Balance, accountability, and responsiveness: lessons about decentralisation. In Evaluation & Development: The InstitutionalDimension, Picciotto R, Wiesner E (eds). Transaction Publishers: New Brunswick, NJ; 83–101.

Sokoloff KL, Engerman SL. 2000. Institutions, factor endowments, and paths of development in the New World. Journal of Economic Perspec-tives 14(3): 217–232.

Stiglitz JS. 1998. Evaluation as an incentive instrument. In Evaluation & Development: The Institutional Dimension, Picciotto R, Wiesner E(eds). Transaction Publishers: New Brunswick, NJ; 287–290.

Superintendencia Forestal. 2001. Informe Anual. Superintendencia General Sistema de Regulacion de Recursos Naturales Renovables: La Paz,Bolivia.

Taut S, Brauns D. 2003. Resistance to evaluation: a psychological perspective. Evaluation 9(3): 247–263.Tendler J. 1995. Good Government in the Tropics. Johns Hopkins University Press: Baltimore, MD.Tendler J, Freedheim S. 1994. Trust in a rent-seeking world: health and government transformed in northeastern Brazil. World Development

22(12): 1771–1791.Weimer DL, Vining AR. 1999. Policy Analysis: Concepts and Practice. 3rd ed. Prentice-Hall: Upper Saddle River, NJ.White H. 1999. Dollars, Dialogue and Development. Sida: Stockholm.World Bank Group. 1994. World Development Report 1994: Infrastructure for Development. Author: Washington, DC.World Bank Group. 1999. World Development Report 1998. Author: Washington, DC.

320 G. GORDILLO AND K. ANDERSSON

Copyright # 2004 John Wiley & Sons, Ltd. Public Admin. Dev. 24, 305–320 (2004)