Understanding the Impact of Accountability Reform on Public Employee Attitudes: The Case of No Child Left Behind Jason A. Grissom Harry S Truman School of Public Affairs University of Missouri Sean Nicholson‐Crotty Department of Political Science and Harry S Truman School of Public Affairs University of Missouri James R. Harrington Harry S Truman School of Public Affairs University of Missouri
35
Embed
Understanding the Impact of Accountability Reform on ...
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Understanding the Impact of Accountability Reform on Public
Employee Attitudes: The Case of No Child Left Behind
Jason A. Grissom
Harry S Truman School of Public Affairs
University of Missouri
Sean Nicholson‐Crotty
Department of Political Science and
Harry S Truman School of Public Affairs
University of Missouri
James R. Harrington
Harry S Truman School of Public Affairs
University of Missouri
1
Over the past three decades, reforms targeted at improving government performance,
accountability, transparency, and customer orientation—sometimes grouped under the moniker
of New Public Management (NPM)—have been implemented in public organizations delivering
all type of services across all levels of government. There is at times disagreement about the
exact reforms that make up NPM or about its continued relevance. Some suggest that
―performance management‖ may have survived as the dominant element in this basket of
reforms and taken its place as the central tenet of modern governance (Kettl and Kelman 2007).
Whatever the name, however, it is widely accepted that performance or results-oriented
management practices that stress employee accountability and borrow tools from the private
sector to incentivize production have come to dominate administrative reform (see Moynihan
2008).
The performance payoffs of these reforms have received considerable scholarly attention
and the findings have been, not unexpectedly, mixed (see for example, Thompson and Rainey
2003; Moynihan and Pandey 2005; Dubnick 2005; Fredrickson 2006). As these reforms have
taken hold in more organizations, and affected more employees, scholars have begun to expand
this exploration to the impact on the attitudes of public workers. While early proponents of NPM
style reforms suggested that they would increase satisfaction, empowerment, and commitment
among public employees (Barzelay 1992; Osborne and Gaebler 1992), the empirical evidence
has been a decidedly mixed bag. Some reforms have been shown to negatively impact public
employee attitudes (e.g., Korunga et al. 2003), while others have been found to be positively
related to satisfaction (e.g., Lee et al. 2006). In some cases, authors have found positive and
negative effects in the same study (Yang and Kassekert 2006). Given that the debate over the
utility and appropriateness of many of these reforms is ongoing, understanding their impact on
2
employee satisfaction and commitment is of significant import to scholars, managers, and
policymakers.
We hope to contribute to this enterprise in this project. We endeavor to show that a
prominent model of job stress from the private sector can be used, following some adaptations to
the unique values of public workers, to predict the impact that performance based reforms will
have on satisfaction among those employees. Specifically, we adapt the Demand, Control,
Support (DCS) model (Krasek and Theorell 1990), which has recently been applied to attitudes
in the public sector (Noblett and Rodwell 2009), to include job security, which has been shown
to be an important predictor of public employee attitudes (Lim 1996). We propose that the
impact of reform on more generalized attitudes like satisfaction should be a function of the sum
of its impacts on an important set of antecedents; an impact that we expect will be moderated by
the effectiveness of the public employee’s manager. We test the utility of that framework in an
examination of the impact of No Child Left Behind—arguably one of the most ambitious
performance based accountability reform ever implemented in this country—on teacher attitudes.
The results suggest that the adapted DCS model may offers a powerful tool for explaining, and
disentangling the components of, the often disparate impact of performance reforms on public
employee attitudes.
Literature on the Impact of Performance Reforms on Employee Attitudes
With the widespread adoption of private sector management practices in public
organizations, often grouped loosely under the names ―New Public Management‖ or
―Performance Based Accountability,‖ scholars have become increasingly interested in the impact
that these reforms have on public employees. This growing literature has investigated the impact
of managing for results, accountability standards, pay-banding, at-will employment and other
3
common NPM reforms on the stress and satisfaction of those working in the public sector.
Interestingly, however, this research has arrived at divergent conclusions regarding that impact.
One body of scholarship has suggested a negative relationship. Authors have found a link
between these reforms and levels of public employee stress (Korunka et al. 2003; Exworthy and
Halford 1999). Work has also suggested that emphases on efficiency and accountability decrease
employee satisfaction (Mikkelsen, Osgard, and Lovrich 2000) and that reforms which emphasize
extrinsic rather than intrinsic rewards may result in reduced levels of organizational commitment
(Young, Worchel, and Woehr 1998; Foster and Wilding 2000). Finally, New Public Management
reforms have been shown to erode professional values among public servants (Pollitt and
Bouckaert 2000; Powell, Brock, and Hinings 1999) and workplace trust (Battaglio and Condrey
2009).
Alternatively, another body of work suggests a positive, or at least more complicated,
relationship between NPM style reforms and employee attitudes in the public service. Yang and
Kaessekert (2006) find that contracting out and the erosion of civil service protections reduce
satisfaction, but that performance-based accountability, pay-for-performance, and
―innovativeness culture‖ can actually produce improvements in public employee satisfaction
(Yang and Kassekert 2006). Lee et al. (2006) find that pay-for-performance schemes are
regularly associated with increased satisfaction in the federal civil service in more than three
decades worth of surveys. Finally, Bertelli (2007) finds that performance-based incentives can
reduce stated turnover intention among some federal employees. Specifically, he finds that
supervisory-level employees who are subject to pay-banding respond positively (i.e., have lower
stated turnover intention) to timely performance incentives, though perceived accountability does
not influence turnover intention. Alternatively, among nonsupervisory personnel, being held
4
accountable for results significantly increases the likelihood that they will state an intention to
leave.
Understanding and Managing the Impact of Performance Reforms
The literature reviewed thus far suggests that performance or accountability focused
reforms, though often bundled together by both scholars and policymakers, can have differential
effects on public employees’ psychological health, job satisfaction, and turnover intention. This
section draws on and expands a theoretical model from the private sector management literature
that has recently been applied to public organizations in order to construct a framework that will
generate predictions about the direction of a reform’s impact on public employees.
In recent years, scholars have applied a number of models from the private management
and occupational health literatures in order to better understand the mechanisms by which
organizational reforms impact the attitudes of public employees (see for example Noblett and
Rowdell 2009). Among the most commonly applied of these has been the Demand-Control-
Support, or Job-Strain, model (Krasek 1979; Krasek and Theorell 1990). The model
hypothesizes an interactive relationship between the demands placed on an individual by her job,
the level of decision making authority that she feels she has, and the support that she receives
from supervisors and coworkers (see van der Doef & Maes 1999 for a review). It predicts high
strain, and the negative psychological consequences that accompany it (e.g. stress, low
satisfaction), when the demands of a job exceed the control and support necessary to meet those
demands. The model has received widespread support in research on private organizations and is
among the most commonly used theoretical approaches in occupational stress research (Fox et al.
1993).
5
The justification for applying this model in the public sector is typically the
organizational change wrought by New Public Management. Scholars suggest that the imposition
of external accountability standards and the new performance-oriented culture, along with the
dwindling or static resources that often accompany NPM reforms, have placed much greater
demands on, and intensified the work of, public employees (Korunga et al. 2003; Schafer and
Toy 1999). They also assume that, despite the rhetoric of decentralization, NPM reforms often
do little to increase the actual decision making authority of line employees and may actually
decrease control by giving more power to external stakeholders (Dixon, Kouzmin, and Korac-
Kakabadse 1998; Hood 1991). While these are reasonable assertions and provide an intuitive
justification for the application of the DCS to public organizations, there has been little empirical
evidence generated regarding the actual impact of NPM style reforms on demand, control, or
support.
Nonetheless, scholars have found some evidence for elements the DCS model in public
organizations that have undergone NPM-style reforms. Noblett et al. (2005) find that job control
and supervisor support have a significant impact on psychological health, satisfaction, and
organizational commitment in what they describe as a ―commercially-oriented‖ public
organization. Similarly, Noblett and Rowdell (2009) find that for police officers in a
metropolitan department that had undergone NPM-style reforms, demands, perceived control,
and support from supervisors influence intrinsic satisfaction, extrinsic satisfaction, and a more
generalized sense of well-being. They did not, however, find an interactive relationship between
these variables. In a more ―customer‖ oriented bureau that included numerous occupational
categories, they found that higher demands reduced extrinsic satisfaction and well-being, higher
control increased intrinsic and extrinsic satisfaction, and supervisor support improved all three.
6
Again, however, they did not find evidence that supervisor support moderates either demand or
control.
Before moving on the ways in which we believe the DCS model can be used to generate
predictions about the impact of performance and accountability reforms on public employee
well-being, we believe that a component needs to be added to that model. While it is reasonable
to expect—as have scholars applying the model—that task demands, sufficient decision
authority, and support from supervisors and coworkers will be key factors in determining public
sector employees’ psychological well-being, there are some additional dimensions that may
contribute to stress among these workers.
The most studied difference between public and private employees is public service
motivation. Research suggests that individuals who choose to go into the public workforce value
the production of public goods or the protection of the public interest to a greater degree than
those who select into private sector employment (Rainey 1982; Perry and Wise 1990; Brewer
2008). It is reasonable to expect then that task demands that are contrary to the public good will
create stress in a public employee who is unable to resolve the conflict. Indeed, the literature on
whistleblowers suggests that this tension is, in part, what motivates individuals to go outside of
traditional lines of authority to change the direction of their organizations (Brewer and Selden
1998). Systematic violations of employees’ public service ethic by the demands made on them
should be, we believe, relatively rare (though see O’Leary 2005) as government agencies are
most often dedicated to furthering the public good. Therefore, we do not treat violations of the
public service ethic as a potential source of public employee stress in our framework.
The other value that we know public sector workers elevate more than their private
counterparts is job security. A long line of research in economics suggests that, all else being
7
equal, those who choose to work for government are more risk adverse than those who select
private firms (see for example Bellante and Link 1981). While work in public affairs has
demonstrated that these differences do not always translate into risk tolerance at the
organizational level (Bozeman and Kingsley 1998) it has confirmed that, at the individual level,
public employees value job security considerably more than their private sector counterparts
(Houston 2000). Unlike the public service ethic, job security is something that can be
systematically threatened by changes to the organizations or systems in which public employees
work. Indeed, reforms enacted in states such as Georgia and Florida and under consideration in
others (e.g., Wisconsin, Kansas, and Ohio) have done so across the board by ending civil service
protections for new hires. Other reforms threaten security for individuals by downsizing
agencies, tying wages or advancement to performance, or allowing for the reorganization of
entire public organizations if they fail to meet performance targets.
Research on public organizations confirms that reduced jobs security is, in part,
responsible for the increase in job stress among public sector employees following the
widespread adoption of NPM-style reforms (VanWart and Berman 1999; Golembewski 1996).
Recent work also suggests that lack of security arising from downsizing within an individual’s
organization produces stress that can, under certain circumstances, impede reform efforts
(Kelman 2006). Thus, we suggest that, in addition to the task demands, the decision authority
necessary to meet those demands, and the level of social and supervisory support, the level of
perceived job security should be considered as a potentially important element in the production
of job stress for public employees.1
1 The actual use of demand, control, support, and security to predict employee stress would be challenging
because the theory suggests that these are interactive and a 4-way interaction is essentially impossible to interpret; though scholars do sometimes report primarily additive results from the model (Noblett and Rowdel 2009).
8
With this addition to the DCS model, we now turn to our framework for understanding
the diverse impacts of performance and accountability related reforms on public employee
satisfaction. If demand, control, social and supervisory support, and security are the component
elements of employee stress and satisfaction in the public service (see Noblett and Rowdel
2005), then the impact of any reform on these larger outcomes should be a function of the impact
of the reform on those component parts. We are particularly interested in the impact on demand,
control, social support, and security, because these are the variables that we believe are most
likely to be directly influenced by reform. We make this argument because what you are asked to
do as an employee will likely change as a result of the adoption of performance-oriented reforms
(see Korunga et al. 2003); the discretion you are afforded to accomplish those tasks may go up
(Osborne and Gaebler 1992) or become even more constrained (Brodkin 2011) depending on the
level to which decision-making is decentralized; scarce resources allocated according to
performance may erode relationships among coworkers who now view peers as competitors or
may bond them more tightly together as ―survivors‖ (Kellman 2006; Brockner et al. 2004); and,
finally, security is likely to be impacted by reforms that erode tenure or tie it to performance.
Alternatively, we view supervisor support as unlikely to be influenced by performance or
accountability reforms. The factors that influence whether an individual is an effective or
supportive leader are more likely to be individual characteristics such as empathy, skill, and
experience.
We do expect supervisor or manager behavior to moderate the impact of
performance/accountability reforms on demand, control, social support, and security. We make
this argument because of the considerable evidence that this moderating role is a key component
of what managers do. Working within the structure of the organization, they influence the ways
9
in which inputs influence employees and, ultimately, are converted by those employees into
outputs (Lynn et al. 1999). Even more germane to our investigation, they make decisions that
buffer the organization and its employees from environmental shocks and moderate the impact of
environmental changes on the attitudes of individuals within the organization (Fredrickson 1999;
Kam and Franzese 2007).
There is considerable anecdotal and descriptive evidence that some managers are better at
these tasks than others (Doig and Hargrove 1987; Holzer and Callahan 1998; Thompson and
Jones 1994). There is also mounting empirical evidence that higher quality managers have a
direct positive impact on employee behavior and organizational performance (Grissom 2011;
Meier and O’Toole 2002; Meier et al. 2007). Research is also beginning to demonstrate that
managerial effectiveness moderates the impact of other managerial activities on performance
(Hicklin et al. 2008; Grissom 2011).
Based on this research, we expect that more effective public managers will positively
moderate the impact of performance and accountability reforms on the task demands, decision
authority, social support, and job security of employees. Taking job control as an example, if a
reform, on average, decreases control it will do so less for employs who work for a highly
effective manager. Alternatively, if the reform increases decision authority then that increase will
grow even larger as managerial effectiveness increases. To put this another way, we expect that
outcomes for an employee affected by a reform will improve (i.e., lower demands, more control,
more social support, more security) as their manager becomes more effective.
Thus, we focus on task demands, decision authority, social support, and job security as
the key antecedents of job stress that will be influenced by performance and accountability
reforms and the interaction of those reforms with managerial effectiveness. We expect that the
10
sum of the impact of a reform across these factors explains its impact on more general employee
attitudes such as job satisfaction and organizational commitment. So, if a new performance
measurement system increases the reporting burden for the employees of an organization but is
not accompanied by discretion in time allocation to meet those new demands, does not produce
any solidary benefits among employees, and has no bearing on job security, then we would
expect a negative impact on satisfaction. If it increased reporting requirements, left discretion
and security unchanged, but fostered a cut-throat work environment among employees who
undercut one another to bolster their own performance reports, we would expect a larger
negative impact on satisfaction.
Alternatively, we might envision scenarios where performance rewarding schemes were
associated with improvements in satisfaction. These improvements might come about if security,
social support, and task demands stayed largely the same but managers now afforded employees
more discretion in accomplishing duties, so long as they met performance goals. If the new
performance culture also encouraged more team production, which increased solidarity among
previously isolated coworkers, then we would expect an even greater association between a
reform and the satisfaction of employees affected by it.
Finally, we believe that in the most likely outcome, reforms might improve one of the
dimensions discussed above while negatively affecting others. So, to take a reform that has been
widely adopted throughout the public sector, Managing for Results (MFR) might increase the
time employees must spend recording and reporting activities to superiors, but, it has been
suggested, will also reduce the red-tape and bureaucratic constraints that reduce employee
autonomy (Barzelay 1992). In this case, the impact of the reform on overall levels of satisfaction
or commitment will be a function of the relative impact that it has on these two subcomponents.
11
If the employee perceives MFR to increases demands only slightly but thinks that it has
significantly improved control, then the effect on satisfaction should be positive. If the size of
those impacts is reversed, then we would expect employee satisfaction to go down as a result of
the reform.
Examining the Impact of No Child Left Behind on Teacher Attitudes
We test the utility of the framework developed above in an analysis of No Child Left
Behind (NCLB) and its impact teacher attitudes. NCLB is arguably one of the largest
performance-based accountability reforms ever implemented in this country, affecting more than
16,000 public organizations. The legislation, which took effect in the 2002-2003 school year,
compelled states to set standards, conduct annual evaluations of student performance linked to
those standards, and to sanction schools that fail to make ―adequate yearly progress‖ toward
meeting them. The sanctions mandated by the law are tied to the continued receipt of Title I
funds and are relatively draconian. If a school fails to make AYP two years in a row, districts
must offer students in that school the opportunity to attend another school and pay the
transportation costs. If the school misses the mark 4 years in a row, it must make ―fundamental‖
staffing and structural changes to address the problem. After a sufficient number of failing years,
the management of the school can be handed over to a private company or the state or the
organization can be reorganized as a charter school.
Limited evidence is beginning to accumulate regarding the impact of these reforms on the
operation and performance of schools and the attitudes of teachers. Dee and Jacob (2010) find
that the policy increased per-student expenditures and the educational qualifications of teachers.
They also find that it caused a reallocation of teaching time toward tested subjects like reading
and away from unmeasured outcomes such as social studies and science. Dee and Jacob (2011)
12
demonstrate that the implementation of the policy has increased average 4th
grade math scores
and 8th
grade scores among traditionally low achieving students on the National Assessment of
Educational Progress. In interviews with teachers in three states, Hamilton et al. (2007) find that
teacher’s felt an increased sense of autonomy. Reback et al. (2011) find lower reported levels of
job security in schools that were close their state’s performance threshold and, thus, in the
greatest danger of failing to make AYP.
Because it sets clear performance standards, mandates meaningful standards for
organizations that fail to meet them, and has affected hundreds of thousands of line bureaucrats
across the country, NCLB implementation is an ideal arena in which to examine the utility of the
framework outlined above. We first use a cross-sectional time-series design to model task
demands, job control, social support, and job security before and after the implementation of
NCLB, allowing principal effectiveness to moderate the impact of the policy. We then take a
more sophisticated approach, modeling the differential impact of NCLB on demand, control,
support, and security in states that had no preexisting state-level accountability systems (treated)
and those that had such systems (untreated) and examine the moderating impact of principal
effectiveness on the policy’s impact in treated states. Finally, we use the results from these
models to predict the impact of NCLB on a more general measure of teacher satisfaction and use
a mediating variables analysis to show the degree to which that impact is actually a function of
its impact on demand, control, support, and security.
Data and Methods
For this study, we built a cross-sectional time series of data on teachers, principals, and
schools spanning four waves of the Schools and Staffing Survey (SASS). SASS is a nationally
representative survey of public school personnel collected approximately every four years. The
13
four waves we utilize were collected during the 1993-94, 1999-2000, 2003-04, and 2007-08
academic years, which means that we have data on two time points prior to the date that No
Child Left Behind took effect and two time points afterward. Throughout the remainder of the
paper we will refer to the survey years by year corresponding to the second year in the survey
wave (i.e., 1993-94 will be ―1994‖).
In selected SASS schools, survey data are collected from the principal and from multiple
randomly selected teachers on such topics as school organization, professional development, and
perceptions of the school climate. Demographic, experience, and educational background data
also are collected. Unique respondent identifiers make teacher responses linkable to their
principals and information on the schools in which they work. Pooling the data across years, we
utilize data on approximately 150,000 teachers. Survey weights are used in all analysis to
account for the complex sampling strategy SASS employs.
Dependent Variables. The primary constructs for which we aim to examine the impact of
No Child Left Behind are demand, control, job support, job security, and job satisfaction. We
measure each at the teacher level using items from the SASS teacher questionnaires. Our
measure of demand is total weekly hours worked, measured as a teacher’s estimate of how many
hours he or she works on all teaching-related duties during a typical week.2 As shown in Table 1,
the mean across years is approximately 49 hours per week.
To capture control, we make use of six items asked in each SASS wave that ask teachers
how much control they feel they exercise in their own classrooms over: selecting textbooks and
2 The questions concerning this variable vary somewhat across SASS waves. In 1994 and 2000, we created this total
from a composite of three questions which asked respondents how many hours they were required to work each week during school hours, how many hours they spend on student interactions outside of school, and how much other time they spent. In 2004 and 2008, they were simply asked to estimate their total hours worked in a typical week. We cannot rule out the possibility that differences in answers between the two sets of years are not due in part to differences in question wording.
14
materials; selecting content, topics and skills to be taught; selecting teaching techniques;
evaluating students; disciplining students; and determining the amount of homework to be
assigned. The scale for each item ranges from ―No control‖ to ―Complete control,‖3 though the
number of points in the scale varies across years. To equate the scales, we converted each one to
a three-point scale for no control, some control, and complete control. Polychoric factor analysis
on the converted items revealed one underlying control factor (Eigenvalue = 3.7), which
Cronbach’s α suggested to have a relatively high degree of reliability (α = 0.78). Factor scores
were used to assign a single control measure to each teacher and then standardized across
observations to facilitate interpretation.
Job support is captured using two items: ―Most of my colleagues share my beliefs and
values about what the central mission of the school should be,‖ and ―There is a great deal of
cooperative effort among staff members.‖ Teachers were asked to respond to each of these
statements using a 4-point Likert scale (strongly disagree, somewhat disagree, somewhat agree,
strongly disagree) each year. Factor testing revealed that these two measures could not be
reasonably combined into one scale,4 so we chose to model each variable separately.
Our measure of job security comes from teachers’ Likert scale responses to the item: ―I
worry about the security of my job because of the performance of my students on state or local
tests.‖ This item was not included on the 1994 survey. A higher value of this variable indicates
greater feelings of job insecurity.
Finally, we measure job satisfaction with the Likert response to: ―I am generally satisfied
with being a teacher at this school.‖ Teachers are quite satisfied in generally, averaging 3.47 of 4
points across years.
3 In 2004 and 2008, the range was “No control” to “A great deal of control.”
Standard errors clustered at the state level. * p<0.10, ** p<0.05, *** p<0.01. All models include teacher and school control variables, state fixed effects, and a
linear time trend.
32
TABLE 3: No Child Left Behind and Management Effects on Job Support
Dependent Variable: Teachers Share Beliefs about Mission Cooperative Effort among Staff