Measuring Edu Adequacy in Public Schools

8/9/2019 Measuring Edu Adequacy in Public Schools

1/35

MEASURING EDUCATIONAL ADEQUACY IN PUBLIC SCHOOLS

Lori L. Taylor

Texas A&M University

College Station, TX 77843

E-mail: [email protected]

Bruce D. Baker

University of Kansas

Lawrence, KS 66045


Arnold Vedlitz

Texas A&M University

College Station, TX 77843


Bush School Working Paper # 580

No part of the Bush School transmission may be copied, downloaded, stored, further transmitted, transferred, distributed,

altered, or otherwise used, in any form or by any means, except: (1) one stored copy for personal use, non-commercial

use, or (2) prior written consent. No alterations of the transmission or removal of copyright notices is permitted.


2/35

Measuring Educational Adequacy in Public Schools*

Lori L. Taylor

George Bush School of Government and Public ServiceTexas A&M University

4220 TAMUCollege Station, TX 77843

Phone: 979.458.3015Fax: 979.845.4155


Bruce D. BakerDepartment of Teaching and Leadership

School of Education

1122 West Campus RoadUniversity of KansasLawrence, KS 66045Phone: 785-864-9844Fax: 785-864-5076


Arnold VedlitzGeorge Bush School of Government and Public Service

Texas A&M University4220 TAMU

College Station, TX 77843Phone: 979.845.2929Fax: 979.862.8856


September 2005

* This research updates and extends a report commissioned by Texas Joint Select Committee for Public SchoolFinance.


3/35

2

Measuring Educational Adequacy in Public Schools

Abstract

An increasing emphasis on educational adequacy has altered the school finance debate

and the research that supports it. Where courts, legislators and litigants were previously guided

by analyses of educational equity, they are becoming increasingly reliant on studies intended to

measure the level of educational funding that can be deemed adequate. Our study systematically

examines the various methodologies used by states and interest groups to measure educational

adequacy, compares the estimates generated by the various studies, and discusses the strengths

and weaknesses of each approach.


4/35

3

Measuring Educational Adequacy in Public Schools

Federal, state and judicial initiatives have recently focused a policy spotlight on

educational adequacy. The No Child Left Behind Act of 2001 requires states to develop

standards for student performance, holds states accountable for adequate yearly progress towards

those standards, and implicitly obliges states to provide the necessary funding. Legislatures in

Arkansas, Kansas, New Hampshire, North Dakota, Texas and Wyoming have commissioned

studies of the level of funding required to meet such standards. State courts in Kansas, New

York, New Jersey, Texas and Wyoming have ordered the states to address deficiencies in not

only the distribution of school funds but also the absolute level of educational funding.

The increasing emphasis on educational adequacy has altered the school finance debate

and the research that supports it. Where they were previously guided by analyses of educational

equity, 1 courts and legislatures are increasingly asking how much does an adequate education

cost?The result is a growing reliance on adequacy studies. For example, the Wyoming court

has focused on critically evaluating the cost basis of that states new cost-based block grant

formula. The New Hampshire school finance formula rests on a study of the basic costs incurred

by successful schools. And, in decisions at least partially influenced by adequacy studies, state

courts in Kansas, New York, and Texas have all ordered large increases in state support for K-12

education.

No consensus has developed about the most appropriate strategy for conducting such

analyses, however. Numerous types of studies have been employed with highly varied results.

Our study systematically examines the various methodologies used by states and interest groups

to measure educational adequacy, compares the estimates generated by the various studies, and

discusses the strengths and weaknesses of each approach.


5/35

4

What is an Adequacy Study?

For purposes of this report, we define an adequacy study as a publicly-reported attempt

by state officials or special interest groups to apply an empirical methodology to estimate the

costs of providing an adequate public education at the elementary and/or secondary level.

Types of Adequacy Studies

Three major types of adequacy studies presently dominate the landscape. Those

categories include:

Average Expenditure Studies. Prior to the 1990s, notions of educational adequacy were

often guided by the average or median expenditures of districts in the prior year. A common

presumption was that median spending is adequate, and that states should strive to bring the

lower half of districts up to the median.2

With increased prevalence of state standards and assessments, consultants and

policymakers in the early 1990s turned their attention to the average expenditures of districts

meeting a prescribed set of outcome standards, rather than the simple average or median of all

districts. This approach was coined the Successful Schools Model.

Successful Schools studies use outcome data on measures such as attendance, dropout

rates, and student test scores to identify that set of schools or districts in a state that meet a

chosen standard of success. Then the average of the expenditures of those schools or districts is

considered adequate (on the assumption that some schools in the state are able to be successful

with that level of funding). Modified Successful Schools analyses include some consideration

of how schools use their resources. This is done in either of two ways. In most cases, analysts

may use data on how schools use their resources to identify and exclude peculiar, or outlier,

schools or districts from the Successful Schools sample. Alternatively, one might seek patterns in


6/35

5

resource allocation to identify those schools that allocate resources in such a way as to produce

particularly high outcomes, with particularly low expenditures.3

Resource Cost Studies. The Resource Cost Model (RCM) is a method that has been

used extensively for measuring the costs of providing educational services (Chambers 1999;

Hartman et al. 2001). The RCM methodology typically involves three steps: (1) identifying

and/or measuring the resources (people, space, time, and stuff) used in providing a particular set

of services; (2) estimating resource prices and price variations from school to school or district to

district; and (3) tabulating total costs of service delivery by totaling the resource quantities and

their prices. RCM has been used for calculating the cost of providing adequate educational

services since the early 1980s (Chambers and Parrish 1982; Chambers 1984).

Two relatively new (circa 1997) variants of RCM have been specifically tailored to

measure the costs of an adequate educationProfessional Judgment-driven RCM and

Evidence-Based RCM. The difference between them lies in the strategy for identifying the

resources required to provide an adequate education. In Professional Judgment studies, focus

groups of educators and policymakers are typically convened to prescribe the basket of

educational goods and services required for providing an adequate education. In Evidence-

Based studies, resource needs are derived from proven effective school reform models. Early

Evidence-Based studies focused on Comprehensive School Reform (CSR) models, such as

Robert Slavins Roots and Wings/Success for All model (Goertz and Malik 1999). More

recently, Evidence-Based analyses have strived to integrate a variety of proven effective input

strategies such as class size reduction, specific interventions for special student populations, and

comprehensive school reform models, rather than relying on a single reform model.


7/35

6

Because Evidence-Based strategies have been recently broadened to include and blend a

variety of reform strategies, we adopt the phrase Evidence-Based rather than cost of

comprehensive school reforms to describe the approach. We note, however, that this may lead

to a blurred distinction between Evidence-Based and Professional Judgment models. One might

assume, for example, that a panel of well-informed professionals would prescribe inputs for

schools based at least partly on the professionals knowledge of research literature on effective

reform strategies. The subtle distinction between this and Evidence-Based analysis is that

Evidence-Based analysis requires an empirical research basis for the recommended resource

configurations. Further, in Evidence-Based analysis, the recommendation is provided by

consultants conducting the cost study and does not typically include panels of experts from

schools and districts in the state.

Cost Function Studies. Cost Function analysis, which is a statistical method used to

measure the systematic relationship between actual expenditures and educational outcomes given

district and student characteristics, is becoming increasingly common among analyses of

educational adequacy. Cost Function analysis is a regression technique specifically designed to

measure the district-by-district differences in costs associated with geographic price variations,

economies of scale, and variations in student need. As such, it can be used not only to predict the

cost of achieving a desired level of outcomes in an average district, but also to generate a cost

index that indicates the relative cost of producing the desired outcomes in each district. For

example, it would likely be found that per pupil costs of achieving target outcomes are higher

than average in small, rural districts, that costs are higher in districts with high percentages of

economically disadvantaged and limited English proficient children, and that costs are higher

where competitive wages for teachers are higher.


8/35

7

As an alternative to regression-based methods, education cost functions may also be

estimated via numerical maximization algorithms such as Data Envelopment Analysis or any of

a variety of algorithms intended to fit non-linear multivariate models. Conceptually, the approach

is the same as when regression-based models are used. The goal is to estimate a model of the

relationships between schooling outcomes, exogenous conditions and historical spending such

that the model may then be used to predict costs of achieving different outcomes. Flexible, non-

linear estimation methods impose fewer restrictions on functional form and do not require that

the researcher specify a priori, the shape of the input-outcome relationship. However,

nonstochastic approaches like DEA can be particularly susceptible to errors arising from the

miss-measurement of inputs and outputs.

Placing the Methods on a Continuum

Adequacy study methods may be generally characterized as resource-oriented or

performance-oriented. This characterization is in part a function of the type of data

incorporated into the analyses. Resource-oriented analyses focus specifically on categories of

educational resource inputs, including numbers of teachers, classrooms of particular dimensions,

or computers and software required for implementing specific programs. Again, most such

studies prescribe resources toward the achievement of specifically identified sets of performance

outcomes. Performance-oriented studies, on the other hand, focus on measures of student

performance outcomes of interest to policymakers, and use either tabulation methods (Successful

Schools) or statistical models (Cost Function) to estimate the costs of achieving those

performance standards. Table 1 summarizes the previously discussed models and their variants


9/35

8

on a continuum from resource-oriented (top) to performance-oriented (bottom) analysis. Figure 1

places existing adequacy studies on that same continuum.

[Table 1 here]

[Figure 1 here]

Professional Judgment analyses where consideration is given only to identifying

resources required for providing particular educational programs, regardless of expected or

desired outcomes, might be considered pure resource-oriented analyses. Such analyses would be

unlikely in the present policy context. Most recent applications of Professional Judgment

analysis have included at least some discussion of the types of performance outcomes that should

result from providing a given set of inputs, most often drawing on outcomes specified in state

standards and accountability systems. Often, resource selection is guided by state curricular

standards promulgated by legislatures or boards of education on the assumption that particular

curricular offerings (core content standards) will lead to desired performance outcomes (often as

measured by standardized assessments on core content e.g. math, reading). Evidence-Based

analyses are resource-oriented similar to Professional Judgment methods in which professionals

are guided by the need to meet certain outcome standards. As with Professional Judgment

analyses, outcome data do not directly influence Evidence-Based analyses.

At the other end of the continuum are education Cost Function and Successful Schools

analyses, where performance outcome data drive the estimation of costs.4

These methods attempt

to estimate directly the costs or expenditures associated with schools and/or districts that achieve


10/35

9

specific educational outcomes. Cost Function analyses differ substantially from Successful

Schools analyses in that they involve much more empirically rigorous attempts to not only

determine what levels of present spending are associated, on average, with a specific set of

outcomes, but also how those levels of spending may vary for districts of different characteristics

serving different student populations.

Toward the middle of the continuum are hybrid methods like Modified Successful

Schools that involve analysis of both student outcomes and the expenditures required to achieve

those outcomes and of how schools and districts internally organize their resources.

Reconciling the Various Approaches

Since the various methodologies are aimed at the same targetidentifying the costs of an

adequate educationthey should lead to similar predictions about costs, all other things being

equal. Ideally, well-informed professionals advising districts on how to meet a specific

performance goal would prescribe the same mix of resources as would Evidence-Based

consultants, and that mix, when evaluated at market prices, would cost exactly as much as

predicted by a cost function.

Different cost estimates arise when all other things are not equal. The scope of

information required to conduct the analysis provides insight into the potential for divergent cost

estimates. Table 2 summarizes the data demands of the various methods. As the table illustrates,

the various methods have very different data needs.

[Table 2 here]

For obvious reasons, all of the performance-oriented methods require some measure of


11/35

10

student outcomes to be able to calculate costs. Professional Judgment and Evidence-Based

approaches have no such requirement. However, in Professional Judgment analysis, researchers

might ask professionals to keep a particular performance goal in mind when forming judgments.

Further, researchers may evaluate and share with professionals data on current performance of

schools and districts at current resource levels. Proponents of Evidence-Based analysis posit that

reform strategies that have produced positive results elsewhere on standardized outcome

measures are most likely to achieve the positive outcomes in the state in question on that states

desired outcome measures. As such, Evidence-Based analysis requires no direct measure of

outcomes within the state in question.

All of the methods, with the exception of the Successful Schools approach, require

information about input prices, particularly educator wages. Ideally, such information represents

price variations outside of school district control. Isolating uncontrollable variations in input

prices can be a major analytic challenge for any adequacy studies.5

Whereas all of the other methodologies require information on input quantities, Cost

Function and Successful Schools analyses require information on total expenditures. (Modified

Successful Schools analysis may require both.) As such, Cost Function analysis and Successful

Schools analysis tend to require less detailed financial data than other approaches. The obvious

trade-off is that these analytic techniques also offer less information about the optimal level of

input quantities.

How Do the Results Vary?

The growing track record on adequacy analysis provides us with increased opportunities

to compare the results of adequacy studies and assess whether certain patterns exist. Table 3

presents a comparative look, with adjusted dollar figures, at selected available state studies.6


12/35

11

Only publicly available, statewide studies that reported district-level cost estimates and were

sponsored by states or interest groups are included in the analysis for table 3. Studies produced

for an academic audience (such as Reschovsky and Imazekis Cost Function analysis of

Wisconsin, 1998), studies that produced cost estimates for only a subset of school districts in a

state (such as the MAP Professional Judgment analysis of Texas) and studies that produced only

school-level cost estimates (such as the Augenblick and Myers Professional Judgment analysis of

South Carolina schools) are excluded.

[Table 3 here]

When constructing table 3, we attempted to `make the findings as comparable as was

feasible. We adjusted dollar figures for year-to-year and state-to-state differences in the price

level using the recently developed National Center for Education Statistics Comparable Wage

Index (Taylor and Fowler 2005).

We focused wherever possible on costs associated with a scale efficient (optimally-sized)

school district and excluded wherever possible any incremental cost associated with special

student populations. In most recent Professional Judgment studies, basic costs were easily

identifiable and most often listed as the total of school and district level costs (before student

need adjustments) of a large prototype district. Basic costs, per se, are the costs of providing core

educational services, assuming no additional student needs. Studies where we were able to

identify basic costs are marked base in the fourth column of table 3.

For other studies, it was not possible to generate basic cost estimates. The basic cost

figures produced by successful schools analyses are typically the average costs, in districts with


13/35


14/35

13

York City in the calculation of average cost for New York State greatly raises the bottom line.

While state average costs for New York State were estimated at $14,083, New York City costs

per pupil were estimated at $14,768 and costs in a district of average needs at $10,753

(unadjusted estimates).

Not only are differences in district characteristics muddying the waters in table 3, but so

are differences in the definition of adequate. All other things being equal, one would expect that

states with higher expectations for there students will have higher estimates of the costs of an

adequate education. The analysis in table 3 cannot control for such differences.

Caveats aside, it is readily apparent in table 3 that studies employing Successful Schools

methods have produced the lowest estimates of the cost of an adequate education (after

adjustments for inflation and regionally price differences). Resource-oriented methods like

Professional-Judgment and Evidence-Based methods produced consistently higher results. Some,

though not all cost function analyses also produced high results. In any state with multiple cost

estimates, the Successful Schools estimate is lower than that generated by any other approach.

However, we stress again that the Successful Schools approach (which by construction uses a

performance standard that some schools already meet) may estimate the cost associated with a

lower performance standard than the one implicit or explicit in the other methodologies.

Differences across Methods

Table 4 summarizes findings of cost studies where the same researchers examined

alternative methods on the same state in the same year.

[Table 4 here]


15/35

14

In four cases, the firm of Augenblick, Myer and Associates of Denver, Colorado

conducted both Professional Judgment and Successful Schools analyses. In all four cases,

Successful Schools analyses produced much lower basic cost figures than Professional Judgment

analyses.

Differences within Methods

Table 5 summarizes cost findings from states where similar methods were performed by

the same and different researchers or policymakers. In cases where multiple estimates are

provided by the same researchers, results may vary for a variety of reasons. For example, several

recent cost function studies estimate costs of achieving different average outcome levels. In New

York State, Duncombe and colleagues provide alternative estimates of costs at performance

levels of 140, 150 and 160 on their composite, 200 point scale outcome index. Similarly,

Reschovsky and Imazeki and Gronberg et al. provide alternative estimates for 55 percent and 70

percent pass rates on the Texas Assessment of Knowledge and Skills, and Gronberg et al.

conducted the analysis at the 55 percent passing level twiceonce with 2002 data and again

with 2004 data.

Augenblick and Colleagues provide multiple cost estimates for Illinois based on different

outcome standards, using single or multiple years of data and including some or all outcome

standards. The higher of the two figures in Table 5 represents the average expenditures of Illinois

school districts which, using 1999-2000 data, had 83% of students meeting or exceeding the

standard for improvement over time. The lower of the two figures is based on the average

expenditure of districts which, using 2000 data only, had 67% of pupils meet or exceed the

standards, and 50% meeting standards on all tests.


16/35

15

[Table 5 here]

Similar issues exist in a series of successful schools cost estimates produced in Ohio a

year earlier. In Ohio, however, estimates were derived and proposed amidst the political process,

with various constituents picking and choosing their data years and outcome measures to yield

the desired result. Two Ohio estimates are provided in the table, but multiple estimates were

actually prepared based on different subsets of districts meeting different outcome standards. The

Governors office chose 43 districts meeting 20 of 27 1999 standards, the Senate selected 122

districts meeting 17 of 18 1996 standards, the House chose 45 districts meeting all 18 original

standards in 1999, and the House again in an amended bill used 127 districts meeting 17 of 18

1996 standards in 1996 and 20 of 27 standards in 1999.

While different outcome measures yield legitimate variations in cost estimates, other less

transparent features of the methodologies may also lead to different cost estimates. The two cost-

function analyses for Texas cover essentially the same school districts in a similar time frame.

However, the studies differ in a number of key respects (see Taylor 2004a) not the least of which

is that the plaintiffs estimate (Imazeki and Reschovsky) is the unweighted average of district

cost projections while the Legislatures estimates (Gronberg et al.) are pupil-weighted averages.

Because Texas has so many small districts with correspondingly high cost projections, this

difference in weighting alone can explain all of the difference in estimated average cost at the 55

percent passing performance standard.7

Findings of reported Professional Judgment and Evidence-Based analyses are often less

directly comparable. In Maryland, for example, the states consultants and special interest

consultants dealt differently with costs associated with special education. MAP, Inc. prepared


17/35

16

estimates for the New Maryland Education Coalition, and included special education costs in all

estimates. The two different estimates provided by MAP represent the low and high estimates of

different professional judgment teams but based on the same outcome objectives. While the

states own professional judgment estimates prepared by Augenblick and Colleagues was lower,

the legislature eventually chose to adopt (for five year phase in) the even lower finding from the

Successful Schools analysis.

Differences across States

Because there are a small number of researchers performing adequacy studies, it is also

possible to compare studies where the same researchers applied the same methods to different

states. Two research teamsAugenblick and Associates and Picus and Associateshave

conducted adequacy studies in multiple states.8

Augenblick and Associates conducted either a

Professional Judgment analysis or a Successful Schools analysis (or both) in more than a dozen

states, while Picus and Associates conducted Evidence-Based analyses for Kentucky and

Arkansas.

The Evidence-Based analyses generate reasonably similar cost projections. After

adjustments for differences in teacher compensation costs between the two states, the two

Evidence-Based cost estimates differ by less than 8 percent.

The Professional Judgment and Successful Schools studies have a much greater range,

but also reflect potentially large differences across states in the definition of adequate. (Recall

that the Evidence-Based approach rests on the consultants definition of adequate.) Basic costs,

which are the costs of providing core educational services in a scale efficient district assuming

no additional student needs, range from $6,921 in Tennessee to $9,259 in neighboring Missouri.


18/35

17

However, it is possible that the 34 percent difference in adjusted basic cost simply reflects higher

educational standards in Missouri.

More striking are the variations across states regarding the impact of school size on the

cost of an adequate education. The consulting teams Successful Schools analyses do not

attempt any adjustment for economies of scale, but their Professional Judgment analyses do

incorporate such adjustments. In each state, the researchers estimated costs of three to five

prototypical districts of varied size, assuming linear changes in costs between the prototypes.

These attempts have produced widely varied results, even in contiguous states. They found that

costs were minimized in districts with 12,500 students (Nebraska), 11,300 students (Kansas),

8,000 students (Tennessee), 5,200 students (Colorado), 4,380 students (Missouri), 1,740 students

(Montana) and 750 students (North Dakota). In Nebraska, a district with 400 pupils had costs

40% above the minimum, but in Missouri, a district with 364 pupils, had costs only nine percent

above the minimum.

Strengths and Weaknesses of the Various Methodologies

As we have demonstrated, there are many analytic approaches to answering the critical

question, What level of public funding is needed to provide an adequate public education? All

of the approaches have strengths and weaknesses in giving decision makers the definitive

information they need to set appropriate funding levels.

Resource-Oriented Strengths

In the policy context, the primary strength of resource-oriented methods, like

Professional Judgment or Evidence-Based analyses, is that the methods are relatively simple and

transparent and produce easily understood results. That is, resource-oriented models appear not


19/35

18

to involve more complex statistical modeling. Of course, well-designed resource-oriented models

require researchers to use statistical modeling to determine market prices for educational inputs,9

and professionals frequently rely on statistical analysis to form their opinions.

Because achieving consensus regarding desired educational outcomes can be difficult and

precise measurement of those outcomes even more complicated, one advantage of resource-

oriented analyses is that they avoid these complexities altogether. Professional Judgment

approaches can also incorporate outcomes that are difficult to measure, while outcome-based

analyses can only estimate the costs associated with measurable outcomes.

Resource-Oriented Weaknesses

In an era of increasing emphasis on educational standards and accountability, it can be

difficult to justify a cost figure for an adequate education, where that cost figure is, at best,

indirectly linked to student outcomes.

Furthermore, analyses that rest on the judgment of a panel of professionals are vulnerable

to the blind spots and biases of individual panel members. If the panel is poorly drawn or

unaware of cost effective educational practices, their cost estimates will be biased.

While proponents of Evidence-Based analysis infer a strong connection between specific

comprehensive school reforms and improved outcomes, research evidence regarding the

effectiveness and more specifically the cost effectiveness of these reforms is mixed at best

(Bifulco et al. 2002; Borman and Hewes 2002; Levin 2002; Borman et al. 2003). Furthermore,

there may be little connection between the outcomes such reform models are proven to

accomplish and the outcomes policymakers hope to achieve.

For practical reasons, resource-oriented analyses rely on a limited set of prototypical

districts, which can lead to problems when actual school districts differ from the prototypes. For


20/35

19

example, it can be difficult to estimate the costs of operating a district with 600 pupils, when

prototypes have been estimated with 200 pupils and 1000 pupils. Similar issues exist in the

accommodation of student needs, where only a limited range of possibilities may be feasibly

represented in the prototypes. The greater the difference between the prototypes and the actual

schools, the greater the margin for error. It can be particularly problematic to estimate costs

when the actual schools differ from the prototypes in more than one dimension, as would occur

when schools were smaller and served more disadvantaged students than the most similar

prototype. Even apparently subtle differences in applying the prototypes to the real world (such

as choosing to interpolate between prototypes linearly instead of nonlinearly) can lead to

significantly different cost estimates.10

Resource-oriented analyses frequently prescribe sharp increases in resource utilization,

but tend to presume that implementing such changes will have no effect on resource prices. If the

increase in demand resulting from the new intensity requirement drives up the price of inputs,

then the total cost predictions from the analysis will be greatly understated.

In summary, to use an analogy, with resource-oriented analysis, you know the mode of

transportation youre going to take, but youre not sure exactly where youre going.

Performance-Oriented Strengths and Weaknesses

The primary strength of performance-oriented models is that they establish a direct link

between education costs and desired outcomes. Understanding the link between costs and

outcomes and designing aid formulas based on this understanding is arguably a critical objective

in an era of increased emphasis on standards and accountability.

Cost Function analysis has the added strength that it is specifically designed to measure

the district-by-district differences in costs associated with the geographic price variations,


21/35

20

economies of scale, and variations in student need. As such, it provides an empirical basis not

only for the basic level of spending, but also for various cost adjustments that must be applied to

that base.

A central difficulty of performance-oriented analysis involves the politics of achieving

consensus regarding important outcomes and the empirics of precisely measuring those

outcomes. Many outcomes that policy-makers consider important may be too difficult to

measure, and that which is measured well may be a biased representation of that which we hope

to achieve. The Cost Function approach is data intensive, requiring high quality measures of

school district performance and expenditures. Many states lack the necessary data to conduct

such analyses. For example, Maryland does not collect detailed data on school expenditures.

Thus, although the state of Maryland was able to identify 104 schools that it considered to be

successful, researchers conducted a Successful Schools analysis on a narrower sample of less

than 60 schools on the grounds that it would be difficult to obtain fiscal data from the full 104

within the time available. Cost Function analyses on the basis of such a small sample would be

problematic.

A difficulty with more complex statistical methods like education Cost Functions is that

both the underlying methodologies and eventual outcomes of those methodologies can be

difficult to understand and difficult to communicate to constituents. The underlying

methodologies rest on theoretical and analytical assumptions with which informed parties may

disagree.

Statistical modeling inherently involves errors of estimation. While other methodologies

are also vulnerable to error and bias, there can be political resistance to methodologies that reveal

the inherent imprecision of social science.


22/35

21

By design, statistical models describe relationships within the experience of the data. It is

problematic to extrapolate beyond that experience to predict the costs associated with a level of

performance that is not regularly achieved, or is not achieved by districts with a particular set of

geographic and demographic characteristics.

While performance-oriented methods like Cost Function analyses estimate a statistical

relationship between spending and outcomes, they do not provide specific insights into how

districts should internally organize their resources to effectively and efficiently produce

outcomes.

In summary, again, with performance-oriented analysis, you know where youre going

and how much money it should take to get there, but youre not quite sure of the best way to go.

Conclusions

Over the last decade, educational adequacy studies have been conducted in many states.

Such studies can be grouped into three broad categories: average expenditure studies, resource

cost studies, and cost function studies. No one approach dominates the others from either a

theoretical or a practical perspective. All have been used by policy-makers, litigants, interest

groups and public managers to influence state education policy.

The lack of consensus regarding the appropriate research methodology would be largely

moot if all of the methods yielded similar predictions about the cost of an adequate education.

The reality is that such congruence of cost estimates is not the case. The choice of research

method has considerable influence on the nature of the predictions. Adjusted for inflation and

regional price variationsbut not for differences in the definition of adequatethe estimated

per-pupil cost of an adequate education for the same state but using different adequacy measures


23/35

22

can range from $6,612 to $10,945 (in 2004 dollars). Furthermore, even when two research

methods yield roughly similar estimates of the cost of an adequate education in the baseline or

average school district in a given state, different methods can yield strikingly different estimates

of the variations in costs associated with district characteristics, like size and student

demographics.

Given the sensitivity of the cost projections to variations in research methodology and

study specifics, states must be sensitive to the methods used in their specific case and the

assumptions underlying that method. If the method chosen affects the amounts identified, states

must be sensitive to study choices and the application of specific approaches, and their

interpretation, in their particular context. One obvious policy response could be to conduct

several studies with different methods. In such a comparative context, researchers should be

forced to demonstrate the robustness of their cost projections to alternative modeling

assumptions, and the relevance of their particular assumptions to specific state contexts.

Policymakers and the courts should evaluate educational adequacy from a variety of

perspectives, with clear enunciation of assumptions and their relevance to a specific states

education, social and economic environment. No single adequacy study, without such

considerations, should move billions of dollars.


24/35

23

Notes

1. For more on educational equity, see Johnston and Duncombe (1999), Moser and Rubinstein,

(2002), Evans et al. (1999), or Wyckoff (1992).

2. For example, a commonly used index of school finance equity/adequacy is the McLoone

index, which compares the average expenditures of the lower 50 percent of children with the

median expenditures. A perfect McLoone index is equal to 1.0, or a situation where no

children fall below the median (50 percent are at the median).

3. For example, early successful schools analyses in Ohio used data on district resource

allocation as a partial basis for modifying the sample of districts to be used for calculating

average costs of achieving standards.

4. An appendix detailing the output measures used in Cost Function and Successful Schools

analyses is available from the authors.

5. For more on input price estimation, see Taylor 2004b.

6. An appendix listing the studies reviewed is available from the authors.

7. Unweighted, the Gronberg et al. estimate at the 55 percent pass rate is $7,375 using 2004

data, and $7,304 using 2002 data (both in adjusted $2004). We thank them for making these

calculations for us.

8. The North Dakota study was formally attributed to Augenblick, Palaich and Associates.

9. It is important to note that one critical phase in well developed resource cost modeling is the

setting of competitive market prices for educational resources and the estimation of how

those prices vary from one district to another in a state. This phase is best performed via

statistical modeling not too unlike Cost Function modeling in its statistical complexity. See

Chambers (1999).


25/35

24

10. In Kansas, for example, differences in aid resulting from applying linear segments between

Augenblick and Myers prototypes and applying a curved expenditure function of similar

high-low range exceed 10 percent across some ranges.


26/35

25

References

Bifulco, Robert, Carolyn Bordeaux, William Duncombe and John Yinger. 2002. Do Whole

School Reform Programs Boost Student Performance? The Case of New York City.

Smith-Richardson Foundation.

Borman, Geoffery D. and Gina Hewes. 2002. The Long-Term Effects and Cost Effectiveness of

Success for All. Educational Evaluation and Policy Analysis 24(4): 24366.

Borman, Geoffery D., Gina Hewes, Laura Overman and Shelly Brown. 2003. Comprehensive

School Reform and Achievement: A Meta-Analysis. Review of Educational Research

73(2): 125-230.

Chambers, Jay G. 1984. The Development of a Program Cost Model and Cost-of-Education

Model for the State of Alaska. Volume II: Technical Report. Associates for Education

Finance and Planning, Inc.

1999. Measuring Resources in Education: From Accounting to the Resource Cost Model

Approach. Working Paper Series, National Center for Education Statistics, Office of

Educational Research and Improvement. Washington, DC: U.S. Department of

Education. Working Paper #1999-16.

Chambers, Jay G. and Thomas B. Parrish. 1982. The Development of a Resource Cost Model

Funding Base for Education Finance in Illinois. Report prepared for the Illinois State

Board of Education.

Evans, William N., Sheila E. Murray and Robert M. Schwab. 1999. The Impact of Court-

Mandated School Finance Reform. In Helen F. Ladd, Rosemary Chalk and Janet Hansen

(eds) Equity and Adequacy in Education Finance: Issues and Perspectives. Committee


27/35

26

on Education Finance, Commission on Behavioral and Social Sciences and Education,

National Research Council.

Goertz, Margaret E. and Edwards Malik 1999. In Search of Excellence for All: The Courts and

New Jersey School Finance Reform. Journal of Education Finance 25 (1): 5-31.

Hartman, William T., Denny Bolton and David H. Monk. 2001. A Synthesis of Two Approaches

to School-Level Financial Data: The Accounting and Resource Cost Model Approaches.

In Selected Papers in School Finance, 200001, edited by W. Fowler. National Center for

Education Statistics, Office of Educational Research and Improvement. Washington, DC:

U.S. Department of Education.

Haveman, M. (2004) Determining the Cost of An Adequate Education in Minnesota.

Implications for the Minnesota Education Finance System. Minnesota Center for Public

Finance Research, Minnesota Taxpayers Association. St. Paul, MN.

Johnston, Jocelyn and William Duncombe. 1998. Balancing Conflicting Policy Objectives: The

Case of School Finance Reform. Public Administration Review 58: 145-155.

Levin, Henry M. 2002. The Cost Effectiveness of Whole School Reforms. Urban Diversity

Series No. 114. Eric Clearinghouse on Urban Education. Institute for Urban and Minority

Education.

Moser, Michele and Ross Rubinstein. 2002. The Equality of Public School District Funding in

the United States: A National Status Report, Public Administration Review, 62 (1): 63-

72, January-February.

Reschovsky, Andrew and Jennifer Imazeki. 1998. The Development of School Finance Formulas

to Guarantee the Provision of Adequate Education to Low-Income Students. In

Developments in School Finance, 1997 (NCES 98-212), edited by William J. Fowler, Jr.,


28/35

27

Washington, DC: U.S. Department of Education, National Center for Education

Statistics.

2001. Achieving Educational Adequacy through School Finance Reform. Journal of

Education Finance 26(4): 373-96.

Taylor, Lori L. 2004a. Estimating the Cost of Education in Texas. Prepared for the Texas Office

of the Attorney General.

2004b. Adjusting for Geographic Variations in Teacher Compensation: Updating the

Texas Cost-of-Education Index. Prepared for Texas Joint Select Committee on Public

School Finance.

Taylor, Lori L. and William J. Fowler, Jr. 2005. A Comparable Wage Approach to a Geographic

Cost Adjustment in Education NCES 2005862. Washington, D.C.: National Center for

Education Statistics.

Wyckoff, James H. 1992. The Intrastate Equality of Public Primary and Secondary Education

Resources in the U.S., 1980-1987. Economics of Education Review 11(1): 19-30.


29/35

28

Table 1. Types of Adequacy Analyses

Model Research Question Methodology

Professional

Judgment

What is the total cost ofproviding students with the

"basket of educational goodsand services" determined to be"adequate" [for achievingspecified outcomes] by apanel of educational experts?

Tabulation of resource quanities(and qualities) and calculation of

total cost of purchasing thoseresources at competitive marketprices

ResourceOriented

Evidence

Based

Professional

Judgment

Is present funding adequate(and/or how much more isneeded) for high poverty andlow performing schools toimplementRoots andWings/Success for All or other

Comprehensive SchoolReforms or combinations ofproven effective strategies(Class Size Reduction)?

Tabulation of resource quantitiesrequired for implementingspecific reform strategies in highpoverty schools

BlendedMethods Modified

(Resource

Analysis)

Successful

Schools

What resource quantities andqualities exist in successfulschools? How much would itcost for other schools to havesimilar resources, orreorganize their resources tobe more similar?

Tabulation of resource quantitiesand qualities of successfulschools and estimation of thecosts of having similar resourcesin other schools

Cost Function What is the cost of achieving atarget set of outcomes, in thedistrict of averagecharacteristics serving thepopulation of averagecharacteristics? How does thecost of achieving that set ofoutcomes vary by district andstudent characteristics?

Formal modeling to determinethe relationship between districtspending and student outcomes,while accounting for factorswithin and outside the control oflocal officials (economies ofscale, competitive wages, studentneeds). Simulation using costfunction to estimate the "cost ofachieving specified outcomes" indistricts with varied

characteristics, serving variedstudent populations.Per

formanceOriented

Successful

Schools

How much do schools thatmeet specific outcome criteriapresently spend?

Calculation of the weighted (byenrollment) average spending perpupil of districts meetingoutcome criteria


30/35

29

Table 2. Data Demands of Various Models

Model OutcomesInput

QuantitiesInputPrices

Expenditures

Professional

Judgment X X

Resourc

e

Oriented

Evidence Based

Professional

Judgment

X X

Modified (Resource

Analysis) Successful

Schools

X X X X

Successful Schools X X

Performance

Oriented

Cost Function X X X


31/35


32/35

State Authors Cost MethodEstimate

Type(a)

Includes

Disabilitie

Tennessee Augenblick and Colleagues Professional Judgment Base

Texas (55%) Reschovsky & Imazeki Cost Function State Mean YES

Texas (55%) Gronberg, Jansen, Taylor & Booker Cost Function State Mean YES Texas (55%) Gronberg, Jansen, Taylor & Booker Cost Function State Mean YES

Texas (70%) Reschovsky & Imazeki Cost Function State Mean YES

Texas (70%) Gronberg, Jansen, Taylor & Booker Cost Function State Mean YES

Washington Ranier Institute Professional Judgment State Mean YES

Wisconsin Institute for Wisconsin's Future Professional Judgment Base

(a) Base = cost of basic programs, assuming 0% additional student needs; Low = average spending for target outcneed districts; Mean = cost of target outcomes in district of state average student and district needs/costs.

(b) We use a Comparable Wage Index to adjust for both inflation and regional variations in labor costs. The estimand Fowler (2005).


33/35

Table 4. Comparison of Findings from Alternative Models where Analyses Were

Performed by the Same Consultants (in adjusted $2004)

State Professional Judgment Successful Schools% Difference

Colorado $7,504 $5,124 46%Kansas $7,577 $5,929 28%Maryland $7,325 $6,612 11%Missouri $9,259 $6,696 38%


34/35

33

Table 5. Comparison of Findings from Similar Models where Analyses were Performed by

Different Consultants or for Different Clients (Adjusted $2004)

Authors EstimateType IncludesDisabilities EstimateYear Estimate

Regionally

&InflationAdjusted

Successful Schools

IllinoisAugenblick and Colleagues Low 2000 $5,594 $6,360

Augenblick and Colleagues Low 2000 $5,965 $6,782

New York

Standard & Poors State Mean YES 2004 $13,420 $12,007

Standard & Poors State Mean YES 2004 $12,679 $11,344

Ohio

Legislature Low 1999 $5,560 $7,093

Legislature Low 1999 $4,446 $5,672Augenblick and Colleagues Low 1996 $3,930 $5,624

Professional Judgment

Maryland

Augenblick and Colleagues Base 2000 $6,612 $7,325

Management, Planning & Analysis, Inc. State Mean YES 1999 $9,313 $10,945

Management, Planning & Analysis, Inc. State Mean YES 1999 $7,461 $8,769

Cost Function

New York

Duncombe & Colleagues (Syracuse U.) State Mean 2004 $14,107 $12,622

Duncombe & Colleagues (Syracuse U.) [140] State Mean 2000 $14,083 $14,563

Duncombe & Colleagues (Syracuse U.) [150] State Mean 2000 $14,716 $15,218

Duncombe & Colleagues (Syracuse U.) [160] State Mean 2000 $15,139 $15,655Texas

Reschovsky & Imazeki [55%] State Mean YES 2002 $6.949 $7,352

Gronberg, Jansen, Taylor & Booker [55%] State Mean YES 2002 $5,950 $6,295


Reschovsky & Imazeki [70%] State Mean YES 2002 $9,787 $10,355



35/35

Figure 1. Continuum of Education Cost Analysis Methods, Adequacy Studies 1995-2004


Wyoming 1997 (s)

Oregon 2000

South Carolina 2000 (s)Maryland 2001(2)

Kansas 2002Nebraska 2003

Indiana 2002Wisconsin 2002Colorado 2003Missouri 2003

Montana 2003

Kentucky 2003North Dakota 2003Washington 2003New York 2004Tennessee 2004

Vermont 2004 (s)


Wyoming 1997 (s)Oregon 2000

South Carolina 2000 (s)Maryland 2001(2)

Kansas 2002Nebraska 2003

Indiana 2002Wisconsin 2002

Colorado 2003Missouri 2003Montana 2003Kentucky 2003

North Dakota 2003

Washington 2003New York 2004

Tennessee 2004Vermont 2004 (s)

Successful Schools

Mississippi 1993(s)Ohio 1999

Maryland 2001Kansas 2002

Louisiana 2001 (s)

Colorado 2003Missouri 2003

New York 2004Vermont 2004 (s)

Successful Schools

Mississippi 1993(s)

Ohio 1999Maryland 2001

Kansas 2002Louisiana 2001 (s)

Colorado 2003Missouri 2003

New York 2004Vermont 2004 (s)

Modified

Successful Schools

Ohio 1999

New Hampshire 1998 (s)Illinois 2001

New York 2004

Modified

Successful Schools

Ohio 1999New Hampshire 1998 (s)

Illinois 2001New York 2004

Evidence Based

New Jersey 1998 (s)Kentucky 2003Arkansas 2003

------------------pending

Wyoming2005/06

Kansas

2005/06

Evidence Based

New Jersey 1998 (s)Kentucky 2003

Arkansas 2003

------------------

pending

Wyoming

2005/06Kansas2005/06

Resource Oriented

Performance Oriented

Cost

Function

New York2004

Texas2004, 2005

Minnesota

2004-------------

pending

Kansas2005/ 06

Cost

Function

New York2004

Texas2004, 2005Minnesota

2004-------------pending

Kansas

2005/ 06

(s) School level costs estimated (not included in Table 3)Includes only statewide cost analyses.

Measuring Edu Adequacy in Public Schools

Documents