Electronic copy available at: https://ssrn.com/abstract=3087370 * * * DRAFT * * * 1 The Problem of Unquantified Benefits Amy Sinden Abstract Many of the objections raised against the use of cost-benefit analysis (CBA) to evaluate government regulation, especially in the environmental context, center around the difficulties involved in quantifying and monetizing regulatory benefits. These difficulties implicate deep theoretical issues that have spawned a massive literature spanning many decades. But the difficulties posed by quantification also raise a straightforward empirical question that has been largely ignored: How often and to what extent does the problem of unquantified benefits actually arise in the practice of CBA? This article presents methods and results of an empirical study aimed at this question. The study examined the CBA’s prepared by EPA in connection with 32 major final rules issued between 2002 and 2012. In 84 percent of the CBAs analyzed, EPA excluded categories of benefits that the agency itself described as either actually or potentially “important,” “significant,” “substantial” because they were unquantifiable due to data limitations. In order to understand the implications of these findings for the debate about CBA more generally, this article lays out an analytic framework for understanding the role that quantification plays in CBA, detailing how significant unquantified benefits constrain the kind of CBA that can be performed, precluding more formal varieties. These results suggest that in developing environmental rules, agencies are rarely ever able to legitimately conduct formal CBA of the sort called for in the relevant executive orders and guidance documents, and that even the more informal varieties of CBA they can conduct will produce only limited conclusions at best. This suggests that the connection between CBA and its normative foundations in efficiency or welfare is even more tenuous than most of its defenders have assumed, and bolsters the case for alternative tools, like feasibility and health-based criteria, that set standards based on the information we have rather than the information we wish we had.
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Electronic copy available at: https://ssrn.com/abstract=3087370
***DRAFT***
1
TheProblemofUnquantifiedBenefits
AmySinden
Abstract
Manyoftheobjectionsraisedagainsttheuseofcost-benefit
analysis(CBA)toevaluategovernmentregulation,especiallyintheenvironmental context, center around the difficulties involved inquantifying andmonetizing regulatory benefits. These difficultiesimplicate deep theoretical issues that have spawned a massiveliterature spanning many decades. But the difficulties posed byquantificationalsoraiseastraightforwardempiricalquestionthathasbeen largely ignored: Howoftenand towhatextentdoes theproblem of unquantified benefits actually arise in the practice ofCBA? This article presents methods and results of an empiricalstudy aimed at this question. The study examined the CBA’sprepared by EPA in connection with 32 major final rules issuedbetween2002and2012. In84percentoftheCBAsanalyzed,EPAexcluded categories of benefits that the agency itself described aseither actually or potentially “important,” “significant,”“substantial” because they were unquantifiable due to datalimitations.
Inordertounderstandtheimplicationsofthesefindingsforthe debate about CBA more generally, this article lays out ananalytic framework forunderstanding therole thatquantificationplays in CBA, detailing how significant unquantified benefitsconstrainthekindofCBAthatcanbeperformed,precludingmoreformal varieties. These results suggest that in developingenvironmental rules, agencies are rarely ever able to legitimatelyconduct formalCBAofthesortcalledfor intherelevantexecutiveordersandguidancedocuments, and that even themore informalvarieties of CBA they can conduct will produce only limitedconclusionsatbest.ThissuggeststhattheconnectionbetweenCBAanditsnormativefoundationsinefficiencyorwelfareisevenmoretenuousthanmostof itsdefendershaveassumed,andbolstersthecase foralternativetools, like feasibilityandhealth-basedcriteria,that set standards based on the informationwe have rather thantheinformationwewishwehad.
Electronic copy available at: https://ssrn.com/abstract=3087370
***DRAFT***
2
TheProblemofUnquantifiedBenefits
AmySinden∗
[Cost-benefit analysis] minimizes decisioncosts through the magic of quantification.Once valuations are obtained from themarketplace and surveys . . . decisions arerelativelyautomatic.
JonathanMasur&EricPosner,AgainstFeasibilityAnalysis,77U.CHI.L.REV.657,700(2010)
When important benefits and costs cannotbe expressed in monetary units, [cost-benefit analysis] is less useful, and it canevenbemisleading, because the calculationof net benefits in such cases does notprovide a full evaluation of all relevantbenefitsandcosts.
OFFICEOFMGMT.&BUDGET,CIRCULARA-4, TOTHEHEADSOFEXECUTIVEAGENCIESAND ESTABLISHMENTS: REGULATORYANALYSIS10(2003)
It’s a simple idea. Before issuing regulations, thegovernmentshouldfirstaddupallthesocialcostsandthesocialbenefits and compare them.1 But the devil is in the details.Drawingmeaningfulconclusionsfromacomparisonofcostsandbenefits is difficult—and sometimes maybe impossible—unlessyou can quantify both sides in a commonmetric. If costs are
∗ThankstoDanCole,DavidDriesen,SanneKnudsen,MichaelLivermore,GregMandel, Catherine O’Neill, Arden Rowell, Sid Shapiro, and ToddWildermuth,andparticipantsinfacultycolloquiaattheUniversityofWashingtonSchoolofLawandTempleLaw forhelpful comments on earlier drafts. Special thanksalsotomysupervisingRAs,MaryJones,MattKaslow,andShanaMiles.
1See,e.g.,RICHARDL.REVESZ&MICHAELA.LIVERMORE,RETAKINGRATIONALITY:HOW COST-BENEFIT ANALYSIS CAN BETTER PROTECT THE ENVIRONMENT AND OURHEALTH13–16(2008);CASSR.SUNSTEIN,THECOST-BENEFITSTATE:THEFUTUREOFREGULATORYPROTECTION19–20(2002).
***DRAFT***
3
measured in dollars, then the best way to accomplish acomparisonistomeasurethebenefitsindollarsaswell.
Andthere’stherub.Whileregulatorycoststendtoinvolvevalues that are relatively easy to measure and express inmonetary terms—the cost of installing a scrubber on asmokestack, for example—regulatory benefits tend to involvethings that are hard to quantify, and even harder tomonetize.2They include things like effects on human health, prematuredeath,degradationofecosystems,extinctionofspecies,andsoon.Andifcostsarecompletely(orrelativelycompletely)monetized,but benefits only partially so, then drawing any meaningfulconclusionfromacomparisonbecomesproblematic. Thisishardlyanewinsight.Indeed,mostofthecriticismsraisedbythosewhoareskepticalofcost-benefitanalysis(CBA)inagencyrulemakingrelateinsomewaytothedifficultiesposedbythe quantification andmonetization of regulatory benefits. Thelist of reasons that benefits may be left unquantified or under-countedinCBAislong.Andmanyofthesereasonsimplicatedeeptheoreticalandnormativeissuesthathavespawnedanextensiveliterature over many decades.3 But the difficulties posed byquantificationalsoraiseastraightforwardempiricalquestionthathasbeenlargelyignored:Howoftenandtowhatextentdoesthe
2 While costs are generally far more amenable to quantified (and
monetized) estimation, the simplicity of quantifying costs should not beoverstated. TheCBAliteraturehasrightlybeencriticizedforpayingtoolittleattention to thedifficultiesattendant tocostestimation. SeeAdamM.Finkel,TheCostofNothingTrumpstheValueofEverything: TheFailureofRegulatoryEconomicstoKeepPacewithImprovementsinQuantitativeRiskAnalysis,4MICH.J.ENVTL.&ADMIN.L.91(2014);seealsoWhitman,531U.S.457,492-93(2001)(Breyer,J.,concurring)(notingthatcostestimatesarenecessarily“speculative,fortheyincludethecostofunknownfuturetechnologies”).
3DOUGLASA.KYSAR,REGULATINGFROMNOWHERE:ENVIRONMENTALLAWANDTHESEARCH FOR OBJECTIVITY 104 (2010); FRANK ACKERMAN & LISA HEINZERLING,PRICELESS: ON KNOWING THE PRICE OF EVERYTHING AND THE VALUE OF NOTHING(2004); SIDNEY A. SHAPIRO & ROBERT L. GLICKSMAN, RISK REGULATION AT RISK:RESTORING A PRAGMATIC APPROACH 32 (2003); ELIZABETH ANDERSON, VALUE INETHICSANDECONOMICS 55–59 (1993);MARKSAGOFF,THEECONOMYOFTHEEARTH:PHILOSOPHY, LAW, AND THE ENVIRONMENT 1–7 (1988); Duncan Kennedy, Cost-BenefitAnalysisofEntitlementProblems:ACritique, 33STAN.L.REV.387,401–07(1981);RobertDorfman,FortyYearsofCost-BenefitAnalysis,inECONOMETRICCONTRIBUTIONSTOPUBLICPOLICY268,268(Richard Stone andWilliamPetersoneds.1978);ARTHURSMITHIES,THEBUDGETARYPROCESSINTHEUNITEDSTATES344–46(1955).
***DRAFT***
4
problemofunquantifiedbenefitsactuallyarise inthepracticeofCBA?4 Asking that empirical question also brings into focus amore prosaic problem that is frequently mentioned but rarelyanalyzedinanydepth—theproblemofinsufficientdata.5Puttingaside the perhaps more intellectually exciting problems ofincommensurability, endowmenteffects,wealtheffects,discountrates,andsoon,benefitsaresometimes(perhapsquiteoften)leftunquantified andunder-quantified inCBA for the simple reasonthattherelevantdatadon’texist.6
CBAskepticsalmostalways list themissingdataproblemin an initial catalogue of the shortcomings of CBA, but thenusuallymoveontotackledeepertheoreticalissues.ProponentsofCBA often acknowledge the problem also, but then shrug andmoveonasthoughitdoesn’treallymatter,or is,perhaps,trivialenough to be safely ignored. But when we tackle the empiricalquestionofthefrequencyandmagnitudeofunquantifiedbenefitsintherealworld,itturnsoutthatthemissingdataproblemlooms
4See, e.g., Robert W. Hahn et al., Assessing Regulatory Impact Analyses:
TheFailureofAgenciestoComplywithExecutiveOrder12,866, 23HARV.J.L.&PUB. POL’Y 859, 869-70 (2000) (“Determining whether the benefits thatagencies chose not to quantify represent a significant portion of the totalbenefits was beyond the scope of this analysis, although it is an importantissue.”).
5ButseeJonathanMasur&EricPosner,UnquantifiedBenefitsandtheProblemofRegulationunderUncertainty,102CORNELLL.REV.87(2016)(reportingresultsofempiricalstudyofCBAsoffederalregulationsinwhichforover74%oftheregulationstheagency“couldnotquantifyalloftherelevantbenefitsorcostsbecauseofempiricaluncertainty—missingdata,modelingdifficulties,orotherrelatedeffects.”).
6 See THOMAS O. MCGARITY, REINVENTING RATIONALITY: THE ROLE OFREGULATORYANALYSISINTHEFEDERALBUREAUCRACY134(1991)(“Inadequatedata,inaccuratemodels, and the infirmities of quantitative analysis all combine toleaveregulatoryanalysisswimminginaseaofuncertainties.”);AlMcGartland,Richard Revesz, et al., Estimating the Health Benefits of EnvironmentalRegulations:ChangesNeededforCompleteBenefitsAssessment,357SCIENCE457(Aug. 4, 2017); Ronnie Levin, Lead in Drinking Water, in RICHARD D.MORGENSTERN, ECONOMIC ANALYSES AT EPA: ASSESSING REGULATORY IMPACT 230(1997)(“Seriousgaps indataandmethodologyconstrain theutilityof [CBA].Typically,onlya fewpotentialhealthorotherbenefitscanbequantified,andeven fewer can be valued monetarily. Consequently, when the sum of thelimitedsubsetofbenefitsthatcanbequantifiedandmonetizedisshowntobelessthantheestimatedcosts,itisoftenimpossibletoconcludeanythingaboutthe relative magnitude of the full benefits.”); David Driesen, Is Cost-BenefitAnalysisNeutral?,77U.Colo.L.Rev.335,371-77(2006)(inempiricalstudyofOIRA review of 25 rules, finding numerous instances in which significantbenefitswereleftunquantified).
***DRAFT***
5
large and, as I argue below, calls into question the intellectualfoundationsofCBAitself. All of this matters, particularly now. The TrumpAdministration has declared war on the regulatory state.7 Aseries of executive orders have promised to reduce regulatoryburdens, and the President has pledged to undo a litany ofObama-eraregulationsaimedatprotectingpublicsafetyandtheenvironment—rulesonclimatechange,8highwaysafety,9workerprotections, 10 wetlands preservation, 11 and a host of otherpressing issues.12 In this war, cost-benefit analysis (CBA) willplay a central role,13 as it has since an earlier icon of anti-
7PhilipRucker&RobertCosta,BannonVowsaDailyFightforDeconstructionoftheAdministrativeState,WASH.POST(Feb.23,2017),https://www.washingtonpost.com/politics/top-wh-strategist-vows-a-daily-fight-for-deconstruction-of-the-administrative-state/2017/02/23/03f6b8da-f9ea-11e6-bf01-d47f8cf9b643_story.html?utm_term=.eb658d117255.
8SeeExec.OrderNo.13,783,PromotingEnergyIndependenceandEconomicGrowth,82Fed.Reg.16,093(Mar.31,2017).
9AlanLevin,Killer-TruckFixHitsRoadblockinTrump’sQuesttoCutRules,BLOOMBERGNEWS,July5,2017,https://www.bloomberg.com/news/articles/2017-07-05/killer-truck-fix-hits-roadblock-in-trump-s-quest-to-cut-rules.
10BarryMeier&DanielleIvory,UnderTrump,WorkerProtectionsAreViewedwithNewSkepticism,N.Y.TIMES(June5,2017),https://www.nytimes.com/2017/06/05/business/under-trump-worker-protections-are-viewed-with-new-skepticism.html.
11SeeExec.OrderNo.13,778,RestoringtheRuleofLaw,Federalism,andEconomicGrowthbyReviewingthe“WatersoftheUnitedStates”Rule,82Fed.Reg.12,497(Mar.3,2017).
12SeeExec.OrderNo.13,771,ReducingRegulationandControllingRegulatoryCosts,82Fed.Reg.9,339(Feb.3,2017);Exec.OrderNo.13,777,EnforcingtheRegulatoryReformAgenda,82Fed.Reg.12,285(Mar.1,2017).
13Forexample,indefendingitsproposaltorepealoneparticularlycontentiousObama-eraruleprotectingwetlands,Trump’sEPAhasalreadyputCBAfrontandcenter,issuinganewCBAthatreducestheObamaadministration’sbenefitsestimate85to90percentbycategorizingpreviouslyquantifiedbenefitsasunquantifiable.TheresultisthatthemonetizedbenefitsoftheObamarulenolongeroutweighthecosts,thuspavingthewayforrepeal.SeeArielWittenberg,TrumpAnalysisSlashesWOTUS’sEconomicBenefits,E&ENEWS,July7,2017,https://www.eenews.net/stories/1060057053. Trump’smostnotoriousexecutiveorderon“regulatoryreform,”E.O.13,771(alsoknownastheone-in-two-outorder)insomewaysrepresentsaradicaldeparturefromtheCBAapproachthathasdominated“regulatoryreform”effortsfordecades.TheTrumporderattemptstoimposeonagenciesacost-onlyregulatorybudgetthatcapstheamountofregulatorycostseachagencycanimposethroughregulationwithnoreferencetoregulatorybenefits.Thisapproachhasbeenroundlycriticizedbyeconomistsandotheracademicsandpolicymakerscoveringawideswathofthepoliticalspectrum.SeeAriannaSkibell,95ScholarsUrgeTrumptoRevamp‘Misguided’2-for-1Order,E&ENEWS,May24,2017,https://www.eenews.net/greenwire/stories/1060055082.But
***DRAFT***
6
regulatoryzeal,RonaldReagan,firstimposedaCBArequirementon federal agencies nearly four decades ago.14 Since then, CBAhas been embraced by both Democratic and Republicanadministrations,butinacademicandpolicycircles,itcontinuestospark fierce debate: Is it a valuable technocratic tool thatharnesses “themagicofquantification” tomeaningfullyevaluatethequalityanddesirabilityofregulations,orasmokescreenthatcloaksagarbage-in-garbage-outanalysis inaveneerofscientificobjectivity? Tackling the question of unquantified benefitsempirically, it turns out, begins to shed new light on thesequestions. So how big is the problem of unquantified benefits?Anecdotal evidence suggests that it may be significant. 15whileitisantitheticaltoandarguablyincompatiblewithCBA,theTrumpAdministrationhasnonethelessrepeatedlyinsistedthatitwillcontinuetorespectandenforcethepre-existingCBAmandate.DominicJ.Mancini,ActingAdministrator,OfficeofInformationandRegulatoryAffairs,GuidanceImplementingExecutiveOrder13771(April5,2017),availableat:https://www.whitehouse.gov/the-press-office/2017/04/05/memorandum-implementing-executive-order-13771-titled-reducing-regulation(“E.O.12,866remainstheprimarygoverningE.O.regardingregulatoryplanningandreview.Accordingly...agenciesmustcontinuetoassessandconsiderboththebenefitsandcostsofregulatoryactions...andissueregulationsonlyuponareasoneddeterminationthatbenefitsjustifycosts.”)Accordingly,astheaboveexamplesillustrate,CBAcontinuestoplayacentralroleinpublicdebatesaboutregulatorydecisionmaking.
14Exec.OrderNo.12,291§2(b),3C.F.R.127,128(1982).SeegenerallyRICHARDL.REVESZ&MICHAELA.LIVERMORE,RETAKINGRATIONALITY:HOWCOST-BENEFITANALYSISCANBETTERPROTECTTHEENVIRONMENTANDOURHEALTH24-29(2008).
15 See Mark L. Plummer, Welcome to the Data-Poor Real World:IncorporatingBenefit-CostPrinciples intoEnvironmentalPolicymaking, 23RES.IN L. & ECON. 103, 127 (2007) (“As is often the case for other forms ofenvironmentalpolicymaking,dataonthemonetarybenefitsofcriticalhabitatdesignationforWestCoastsalmonandsteelheadwereinsufficienttopursueastandardbenefit-costapproach.”).SeealsoANDREARENDA,IMPACTASSESSMENTINTHEEU:THESTATEOFTHEARTANDTHEARTOFTHESTATE 62 (2006) (Reporting,basedonascorecardanalysisofthefirst70CBAscompletedbytheEuropeanCommission, that “some” benefitsweremonetized in only 28%of CBAs, and“allornearlyall”benefitswerequantifiedinonly14%);StuartShapiro&JohnF. Morrall III, The Triumph of Regulatory Politics: Benefit-Cost Analysis andPoliticalSalience,6REG.&GOVERNANCE1,5(2012)(inanempiricalstudyof109CBAsbyfederalagencies,“many”didnotmonetizeallthebenefits”);REVESZ&LIVERMORE, supra note 1 at 11 (identifying “ignored benefits” as one of the“substantivebiases” inCBA thatmake it lessprotectiveof the environment);DanielH.Cole,Law,Politics,andCost-BenefitAnalysis,64ALA.L.REV.55,61n.31 (2012)(“As a practical matter, non-market environmental goods are stillfrequentlyassignedavalueofzerobecausemanyagencyCBAs,includingthoseoftheEPA,excludethemoredifficulttoevaluateEnvironmentalbenefits . .of
***DRAFT***
7
Numerous case studies of individual CBAs show large andsignificant aspects of benefits that are left uncounted. CassSunstein, forexample, found that in itsCBAon theregulationofarsenicindrinkingwater,EPAleftunquantifiedtheeffectsoffiveof the seven different kinds of cancer associated with arsenic,alongwith a host of other health effects, including “pulmonary,cardiovascular, immunological, neurological, and endocrineeffects.”16The CBA accompanying EPA’s 2011 mercury and airtoxics rule for powerplantsmonetized only onenarrowhumanhealth endpoint: IQ losses suffered by children exposed tomercury in utero when their mothers ate fish, leaving outnumerous other impacts, including other IQ losses, otherneurological effects, potential cardiovascular, genotoxic, andimmunotoxic effects, all ecological effects, and all other toxicsbesidesmercury.17And even for that one narrow endpoint theydid include (consumption of fish by pregnant women) theyexcluded all exposures from commercially caught fish and fromfish caught in non-U.S. waters. EPA’s CBA of its rule governingcoolingwater intakesatpowerplantswasroundlycriticized forleavingunquantifiedthesubstantialecologicaleffectsof theruleaswellasitsimpactsonnumerousimportantaquaticspecies,andfor counting the impacts on just two percent of the fishpopulationsitdidinclude.18regulatoryproposals.”);RichardParker,GradingtheGovernment,70U.Chi.L.Rev. 1345, 1383 (2003)(noting that many CBAs either don’t quantify anybenefitsof fail toquantify“wholecategoriesof importantbenefits”);Masur&Posner,supranote5,at12-15(inempiricalstudyof106agencyCBAs,only2fullyquantifiedbothcostsandbenefits,36didnotquantifyanybenefits,and48partiallyquantifiedcostsandbenefits).
16Cass R. Sunstein,TheArithmetic ofArsenic, 90 GEORGETOWNL. J. 2255,2272-74 (2002); Thomas O. McGarity, Cass Sunstein’s Fuzzy Math, 90GEORGETOWNL.J.2341(2002).
17EPA,Regulatory ImpactAnalysis for theFinalMercuryandAirToxicsStandards, EPA-452/R-11-011 (Dec. 2011); David A. Evans, The Clean AirMercury Rule, in REFORMING REGULATORY IMPACT ANALYSIS 82, 99 (WinstonHarringtonetal.eds.2009);CatherineA.O’Neill,TheMathematicsofMercury,in REFORMING REGULATORY IMPACT ANALYSIS 108, 112, 118-19 (WinstonHarringtonetal.eds.2009)(callingtheCBAofEPA’s2005CleanAirMercuryRule“acompletecost-incompletebenefitanalysis.”).
18See Amy Sinden, Cost-Benefit Analysis, Ben Franklin, and the SupremeCourt,4U.C.IRVINEL.REV.1175(2014). TheEPAexpressedconcernfromtheoutsetofthisrulemakingthatCBAsundertheCleanWaterActhavegenerallybeen“limitedintherangeofbenefitsassessed,”thus“hinder[ing]EPA’sabilityto compare... benefits and costs... comprehensively.” National PollutantDischarge Elimination System—Proposed Regulations to EstablishRequirements for Cooling Water Intake Structures at Phase II ExistingFacilities, 67 Fed. Reg. 17,191 (proposed Apr. 9, 2002); see also Control of
***DRAFT***
8
In order to tackle the question of unquantified benefitsmoresystematically,Iconductedanempiricalstudyofasetof32CBAsconductedbyEPAoverarecentten-yearperiod.IchosetofocusonEPAbecauseitistheagencythatisusuallyheldupasthegoldstandard foragencyconductofCBA. Thedataset includedtheCBAsconductedbyEPAinconnectionwitheachofthemajorrules(ruleswitheffectsontheeconomyof$100millionormore)issuedbetween2002and2012.
While this empirical project has embedded within it aparadox—it seeks to measure what is, by some definitionsimmeasurable—Iwasnonethelessabletouncoversomeevidenceas to the magnitude of the benefits left unquantified in theseCBAs. In 27 out of 32 CBAs analyzed (84%), EPA excludedcategories of benefits that the agency itself described as“important,” “significant,” or “substantial” because they wereunquantifiableduetodatalimitations.
Indeed, in certain instances, the monetized benefitsestimateleftouttheveryharmatwhichtherulewasaimed.Sixofthe rules were specifically aimed at reducing emissions ofhazardousairpollutantsandyettheCBAsfailedtomonetizeanyof the benefits of reducing hazardous air pollutants. All of themonetized benefits came instead from the salutary fact thatemissions controls aimed at reducing hazardous air pollutantsalso happen to produce the ancillary benefit of reducing adifferentpollutant:particulatematter.
While admittedly preliminary, this data suggest that theproblemofunquantifiedbenefitsisabigonethatdeservesmoreattentionthanithasreceived. Oneconsequence, forexample,of
EmissionsofAirPollutionfromLocomotiveEnginesandMarineCompression-IgnitionEnginesLessThan30LitersperCylinder;Republication,73Fed.Reg.37,182 (proposed Apr. 3, 2007)(“Limitations of the scientific literature oftenresult in the inability to estimate quantitative changes in health andenvironmental effects, such as potential increases in premature mortalityassociated with increased exposure to carbon monoxide. Deficiencies in theeconomics literature often result in the inability to assign economic valueseventothosehealthandenvironmentaloutcomeswhichcanbequantified.”). Oneofthegreatestpublichealthtriumphsofalltime,EPA’sphase-outof lead in gasoline, might never have happened had EPA been required toproduceaquantifiedCBAtodefenditsrule.Ironically,thedatathateventuallyallowed epidemiologists to quantify the dramatic impact that spewing leadfrommillions of car tailpipes had on children’s healthwere only possible toproduceoncethephase-outwasalreadywellunderway,creating,inessence,areal-world control group. See Frank Ackerman, Lisa Heinzerling, & RachelMassey,ApplyingCost-Benefit toPastDecisions:WasEnvironmentalProtectionEveraGoodIdea?,57ADMIN.L.REV.155,160-172(2005).
***DRAFT***
9
significantbenefitsremainingunquantified is that formalCBAofthe sort called for in the executive orders and guidancememosgoverningagencyCBAisimpossible. Ratherthanidentifyingtheefficient level of regulation, the analyst can draw only limitedconclusions. Accordingly, theseresultssuggest that formalCBAis even further unmoored from its foundations in welfareeconomicsandKaldor-Hicksefficiencythanmostofitsdefendershaveassumed.
At least in the environmental area, there are otherstandards—feasibility and health-based standards, in particular—with long track records in agency practice that don’t requirecomprehensivemonetizationofregulatorybenefits. Whilethesehavebeencriticizedforbeinginsufficientlygroundedinefficiencyand welfare economics, if CBA’s own grounding in efficiency isitself called intoquestion, then itno longerhas that leg-upoveralternativetools.Andperhapsthesealternativesdeserveasecondlook.Thereissomesense,afterall,insettingstandardsbasedonthe information youhave, rather than the information youwishyouhad.
Thispaperproceedsinthreeparts.PartIbeginsbylayingout an analytic framework for understanding the role thatquantificationplaysinCBA.Attheoutsetitisimportanttoclarifythat a variety of methods are often lumped together under theumbrellaterm,“cost-benefitanalysis,”notallofwhichrequirethequantification or monetization of benefits. These range from avery informal comparison of pros and cons described in onlyqualitative terms, to a highly technical economic analysis thatidentifies the point of equivalence between marginal costs andbenefitsandrequirescompletemonetization.19Ontheformalendofthisspectrum,economicCBAfindsitsnormativegroundingineconomictheoryandinparticulartheideaofefficiency,orsocialwelfare maximization. This framework sets the stage forunderstanding the role that quantification and monetization ofbenefits plays in various forms of CBA and the constraints thatsignificant unquantified benefits place on the kind of CBA thatthatagenciescanmeaningfullyperform.
Part I goes on to examine the lawof CBA—primarily theexecutive orders and guidance documents that govern federalagencyconductofCBA—inordertoexplorethelegalconstraintsthesedocumentsimposewithrespecttothelevelofformalityand
19This part draws on ideas exploredmore fully inmy previous article,
Formality and Informality in Cost-BenefitAnalysis.Amy Sinden, Formality andInformalityinCost-BenefitAnalysis,2015UTAH.L.REV.93.
***DRAFT***
10
quantificationexpectedorrequiredofagencies.Itturnsoutthatformal CBA, grounded in economic theory, is held up as theexpectation and the norm. Part I concludes by reviewing theexisting literature, which, while frequently acknowledging theproblemofunquantifiedbenefits,hasgivenitverylittlesustainedattention.
PartIIexplainsthemethodsusedandtheresultsobtainedin the empirical study, including both the overall quantitativeresults as well as qualitative descriptions of some of theindividual CBAs underlying those results. Finally, Part III bringstheempiricalresultsandtheconceptualframework(PartsIIandI)togethertoarguethattheproblemofunquantifiedbenefitsisasignificant one that raises fundamental concerns about thesoundness of CBA’s normative foundations. The results of theempirical study suggest that in developing environmental rules,agenciesarerarelyeverable to legitimatelyconduct formalCBAof the sort called for in the executive orders, and that even themore informal varieties of CBA they can conduct will produceonly limited conclusions at best. This suggests that theconnection between CBA and its normative foundations inefficiency or welfare is even more tenuous than most of itsdefendershaveassumed, and that alternatives toCBA thathavebeen criticized for not closely tracking efficiency deserve asecond,lessjaundicedlook.
I.BACKGROUND ThisPartreviewsfirst(inI.A)thevarietyofformsthatCBAcantakeonaspectrumfrominformaltoformal,anddescribesthecommongroundinganddefenseofformalCBAinwelfareeconomicsaswellastheprocessesbywhichregulatorybenefitsaretypicallyquantifiedandmonetized.PartI.BthenexaminesthelegalconstraintsimposedonagenciesbytheexecutiveordersandguidancedocumentsgoverningtheCBArequirement.Finally,PartI.CexaminesthesmallamountofpreviousliteraturethathastakenanempiricalapproachtoquantificationinCBA.
A. FormalandInformalCBA
Theterm“cost-benefitanalysis”canrefertoanydecision-makingtoolthat involvesweighingandcomparingthecostsand
***DRAFT***
11
the benefits of a course of action.20 There are many differentvarietiesofCBAthat fallonaspectrumfrominformal to formal.Andnotallofthemrequirethequantificationormonetizationofbenefits (or costs). In previous work I have explored thisspectruminsomedepth,developingatypologyofthevarietiesofCBAandconsideringsomeoftheimplicationsofthattypologyfortheongoingdebateabouttheuseofCBAinagencyrulemaking.21Here, it will be sufficient to merely touch on some of thehighlightsofthatanalysis.
On the informal end of that spectrum is what I havepreviously called “BenFranklinCBA,”which involves essentiallydrawing a line down the center of a page, listing pros and consqualitatively described in each column, and then performing anadhoc,intuitivecomparison.OntheotherendofthespectrumiswhatIcall“EconomicCBA,”ahighlytechnicalandformalanalyticmethod grounded in economic theory that attempts to fullyquantify and monetize all of the social costs and benefits of awhole range of regulatory options and then, by calculating thepointatwhichthemarginalbenefitscurveintersectsthemarginalcosts curve, identifies the economically efficient level ofregulation.
1. TheNormativeGroundingofFormalCBAinWelfare
Economics
IrefertotheCBAonthemostformalendofthespectrumas “EconomicCBA”preciselybecause this formofCBA isalmostalwaysnormativelyanchoredinwelfareeconomics.Thatis,itismost often defended as a good decisionmakingmethod on thegrounds that it identifies the economically efficient level ofregulation—in the Kaldor-Hicks sense. 22 A governmentregulationmeets thecriterionofKaldor-Hicksefficiency if those
20Richard A. Merrill, Risk-Benefit Decisionmaking by the Food and DrugAdministration, 45 GEO.WASH.L.REV.994, 996 (1977) (“Risk-benefit analysisincludesanytechniqueformakingchoicesthatexplicitlyorimplicitlyattemptstomeasurethepotentialadverseconsequencesofanactivityandtopredictitsbenefits.”). Cf. Steven Kelman, Cost-Benefit Analysis: An Ethical Critique,REGULATION, Jan./Feb. 1981, at 33 (“At the broadest and vaguest level, cost-benefitanalysismayberegardedsimplyassystematicthinkingaboutdecision-making.”).
21SeeSinden,supranote19.22See ANTHONY E. BOARDMAN ET AL., COST-BENEFIT ANALYSIS: CONCEPTS AND
PRACTICE 32 (4th ed. 2011); E.J.MISHAN&EUSTONQUAH,COST-BENEFITANALYSIS(5th ed. 2007); RICHARD A. POSNER, ECONOMIC ANALYSIS OF LAW 17-20 (8th ed.2011).
***DRAFT***
12
whostandtobenefit fromtheregulationcouldfullycompensatethosewhostandtolosefromitandstillbebetteroff.23
Economic CBA aims to identify the regulatory alternativethatisoptimallyefficientintheKaldor-Hickssensebyfindingtheregulatoryalternativethatmaximizesoverallnetsocialbenefittoall members of society in the aggregate. 24 This involvesestimating the total social costs and total social benefits of awholerangeofincrementallyvaryingalternativeregulationsandfinding thepointofnetbenefitsmaximization(which isalso thepoint at which marginal benefits are just equal to marginalcosts).25Makingthisprecisecalculationrequiresthatbothcostsandbenefitsbeexpressedinthesamemetric(whichinpracticeismoney). The Kaldor-Hicks test runs up against salient moralobjections stemming from the potential disconnect betweenindividual preferences and actual welfare (the drug addict’spreferencefordrugs,forexample),anditsfailuretoensureactualcompensation of losers by winners. 26 Formal cost-benefitanalysis essentially adapts the Kaldor-Hicks test to real-worldimplementationbyusingmoneyasthemeasureofwelfare,butinso doing it generates additional objections relating to wealtheffects, the willingness-to-pay standard, and the devaluation offuturebenefits. MostCBApractitioners simply ignore theseobjections orcarry on in spite of them. But some proponents of CBA inregulatory decisionmaking have in recent years sought todecouple it to some degree from its normative foundations inKaldor-Hicks efficiency. Themost notable example isMatthewAdlerandEricPosner’sgroundbreakingbook,NewFoundationsof
23See E.J. MISHAN, COST-BENEFIT ANALYSIS 390 (1976); BOARDMAN ET AL.,
supranote22at32.24EDITH STOKEY&RICHARD ZECKHAUSER, A PRIMER FOR POLICY ANALYSIS 137
(1978); BOARDMANET AL., supra note 22, at13, 33; OFFICEOFMGMT.&BUDGET,CIRCULAR A-4, TO THE HEADS OF EXECUTIVE AGENCIES AND ESTABLISHMENTS:REGULATORYANALYSIS9–10(2003).
25EDWARDM.GRAMLICH,AGUIDETOBENEFIT-COSTANALYSIS1 (2ded.1990);TOMTIETENBERG,ENVIRONMENTALANDNATURALRESOURCEECONOMICS 25, 66 (5thed.2000);RichardD.Morgenstern,ConductinganEconomicAnalysis:Rationale,Issues, and Requirements, in ECONOMICANALYSES AT EPA: ASSESSINGREGULATORYIMPACT25,40(RichardD.Morgensterned.,1997).
26SeeKysar,supranote3;MATTHEWD.ADLER&ERICA.POSNER,NEWFOUNDATIONSOFCOST-BENEFITANALYSIS125,142-46(2006);ArthurA.Leff,EconomicAnalysisofLaw:SomeRealismAboutNominalism,60VAL.REV.451(1974);DANIELKAHNEMAN&AMOSTVERSKY(EDS.),CHOICES,VALUES,ANDFRAMES(2000).
***DRAFT***
13
Cost-Benefit Analysis,27widely viewed as the most sophisticatedandcredibledefenseofCBAintheliteraturetodate.28Init,theyrejecttheKaldor-Hickstestas“simplynotanattractivenormativecriterion”andseektotetherCBAinsteadtoabroaderutilitariannotion of overallwelfare.29But the tether is a long one. Ratherthan conceptualizing it as a direct measure of overall welfare,theydefendCBAasa“decisionprocedure”thatprovidessimplya“roughandreadyproxy” foroverallwelfare.30Thisallows themto largely sidestep many of the central critiques of welfareeconomics, although some they address through proposedmodificationsofstandardCBA,likethelaunderingofpreferencesto eliminate distorted preferences and distributionalweightingstocounteractwealtheffects.31 In sum, the kind of CBA that emerges out of welfareeconomicsisgroundedinthenotionofKaldor-Hicksefficiencyorwelfare maximization. It is also highly formal, complex andtechnical, and requires both costs and benefits to be quantifiedandexpressedinmonetaryterms.
2. Quantification
Formal economic CBA requires an arithmetic calculation.Ideally,theanalystcanplotmarginalcostsandmarginalbenefitsforawholerangeofincrementallyvaryingregulationsonagraphandlocatewherethetwocurvescross. Or,ataminimum,sheisabletosubtracttotalcostsfromtotalbenefitsinordertocalculateasinglenumberrepresentingnetbenefitsforeachalternativeandtoidentifythealternativewiththelargestnetbenefits.Thiskindof calculation requires both costs and benefits to be quantifiedandtobeexpressedinthesamemetric.Inpractice,thatmetricisalwaysmoney.
What we might generically refer to as quantification is,then,actuallya two-stepprocess. First, the relevantvaluemustbe quantified (e.g., lives saved, acres of wetlands preserved,poundsofpollutionaverted).Second,thatquantifiedvaluemustbe translated into monetary terms. Note that I use the term“quantify”toreferspecificallytothefirststep,butalsosometimes
27ADLER&POSNER,supranote26at23;seealsoCASSR.SUNSTEIN,THECOST-BENEFITSTATE:THEFUTUREOFREGULATORYPROTECTION19–20(2002).
28 Amy Sinden, Douglas A. Kysar, & David M. Driesen, Cost-BenefitAnalysis:NewFoundationsonShiftingSand,3REG.&GOVERNANCE48,50(2009).
29ADLER&POSNER,supranote26,at23.30Id.at25.31Id.at149-153.
***DRAFT***
14
(as in the title of this article and this subsection) to refergenerically to thewhole two-stepprocess. These two steps aredescribedinmoredetailbelow.
a. QuantitativeRiskAssessment
Inthecontextofhumanhealth(wherethevastamountof
the energy and resources aimed at providing monetized valuesforCBAhasbeenaimed),thefirststephasbeenformalizedintoaprocesscalledQuantitativeRiskAssessment.32QuantitativeRiskAssessment involves four stages: 1) hazard identification, 2)dose-response evaluation, 3) exposure assessment, and 4) riskcharacterization.33 Hazard identification is typically conductedthroughepidemiologicalstudiesorlong-termanimalbioassaysinordertodeterminewhetherthere issomecausal linkbetweenagivenpollutantandsomeadversehealtheffect.
Dose-response evaluation is the process by which thatcausal relationship is reduced to quantitative terms. Thisrequires that the relevant epidemiological and animal bioassaystudies be of sufficient number and quality to support aquantitative estimate of the likely incidence of disease atparticular dose or exposure levels. In most instances, theexposure level for which the studies provide direct evidence ofdisease incidence are substantially higher than the levelsanticipated to occur from pollution. Dose-response evaluationthenrequiresmakingcertainassumptionsinordertoextrapolatethe dose-response curve to lower exposure levels. Where thestudies are all or primarily animal bioassays, the analyst alsoneedstomakeasetofassumptionsinordertoextrapolatefromanimaldatatolikelyeffectsonhumans.34
Exposure assessment involves estimating the extent towhich human populations will be exposed to a particularhazard.35 This requires data or modeling of ambient levels ofpollution and how these are affected by weather patterns etc.
32SeeEPA’sRiskAssessmentPortalformoreinformation.
https://www.epa.gov/risk.33SeeAlonRosenthal,GeorgeM.Gray&JohnD.Graham,Legislating
AcceptableCancerRiskfromExposuretoToxicChemicals,19ECOLOGYL.Q.269(1992);NAT’LRESEARCHCOUNCIL,RISKASSESSMENTINTHEFEDERALGOVERNMENT:MANAGINGTHEPROCESS3(1983).
34WendyE.Wagner,TheScienceCharadeinToxicRiskRegulation,95COLUMB.L.REV.1613(1995).
35CatherineA.O’Neill,Exposed:AskingtheWrongQuestioninRiskRegulation,48ARIZ.STATEL.J.703,713(2016).
***DRAFT***
15
Thismayrequiredataabouthowmanypeoplelivenearafacilitythatemitsharmfulpollutantsintotheairandhowmuchtimetheyspend outside. Or it might require data on how many peopledrink water from particular groundwater sources likely tobecome contaminated or eat fish caught from contaminatedwaters,alongwithdataabouthowtoxinsenterthefoodchainandbecomeconcentratedinthetissuesofparticularfishspecies,andsoon.
Finally,theriskcharacterizationstageinvolvescombiningtheresultsfromthesecondandthirdstagesinordertoderiveanumerical estimate of population health risk. This essentiallymeansmultiplyingthedoseresponseratiobytheexposurelevel.This isusuallyexpressedasan individual’s incremental increasein the risk of dying from some particular disease (or of simplycontracting thatdisease)at someparticular levelof exposure tothepollutant inquestion. Thatnumbermightbetranslated intoan amount of harm to society as awhole bymultiplying by thenumber of people expected to be exposed at that level. In thiswayananalystcancalculateatotalnumberofdeathsaparticularincreaseinsomepollutantmightbeexpectedtocauseoversomedesignatedgeographic areaand timeperiod—or conversely, thenumberof lives thatwouldbe savedby a regulationdecreasinglevelsofthatpollutant.
Thus, through Quantitative Risk Assessment, an analystcan arrive at some number representing a society wide benefitexpectedtoaccruefromsomeparticularincrementofregulatoryprotection: lives saved, casesof chronicbronchitis averted, andso on. In attempting to quantify ecological harms, an analystmightwellgothroughanalogoussteps,thoughtheprocessisfarless formalizedandroutinized than ithasbecome in thehumanhealth sphere. Rather than quantifying lives saved or illnessesaverted, theanalystmight comeupwithanestimateof acresofwetlands or habitat preserved, numbers of fish deaths averted,andsoon.36
But these values are inevitably expressed in units of“apples” that cannotbedirectly comparedwith the “oranges”ofsocialcosts.Thus,aformalEconomicCBArequiresthefinalstepof monetization: translating a quantified estimate of socialbenefit intomonetary terms,so that itcanbedirectlycomparedtosocialcosts.Thenextsectiondescribesthisprocess.
36Seegenerally,EPA,EcologicalRiskAssessment,https://www.epa.gov/risk/ecological-risk-assessment(lastvisitedAug.9,2017).
***DRAFT***
16
b. Monetization
In some instances, where the quantified benefit involves
some good traded in markets, monetization is relativelystraightforward. Where, for example, the benefit involvesimprovingacommercialfisheryandtheincreaseinthenumberoffish expected to be caught by commercial fishermen has beenestimated,itisarelativelysimplemattertomultiplythenumberoffishbytheretailpriceofthatspeciesoffishinordertoobtainamonetaryvalueforthatbenefit.37
More often, however, environmental benefits involve non-market goods for which divining a monetary value is far lessstraightforward. Whatisthedollarvalueofsavingahumanlife,preventingapainful anddebilitating illness, orpullinga speciesback from the brink of extinction? Some argue that the entireenterpriseoftryingtomonetizesuchvaluesismisguidedbecausetheyarefundamentallyincommensurablewithmoney.
Thatis,wemayviewitassimplywrong,descriptivelyand/ornormatively, totrytomeasuresomegoods inmonetaryterms.38Many people, for example, balk at the prospect of attaching adollar figure to the loss of a human life, the destruction of apristinenaturalarea,theextinctionofaspecies,oraregulation’simpactonthedignityofpeopleinwheelchairs,becausetheyviewsuch a measure as flattening and cheapening the richness anddiversity of human experience in a way that at best provides agrossly incomplete and inaccurate representation of humanvalues and atworst leads tomorally unjustified outcomes.39Ortheymayviewmonetizationof suchvalues aswrongbecause itconfuses thepeople’s preferences as consumerswith the valuestheyholdascitizens.40
Despite these objections, economists have developed anumberof techniques for trying todivine themonetaryvalueof
37Intheory,it’snotquitesosimple.Thesocialbenefitshouldincludenot
justprice,butconsumersurplusinordertocapturealloftheincreaseinsocialwelfare.Butthisdetailisroutinelyignored.Priceistakentobeareasonableapproximationofactualincreaseinwelfare.SeeMARKSAGOFF,PRICE,PRINCIPLE,ANDTHEENVIRONMENT(2004).
38SeegenerallyELIZABETHANDERSON,VALUEINETHICSANDECONOMICS(1993);CassR.Sunstein,IncommensurabilityandValuationinLaw,92MICH.L.REV.779,785–86(1994):LawrenceH.Tribe,WaysNottoThinkAboutPlasticTrees:NewFoundationsforEnvironmentalLaw,83YALEL.J.1315(1974).
39See,e.g.,Kelman,supranote20at33-36.40MARKSAGOFF,THEECONOMYOFTHEEARTH93–94(1988).
***DRAFT***
17
thingsnottradedinmarkets.41“Revealedpreference”techniquesattempttoinferadollarvaluefornonmarketgoodsbyobservingthings thatare traded inmarkets and are thought to reflect (or“reveal”)theunpricedvalue.42
These techniques often take the form of hedonic valuationmethods,whichattempttodisaggregateenvironmentalorhealthattributes fromother goodswithwhich they arebundled in themarket in order to infer the value people place on them. Forexample,economistscomparethewagespaidtoworkersin jobsassociatedwithahighriskofdeathtothewagespaidtoworkersinlessriskyjobsinordertoinferthedollarvaluepeopleattachtoavoiding some particular risk of death. They can then use thisvalue to calculate the “value of a statistical life” (VSL), a dollarvalue associated with the avoidance of one death in thepopulationasawhole.(Iftheaveragepersoniswillingtogiveup$10 in wages, for example, to avoid a one-in-a-million risk ofdeath, theVSLwouldbe$10million.) Alternatively,economiststrytomeasurehowmuchvaluepeopleattachtounspoiledopenspace by comparing the prices of properties located adjacent tosuchareaswiththosethatarenot.43
Theotherprimaryrevealedpreference technique, the travelcost method, involves inferring the value of environmentalamenitiesfromthecostspeopleincurtotraveltothem.Thus,aneconomistmightmeasuretherecreational“usevalue”attachedtonatural resources by measuring the admission fees and travelcostshikerspaytovisitanationalpark.44
Alternatively,wherevaluescan’tbe“revealed”throughactualmarket transactions, economists turn to “stated preference”
41SeegenerallyDAVIDW.PEARCE&ANILMARKANDYA,ENVIRONMENTALPOLICYBENEFITS:MONETARYVALUATION(1989)(discussingvariousdirectandindirectbenefitvaluationtechniques,includinghedonicandcontingentvaluationmethods).
42 See generally David S. Brookshire et al., Valuing Public Goods: AComparisonofSurveyandHedonicApproaches, 72 AM.ECON.REV. 165 (1982);seealsoBOARDMANETAL.,supranote22,at353–57;PhilipE.Graves,Benefit-CostAnalysis of Environmental Projects: A Plethora of Biases Understating NetBenefits,3J.BENEFIT-COSTANALYSIS1,12–19(2012).
43E.g., Richard Ready & Charles Abdalla, The Impact of Open Space andPotential Local Disamenities on Residential Property Values in Berks County,Pennsylvania (Pa. State U. Dep’t of Agric. Econ., Soc., & Educ. Staff Paper No.363, 2003), http://aese.psu.edu/directory/aic/the-impact-of-open-space-and-potential-local-disamenities-on-residential-property-values-in-berks-county-pennsylvania/view.
44SeeShi-Ling Hsu & John Loomis,ADefenseofCost-BenefitAnalysis forNaturalResourcePolicy,32ENVTL.L.REP.10,239,10,242(2002);BOARDMANETAL.,supranote22at358-65.
***DRAFT***
18
methods.“Contingentvaluation”surveys,forexample,attempttomeasure people’s willingness to pay for nonmarket goods bysimplyasking them.45Theseareessentially sophisticatedpublic-opinion polls that give respondents information about aparticular natural resource or medical condition and then askthem how much they would be willing to pay to preserve theresource or avoid the disease. One such survey, for example,concludes that California households are, on average, willing topay $18.14 per year to increase graywhale populations by 100percent.46Another pegs the average person’swillingness to paytoavoidcontractingchronicbronchitisat$457,000.47
All of these methods are controversial and produce highlycontestable results. 48 One problem, for example, is theendowmenteffect.Eventhoughmeasuringwillingnesstopay(tobuy)versuswillingnesstoaccept(tosell)yieldsdifferentvaluesfor the same good, economists have yet to come up with anyprincipled basis for choosing between the two. This makesstated-preference surveys,which are almost always designed tomeasure willingness to pay, vulnerable to criticism that theyunderestimatethevaluestheytrytomeasure.
c. ATypologyofUnquantifiedBenefits
In sum, as the preceding pages have set forth, translating
regulatorybenefitsintomonetarytermsinvolvesseveraldiscretesteps,eachofwhichposessignificantchallenges.Obstaclesatany
45See BOARDMANET AL., supra note 22 at 372-405; Hsu & Loomis, supranote41,at10,242;ThomasH.Stevensetal.,MeasuringtheExistenceValueofWildlife: What Do CVM Estimates Really Show?, 67 LAND ECON. 390, 392–97(1991). For a critique, see generally John M. Heyde, Is Contingent ValuationWorththeTrouble?,62U.CHI.L.REV.331(1995).
46 John B. Loomis & Douglas M. Larson, Total Economic Values ofIncreasingGrayWhalePopulations:ResultsfromaContingentValuationSurveyofVisitorsandHouseholds,9MARINERESOURCEECON.275,282tbl.1(1994).
47See W. Kip Viscusi et al., Pricing Environmental Health Risks: SurveyAssessmentsofRisk-RiskandRisk-DollarTrade-OffsforChronicBronchitis,21 J.ENVTL.ECON.&MGMT.32,47,50(1991).
48SeeDAVIDW.PEARCE&R.KERRYTURNER,ECONOMICSOFNATURALRESOURCESAND THE ENVIRONMENT 141–58 (1990); Leonard Shabman & Kurt Stephenson,EnvironmentalValuationandItsEconomicCritics,126J.WATERRESOURCESPLAN.&MGMT.382,382–84(2000).
Aprominentexample isEPA’sCoolingWater Intakerules, forwhich theagencyhasstruggledforyears(notparticularlysuccessfully)tocomeupwithnoncontroversialmethodsformonetizingtheharmstoaquaticorganismsandecosystems againstwhich the rule is aimed. See Sinden,BenFranklin, supranote18.
***DRAFT***
19
ofthesestepsalongthewaymayresultinanagencybeingunabletomonetizearegulatorybenefitinawaythatallowsforaformalCBA. Accordingly, we can sketch out a typology of potentialreasonsthatquantificationmightfail:
1) Insufficientinformationtoconduct:
a. Hazardidentification(establishingacausallinkbetweentheregulatedactivityandhuman/ecologicalhealth)
b. Dose-responseevaluation(collectingsufficientdatatocrediblyestablishadose-responsecurve,describingthecausallinkinquantitativeterms)
c. Exposureassessment(establishinghowmanypeople—orecosystemcomponents—arelikelytobeexposedtoaparticularhazardandtowhatdegree,basedonactualdataormodelingtechniques)
2) Insufficientdata/modelstomonetize
To these, we can add another: The agency might considerbenefitsunquantifiablebecauseofcommensurabilityconcerns.Itmay take the position that a particular benefit is, as aphilosophical matter, simply not reducible to monetary terms.Finally, it is perhaps worth separating out lack of money orresourcesasaseparatereasonthatmightbecitedinconjunctionwithanyofthefirstfour:
3) Incommensurability4) Lackofmoney,time,and/orresources
Whileananalystmaynotalwaysdifferentiateamongallofthesereasons—particularlyamongthefirstthree(1a-c)—thistypologywillnonethelessbehelpfultokeepinmindasweproceed.
3. TheFormalitySpectrum
On the other end of the spectrum from the formalEconomic CBA described above, is the intuitive comparison ofqualitatively described pros and cons, which I have called “BenFranklinCBA.”49Basedonthatspectrum,thissectiondescribesatypologyofformalityandinformalityinCBA.
49SeeSinden,Formality,supranote19;Sinden,BenFranklin,supranote18.
***DRAFT***
20
WecanconceptualizetheCBAspectrumasbreakingdowninto three distinct but related axes, each ofwhich extends frominformalityonthelefttoformalityontheright:
FIGURE1.Theformality-informalityspectrum50
As illustrated by the light gray lines between axes, thethreeaxesarerelatedsuchthatamovealongonemayrequireorimplyaparallelmovealonganother.Where,forexample,allcostsandbenefitsareonlydescribedinqualitativeterms(theleft-mostposition on Axis 1), then the analyst can perform only a roughcomparison (left-most position onAxis 2) and is likely to applythe analysis to only a single alternative or a handful ofalternativesatmost(lefttwoboxes,Axis3).Attheotherextreme,where all costs and benefits are fully monetized (right-mostpositiononAxis1),thenananalystcaneitherpreciselycomparethemforasinglealternative(middleboxonAxis2;left-mostboxon Axis 3), or, if evaluating a whole range of incrementallyvaryingalternatives(right-mostboxonAxis3),shecanperformaformal Economic CBA, pinpointing the economically efficient
50FromSinden,Formality,supranote19,at113.
***DRAFT***
21
alternative for which marginal costs are just equal to marginalbenefits(right-mostboxonAxis2).
Where some but not all benefits are monetized(intermediatepositionsonAxis1),theanalystmayemployalessprecisebalancingtest(intermediateleftpositionsonAxis2),suchas a “wholly disproportionate” standard. Or shemay conduct a“break evenanalysis,”which involves calculating the amountbywhich the monetized benefits estimate falls short of themonetizedcostsestimateandthenmakinganintuitivejudgmentaboutwhethertheunquantifiedbenefitsarelikelybigenoughtomakeupthedifference.(Moreonthislater.)
4.Standard-settingvs.Litmus-testCBA
Oneimportantinsightthatarisesfromthistypologyisthatformal and informal CBA play fundamentally different roles indecisionmaking. Formal Economic CBAmeasures the costs andbenefits of many incrementally differing alternatives and thenchoosesoneperfectoptionfromthewholerange.Accordingly,ithasthecapacity(theoretically,atleast)tolocatethepreciselevelof regulatory stringency that will achieve economic “efficiency”(maximizationofoverallsocialwelfare).AninformalCBA,ontheother hand, simply provides a binary go-or-no-go answer for asingleoption.51Andwhileitmaygivepolicymakersavagueideaofwhetheragivenregulationisdesirableinrelationtothestatusquo, it can’t tell them whether it is efficient or optimal in anysense.52
51See Jonathan Cannon, The Sounds of Silence: Cost-Benefit Canons inEntergy Corp. v. Riverkeeper, Inc., 34 HARV. ENVTL. L. REV. 425, 454 (2010),(describing informalCBA,whathecalls “theweak formofCBA,”asa tool for“screen[ing] for irrational outcomes”); Daniel H. Cole, Toward a PoliticalEconomyofCost-BenefitAnalysis, 3 (Ind. Legal Stud. Res. PaperNo. 1954892,2011), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1954892##(noting that CBA “is viewed as a kind of filter designed to capture welfare-reducing proposals, while allowing welfare-enhancing proposals to passthrough”); see also BOARDMAN ET AL., supra note 22, at 13 (distinguishingbetweenthedecisionruleforlitmus-testCBAandmoreformalCBA);RICHARDJUST ET AL., THEWELFARE ECONOMICS OF PUBLIC POLICY: A PRACTICAL APPROACH TOPROJECTANDPOLICYEVALUATION 642 (2004) (arguing forwelfaremaximizationapproachtoCBA).
52SeeTIETENBERG,supranote25,at66(observingre:litmus-testCBAthat“[w]hile [this test] guarantee[s] that no activitywhich confersmore costs onsocietythanbenefitswillbeundertaken,[it]do[es]notguaranteeefficiency...[E]fficiencyisattainedwhenthemarginalvalueofbenefitsequalsthemarginalvalue of costs”); Nathaniel O. Keohane, The Technocratic and DemocraticFunctions of the CAIR Regulatory Analysis, in REFORMING REGULATORY IMPACT
***DRAFT***
22
Thus, at the formal end of the spectrum, CBA acts as astandard setting tool, telling the agency exactlywhere, among awhole range of options, to set the regulatory standard.53 MoreinformalCBAs,on theotherhand,actasasortof litmus test—asecondary check on a standard setting decision that has beenmade initially by other means.54 These informal, “litmus-testCBAs”includethekindthatagenciesperhapsmostfrequentlydo,where the analyst simply askswhether benefits outweigh costsforasinglealternative.55
Itisworthnotingthatalitmus-testCBAprovidesaprettyinaccurateproxyforefficiency.56While it isprobablytruethataregulation that flunks a litmus-test CBA (produces more totalcosts than total benefits) is inefficient, the converse is not true.Justbecauseasingleregulationpassesalitmus-testtestCBAdoesnotnecessarilymean that it isefficient. Indeed,a regulationcanpass such a test and be very far indeed from the point ofefficiency.
If, for example, the efficient level of regulation wouldreduceaggregateemissionsofsomepollutantfrom100to25tonsand produce $15 billion in net benefits, a far less stringentregulation thatreducedemissionsonly from100 to99 tonsandproducedonly$1billioninnetbenefitswouldstillpassa litmustest CBA with flying colors. But it would be very far from theefficientlevelofregulation,producingonlyasmallfractionoftheemissions reductions and net benefits that would be possibleundertheefficientregulation.
To see the same point graphically, imagine a scenario inwhich reducing pollution levels by some small amount will befairly inexpensive anddeliver substantial benefits butwhere, aspollutantlevelsarereducedfurtherandfurthertowardzero,the
ANALYSIS 49 (Winston Harrington et al. eds., 2009) (“Simply calculating totalbenefitsandcostsdoesnotshedlightonmarginalbenefitsandcosts,which–asanyeconomicsstudentknows—mustbeequatedtosatisfyefficiency.”).
53See,e.g.,Keohane,supranote52,at47(notingthataCBAthatconsidersonlyoneoption“fails tomeet themostbasicrequirementofsoundeconomicpolicyanalysis:namely,theconsiderationofmultiplealternatives.”)
54Id.(“Adocumentthatconsidersthecostsandbenefitsoftheproposedpolicyonlyrelativetothestatusquocannotpossiblyhavebeenusedtodesignthatpolicy.”).SeealsoDavidM.Driesen,TwoCheersforFeasibleRegulation,35HARV. ENVTL. L. REV. 313, 320 (2011) (criticizing Jonathan Masur and EricPosnerforconfusingthesetwodifferentformsofCBA).
55Keohane,supranote52,at34(notingthatEPA’sCBAfortheCleanAirInterstate Rule was “essentially an up-or-down assessment of the final ruleversusthestatusquo”).
56Seesupranote52.
***DRAFT***
23
marginalcosts(thecost foreachunitofpollutantremoved)willgradually increaseand themarginalbenefitsgraduallydecrease.This isa fairlyreasonableassumptionthatprobablycaptures,atleast in general terms, how marginal cost and benefits curvesmost frequently behave. 57 On such assumptions, a stylizedversionof themarginal costandmarginalbenefits curvesmightlooksomethinglikethosedepictedinFigure2:
FIGURE2:Litmus-testversusstandard-settingCBAasameasureofefficiency
Thetotalcostsandbenefitsforanyparticularlevelofpollutioncontrol(anypointalongtheX-axis)istheareaunderthecurve.Atthepointofefficiency,totalbenefitsclearlyexceedtotalcosts(uptodottedline“e,”theareaundertheredcurveisbiggerthantheareaunderthebluecurve).Atanypointtotheleftofe,totalbenefitswouldstillexceedtotalcosts,butnotbyquiteasmuchastheywouldatpointe.Onecanalsoseeintuitivelythatregulatinguptopointzcreatesanareaunderthebenefitscurvethatisroughlyequalinsizetotheareaunderthecostcurve.Thus,regulatingtothatpointwouldproducezeronetbenefit:totalcostswouldbejustequaltototalbenefits.Atanymorestringentlevelofregulation(totherightofpointz),theareaunderthebenefitscurvewouldbesmallerthantheareaunderthecostscurve,meaningthattotalcostswouldexceedtotal
57SeeOMBCircularA-4,supranote24,at8(“[M]arginalcostsgenerally
increasewithstringency,whereasmarginalbenefitsmaydecrease.”).
***DRAFT***
24
benefits(netbenefitswouldbenegative).Thus,anyregulationanywherewithintherangeofthepurpleshadedboxwouldpassalitmus-testCBA.Andthatincludeslevelsofregulationbothfarmorestringentandfarmorelaxthantheefficientlevel(e).
In sum, there are multiple varieties of CBA that can beconceptualized on a spectrum from informal to formal.Conceptualizing CBA in this way brings into focus importantdistinctions between informal and formal types of CBA. Inparticular,ithighlightsthefactthatinformalandformalCBAplayvery different roles in the decision making process. FormalEconomic CBA operates (in theory) as a standard-setting tool,allowing the analyst to choose the maximally efficient level ofregulation from a whole range of alternatives. Less formal“litmus-test” varieties of CBA, on the other hand, act only as asecondary filter on a standard setting decision that has beeninitially made by other means. While formal Economic CBAidentifies the efficient level of regulation, litmus-test CBAsprovideaverypoorproxy forefficiency. Itwillbe important tokeepinmindtheselimitationsoninformallitmus-testCBAasweproceed.
One question that these important distinctions betweenformal and informal varieties of CBA raises is which of thesevarieties are federal agencies required or expected to perform?Thenextsectiontakesupthisquestion.
B. Agencies’LegalObligationsRegardingFormalCBA MostagenciesperformCBAnotpursuanttoanystatutorycommand,butunderasetofexecutiveordersthatrequirefederalagencies to perform CBA on all “major” regulations (i.e., thosecosting at least $100 million per year).58Indeed, most of ourenvironmental health and safety statutes actually eschew CBAanddirectagenciestousesomealternativestandardsettingtoolinstead.59Themostprominentofthesealternativesarefeasibilitystandards,whichdirect the agency to set standards at themoststringent level that is economically and technologically feasible,and health-based standards, which direct agencies to setstandards at the level requisite to protect public health and/or
58See Exec. Order No. 12,866 § 1(b)(6), 58 Fed. Reg. 51,735, 51,736(1993).
59SeeSIDNEYA.SHAPIRO&ROBERTL.GLICKSMAN,RISKREGULATIONATRISK:RESTORINGAPRAGMATICAPPROACH32(2003);ThomasO.McGarity,Media-Quality,Technology,andCost-BenefitBalancingStrategiesforHealthandEnvironmentalRegulation,46LAW&CONTEMP.PROBS.159,160–61(1983).
***DRAFT***
25
the environment, without considering costs.60 This reflects thesignificant congressional skepticism toward CBA that held swayin the 1970s, when most of these statutes were passed—askepticism that stemmed in large part from the same concernsabouttheundercountingofbenefitsthatanimatethisstudy.61
As noted in the introduction, President Reagan firstimposedaCBArequirementonagenciesthroughexecutiveorderin 1981, after sweeping into office on promises of economicstimulus through deregulation. 62 Some version of thatrequirement has persisted, through Democratic and Republicanadministrations, ever since. In cases of conflict, the statutorystandard, of course, prevails over the executive orders’ CBArequirement.Soagenciesoftenfindthemselvesintheanomalouspositionofhaving toperformaCBAunder theexecutiveorders,uponwhichtheyarenotpermittedtorelyintheiractualdecisionmaking.63
Despite the primacy of these statutory commands,however,CBAremainsanimportanttopicofstudyanddiscussionforseveralreasons.First,Republicanshaveformanyyearsbeentrying to pass a statutory CBA “super-mandate,” which wouldeffectivelyre-writeall thestatutes thatcurrentlydirectagenciesto base regulatory decision making on other standards, andinstead create a statutory command making CBA the rule ofdecision. Indeed, in 1995, such a bill camewithin two votes ofsurvivingaSenate filibuster tobecome law.64Anumberofsuchbillsarecurrentlypendingin(orhavealreadypassed)theHouseand, with Republicans control of both Congress and the White
60AmySinden,Cost-BenefitAnalysis,inEDWARDELGARENCYCLOPEDIAOFENVIRONMENTALLAW,VOLII,ENVIRONMENTALDECISIONMAKING(Glicksman&Paddockeds.,2016).
61SeeROBERTV.PERCIVALETAL.,ENVIRONMENTALREGULATION:LAW,SCIENCE,ANDPOLICY363–64(4thed.2003);SUBCOMM.ONOVERSIGHT&INVESTIGATIONSOFTHECOMM.ONINTERSTATE&FOREIGNCOMMERCE,94THCONG.,FEDERALREGULATIONANDREGULATORYREFORM510–15(Comm.Print1976).
62Exec.OrderNo.12,291§2(b),3C.F.R.127,128(1982).SeealsoREVESZ&LIVERMORE,supranote1at,24-25.
63See,e.g.,EPA,FinalRule:PrimaryNationalAmbientAirQualityStandardsforSulfurDioxide,75Fed.Reg.35520,35587(June22,2010)(“EPApreparedaRegulatoryImpactAnalysis(RIA)ofthepotentialcostsandbenefitsassociatedwiththisaction.However,theCAAandjudicialdecisionsmakeclearthattheeconomicandtechnicalfeasibilityofattainingthenationalambientstandardscannotbeconsideredinsettingorrevisingNAAQS,....Accordingly,althoughanRIAhasbeenprepared,theresultsoftheRIAhavenotbeenconsideredbyEPAindevelopingthisfinalrule.”).
64TheRiskAssessmentandCost-BenefitActof1995,H.R.9,TitleIIDivisionD,104thCong.(1995).
***DRAFT***
26
House, have a realistic chance of passing and being signed intolaw.65 Second, theSupremeCourthas in recentyears shownanincreasing willingness to read ambiguous statutes to allow, orpossiblyevenrequire,CBA.66
Third, the executive order CBA requirement has been inplacenow fornearly fourdecades. Andover that time,perhapsspurred on by institutional pressures from the small group ofeconomists at the White House’s Office of Information andRegulatory Affairs (OIRA) who are tasked with overseeing andadministeringit,theCBAmandatehasbeguntohaveaneffectonagency culture. Lisa Heinzerling, for example, who served asAssociate Administrator of EPA’s Office of Policy during theObamaadministration,reportsthat“OIRA’sfinecost-benefitsieveleadsEPApersonnel to bedeeplywary of developing rules thathaveveryhighcostsinrelationtotheirquantifiedandmonetizedbenefits.” 67 Thus, there is reason to believe that the CBArequirement has an informal effect on agency decision makingthatgoesbeyonditsformallegaleffect.
WithrespecttotheformalityoftheCBArequired,theCBAexecutive orders and guidance documents interpreting themevidenceatension:Ontheonehand,theyclearlyholdupformalCBA as the goal and the norm. On the other hand, theyacknowledge the inevitability of unquantified benefits andinstruct agencies to take thosebenefits into account. This is anawkwardtensionbecause,asnotedabove,whereanysignificant
65See,e.g.,TheRegulatoryAccountabilityActof2017,H.R.5,115thCong.;
seegenerallyAssessingtheRegulatoryAccountabilityAct,TheRegulatoryReview(May30,2017),https://www.theregreview.org/2017/05/30/assessing-regulatory-accountability-act/.
66Michiganv.E.P.A.,135S.Ct.2699(2015);seealsoAmySinden,A“Cost-BenefitState?”ReportsofItsBirthHaveBeenGreatlyExaggerated,46ENVTL.L.REP.10,933(2016).
67 Lisa Heinzerling, Inside EPA: A Former Insider’s Reflections on theRelationship Between the Obama EPA and the Obama White House, 31 PACEENVTL. L. REV. 325, 352 (2014); see also CASS R. SUNSTEIN, VALUING LIFE:HUMANIZINGTHEREGULATORYSTATE,37(2014)(“Withinthefederalgovernment,agenciesareacutelyawareofthecost-benefitrequirementofExecutiveOrder13563andExecutiveOrder12866.”);CassR.Sunstein,TheRealWorldofCost-BenefitAnalysis:Thirty-SixQuestions(andAlmostasManyAnswers)8(HarvardLaw Sch. Pub. Law& Legal TheoryWorking Paper No. 13-11, 2013) (notingthat where a regulation’s monetized benefits are less than monetized costs,“the agency is unlikely to attempt to go forwardwith this regulation”); RenaSteinzor,TheCaseforAbolishingCentralizedWhiteHouseRegulatoryReview,1MICH.J.ENVTL.&ADMIN.L. 209, 243–44 (2012) (discussing dynamic set up bycentralized reviewof agency rulesbyOIRA, as givingOIRA significantpowerandswayoveragencyrulemaking).
***DRAFT***
27
portion of benefits are unquantified, formal economic CBA isactuallyimpossibletoperform.
The primary CBA executive order in place today, E.O.12,866, was signed by President Clinton in 1993.68 It replacedReagan’s 1981 order, but, while arguably a little kinder andgentler, kept the basic CBAmandate in place. Like the Reaganorder before it, the Clinton order requires agencies to choose“amongalternativeregulatoryapproaches”soasto“select thoseapproaches thatmaximize net benefits.”69 This reference to netbenefitsmaximizationclearlysetsupformalEconomicCBAasthegoal. On the other hand, the Clinton Order also makes severalreferences to the difficulties inherent in attempting to quantifycertainvalues,directingthatcostsandbenefits“beunderstoodtoinclude both quantifiable measures (to the fullest extent thatthesecanbeusefullyestimated)andqualitativemeasuresofcostsand benefits that are difficult to quantify but nonethelessessentialtoconsider.”70
President Obama considered revoking Executive Order12,866 soon after coming into office in 2009 and even solicitedpubliccommentonthe idea.71Ultimately,he left thepriorOrderin place, however, and instead simply issued Executive Order13,563, “Improving Regulation and Regulatory Review,” which
68Exec.OrderNo.12,866§1(b)(6),3C.F.R.638,639(1994),reprintedas
amendedin5U.S.C.§601app.At88–92(2012).69 Exec. Order No. 12,866, § 1(a), 58 Fed. Reg. 51,735 (1993). A
subsequentsectionof theExecutiveOrderalsorequirestheagencytosubmitto OIRA “[a]n assessment, including the underlying analysis, of costs andbenefits of potentially effective and reasonably feasible alternatives to theplannedregulation.”Id.at§6(a)(3)(C)(iii).TheReaganOrdersimilarlystatedthat “[r]egulatory objectives shall be chosen tomaximize the net benefits tosociety” and “[a]mong alternative approaches to any given regulatoryobjective,thealternativeinvolvingtheleastnetcosttosocietyshallbechosen.”Exec.OrderNo.12,291,§2(c),(d),46Fed.Reg.13,193(1981).
70Exec. Order No. 12,866, § 1(a), 58 Fed. Reg. 51,735, (1993). In otherlanguage, theorderdirectsagencies tousea litmus-testbalancing test ratherthan an economic (netwelfaremaximization) test: “Each agency shall assessboththecostsandthebenefitsoftheintendedregulationand,recognizingthatsomecostsandbenefitsaredifficulttoquantify,proposeoradoptaregulationonly upon a reasoned determination that the benefits of the intendedregulation justify its costs.” Exec. Order No. 12,866 § 1(b)(6), 58 Fed. Reg.51,735,51,736(1993).
71See Memorandum: Regulatory Review, 74 Fed. Reg. 5977 (Jan. 30,2009)(directingOMBtoproducerecommendationsforanewexecutiveorderon regulatory review); OMB, Federal Regulatory Review, Request forComments, 74 Fed. Reg. 8819 (Feb. 26, 2009) (OMB requesting publiccommentonthoserecommendations).
***DRAFT***
28
“supplements and reaffirms” Executive Order 12,866.72 ThisOrder reiterates some of the key provisions in Executive Order12,866, including language that points both toward a simplelitmus-testCBA(requiringashowingthataregulation’s“benefitsjustify its costs” recognizing that “some benefits and costs aredifficult to quantify”) and toward amore formal Economic CBA(requiring agencies to “select, in choosing among alternativeregulatory approaches, those approaches that maximize netbenefits”). 73 But in language that shifts even further towardformalitywithnoanalogueintheClintonOrder,theObamaorderalso unambiguously holds out full quantification andmonetizationasthegoal,statingthat“eachagencyisdirectedtousethebestavailabletechniquestoquantifyanticipatedpresentand future benefits and costs as accurately as possible.”74 Itfollowsthatstatementwithanacknowledgmentofthedifficultiesinherent in attempting toquantify certain values, but ultimatelymakes the directive that agencies discuss unquantifiable valuespermissive,notmandatory:“Whereappropriateandpermittedbylaw,eachagencymayconsider(anddiscussqualitatively)valuesthat are difficult or impossible to quantify, including equity,humandignity,fairness,anddistributiveimpacts.”75
TheCBArequirement in theseExecutiveOrdershasbeenfurtherrefinedandclarifiedinOMBCircularA-4, issuedbyOIRAin 2003.76Like the executive orders, this document is clear insettingupformalEconomicCBAasthegoal,statingandthat“[b]ymeasuring incremental benefits and costs of successively morestringentregulatoryalternatives,youcanidentifythealternativethatmaximizesnetbenefits.”77ItalsonotesthatCBAsshouldbe
7276Fed.Reg.3821(2011).73Id.at§1(b)(1).74Id.at§1I.75Id.President Trumphas kept E.O. 12,866 in place and has not issued
any supplementary orders specifying how agencies are to conduct CBA. HisExecutive order on “regulatory reform” (E.O. 13,771), requiring agencies towithdraw two existing regulations for every new one they promulgate, hasstirred up considerable controversy for taking a very different approach tocontrollingregulatorycosts. But insubsequent implementingguidance,OIRAhasmadeclear that it expectsagencies to continue toadhere toE.O.12866’sCBArequirementandOMBCircularA-4.Seesupranote13.
76OMBCIRCULARA-4,supranote24.77Id. at 10. Earlier, it makes a similar point, stating that “[w]here all
benefits and costs can be quantified and expressed inmonetary units, [CBA]provides decision makers with a clear indication of the most efficientalternative, that is, the alternative that generates the largest net benefits tosociety.”Id.at2.
***DRAFT***
29
“consistent with economic theory,”78 and repeatedly uses thelanguageofeconomics,makingseveralreferencesto“market”or“economic efficiency,”79and directing agencies tomeasure costsandbenefits in termsof “opportunitycosts”and“willingness-to-pay.”80 Finally, it clearly contemplates full monetization as thegoal and the norm: “A distinctive feature of BCA [Benefit-CostAnalysis] 81 is that both benefits and costs are expressed inmonetaryunits,whichallowsyoutoevaluatedifferentregulatoryoptionswithavarietyofattributesusingacommonmeasure.”82
On the other hand, Circular A-4 also acknowledges that“[i]twillnotalwaysbepossibletoexpressinmonetaryunitsallofthe importantbenefits and costs” and that “[w]hen it isnot, themostefficientalternativewillnotnecessarilybetheonewiththelargestquantifiedandmonetizednet-benefitestimate.”83Insuchcircumstances, the Circular directs agencies “to exerciseprofessional judgment in determining how important the non-quantified benefits or costsmay be,” and, to carry out a break-even analysis if they are determined to be “important.” 84Although, at another point, it acknowledges that in suchcircumstancesCBA “is less useful,” and “can evenbemisleadingbecause the calculation of net benefits in such cases does notprovideafullevaluationofallrelevantbenefitsandcosts.”85
78Id.at21,23.79Id.at6,14.80Id. at 18. See also id. at 33-34 (requiring costs and benefits to be
discountedatboth3and7percent).81The term “benefit-cost analysis” (BCA) has the samemeaning as CBA
andispreferredbyanumberofCBA’sproponents.See,e.g.,KennethJ.Arrowetal.,IsThereaRoleforBenefit-CostAnalysisinEnvironmental,Health,andSafetyRegulation?,272SCIENCE221(1996).
82OMBCIRCULARA-4,supranote24at10.Seealso id. at27 (“Youshouldmonetizequantitativeestimateswhereverpossible.”).
83Id.at2.84Id.85Id. at 10. Similar references to unquantified benefits are peppered
throughout the document. For example, at another point, the Circular says“you should be able to assess quantitatively costs and benefits,” but thenimmediatelygoesontoqualifythatstatement:“Acompleteregulatoryanalysisincludes a discussion of non-quantified as well as quantified benefits andcosts.” Id.at3. Seealso id.at18(directingagenciesto“describebenefitsandcosts you cannot quantify”); id. at 26-27 (“Sound quantitative estimates ofbenefitsandcosts,where feasible,arepreferable toqualitativedescriptions. ..However,someimportantbenefitsandcosts(e.g.,privacyprotection)maybeinherently too difficult to quantify or monetize given current data andmethods.”).
***DRAFT***
30
EPA’s Guidelines for Preparing Economic Analyses aresimilarlygearedtowardahighly formalCBA.86Fromtheoutset,the introduction frames the CBA enterprise in the language ofeconomic theory: “[The Potential Pareto] criterion is thefoundationofBCA,requiringthatapolicy’snetbenefitstosocietybe positive. . . The policy that maximizes net benefits isconsidered the most efficient.” 87 And a detailed appendixprovidesatextbookintroductiontothefundamentalsofeconomictheory. 88 Like Circular A-4, EPA’s Guidelines require that“[b]enefits and costs should be reported in monetary termswheneverpossible”andthat“[b]enefitsandcoststhatcannotbemonetizedshould, ifpossible,bequantified.” Theyacknowledgethat “[i]n reality . . . there are often effects that cannot bemonetized, and the analysis needs to communicate the fullrichnessofbenefitandcostinformationbeyondwhatcanbeputin dollar terms.” 89 But ultimately, despite these caveats, theGuidelines take a hard line, requiring a strict numericalcomparison of costs against benefits in order “to determine aregulation’snetbenefits,evenifimportantbenefitsorcostscannotbemonetized.”90
In sum, the CBA executive orders and the guidancedocumentsinterpretingthemclearlysetupformalEconomicCBAas the goal. Theymake repeated reference to economic theory,net benefitsmaximization, and to the importance of quantifyingcostsandbenefits“asaccuratelyaspossible.”91Atthesametime,there’s a tension running throughout these documents: Whiletheyholdup formalCBAas thegoal and thenorm, theyalsoall
86EPA,GUIDELINESFORPREPARINGECONOMICANALYSES, 240-R010-001 (Dec.2010),http://yosemite.epa.gov/ee/epa/eed.nsf/pages/guidelines.Html.
87Id. at 1–4. Starting in 1983, EPA issued a series of Guidelines forpreparing CBAs. The agency released its most recent version in December2010. This document was prepared by economists at EPA and subsequentlypeerreviewedbyEPA’sScienceAdvisoryBoard.
88Id.atAppendixA.89Id.at11-2.90Id.at11-2(emphasisadded). Incontrast, theEuropeanCommission’s
ImpactAssessmentGuidelinesmakea findingthat“themostsignificantpartofboth costs and benefits can be quantified and monetized” a prerequisite torequiring full CBA. European Commission, Impact Assessment Guidelines 45(Jan. 2009). The European Commission’s Impact Assessment process alsoemploys something called “the principle of proportionate analysis” whichessentiallyvaries thedegreeofquantificationandmonetizationrequired inaCBA(i.e.formalityalongAxis1)accordingtothesignificanceoftheaction.Seeid. at 13; Anne Claartje Margreet Meuwese, Impact Assessment in EULawmaking62(Feb.2008).
91E.O.13,563,76Fed.Reg.3821(2011)at§1(c).
***DRAFT***
31
acknowledge that, in at least some instances, quantificationand/ormonetizationofallcostsandbenefitswillbeimpossible.
As to what to do when important benefits areunquantifiable, themessage ismixed: TheOMBCirculardirectsagencies to conduct a break-even analysis, but then later statesthattheuseofCBAinsuchcircumstancesmaybe“misleading.”92The EPA Guidelines direct the agency to “communicate the fullrichness” of unquantified benefits, but then—in an instructionthat seems to at best undermine that directive and at worseproduce highly misleading results—also requires the analyst tocalculate“netbenefits,evenifimportantbenefitsorcostscannotbemonetized.”93In any event, it appears clear that the executiveorders and guidance documents put significant pressure onagenciestoproducequantifiedestimatesofcostsandbenefits.
C. PreviousLiterature TheproblemofunquantifiedbenefitsisasubsetofamuchlargersetofproblemsthatleadgenerallytotheundercountingofbenefitsinCBA.Indeed,mostofthetheoreticalobjectionstoCBAinvolve problems that often cause benefits estimates to be toolow. Theseincludetheproblemsofdiscounting,theendowmenteffect,andwealtheffects.94Theseissues,alongwiththeproblemof incommensurability, implicate deep philosophical andnormative issue and have generated a substantial literaturespanningmanydecades.95Benefitsthatremainunquantifieddueto data deficiency, in contrast, have generally received littlesustainedattentioninthedebateoverCBA.96
92OMBCIRCULARA-4,supranote24,at10.93EPAGUIDELINES,supranote86,at11-2(emphasisadded).94Theoretically,wealtheffectscouldcausebenefitsestimatestobetoo
highortoolow,dependingonwhetherthebeneficiariesarerichorpoor.Sinceenvironmentaldegradationandhealthandsafetyhazardstendtodisproportionatelyimpactpoorcommunities,wealtheffectsoftenskewbenefitslower.
95Seesupranote11;seealsoGraves,supranote42.96ButseeCassR.Sunstein,TheLimitsofQuantification,102CALIF.L.REV.
1369 (2014); (focusing on the problem of benefits that can’t be quantified“simplybecausewelackrelevantinformation”);RichardL.Revesz,QuantifyingRegulatoryBenefits, 102CALIF.L.REV.1369,at1376 (2014);DanielA.Farber,BreakingBad?TheUneasyCaseforRegulatoryBreakevenAnalysis,102CALIF.L.REV. 1469 (2014); Lisa Heinzerling, Quality Control: A Reply to ProfessorSunstein,102CALIF.L.REV.1457(2014);Masur&Posner,supranote5.
***DRAFT***
32
CBA skeptics often mention the problem in passing butthen promptly move on to more meaty theoretical problems.97Proponents,ontheotherhand,oftenacknowledgetheprobleminintroductoryremarksbutthenassumeitawayintheiranalysis.98Most seem to implicitly assume that inmost cases unquantifiedbenefits are insignificant and do not prevent a meaningfulcomparison of monetized benefits to monetized costs.99Theymay, for example, in discussing a particular CBA, acknowledgethat significant categories of benefits were omitted from thebenefitscalculationbut then insist that the(relativelycomplete)costs estimate can still be subtracted from the (relativelyincomplete)benefitsestimatetoobtainameaningfulmeasureofnet benefits. This constitutes a corruption of CBA that I havepreviously dubbed “false formality,” in which the analystinappropriatelyand illogically combinesan informalpositiononAxis1withaformalpositiononAxis2.100 Thiskindoffalseformalityoccurseveryyearinanannualstatutorilyproscribed ritualbywhichOIRAreportson the costsandbenefitsoffederalregulation.101Inthisreport,OIRAaddsupall themonetizedcostandbenefit figurescontained in theCBAsfor major regulations issued by federal agencies during theprevious ten-yearperiodand calculates anumberpurporting torepresent the net benefits of all federal regulation. The 2014report, for example, announced that the annual net benefits offederalregulationduringPresidentObama’sfirsttermwere$200
97See,e.g.,AmySinden,InDefenseofAbsolutes:CombattingthePoliticsof
PowerinEnvironmentalLaw,90IOWAL.REV.1405,1425(2005).98See,e.g., JohnGraham,TheEvolvingRegulatoryRoleoftheU.S.Officeof
ManagementandBudget,1REV.OFENVTL.ECON.&POL’Y171,188(2007)(“Theinformation base on which we made multibillion-dollar decisions was oftenremarkablyslim.”).
99See, e.g., sources cited infra notes – to – and accompanying text;JonathanMasur&EricPosner,AgainstFeasibilityAnalysis,77U.CHI.L.REV.657,700 (2010) (“[Cost-benefit analysis] minimizes decision costs through themagic of quantification. Once valuations are obtained from themarketplaceand surveys—fixed costs that can be spread across multiple regulations—decisionarerelativelyautomatic.”).
100Sinden, Formality, supra note 19, at 93; Sinden, Ben Franklin, supranote18,at1177.
101 Office of Management and Budget website,https://www.whitehouse.gov/omb. OMB is required under the RegulatoryRight-to-KnowActtosubmitthisreportannuallytoCongress.31U.S.C.§1105note (enacted as Section 624 of the Treasury and General GovernmentAppropriationsActof2001).OMBdelegatesthetasktoOIRA.
***DRAFT***
33
billion.102 You have to read several pages further ahead beforeyou see the important caveat that these results “are neitherprecise nor complete,” and that “it is not always possible toquantifyortomonetizerelevantbenefitsorcostsofrulesinlightoflimitsinexistinginformation.”103 Outsideobserverscompoundtheproblembycontinuingtoignorethelimitationsposedbyunquantifiedbenefits.HahnandTetlock, for example, looked at OIRA’s 2006 report to Congressandconcludedthatfor“the95majorrulesfrom1995to2005forwhich substantial benefits and costs were monetized in theregulatoryanalysis,...14ofthe95[were]likelytofailabenefit-cost test.”104 They offered no explanation for their confidencethat the “substantial”monetizedbenefitswere completeenoughto warrant a strict comparison of costs to benefits and theconclusion that a rule “failed” a CBA when the (presumablyrelatively complete) costs estimate outweighed the (possiblypartial)benefitsestimate.TheReportitselfsimplystatedthat“inmanyinstances,agencieswereunabletoquantifyallbenefitsandcosts . . . [and that] [t]he monetized estimates . . . present[ed]necessarily exclude these unquantified effects.”105 Earlier in thesame article, Hahn and Tetlock themselves “readilyacknowledge[d]” the “difficulties inplacingamonetaryvalueoncertain key benefits,” and “le[ft] open the possibility that inparticular cases unquantifiable costs or benefits may tip thebalance.” But despite this disclaimer, they failed to note theinconsistency between the potential existence of significantunquantified benefits and the drawing of definitive conclusionsaboutnetbenefits.106
1022014OFFICEOFMGMT.&BUDGET,EXEC.OFFICEOFTHEPRESIDENT,ANN.REP.TOCONGRESSONTHEBENEFITSANDCOSTSOFFED.REG.ANDUNFUNDEDMANDATESONSTATE,LOCAL,ANDTRIBALENTITIES,at3[hereinafter“2014REP.TOCONGRESS”];seealsoSUNSTEIN,VALUINGLIFE,supranote67,at36(usingnetbenefitsfiguresfromthe OMB annual reports to assert that the net benefits of all economicallysignificantregulationsissuedduringPresidentObama’sfirstthreeyears(whenSunsteinwasOIRA administrator)were25 times as high as during theBushAdministrationandsixtimesashighasduringtheClintonadministration).
1032014REP.TOCONGRESS,supranote102,at6-7;seealsoid.at19(“manyofthesemajorruleshaveimportantnon-quantifiedbenefitsandcosts”).
104Robert W. Hahn & Paul C. Tetlock, Has Economic Analysis ImprovedRegulatoryDecisions?,22J.ECON.PERSPECTIVES67,71(2008).
1052006 OFFICEOFMGMT.&BUDGET,OFFICEOF INFO. ANDREG.AFFAIRS,ANN.REP.TOCONGRESSONTHECOSTSANDBENEFITSOFFED.REG.ANDUNFUNDEDMANDATESONSTATE,LOCAL,ANDTRIBALENTITIES,at2n.9.
106Hahn & Tetlock, supra note 104, at 69. See also RobertW. Hahn &RobertE.Litan,CountingRegulatoryBenefitsandCosts:LessonsfortheUSandEurope,8 J.INT’L.ECON.L.473,483-84(2005) (acknowledging, inastudyof5
***DRAFT***
34
A similar disconnect appears in an empirical study byHahnandDudleythatexamined74EPACBAspublishedbetween1982 and 1999 with the aim of assessing their quality.107 Notsurprisingly, inmany instances, benefitswere notmonetized atall. But fifty percent of the CBAs in the samplemonetized “atleast some benefits.”108 The study made no assessment of themagnitude or significance of the benefits left unmonetized.Instead,bythenextpage,theauthorsappearedtohaveforgottenthat there were any unquantified benefits at all, faulting theagencies fornot calculatingnetbenefits inall those instances inwhich“atleastsomebenefits”werequantifiedormonetized:
Of the rules in the sample thatquantifiedbenefits,only 74 percent calculated . . . net benefits. Thissuggeststhatcomparisonsofcostsandbenefitsarenot occurring in a largenumberof cases forwhichthenecessarydataareactuallyavailable.109
But, of course,where themost that can be said is that “at leastsomebenefitsweremonetized,”thereisnoreasonatalltobelievethat “the necessary data are actually available” to calculate netbenefits. Ina2012study,ShapiroandMorall setout todeterminewhether rules with good CBAs have higher net benefits. Theyconducted an empirical analysis of 109 rules issued by federalagencies between 2000 and 2009.110 They restricted their dataset to rules “that included data on the monetized costs andbenefitsofregulation,”butgavenoindicationwhethertheseCBAsmonetized all or even a significant portion of costs andbenefits.111Nonetheless,theauthorscalculatedthe“netbenefits”ofeachrulebasedon thenumbersprovidedby theagencies(orOIRA)andmadethat their “keydependentvariable.” TheythenOMBannualreports,that“mostregulations[inonecase23of34]eitherdonotprovideenoughinformationtocomparecostsandbenefits,orthereisa largeenoughrangeofuncertaintyintheagencies’estimatestoputtheregulationsinagrayarea,”butsubsequentlyasserting“[a]ggregatenetbenefitscanprovideusefulinformationonwhetheraparticularsetofregulationsorprogramsareenhancingeconomicwelfare”).
107 Robert W. Hahn & Patrick M. Dudley, How Well Does the U.S.GovernmentDoBenefit-CostAnalysis?,1REV.OFENVTL.ECON.&POL’Y192(2007).
108Id.at200.109Id.at201(emphasisadded).110StuartShapiro&JohnF.MorrallIII,TheTriumphofRegulatoryPolitics:
Benefit-CostAnalysisandPoliticalSalience,6REG.&GOVERNANCE189(2012).111Id.at194.
***DRAFT***
35
“scored”eachCBAalonga6-pointscalebyansweringasetofsixyes/no questions. One question related to quantification/monetizationandsimplyasked“[d]oestheanalysisquantifyandmonetizebenefitsandcostsofproposedaction?”112Toreceivea“yes” on this question “monetization did not need to becomprehensive, justthoroughenoughtodemonstratethatmajorbenefitsandcostshadbeenmonetized.”113Theythencomparedtheir calculated “net benefits” with the quality scores each rulereceived on the 6-point scale to determinewhether theirwas acorrelation between the quality of the analysis and the netbenefitsoftherule.
Treating the “net benefit” numbers that they calculatedaccordingtotheagency’smonetizedcostandbenefitestimatesasthe “key dependent variable,” of course, required an implicitassumption that those numbers actually bore some relationshipto the actual net social benefits of the rules, which, in turn,required an implicit assumption that anybenefits (or costs) leftunquantified or unmonetized were insignificant. The authorsappeared surprised and perhaps a little disappointed by theirresults: “Weare forcedtoconcludethat the levelofdetail in theanalysis has little impact on the net benefits of the rule.”114Nowhere was the possibility entertained, however, that theirmeasureoftherules’“netbenefits”mightitselfbeunreliableandbear no consistent relationship to the actual net benefits of therulestosociety. CassSunsteinappearedtomakeasimilaranalytic leap inhis recent article on unquantified benefits, when he mentionedoffhandinafootnote,thatwheresomebenefitsareunquantified,agencies should “to the extent permitted by law, . . . select theapproachthatmaximizesnetbenefits.”115Yet,inthoseinstancesin which the benefits left unquantified are significant, ameaningful calculation of net benefits for any alternative is, ofcourse, impossible, as is a determination of the point of netbenefitsmaximization.116
112Id.at195.Theotherquestionsaskedaboutastatementofneedforthe
regulation, analysis of alternatives, discounting, and presentation ofuncertainties.Id.
113Id.at196.114Id.at197.115Sunstein,LimitsofQuantification,supranote96,at1392n.79.116ThiswouldonlymakesenseifoneadoptsArdenRowell’spositionthat
non-monetizablebenefitsshouldbeentirelyignoredinCBA,SeeArdenRowell,PartialValuationinCost-BenefitAnalysis,64ADMIN.L.REV.723,741(2012)(arguingthatwherebenefitsareunquantifiableduetoincommensurability,
***DRAFT***
36
Even those who recognize the possibility of significantunquantifiedbenefitsasaproblemtendtodiscountthefrequencyofitsoccurrence,treatingitastheunusualorexceptionalcase.117Thistypicallyleadstotheoptimisticsuggestionthattheproblemcan be solved simply by a renewed commitment to researchaimedatquantifyingunquantifiedbenefits.118 One exception is a recent article by JonathanMasur andEric Posner, Unquantified Benefits and Bayesian Cost-BenefitAnalysis.119 In it, they described the results of an empiricalanalysisoftheCBAsassociatedwith106majorregulationsissuedbyfederalagenciesfrom2010through2013.Theyfoundthattheagencieswereonlyableto“fullyquantify”thecostsandbenefitsfor two rules. For 48 theywere able to partially quantify costsand benefits. And for 56, the agencywas unable to attach anynumber to costs or benefits or both. (Most of those—47—involved entirely unquantified benefits.) In over 74 percent oftheregulations,thereasonforthelackofquantificationwaswhatMasur and Posner called “empirical uncertainty—missing data,modelingdifficulties,orotherrelatedeffects.”120
Masur and Posner made no effort to systematicallyevaluatethemagnitudeoftheunquantifiedbenefits,thoughtheydid remark that “in somecases, it appears that theunquantifiedbenefitscouldbequitelarge.”121Despitetheapparentmagnitudeand pervasiveness of the problem,Masur and Posner remainedoptimistic about the ability of agencies to quantify the
theyshouldsimplybeexcludedandCBAconductedusingonlymonetizablecostsandbenefits:“thereisnoroomtoallownon-monetizablebenefitstoaffecttheoutcomeofamonetarycost-benefitanalysis.”).Butthat’snotapositionSunsteinappearedtoendorseintherestofhisarticle.Hedoes,however,atalaterpointinthearticle,acknowledgethatwheresignificantbenefitsareunquantifiedtheagencymight“haveagreatdealofdifficultyindecidingwhichapproachmaximizesnetbenefits.”Sunstein,LimitsofQuantification,supranote96,at1394.
117See Farber, supranote96, at1485 (“Even if . . . arguments [formorequalitative approaches] are rejected as applied to more typical regulatoryproblems. . .theymayhaveadditionalforceinsituationswhereuncertaintiesadmittedlyloomlarge.”).
118 Revesz, supra note 96; John Graham, Saving Lives throughAdministrative Law and Economics, 157 U. PENN. L. REV. 395, 526 (2008).Graham also suggests adopting a star rating system indicating the relativeimportanceofunquantifiedbenefits,seeid.at524-26;JohnGraham,ManagingtheRegulatory State: TheExperienceof theBushAdministration, 33 FORDHAMURBANL.J.953,992-93(2006).
119Masur&Posner,supranote5.SeealsoMcGartland,supranote6.120Id.at15.121Id.at19.
***DRAFT***
37
unquantifiedbenefitsbyuncoveringandanalyzingmoreexistingdata, generating new data, and where those approaches fail,employing Bayesian analysis tomake reasonable guesses aboutthe benefits of regulation “based on the experience and latentknowledgeoftheagencystaff.”122 Thisliteraturebegsanimportantempiricalquestion:HowoftenarethebenefitsthatagenciesleaveunquantifiedinCBAofasufficientmagnitude that theycannotsimplybedismissedasdeminimis?ThenextsectiondescribesanempiricalstudyinwhichIattempted to compile at least some preliminary data on thatquestion.
II.METHODSANDRESULTS
InordertogainabetterunderstandingoftherolethatunquantifiedbenefitsactuallyplayinthepracticeofCBA,Iconductedanempiricalstudyof32CBAsofmajorrulesconductedoveraten-yearperiod.UnlikeMasurandPosner,whoincludedintheirdatasetallfederalagenciesissuingmajorregulations,Iconfinedmystudytoasingleagency:EPA.IchosetofocusonEPAbecauseitisoftenheldupasthefederalagencymostpracticedinperformingCBAsandmostsophisticatedinitsapproach.123Internationally,theU.S.is,inturn,heldupbyanalystsinothercountriesastheworldleaderinthepracticeofCBA.124Accordingly,whilethedirectimplicationsofmyfindingsrelatespecificallytoEPA,theyalsohavelikelyindirectimplicationsfortheuseofCBAinagencyrulemakingmoregenerally,atleastintheenvironmentalarena.
A. TheDataSet
The data-set with which I began consisted of the CBAsconductedbyEPAinconnectionwith37majorfinalrulesissuedbetweenOctober1,2002andSeptember30,2012.Iusedthelistof rules identified in theOMB’s2013annual report toCongress
122Id.at5.123See,e.g.,Masur&Posner,supranote5,at23-25(holdingupEPAasone
of3federalagenciesmostpracticedatCBAandproducingmorequantifiedestimatesofbenefitsthanotheragencies).
124RENDA,supranote15,at7-8(callingtheUSexperiencewithCBA“thepolestar”forEUpolicymakers).
***DRAFT***
38
ontheCostsandBenefitsofFederalRegulation.125Forpurposesofthisreport,OMBdefinesamajorruleasonethatisexpectedtohaveanannualeffectontheeconomyof$100millionormore,orexhibitssomeotherindiciaofsignificanteconomicimpact.126
I excluded from the data set five rules that werequalitatively different in kind from the others. Onewas issuedjointly with the Department of Transportation;127I excluded itsimplytokeepthestudycleanly focusedonEPA.Theother fourwere deregulatory in nature.128 In deregulatory rules, the costsandbenefitsareessentiallyflipped.Coststoindustrythatwillbeavoided due to the removal of regulatory restrictions becomebenefits. Any benefits to public health or safety that will beforegone due to the loosening of regulation become costs.Accordingly, the kinds of challenges that ordinarily makequantification of regulatory benefits so difficult in theenvironmentalcontextaremissing insuchrules(orariseonthecost side instead). This renders deregulatory rules qualitativedifferentinkindfromotherrules.
This left 32 rules in the finaldata-set. Of those32 rules,the vast majority (24) were Clean Air Act Rules. Four werepromulgated under the Clean Water Act, two under the SafeDrinkingWaterAct,andtwoundertheToxicSubstancesControlAct. B.Axis1:Quantification
1252013OFFICEOFMGMT.&BUDGET,EXEC.OFFICEOFTHEPRESIDENT,ANN.REP.
TOCONGRESSONTHEBENEFITSANDCOSTSOFFED.REG.ANDUNFUNDEDMANDATESONSTATE,LOCAL,ANDTRIBALENTITIES,at3[hereinafter“2013REP.TOCONGRESS”].
1262013REP.TOCONGRESS,supranote125.Forpurposesofthereport,OMBdefinesmajorruletoincludeanyrulethat:1)fallsinthedefinitionof“majorrule”undertheSmallBusinessRegulatoryEnforcementFairnessActof1996,5U.S.C.§804(2);2)meetstheanalysisthresholdundertheUnfundedMandatesReformActof1995,2U.S.C.§1532(a);or3)fallsinthedefinitionof“economicallysignificant”undersection3(f)(1)ofE.O.12866.
127 2017 and Later Model Year Light-Duty Vehicle Greenhouse GasEmissions and Corporate Average Fuel Economy Standards, 77 Fed. Reg.62,624(Oct.15,2012)(codifiedat40C.F.R.§85).
128 Oil Pollution Prevention; Spill Prevention, Control, andCountermeasure (SPCC) Rule-Amendments for Milk and Milk ProductContainers,76Fed.Reg.21,652 (April18,2011)(codifiedat40C.F.R.§112);OilPollutionPrevention;SpillPrevention,Control,andCountermeasure(SPCC)Rule-Amendments,74Fed.Reg.58,784(Nov.13,2009)(codifiedat40C.F.R.§112);RevisionstotheDefinitionofSolidWaste,73Fed.Reg.64,668(Oct.30,2008) (codified at 40C.F.R. §260);Oil PollutionPrevention; Spill Prevention,Control, and Countermeasure Plan Requirements-Amendments, 71 Fed. Reg.77,266(Dec.26,2006)(codifiedat40C.F.R.§112).
***DRAFT***
39
1.SignificantCategoriesofBenefitsUnquantified
All of the 32 CBAs in that final data set left multiplecategories of benefits unmonetized. None indicated that themonetized benefits estimate was complete or included allsignificant benefits. In two instances, the agencywas unable tomonetizeanybenefitsatall.In27outof32(84%),EPAexcludedfrom itsmonetized benefits estimate categories of benefits thatthe agency itself described as either actually or potentially“important,” “significant,” “substantial,” or (in one case) “manytimes larger” than the quantified benefits.129 In 15 cases, thatcharacterization was unequivocal. In the other 12 cases, thecharacterization of the magnitude of the omitted benefitcategorieswasaccompaniedbysomedegreeofequivocatione.g.,“potentially,” “likely to be,” or “possibly” significant.130 Theseresultsaredepictedgraphicallybelowinfigure3:
129See,e.g., EPA,Regulatory ImpactAnalysis (RIA) forExistingStationary
Compression Ignition Engines NESHAP 7-11 (Feb. 2010) [hereinafter“StationaryCINESHAPRIA”](“Data,resource,andmethodological limitationsprevented EPA from quantifying or monetizing the benefits from severalimportant benefit categories, including benefits from reducing carbonmonoxide and hazardous air pollutants, ecosystem effects, and visibilityimpairment.”).ThisisconsistentwithMasurandPosner’sfindingsthatinsomecases, at least, “it appears that the unquantifiable benefits [of federalregulations]couldbequitelarge.”SeeMasur&Posner,supranote5,at19.
In one of those cases, (2060-AM82) the RIA used a benefits transferapproach–sothattheRIAitself(whichwasshort)didnotmakeastatementabout unquantified benefits, but it incorporated by reference the RIA fromanotherrule,whichmademanysuchstatements.
130 In a few instances, the language used was slightly different butamountedtothesamething.SeeRegulatoryImpactAnalysisoftheFinalCleanAir Mercury Rule (March, 2005) (“the benefits of reduced cardiovasculareffects (from fatalandnon-fatalheartattacks) ifquantifiedcouldpossiblybemany times larger than those we are able to quantify in this section of thereportduetothepotentialformortalityeffects(monetizedwiththevalueofastatistical lifewhichismuchhigherinvaluethanIQloss)”;NationalPollutantDischarge Elimination System—Final Regulations to Establish RequirementsforCoolingWaterIntakeStructuresatPhaseIIExistingFacilities,69Fed.Reg.41,576, 41660-61 (2004) (discussing in several places the “importance” ofnon-usebenefits,whichtheagencywasunabletoquantify).
***DRAFT***
40
FIGURE 3. CBA’s of “economically significant” EPA rules: Oct. 1,2002throughSept.30,2012. EPAuseda“+B”torepresenttheunquantifiedbenefits inthefinaltallyonlysporadically,insixinstances.Oneofthosesix,discussed below, was among the five CBAs in which did notexplicitlycharacterizeunquantifiedbenefitsassignificant.
For most of the rules in the data set, the monetizedbenefits exceeded the monetized costs. But in ten instances,monetized benefits fell short of costs, either definitively, or,wherecostsandbenefitswereexpressedinranges,thoserangesoverlappedtocreateatleastthepossibilityofabenefitsshortfall.In each of these instances, the shortfall was substantial (morethan40percentoftotalcosts)inatleastpartoftherange.Butinsevenoutof the ten instances, theagencydescribed themissingbenefitsas “important,” “significant,”or “substantial,” suggestingan inconclusive litmus-test CBA. Nonetheless, the agency onlyconductedabreak-evenanalysisfortwoofthetenrules. 2.ReasonsforLackofQuantification
First, itmaybehelpful to recall the typologyof potential
reasonsforlackofquantificationdevelopedinPartI.A.2:
1) Insufficientinformationtoconduct:
15
12
5
"Important,""signizicant,"or"substantial"categoriesofbenezitsunquantizied
"potentially""important,""signizicant,"or"substantial"benezitsunquantizied
magnitudeofunquantiziedbenezitsnotdescribed
***DRAFT***
41
a. Hazardidentificationb. Dose-responseevaluationc. Exposureassessment
2) Insufficientdata/modelstomonetize3) Incommensurability4) Lackoftime,money,orresources
For all of the rules in the data set, EPA cited lack of
information,dataormethods(categories1and2)astheirreasonforfailingtoquantifycertainbenefits.Surprisingly,wefoundnoreferencesatalltoincommensurabilityortheintangiblenatureofthebenefits.131
Some described the reasons only in generic terms (e.g.,“methodologyanddata limitations”132)so that itwas impossibleto discern the specific obstacles that prevented quantification.But some were more specific. Categories 1c (insufficientinformationtoconductexposureassessment)and2(insufficientdata or models to monetize) were the most commonly citedspecific reasons (cited12and13 timesrespectively). EPAcitedinsufficientinformationtoconducthazardidentificationordose-response evaluation less frequently (5 times and 9 times,respectively). In 14 instances, EPA also cited time and/orresourcelimitationsasreasons.
Figure4,below,summarizestheseresults:
131ThisisgenerallyconsistentwithMasurandPosner’sfindingsthatin74percentofinstancesinwhichagenciesfailedtofullyquantifybenefitsorcosts,theycited“empiricaluncertainty”asthereason.Agency’sassertedbenefitstobe“unquantifiableinprinciple”inonly9outof104instances.Masur&Posner,supranote5,at104.132See,e.g.,EPA,RegulatoryImpactAnalysisfortheFinalMercuryandAirToxicsStandards68(Dec.2011).
***DRAFT***
42
Figure4.NumberofCBAscitingvariousreasonsforlackofquantificationofbenefits.(Note:Numbersdonotsumto32becauseinmostinstances,EPAcitedmultiplerationales.)
C.Axis3:AlternativesThe number of alternatives for which cost and benefits
were estimated ranged from one to 12, with the vast majorityfallingonthelowendofthatspectrum. Figure5,below,depictsthese results graphically. For nearly half of the rules (14), theCBAexaminedonlyasinglealternative, that is, theregulationaspromulgated. In ten instances, the number of alternatives wasbetween2and5.(Thehighestnumberofalternativeswas12.)
***DRAFT***
43
Figure5.NumberofCBAsconsideringvaryingnumbersofalternatives.D. DiggingDeeper:TheStoryBehindtheNumbers
1. TheOutsizedRoleofParticulateMatterandthe
UndersizedRoleofToxics
Oncewestartdiggingbeneaththenumberstolookatthekindsofbenefits thatwerequantifiedand leftunquantified,oneparticularbenefitcategorystandsoutheadandshouldersabovethe rest: particulate matter—or, more specifically, the publichealthbenefitsofreducingparticulatematter.Ofthe24CleanAirAct rules in the sample, 10 quantified only particulate matterbenefits,eventhough,ineachinstance,therulewasalsoexpectedto reduce multiple other harmful pollution agents—typicallyozone, sulfur dioxide, oxides of nitrogen, volatile organiccompounds,and/orhazardousairpollutants.
Indeed, in many instances, the particular pollutants atwhich therulewasspecificallyaimedwerenotmonetizedatall.In these instances, all of thebenefit dollarswere attributable totheancillarybenefits(or“co-benefits”)arisingfromthefactthatthemeasurestakentoreducethetargetedpollutantswouldalsohappentoreduceparticulatematterlevels.Thiswasparticularlytrue of the rules aimed at air toxics, or, in the parlance of theCleanAirAct,“HazardousAirPollutants”(alsoknownas“HAPs”).Ten of the rules in the final data set were Clean Air Act rules
***DRAFT***
44
specificallyaimedatthecontrolofHAPs.Ironically,withonlytwoexceptions, these CBAs left completely unmonetized all of thebenefits of reducing the very harm against which the standardwas aimed: HAPs. Those that monetized any benefits at allmonetizedsolely theco-benefitsofparticulatematter reductionsthathappentooccurasabyproductofpollutioncontrolsaimedatHAPs.133
The HAPs for which benefits remained unquantifiedincludealonglistofchemicalsknowntobedangeroustohumanhealth, like benzene, formaldehyde, polycyclic organic matter(POM), toluene,MTBE,arsenic,chromium, lead,andmanymore.Many of these compounds have been listed for many years asknownorprobablecarcinogens.Manyarealsolinkedtoavarietyof other impacts on human health, including immunotoxicity,genotoxicity, nervous system disorders, respiratory effects, andreproductiveanddevelopmentaleffects. Throughdynamicsthatareevenlesswellstudied,thesecompoundsalsohavedeleteriouseffects on other species (plant and animal) and therefore onecosystemstructureandfunction.134
The two instances inwhich EPA did include some of thebenefitsofHAPreductionintheirbenefitsestimatewerethetwoiterations of the EPA’s rule limiting emissions of mercury andother toxics frompowerplants (theBushadministration’sCleanAirMercury Rule and the Obama administration’sMercury andAirToxicsStandards).135IntheCBAsaccompanyingeachoftheserules, EPA did monetize some of the benefits of reducingemissions of one of theHAPs (mercury), but itwas only a verysmall slice of those benefits. First, while there is evidence thatmercurycausesarangeofnastyimpactsonhumanhealth—fromdecreased neurological function and loss of finemotor skills toadverse cardiovascular effects, genotoxic effects, andimmunotoxic effects—EPA included just one endpoint in itsanalyses: decreases in IQ in children exposed prenatally to
133Ofthe8CBAsthatleftHAPscompletelyunmonetized,twofailedto
monetizeanybenefitsatall.Theother6monetizedonlyPMco-benefits.134SeeEPA,RegulatoryImpactAnalysisfortheStationaryInternal
CombustionEngine(RICE)NESHAP7-3(Feb.2004).135EPA,RegulatoryImpactAnalysisfortheFinalMercuryandAirToxics
Standards,EPA-452/R-11-011(December2011);EPA,RegulatoryImpactAnalysisoftheFinalCleanAirMercuryRule,EPA-452/R-05-003(March2005).SeealsoCatherineA.O’Neill,TheMathematicsofMercury,inREFORMINGREGULATORYIMPACTANALYSIS108(WinstonHarringtonetal.eds.,2009);LisaHeinzerling&RenaSteinzor,APerfectStorm:MercuryandtheBushAdministration,PartII,34ELR10,485,10,489(2004).
***DRAFT***
45
mercury from theirmother’s consumption of fish.136Yet, there’sreasontobelievetheothereffectsmaywellbesubstantial.
One study suggests that the cardiovascular benefits ofmercuryreductions,forexample,maybeontheorderof17timesthe IQ benefits. 137 Second, even though “commercial fishconsumption constitutes a large portion of exposure tomethylmercury,”138EPAreducedthepopulationaccountedfor inits analysis by a factor of fifteenby counting only consumers ofrecreationally caught fish from inland waters.139 Third, in astartling descent into Alice-in-Wonderland logic, the agencydiscounted the benefit of reducing mercury poisoning (andavoiding IQ loss) to account for the fact that had the benefitedkidsactuallylostIQpointsduetomercurypoisoning,theycouldhavesavedmoneyontuition,becausekidswithlowerIQsattendfeweryearsofschool.
In sum, there aremany reasons to conclude that, inbothinstances,thedollarfigurestheagencyarrivedatforthebenefitsof mercury reduction significantly under-estimated the actualbenefits.
2. MissingBenefits:EcologicalEffects
Ecologicalbenefitswere similarlyunder-quantified. Only
five CBAs (four of them Clean Water Act rules) included any
136SeeEPA,RegulatoryImpactAnalysisfortheFinalMercuryandAir
ToxicsStandards4-30(Dec.2011);U.S.EPAOfficeofAirQualityPlanningandStandards,RegulatoryImpactAnalysisoftheFinalCleanAirMercuryRule,EPA-452/R-05-003,3-10to3-14(March2005)[hereinafter“MercuryRIA”],availableat:http://www.epa.gov/ttn/ecas/regdata/RIAs/mercury_ria_final.pdf.Mercuryemittedfrompowerplantsiscarriedbywindsthroughtheairandeventuallydepositedonwaterandland.Onceinthewater,somemercuryistransformedtomethylmercury,ahighlytoxicformofthechemical,whichisingestedbyorganismslowontheaquaticfoodchainandeventuallybioaccumulatesinfish.Thereisconsiderableevidencethatchildrenexposedtomercuryinuterofromtheirmothers’consumptionofcontaminatedfishexhibitdecreasesinIQ.
Mercuryobviouslycanalsobeexpectedtohaveimpactsonotherspeciesandecosystems.EPAmadenoefforttoquantifytheecologicaleffectsofmercuryemissions,however.SeeMercuryRIAat2-8.
137SeeGLENNRICE&JAMESK.HAMMITT,ECONOMICVALUATIONOFHUMANHEALTHBENEFITSOFCONTROLLINGMERCURYEMISSIONSFROMU.S.COAL-FIREDPOWERPLANTS189(HarvardCenterforRiskAnalysis,Feb.,2005)(estimatingIQbenefitsatbetween$194millionand$288millionannually,andcardiovascularbenefitsatbetween$3.3billionand$4.9billionannually).
138SeeMercuryRIAat4-1.139Seeid.at4-46.
***DRAFT***
46
quantification of ecological benefits at all, and thesewere in allcases incomplete. Twenty-one contained some qualitativediscussionofecologicalbenefits,rangingfromagenericmentionof the fact thatsuchbenefitsmightexist, toanextensive (multi-page) qualitative description of ecological benefits butmade noefforttoquantify.AndfourCBAsmadenomentionofecologicalbenefits at all. (A search for the terms “ecology,” “ecological,” or“ecosystem”cameupempty.)
3. TheOtherFiveRules
As noted above, none of the CBAs contained quantified
benefits estimates that were complete or near complete. Therewere five that did not specifically characterize unquantifiedbenefits as “significant,” “important,” or “substantial,” but eventhese presented monetized estimates that left out wholecategories of benefits. And when one digs into the details ofthese, it appears at least plausible that the benefits leftunquantifiedintheseCBAswerealsosignificant.Theagencyjustdidn’thappentodescribetheminpreciselythoseterms.
Two involved rules aimed at reducing levels of airbornelead—theNational Ambient Air Quality Standard for lead and aruleamendingtheregulationofrenovationsinvolvinglead-basedpaint.140Airborneleadisassociatedwithawholehostofadversehealthimpacts,includingreproductive,developmental,cognitive,neurobehavioral, cardiovascular, renal, immunotoxicity,genotoxicity,andcarcinogeniceffects.141ButineachofthesetwoCBAs,onlyanarrowsliceoftheseregulatorybenefitswasactuallymonetized.
Fortheleadpaintrule,theCBAmonetized“avoidedlossesinexpectedearningsduetoIQdrop[inchildrenundertheageof6],” and avoided medical costs for cardiovascular diseases inadults,butleftoutallotherhealtheffects,manyofwhicharevery
140EPA,FinalRule,Lead;AmendmenttotheOpt-outandRecordkeeping
ProvisionsintheRenovation,Repair,andPaintingProgram,75Fed.Reg.24802(May6,2010);EPA,FinalRule,NationalAmbientAirQualityStandardsforLead,73Fed.Reg.66964(Nov.12,2008).
141EPA,EconomicAnalysisfortheTSCALeadRenovation,Repair,andPaintingProgramOpt-outandRecordkeepingFinalRuleforTargetHousingandChild-OccupiedFacilities,at5-12n.5(Apr.2010)[hereinafter“LeadOpt-outEA”];EPA,RegulatoryImpactAnalysisoftheProposedRevisionstotheNationalAmbientAirQualityStandardsforLead(Oct.2008)[hereinafter“LeadNAAQSRIA”]at5-10to5-11.
***DRAFT***
47
wellestablished.142 Italso leftoutcertaingroupsofpeoplewhowouldreceiveIQandcardiovascularbenefits.Forexample,itdid“not include individuals living in adjacent (detached) houses orchildrenwhospendtimeinafriendorrelative’shouserenovatedunder the rule.”143 Nor did the number for IQ benefits includepeople other than children under 6.144 The CBA for the leadNAAQS monetized only avoided IQ losses for children under 7(basedon future earnings impacts) aswell as the co-benefits ofreducingdirectemissionsofparticulatematter.145(Airborneleadusuallytakesparticulateform.)Butalloftheotherhealtheffectsofleadwereleftout.146
Two of the rules involved measures to reduce waterpollution.IntheCBAfortherulesettingwaterqualitystandardsfor nitrogen and phosphorous in Florida, EPAwent through anelaborate process to try to monetize the largely ecologicalbenefits using contingent valuation surveys.147 But the agencycandidlyacknowledgedthat“these . . .estimatesdidnotaccountfor all potential economicbenefits.”148 Indeed,EPAappeared tohave so little confidence in the accuracy of this estimate, that itnever actually put it in a side-by-side comparisonwith costs.149
142LeadOpt-outEA,supranote141,at5-20;seealsoid.at5-12n.5.143Id.at5-6(“Thepopulationgroupsdiscussedbelowdonotreflectallof
theindividualsprotectedbyremovingtheopt-out,butinsteadrepresentgroupsforwhichcalculationscanbereadilymade.”).
144Id.at5-22;73Fed.Reg.at2481.145LeadNAAQSRIAatES-7,ES-10.146Id.Themonetizedbenefitsweresmallerthanthemonetizedcosts
(thoughtheagencywasnottechnicallysupposedtoconsiderthatfactinitsdecisionmaking.)Seeid.at1-1;Whitmanv.AmericanTruckingAss’ns,531U.S.457,468,471(2001).
147Thisprocessinvolvedtranslatingexpectedimprovementsinwaterqualityintoasinglenumerical“waterqualityindex,”andthenusingcontingentvaluationsurveystoestimatehowmuchresidentsofFloridawouldbewillingtopayforimprovementsinthatindex.SeeEPA,EconomicAnalysisofFinalWaterQualityStandardsforNutrientsforLakesandFlowingWatersinFlorida13-1(Nov.2010)[hereinafter“FloridaWQSEA”].
148Id.atES-12.SeealsoWaterQualityStandardsfortheStateofFlorida’sLakesandFlowingWaters;FinalRule,75Fed.Reg.75,762,75,802(2010)(samelanguage).
149EPAseemedtoviewitinsteadassimplyatoolgiveasenseofthegeneralmagnitudeofthepotentialbenefits.SeeFloridaWQSEAsupranote147,atES-12(“Althoughthesemonetizedbenefitsestimatesdonotaccountforallpotentialeconomicbenefits,theyhelptodemonstratetheeconomicimportanceofrestoringandprotectingFloridawatersfromtheimpactsofnutrientpollution.”);75Fed.Reg.at75802(samelanguage).Onerespectinwhichthemonetizedbenefitsestimatewasincompletewasitsgeographicscope:
***DRAFT***
48
Had it done so, the comparison would have shown monetizedcosts outweighingmonetized benefits. The fact that the agencydecided to go forward despite this is perhaps implicit evidencethatitviewedtheunquantifiedbenefitsassubstantial.150
The CBA accompanying EPA’s rule aimed at waterpollutionfromConcentratedAnimalFeedingOperations(CAFOs)alsoreliedoncontingentvaluationstudiestoquantifybenefitstosurface water quality and reduced contamination of privatedrinkingwaterwells.151Additionally, itmonetizedsomebenefitsinvolving commercial goods—reduced die-offs of cattle fromdrinking impaired waters and improved commercial shellfishharvesting.ButthelistofbenefitcategoriesthatEPAwasunableto monetize due to data limitations and monetizationdifficulties152waslongandencompassedbothhumanhealthandecological effects, including “reduced human illness due topathogen exposure,” and “reduced eutrophication of estuaries,”amongmanyothers.153 Indeed,EPA thought theseunmonetized
Inanalyzingbenefitsoftherule,EPAestimatedbenefitsfromnutrientreductionstoFloridahouseholdsonly.Althoughresidentsofotherstatesmayholdvaluesforwaterresourcesoutsideoftheirhomestate,ifsuchresourceshavepersonal,regional,ornationalsignificance,EPAdidnothavesufficientinformationtoestimateWTPforwaterqualityimprovementsinFloridaforoutofstateresidents.Asaresult,thepopulationconsideredinthebenefitsanalysisoftheruledoesnotrepresentallthehouseholdsthatarelikelytoholdvaluesforwaterresourcesinthestateofFlorida.EvenifperhouseholdWTPforout-ofstateresidentsaresmalltheycanbesubstantialintheaggregateifthesevaluesareheldbyasubstantialfractionofthepopulation.
FloridaWQSEA,supranote147,at13-16(emphasisadded).150TheprovisionsoftheCleanWaterActunderwhichthisrulewas
promulgatedarenotamongthose,discussedsupranotes–to–andaccompanyingtext,thatclearlyprohibittheagency’srelianceonCBAinsettingstandards.See33U.S.C.§1313.
151EPA,EnvironmentalandEconomicBenefitAnalysisofFinalRevisionstotheNationalPollutantDischargeEliminationSystemRegulationandtheEffluentGuidelinesforConcentratedAnimalFeedingOperations3-9,4-17to4-25(Dec.2002).
152NationalPollutantDischargeEliminationSystemPermitRegulationandEffluentLimitationGuidelinesandStandardsforConcentratedAnimalFeedingOperations(CAFOs);FinalRule,68Fed.Reg.7,176,7,240(2003).
153AsdescribedbyEPAinthepreambletotherule,“Unqualifiedbenefitsincluded:
***DRAFT***
49
benefitsweresignificantenoughthat ituseda“+B”torepresenttheminthefinaltally.154
Finally, the last of these ruleswas the 2008New SourcePerformanceStandardsforpetroleumrefineriesundertheCleanAir Act, which reduced emissions of particulate matter (PM),sulfur dioxide (SO2), Nitrous oxides (NOx), and volatile organiccompounds (VOCs). But the only benefits that were quantifiedwere those stemming from reduced PM2.5—that which wasdirectly emitted as well as that which was formed in theatmosphereby“precursoremissions,”SO2,NOx,andVOC’s.ThisleftoutnotonlythedirecthealthandecologicalbenefitsofPM10,SO2, NOx and VOCs, (which include a number of toxiccompounds),butalsotheimportanthealtheffectsassociatedwithozone, theprimarycomponentofsmog,which isknowntohavesignificantadverseeffectsonpublichealthandwhichisformedintheatmospherebychemicalreactionsinvolvingNOxandVOCs.155
In sum, even with respect to those five rulemakings inwhichEPAdidnotactuallydescribetheunquantifiedbenefitsas“significant,”“important,”or“substantial,”themonetizedbenefitsestimateswerefarfromcomplete.Theyleftoutwholecategories
• Humanhealthandecologicaleffectsofmetals,antibiotics,hormones, salts, and other pollutants associated withCAFOmanure
• Eutrophication of coastal and estuarine waters due toboth nutrients in runoff and deposition of ammoniavolatilizedfromCAFOs
• Reducedhumanillnessduetopathogenexposureduringrecreationalactivitiesinestuariesandcoastalwaters
• Improvements to soil properties due to reduced over-application of manure, together with increased acreagereceivingmanureapplicationsatagronomicrates
• Reduced pathogen contamination in private drinkingwaterwells
• Reduced cost of commercial fertilizers for non-CAFOoperations
Id.154EPA,EconomicAnalysisoftheFinalRevisiontotheNationalPollutant
DischargeEliminationSystemRegulationandtheEffluentGuidelinesforConcentratedAnimalFeedingOperationsES-18,TableES-5(Dec.2002).
155EPA,FinalRule:StandardsofPerformanceforPetroleumRefineries,75FR35838,35862(2008)(“Itshouldbenotedthattherangeofbenefitsestimatesprovidedabovedoesnotincludeozone-relatedbenefitsfromthereductionsinVOCandNOemissionsexpectedtooccurasaresultofthisfinalrule,nordoesthisrangeincludebenefitsfromtheportionoftotalPMemissionsreductionthatisnotPM2.5.Wedonothavesufficientinformationormodelingavailabletoprovidesuchestimatesforthisrulemaking.”).
***DRAFT***
50
of benefits, which may well have been significant, even if notexplicitlydescribedbyEPAinthoseterms.
III.IMPLICATIONS:THEPROBLEMOFUNQUANTIFIEDBENEFITS
ThesefindingsstronglysuggestthatinthepracticeofCBA,unquantified benefits occur with significant frequency andmagnitude.Thenextquestionis,doesitmattertotheenterpriseofCBA? Thediscussion inPart I about themultiplevarietiesofCBA suggests that it may not matter, depending on the kind ofCBA the analyst is pursuing. Some varieties of CBA at theinformal end of the spectrumdon’t require quantification at all.As we have also seen, however, the CBA executive orders andaccompanying guidance charge the federal agencies withconductingformalEconomicCBA.And,forformalEconomicCBA,unquantifiedbenefitsposefundamentalproblems.
A. TheConstraintsImposedbyUnquantifiedBenefits
If we think in terms of the diagram in Figure 1 (PartI(A)(3)),wecansee thatsignificantunquantifiedbenefitsofanytypeputananalysis toward the left endofAxis1and thereforeconstrainthetypeofCBAthatananalystcanperform. AformalEconomic CBA that calculates the point of economic efficiencywhere marginal costs and benefits are equal, for example, isimpossible in such circumstances. Indeed, no standard settingCBAof anyvariety ispossible. Significantunquantifiedbenefitsconstraintheanalystinsteadtoalitmus-testCBA(atbest).156
Anotherway to see this is to beginwith the observationthatsignificantunquantifiedbenefitspreventacalculationofnetbenefits for any alternative. Imagine the most favorable case,where the partial estimate of benefits exceeds the completeestimate of costs.157 Here the analysis can adopt the middleposition on Axis 2 and apply a benefits-exceed-costs test, sincethe analyst can comfortably infer that the true benefits also
156 Uncertainty in benefits estimation poses a related problem and
similarly leads to constraints on the formality of the CBA that can bemeaningfullyperformed. Forexamples, seeDavidM.Driesen,TwoCheersforFeasibleRegulation:AModestResponsetoMasurandPosner,35HARV.ENVTL.L.REV.313,330(2011).
157Thiswastrueformostoftherulesinthedataset.Seesupranotes–to–andaccompanyingtext.
***DRAFT***
51
exceed the true costs.158 But she cannot know by how muchbenefitsexceedcostsandthereforecannotcalculatenetbenefits.Sincetheanalystcannotcalculatenetbenefitsforanyalternative,she also cannot use CBA to locate the net-benefits-maximizingalternative;onlyalitmus-testCBAispossible.159
In the data set examined above, then, EPA would havebeenlimitedtoalitmus-testCBAatbestinatleast84percentofthecases.160Accordingly,theresultsofthisstudysuggestthatallof the distinctions described above between a more formalstandard setting CBA on one hand and a less formal litmus-testCBA on the other are of particular importance to the actualpractice of CBA at EPA. Moreover, EPA’s analysis is in mostinstances hobbled by the substantial limitations of litmus-testCBA, includingthefactthat litmus-testCBAprovidesonlyaverybluntinstrumentforevaluatingtheefficiencyofregulations.
And, of course, as noted above, a successful litmus-testCBAisonlythebestcasescenario.161Ifthe(complete)costsare
158This, of course, assumes that one is comfortablewith themonetizedvaluesassignedtobeginwith.
159 This constraint on CBA with unquantified benefits is sometimesoverlooked.See,e.g.,Sunstein,LimitsofQuantification,supranote96,at1392,n.79 (suggesting that an analysis might be able to “select the approach thatmaximizesnetbenefits”evenwheresignificantbenefitsareunquantified),butseeid.1394(recognizingthatanagency“mighthaveagreatdealofdifficulty”using a break-even analysis to “decid[e] which approach maximizes netbenefits.”).
160Indeed,thepercentagecouldbeevenhigher.Recallthateveninthe5casesinwhichEPAdidnotexplicitlydescribetheunquantifiedbenefitsas“important,”“significant,”or“substantial,”itappearedthattheymightwellhavebeen.Moreover,EPAwasalsolimitedinmostcasestoa“litmustestCBA”becauseofthesmallnumberofalternativestheagencyconsidered.Seesupranotes–to–andaccompanyingtext.
161Rather than acknowledge this challenge, agencies often simply zerooutunquantifiablebenefits.Indeed,instrikingdowntheasbestosban,theFifthCircuit Court of Appeals encouraged the EPA to do exactly that when itfamouslysaid“Unquantifiedbenefitscan,attimes,permissiblytipthebalanceinclosecases.Theycannot,however,beusedtoeffectawholesaleshiftonthebalancebeam.”CorrosionProofFittingsv.EPA,947F.2d1201,1219(5thCir.1991).Wherebenefitsarenon-monetizablesolelyduetoincommensurabilityproblems,ArdenRowell takesasimilarview,arguing that theunmonetizablebenefits should be excluded from the analysis altogether. See Rowell, supranote 116, at 741 (“[T]here is no room to allow non-monetizable benefits toaffect theoutcomeofamonetarycost-benefitanalysis.”). This isnotbecausethey are “worthless; it is because monetary cost-benefit analyses deal withmoney,andnon-monetizablebenefits,bydefinition,havenovaluethatcanbeexpressedindollars.”Id.at732.Sheisalsocarefultomakeclear,however,thatshe does not necessarily think CBA “should be the sole determinant of legalpolicy.” Id.at741.Thus, inherview, incommensurablenon-monetizedvalues
***DRAFT***
52
greaterthanthe(incomplete)benefits,itismuchhardertoreacha definitive conclusion.162 The unmonetized benefits might ormight not be big enough to make up the difference. Thus, acomplete-costs-partial-benefits CBA produces an asymmetry: Ifthemonetizedbenefitsexceedthemonetizedcosts, itprovidesadefinitive result (at leastwith respect to litmus-testCBA),but ifcostsexceedbenefits,itdoesnot—oratleast,itprovidesahardercase.163
In the data set described above, monetized benefitsexceededmonetizedcosts inmost instances,but therewerestilltencasesinwhichtheydidnot,andsevenofthosewereinthesetof CBA’s that explicitly labeled the unquantified benefits as“significant,”“important,”or“substantial.”Thus,inatleastseveninstances (and probably more),164EPA had no way of knowingwhether unquantified benefitswould be large enough to tip thescalesandwasthereforeunabletoreachadefinitiveresult,evenunderalitmus-testCBA.
Thesearethehardercases.OIRAencouragesagenciestoconduct what it calls a “break-even” or “threshold” analysis inthesecircumstances,asthenextsectiondiscusses.165
B. Break-evenAnalysis
canbeconsideredbyagencies, justnotwithin“theledgersofmonetizedcost-benefitanalysis.”Id.at732.
162See Driesen, Neutral, supra note 6, at 401; Ronnie Levin, Lead inDrinkingWater,inECONOMICANALYSISATEPA:ASSESSINGREGULATORYIMPACT205,230 (Richard D. Morgenstern, ed. 1997). There are arguments against thisfrom both ends of the spectrum. Some argue that a determination aboutwhether benefits “justify” or “outweigh” costs can still be made wheresignificantbenefitsareunquantifiedbyperformingarough,intuitiveapples-to-oranges comparison. See infra notes – to – and accompanying text. ArdenRowell, on the other hand, argues that unmonetizable benefits are irrelevantandshouldbeexcludedfromCBA.SeeRowell,supranote116,at741(arguingthatwherebenefitsareunquantifiableduetoincommensurability,theyshouldsimply be excluded and CBA conducted using only monetizable costs andbenefits: “there is no room to allow non-monetizable benefits to affect theoutcomeofamonetarycost-benefitanalysis.”).
163 And of course, in such an instance there can be no meaningfulcalculationofnetbenefitsunlessitweretosetalowerbound.
164EveninthecasesinwhichEPAdidnotexplicitlylabeltheunquantifiedbenefitsassignificant,therewasreasontobelievethattheymightbe.SeesupraPartII.D.3.
165OMBCIRCULARA-4,supranote24,at2;seegenerallySunstein,LimitsofQuantification, supra note 96; Revesz, supranote 96; Farber, supranote 96;Heinzerling,supranote96.
***DRAFT***
53
Break-even (or “threshold”) analysis subtracts theincompletebenefitsestimate fromthe(complete)costsestimateandthenaskstheanalysttomakeanintuitivejudgmentwhetherthe remainingunquantifiable benefits are likely large enough tomakeupthedifference.166
Breakeven analysis essentially involves the analyst in akindof intuitive,apples-to-orangesbalancing—askingwhetheraset of benefits described in qualitative and/or non-monetizedquantitative terms is likely to outweigh some dollar cost. CassSunstein, however, argues that agencies can employ certainmethods to bring more “discipline” and “rigor” to breakevenanalysis, so that it is not based only on “intuition, dogma andanecdote.”167
Sunsteinsuggeststwospecificmethodsforaccomplishingthis. First, the analyst can try to establish monetized upperand/or lower bounds on the benefits estimate. If the datathemselvesdonotallowforsuchestimates,hesuggests that theanalyst try to create such upper/lower bounds (or floors andceilings) by analogizing to other values for which monetizedvalueshavebeendeveloped.Thus,forarulethatwouldpreventone hundred people from becoming partially paralyzed, forexample, onemight create anupperboundbenefits estimatebyanalogy to the value of a statistical life, which most agenciescurrentlysetaround$9million.Underthislogic,sincepreventing100 deaths would be valued at $900 million, and since partialparalysis presumably imposes less of awelfare loss than death,$900 million would be identified as the upper bound forestimatingthevalueofpreventing100casesofpartialparalysis.
Second, the analyst can use quantification to break theproblemintosmallerunitsthatmightallowformoremeaningfulintuitive judgment. 168 Sunstein gives as an example, a
166OMB CIRCULAR A-4, supra note 24, at 2. Lisa Heinzerling argues that
breakeven analysis has been selectively deployed by OIRA to “relax theevidentiaryrequirementsofthecost-benefittest,”butonlyforcertainfavoredtypesof rules. It isusedoften to loosen therequirements for terrorismandtransportation safety rules, but only rarely for environmental rules.Heinzerling,supranote96,at1459.
167Sunstein,LimitsofQuantification,supranote96,at1380.168ProfessorSunsteinwarnshereabouttheperilsofwhathecalls“micro-
disaggregation,” which he argues can “mask” the truemagnitude of costs orbenefits.Id.at1402(“Itiseasytoframeanexpensiveregulationintermsthatmakeitappearappealing—asin,forexample,theideathatforaparticularrule,onethatreducesrealrisks,everyAmericanwillhavetopaymerely$4perday.. . . sucha rulewouldcostover$360billionperyear, anda ruleof thatkind
***DRAFT***
54
hypothetical regulation thatwould cost $200million to providemodest improvements inwaterqualitywithnoramificationsforhumanhealth.Iftheanalystknewtheregulationwouldaffect20water bodies and knew something about those water bodies—that they were relatively small and had little recreational oraesthetic value—then she might be able to make the intuitiveapples to oranges judgment more meaningful by breaking theproblemdownandconceptualizingitonaper-water-bodybasis.(Here,Sunsteinassertsitwouldbeeasytoconcludethatitwouldnot be worth spending $10 million per water body for suchmodestimprovements.Butonecaneasilyspinscenariosinwhichsuch a conclusionmight bequestionable. Imagine, for example,that thewaterbodies provide breeding grounds for a species ofdragonfly that provides free ecological services in the form ofmosquitocontroltoanearbyheavilypopulatedarea.)169
Eachofthesemethods,ofcourse,requiressomeabilitytomeaningfully quantify benefits (just not in a monetary metric).Sunstein’snotionof“disciplining”theanalysisormakingitmore“rigorous” appears to involve primarily shoehorning someamountofquantificationbackintotheanalysis.170Wherethatisnot possible, or where the quantification that can beaccomplished is not of a type that facilitates meaningfulcomparison to costs, he acknowledges that “breakeven analysismay not be a great dealmore than a conclusion or a hunch.”171But even here, he insists that breakeven analysis is nonetheless“helpful” because “it explains what information is missing andwhysomecasesareespeciallydifficult.”172
How often such guideposts will actually be available to“discipline” the analysis is unclear. In the data set describedabove,EPAonlyconductedabreakevenanalysisintwooftheteninstances in which it was called for (where costs exceededbenefits). 173 In one, guideposts along the lines described bywould have adverse economic consequences whether or not it is ultimatelyjustified.”).
169Sunstein, Limits ofQuantification, supra note 96, at 1387-88; but seeFarber,supranote96.
170To this extent, for at least some kinds of benefits, his argument isvulnerabletocriticismonincommensurabilitygrounds.SeeHeinzerling,supranote96,at1464(“Subjectingrulesthatspringfromstatutesthatareallabouthumandignitytocost-benefitanalysiswithadollopofdignitythrownintothemix,disrespectsboththestatutoryschemeanddignityitself.”).
171Sunstein,LimitsofQuantification,supranote96,at1404.172Seeid.at1393.173ThereweretwootherinstancesinwhichEPAconductedabreakeven
analyses,buttheseanalyseswereaimedataccountingforuncertaintyinthe
***DRAFT***
55
Sunsteinwereavailable,andwhiletheybynomeansdefinitivelyresolvedtheuncertainty, theydidprovideat leastsomesupportfortheagency’sconclusionthat“thereisareasonablechancethatthebenefitsoftheseruleswouldexceedthecosts.”174Intheother
benefitsestimateratherthanunquantifiedbenefits.SeeEPA,EconomicAnalysisfortheFinalLongTerm2EnhancedSurfaceWaterTreatmentRule8-10to8-11(Dec.2005);EconomicAnlaysisfortheFinalStage2DisinfectantsandDisinfectionByproductsRule(Dec.2005).ThesewerebothSafeDrinkingWaterActregulationsaimedatpreventingcontaminationofpublicwatersystemsbyCryptosporidiumandothermicrobialpathogens.Inbothinstances,themonetizedbenefitsestimateexceededthemonetizedcostestimatebyasizablemargin.Butbecauseoftheconsiderableuncertaintiesassociatedwiththebenefitsestimates(significantlyhigherthanforthecostestimate),EPAusedbreakevenanalysestoshowthatevenatthe5%and95%confidenceintervals,benefitswere,undermostassumptions,bigenoughtoexceedcosts.
174EPA,RegulatoryImpactAnalysis,FinalNewSourcePerformanceStandardsandAmendmentstotheNationalEmissionsStandardsforHazardousAirPollutantsfortheOilandNaturalGasIndustry6-3(April2012).ThisrulesetairpollutionstandardsforoilandnaturalgasproductionfacilitiesandprovidedatleastsomeguidepostsofbothtypesdescribedbySunstein.First,whileEPAdidnothavesufficientinformationonthelocationsofaffectedfacilitiestoperformandexposureassessment,EPAwasabletoestimatequantitativelytheaggregateamountsbywhichemissionsofvariouspollutantswouldbereduced(12,000tonsofHazardousAirPollutants,190,000tonsofVolatileOrganicCompounds(VOCs),andsoon).Itusedthesenumberstobreaktheproblemintosmallerunits,bycalculatingthedollarsofbenefiteachtonofpollutionreductionwouldhavetoproduceinordertobreakeven.Thiswascomplicatedbythefactthatmultiplepollutantswereinvolved.SoEPAtreatedeachpollutantseparately,calculatingthedollarsofbenefitpertoneachpollutantreductionwouldhavetoproduceinordertobreakeven,holdingthebenefitsoftheotherpollutantsatzero.(e.g.,“[i]fweassumethehealthbenefitsfromHAPemissionreductionsarezero,theVOCemissionswouldneedtobevaluedat$2,900perton...forthebenefitstoexceedthecosts.”Id.at6-1.)Theagencyacknowledged,however,that“[o]fcourse,itisinappropriatetoassumethatthevalueofreducinganyofthesepollutantsiszero,”andthusnotedthat“therealbreak-evenestimateisactuallylowerthantheestimatesprovidedabovebecausetheotherpollutantseachhavenon-zerobenefitsthatshouldbeconsidered.”Id.
Thesecond“guidepost”EPAemployedtooktheformofupperandlowerbounds.Theagencycomparedbreakevendollar-per-tonvaluestotheresultsofstudiesthathadcalculatedapproximatepertonbenefitvaluesforreductionsofthesamepollutant.ForVOCreductions,forexample,therelevantstudyreportedtheassociatedPM2.5healthbenefitsasrangingfrom$280to$7,000perton.Thatrangewasobviouslyquitelargeandnotconclusive,sincethebreak-evenamount($2,900)wasinthemiddle.Furthermore,sincethebenefitamountdependsonpopulationsizeinthelocalityaffectedandthestudyresultsweredrawnfromadifferentsetofgeographicallocations(eighturbanareas)thanthoseaffectedbytherule,EPAhadlittleconfidenceintheaccuracyofthoseupperandlowerboundsforthis
***DRAFT***
56
instance—EPA’s rule aimed at limiting the number of fish andotheraquaticorganismskilledbycoolingwaterintakestructuresatpowerplants—nosuchguidepostswereavailable.Thebenefitsthat EPA left unquantified for that rule included, for example,wholecategoriesofaquaticorganismsforwhichitsimplyhadnodata,someofwhichwerespeciesthatmightplaycrucialrolesinthe food chain and other aspects of the aquatic ecosystem—phytoplankton, zooplankton, endangered sea turtles, and evencertain commercially valuable species, like shrimp, lobsters,crabs, andmussels.175ButEPAsimplydidnothavedata toevenbegin to quantify any aspect of the losses to these species thatwouldhavebeenavertedbytherule.
Even where such guideposts are available, pushinganalysts to look for them could in some instances becounterproductive. DanFarberworriesthat“[i]nsituationsthatinvolve the most moral perplexity—situations with unknownrisks of catastrophic outcomes or intangible values like humanindignity—the worst mistake of all may be to think that thesolution is too easy.”176 Farber worries that Sunstein may beputtingmoreconfidencethaniswarrantedinthe locationof theerrorbars.177Andheworriesinparticularabouttailrisks—thateven where the most probable level of harm avoided might bewellbelowthecost,thereisasmallbutsignificantprobabilityofcatastrophic harm that would dwarf the costs. By pushinganalysts to create upper and lower bounds on their estimates,Sunstein’svisionofbreakevenanalysismightencourageanalyststocutthetailsoftheprobabilitydistributioncurvetooshort,thusfailingtoaccountforunlikelybutdisastrousoutcomes.
Inanyevent,whilereasonablemindscanobviouslydifferon theutilityofbreak-evenanalysis, it certainly confronts somesignificant challenges. And all agree that at least in some
context,andconcludedthatthestudyresultswere“notappropriatetocalculatemonetizedbenefitsoftheserules,evenasaboundingexercise.”Id.at4-21(“Whiletheserangesofbenefit-per-tonestimatesprovideusefulcontextforthebreak-evenanalysis,thegeographicdistributionofVOCemissionsfromtheoilandgassectorarenotconsistentwithemissionsmodeledinFann,Fulcher,andHubbell(2009).”)Nonetheless,theagencyconcludedthat“evenifVOCemissionsfromoilandnaturalgasoperationsresultinmonetizedbenefitsthataresubstantiallybelowtheaveragemodeledbenefits,thereisareasonablechancethatthebenefitsoftheseruleswouldexceedthecosts,especiallyifwewereabletomonetizeallofthebenefitsassociatedwithozoneformation,visibility,HAP,andmethane.”Id.at6-3.
175SeeSinden,BenFranklin,supranote18,at1196.176Farber,supranote96.177Id.at1474-75.
***DRAFT***
57
instances, significant unquantified benefits will render even alitmus test CBA inconclusive (and a standard setting CBAimpossible). C.Implications
Thus,significantamountsofunquantifiedbenefitsimposeserious constraints on the kind of CBA an analyst can perform.Therootoftheproblemistheinabilitytomeaningfullycalculatenet benefits where significant benefits are unquantified. Themost one can definitively say in such circumstances is thatbenefits outweigh costs—if it so happens that the (incomplete)benefitsestimateexceedsthe(complete)costsestimate.Buteventhen, one cannot say by how much benefits exceed costs (i.e.,whatthenetbenefitsare).Andwithoutameaningfulmeasureofnet benefits, it is impossible to identify the alternative thatmaximizes net benefits and so impossible to conduct a formalEconomicCBA.
Thismeansthat forat least27outof the32EPArules inmy data-set, any meaningful calculation of net benefits wasimpossible. Even had the agency expended the substantialresourcesnecessary togeneratecostandbenefitestimates foralarge number of alternatives, those would have been no morethanpartialestimates,andaformalEconomicCBAthatidentifiedthe efficient level of regulation was therefore a logicalimpossibility. ThebestEPAcoulddowasa litmus-testCBA. Asthe discussion in Part I.A.4 above demonstrates, however, alitmus-test CBA is a very blunt instrument for locating theefficientlevelofregulation.Allittellsyouisthatyouaremovinggenerallyinthedirectionofefficiency,butnothowcloseyouare.Indeed,a regulationmaypassa litmus-testCBAandyetbeveryfarfromefficiency.
And even a successful litmus-test CBAwas contingent onthe happenstance of monetized benefits exceeding monetizedcosts.Totheextentthenumberscameouttheotherway(astheydid in at least seven instances in my data-set), the agency waslimitedtoconductingabreak-evenanalysis—acrudeinstrumentdependent on intuition and guesswork in the absence ofmeaningful information that is, at best, a far cry from thepurported accuracy and objectivity that ordinarily makes CBAseem attractive in the first place, and, at worst, a recipe forcontroversy,confusion,andpoliticalmanipulation.178
178Sinden,InDefense,supranote97,at1454-57.
***DRAFT***
58
Thus, the results of this study suggest that unquantifiedbenefitsduetodatalimitationsposerealproblemsforCBA.AndwhilethisstudywaslimitedtoEPA,becauseEPA’spracticewithrespect to CBA is so often held up nationally and eveninternationally as a model, these findings have implications fortheuse of CBA in agency rulemakingmore generally, at least intheenvironmentalarena.Accordingly,thereisreasontosupposethat the problems revealed here occur no less—and quitepossiblymore—inotheragencies,bothhereandabroad. Totheextentthatitso,thesedatapaintapictureofthepracticeofCBAthatisfarlesssanguinethantheusualaccount.
First, these results suggest that the presence ofunquantified benefits of a magnitude that rises above a deminimisthresholdmaywellbeasystemicandpervasiveproblemthatcannotbe ignored inbroaderconversationsaboutCBAasaregulatorytool.
Second, and more specifically, these results suggest thatmostof the time,due to thepresenceofsignificantunquantifiedbenefits, the kind of formal economic standard-setting CBA thatthe executive orders and guidance documents hold up as thenorm is quite simply impossible to perform or evenapproximate.179Theagenciesareinsteadconfinedto litmus-testCBA,atbest,andabreak-evenanalysisatworst. Byrenderingaformalstandard-settingCBAalogicalimpossibility,thepervasiveexistence of significant unquantified benefits further distancesCBA from thenormative criteria towhich it ismoored,whetherKaldor-HicksefficiencyorAdlerandPosner’swelfarism.180
179UnlessoneassumesthatEPAisnotbeingforthrightaboutitsabilitytoquantifythesebenefits.SeeMasur&Posner,supranote5,at4(suggestingthatagenciesmayhaveanincentive“toclaimunquantifiedbenefitsevenwhentheycanbequantified”inordertoarguethattheyarelargeenoughtotipthebalance).
180These results also suggest at least the possibility that beyond theunquantified benefits identified and described by analysts conducting CBAs,there may also be a significant number of what we might call “unknownunknowns”—impactsofwhichwearesimplyunaware,andaboutwhichwedonotevenknowtotrytocollectdata.DonaldRumsfeld,Sec'y,Dep'tofDef.,DoDNews Briefing (Feb. 12, 2002) (transcript available at:http://archive.defense.gov/Transcripts/Transcript.aspx?TranscriptID=2636).See Graves, supra note42, at11 (noting that environmental regulationshavewide variety of benefits beyond mortality and morbidity that are usuallyignored, including “ecosystem improvements, agricultural cropyieldbenefits,materialdamagereductions(e.g.housepaintingwith lessfrequency)benefitsfor pets, aswell as aesthetic effects (e.g., smells, visibility)”). If, as this studysuggests,theknowngapsinourknowledgebasearesufficientlypervasiveandwidespread to produce “significant,” “substantial,” and “important”
***DRAFT***
59
As detailed above, Adler and Posner shored up thenormative defense of CBA ten years ago by framing it not as adirect measure of efficiency or welfare, but rather as adecisionmaking tool that tracked overall welfare reasonablywell—at leastbetterthananyof thealternatives.181Drawingonthe legal literature about rules versus standards, Adler andPosner had the crucial insight that a decisionmaking tool thatmeasured welfare directly in theory would fail to do so inpractice, much as a legal standard that directly implements alegislativegoal (“Motoristsmustdrivesafely”)mayoftenbe lesseffective at achieving that goal than a specific rule (“Motoristsmustnotdrivefasterthanfifty-fivemilesperhour.”).182Therule“deviates,insomeinstances,fromtheunderlyinglegislativegoal,but it is easier to apply.”183 Like rules, they argued, CBAmayactually be more successful in practice at tracking the moralcriterionwe care about precisely because it is one or two stepsremovedfromthatcriterion.184
InAdlerandPosner’sview,“directimplementation”ofthewelfarist criterionwould inevitably flounder on awhole host ofobstacles:
unquantified benefits most of the time, it may be reasonable to assume theexistence of additional knowledge gaps aboutwhichwe are not even aware.But see Sunstein, Limits ofQuantification, supra note 96, at 1380 (suggestingthatsuchcircumstancesarerare).
Itisnothardtoimagine,forexample,thatatpresent,manyconsequencesofecosystemdegradationremainofftheradarscreenasunknownunknowns.Both the services that ecosystems provide to humans, as well as theconsequences to ecosystems of pollution and other forms of environmentaldegradation remain vastly understudied and inadequately understood,as dothehumanhealtheffectsofanenormousnumberofchemicalcompounds.SeeJodyFreeman&AndrewBuzman,ClimateChangeandU.S.Interests,109COLUM.L. REV. 1531, 1556-60 (2009) (valuation of biodiversity loss from climatechangeleftoutofmosteconomicmodelsandveryproblematicinthosemodelsthatattemptit);MichaelLivermore,CanCost-BenefitAnalysisofEnvironmentalPolicyGoGlobal?19N.Y.U.ENVTL.L.J.146,172(2011) (discussing the limitedvalueofCBAincontextsinvolvingimpactstonaturalresourcesandecosystemsbecause of the “particular difficulties” of valuation in this sphere). Whereunquantified benefits take the form of unknown unknowns, CBA may failaltogether—that is,meaningful comparisonof costs andbenefits, evenof theinformal, Ben Franklin variety, becomes impossible. Sinden, Formality, supranote19at116.
181ADLERANDPOSNER,supranote26.182Id.at65.183Id.184Id.at65-68.
***DRAFT***
60
To begin, the “direct implementation” procedurewould have massive decision costs, both directcosts...anddelaycosts....[Itwould]tellagenciesto collect all information relevant to determiningtheprobabilityofthedifferentpossibleoutcomesofpolicy choices and to determining the level ofoverall welfare in each outcome . . . [It] is alsopoorly designed to check decision-maker error oropportunism.185
Inshort,“directimplementation”wouldbesocomplicatedthatitwouldbehopelesslyexpensiveandtimeconsumingtocarryout,nontransparenttoallbutthetechnicallytrained,and,asaresult,vulnerabletopoliticalmanipulationandcorruption.
Thus, the argument walks a precarious tightrope: tosucceed,CBAmustbebothdistantenoughfromactualwelfaretoavoid thepitfallsof “direct implementation”yet closeenough toconvincingly and meaningfully provide a proxy. 186 One canquestion whether CBA actually does avoid the pitfalls of directimplementation or is itself hopelessly complex, expensive, non-transparent, and vulnerable to manipulation, 187 and manyhave.188Butformypurposeshere,theconcernistheothersideofthe tightrope: whether CBA in practice provides a sufficientlyaccurateproxyforoverallwelfaretoprovidemeaningfulresults.AdlerandPosnerpositthatitdoes,andresttheirentireargumentonthatpremise.Astheyultimatelyacknowledgeattheendofthebook, however, this question of the accuracy of CBA in trackingwelfare “is at bottom an empirical question” that their largelytheoreticalbookcannotanswer.189
InthestudydescribedaboveIhaveattemptedtoapproachthatempiricalquestion.Andtheresults,thoughpreliminary,arenot promising forCBA. They suggest that CBA inpractice faresquitepoorlywithrespecttoitsaccuracyintrackingwhatwecareabout,whether that’s overallwelfare or Kaldor-Hicks efficiency.Certainly, this study involved an admittedly small sample andproduced results that undoubtedly need to be tested by furtherempirical work. Nonetheless, the results paint an intriguingpictureofCBAintherealworldaseven furtherunmooredfrom
185Id.at66-67.186Sinden,Kysar,&Driesen,supranote28,at48,58.187Seeid.at58.188See,e.g.,Sinden,InDefense,supranote97.189ADLER&POSNER,supranote26,at190.
***DRAFT***
61
its normative foundations than most of its defenders haveassumed.
Somewillundoubtedlyrespondtothesefindingsbycallingon agencies to devote more personnel and resources to fixingCBA,bydoingthestudies,gatheringthedata,anddevelopingthemodels necessary to quantify the unquantified benefits. 190Althoughthequestionofthefeasibilityofsuchanenterprisewasnotthespecificfocusofthisstudy,iftheCBAsinthissampleareany guide, it appears that this will be no easy (or inexpensive)task.
Thebenefitsleftunquantifiedarenumerous,wide-ranginganddiverse.Theyincludemanyeffectsonecosystemsandhumanphysiologyforwhicheventhebasiccausalmechanismsareonlydimly understood. Meanwhile, the vast majority of quantifiedbenefits are attributable to a single air pollutant, particulatematter, forwhichwe have been fortunate to have an unusuallylarge and extensive network of monitors generating data fordecades—datathatgenerationsofscientistshavenowminedforan abundance of studies on the health effects of this singlepollutant. Incontrast, fornumerousotherpollutants (including,forexample, the189hazardousairpollutants listed intheCleanAir Act) the information base is slim to nonexistent. Thus, anysuggestion that CBA should be rehabilitated faces difficultquestions about whether such an enterprise is feasible at allwithin any reasonable time frame, and, if so, whether thenecessary commitment of resources would be worth theopportunitycosts.
Another approach takes the view that you setenvironmentalstandardswith the informationyouhave,not theinformation you wish you had. CBA is not the only standard-
190Revesz,supranote96;Graham,supranote118,at526.Masurand
Posnersuggestthatagenciesactuallyfaceincentivestounder-quantifyinordertouseunquantifiedbenefitsasajustificationtopromulgaterulesthatotherwisefailCBA.SeeMasur&Posner,supranote13,at4.Inpartonthatbasis,theyarguethatagenciesshouldmoreaggressivelyuseexistinginformationtoquantifybenefits,and,failingthat,pursueadditionalstudies,and/oremployBayesianprobabilitiestomakereasonableguessesbasedonstaffintuitionandexpertise.Inpreviouswork,Ihavesuggestedthatagenciesseemtofacetheoppositeincentive—toquantifymoreandmoreinordertogaincredibilitywiththepublicandOIRA.SeeSinden,Formality,supranote19.Thatpulltowardformalityandquantificationcangetagenciesintotrouble.SeeSinden,BenFranklin,supranote18.IworrythatresortingtoBayesianprobabilitieswouldinviteimproperpoliticalmotivationsintotheanalysisoratleastperceptionsofthat,andultimatelymireCBAinintractablecontroversyandgridlock.
***DRAFT***
62
setting method in the book. The two most prominentalternatives,feasibilityandhealth-basedcriteria,bothhavealonghistory of implementation at EPA191and require a substantiallyless extensive data set. In particular, neither standard requiresmonetizationofbenefits.
Feasibility analysis aims at locating the most stringentlevel of pollution control or safety that is technologically andeconomically feasible.192 It typically begins with a thresholdfinding of regulatory benefit, but this is usually just a cursorylook, without detailed explication or quantification, simply toensure that there is some harm worth regulating (i.e., thatregulation will produce some significant benefit). The focus isinsteadonthecostsideoftheequation,wherefeasibilityanalysisperforms a fairly granular and often quantified analysis of thecosts and technological feasibility of pollution controltechnologies. Rather than comparing costs to benefits, as CBAdoes, it keeps the focus on costs, gaging their magnitude bycomparing them to the overall financial capacities of theindustry. 193 Thus, feasibility analysis avoids the necessity ofcomprehensively quantifying and monetizing benefits byessentially assuming—based on a cursory threshold finding ofsomebenefit—thatbenefitssufficienttojustifycostsactuallyexist(atleastuptothefeasibilitylimit).
Health-basedstandardsettingfocusessolelyonregulatorybenefits, but two aspectsmake it far less demanding than CBA.First,itisnotasdependentoncomprehensivemeasurementofall
191SeeMichaelA.Livermore&RichardL.Revesz,RethinkingHealth-Based
EnvironmentalStandards,89N.Y.U.L.REV.1184,1190(2014).192SeegenerallyDavidM.Driesen,DistributingtheCostsofEnvironmental,
Health,andSafetyProtection:TheFeasibilityPrinciple,Cost-BenefitAnalysis,andRegulatoryReform,32B.C.ENVTL.AFF.L.REV.1(2005).JasonR.Bent,HealthTheft,48CONN.L.REV.637(2016);DovWaisman,EquityandFeasibilityRegulation,50U.RICH.L.REV.1263(2016);DavidM.Driesen,TwoCheersforFeasibleRegulation:AModestResponsetoMasurandPosner,35HARV.ENVTL.L.REV.313(2011);JonathanS.Masur&EricA.Posner,AgainstFeasibilityAnalysis,77U.CHI.L.REV.657,669(2010);SidneyA.Shapiro&ThomasO.McGarity,NotSoParadoxical:TheRationaleforTechnology-basedRegulation,1991DUKEL.J.729(1991);WendyE.Wagner,TheTriumphofTechnology-basedStandards,2000ILL.L.REV.83(2000).
193Note,too,thatfeasibilityanalysisdefinescostsmorenarrowlythandoesformalCBA,countingsimplycompliancecostsimposedontheregulatedindustry,ratherthanformalCBA’sattempt—atleastintheory—tocountallcoststosocietyasawhole.
***DRAFT***
63
social benefits for its success. And second, it does not requireregulatorybenefitstobeconvertedintomonetaryterms.194
Bothofthesealternativeshavebeenfrequentlydismissedasinsufficientlytetheredtoanefficiencyorwelfaristcriterion.195But if, in practice, CBA itself is unmoored from thosenormativefoundations, then that objection no longer holds and CBA’scompetitors perhaps deserve a second look. In short, if closeadherencetoanidealstandardofefficiencyorwelfareturnsoutin practice to be a holy grail, then perhapswe need a differentmeasuringrod.
CONCLUSION
Benefits that are left unquantified due to the prosaicproblemofinadequatedatahavebeenfrequentlymentionedbutrarely analyzed in any depth in the vast literature on CBA inenvironmental rulemaking. Yet the study results describedabove—categories of benefits described by the agency as“important,” “significant,” “substantial” in 84 percent of CBAs—suggest that unquantified benefits due to data deficiency is asystemicandpervasiveproblemthatwecanno longer ignore inbroader conversations about CBA as a regulatory tool. Theseresultsalsosuggestthatmostofthetime,duetothepresenceofsignificant unquantified benefits, the kind of formal Economicstandard-setting CBA that the executive orders and guidancedocuments hold up as the norm, is quite simply impossible toperformorevenapproximate.Theagenciesareinsteadconfinedto litmus-test CBA at best (a blunt instrument for identifyingefficientregulations)andabreak-evenanalysisatworst(acrudeinstrumentdependentonintuitionandguesswork).Finally,morebroadly, these results suggest that the link between CBA andefficiency or welfare may be even more tenuous than most ofCBA’s supporters have assumed. This in turn suggests thatperhaps a fresh look at CBA’s primary competitorsmight be inorder.
194SeegenerallyLivermore&Revesz,supranote191;Sinden,InDefense,supranote97.
195JonathanMasur&EricPosner,AgainstFeasibilityAnalysis,77U.CHI.L.REV. 657, 705 (2010) (arguing that there is no “theoretically coherentnormativebasisforfeasibilityanalysis);ADLER&POSNER,supranote26,at73-100(arguingthatCBAismerelyaproxyforwelfare,butthatittrackswelfaremoreaccuratelythananyofitscompetitors(exceptperhapsinformalCBA)).
Related Documents
