Science and uncertainty in environmental regulation: insights from the evaluation of California’s Smog Check program

eScholarship provides open access, scholarly publishingservices to the University of California and delivers a dynamicresearch platform to scholars worldwide.

University of California TransportationCenter

University of California

Title:Science and Uncertainty in Environmental Regulation: Insights from the Evaluation of California'sSmog Check Program

Author:Bedsworth, Louise Wells, University of California Transportation CenterKastenberg, William E, University of California Transportation Center

Publication Date:03-01-2003

Publication Info:University of California Transportation Center, UC Berkeley

Permalink:http://escholarship.org/uc/item/5nh3w7s8

Abstract:Environmental decision making is a complex process confounded by technical uncertainty,political pressure, and social interests. New calls for environmental decision-making frameworksemphasize the need for an holistic approach that incorporates technical and non-technicalexpertise, and participation by all affected and interested parties. In this paper, we analyze theevaluation of an environmental regulatory program to characterize the interaction of science andpolicy and the processing of uncertainty using concepts from science and technology studies.This demonstrates the influence of institutional goals and commitments on the uptake and use ofscience and the processing of uncertainty in the regulatory process. We discuss the implicationsof such analyses on the development of new environmental decision-making frameworks.

Science and Uncertainty in EnvironmentalRegulation: Insights from the Evaluation ofCalifornia’s Smog Check Program

Louise Wells BedsworthWdham E Kastenberg

UCTCNo. 617

The Uzfiversity of CaliforniaT ranspo~tatlon Center

Universl~ of CaIL~rnig]?erkeley, CA 9~20

The University of CaliforniaTransportation Center

The Umvermw of Ca/iformaTranspormtaon Center (UCTC),sane of ten regional ~.xt~mandated by Congress andestablished in Fall 1988 tosuppor~ research, educauon,and ~rmmng it, surface t~,ms-pormtmn. The UC Center

serves federal Region IX and

ts st.xpponed by rrmtchmg

grants from the U.S. Depart-

ment of Transportmaon, the

California Department of

Transportataon (Cakrans), and

the Urdverstty

Based on the Berkeley

Campus, UCTC draws upon

existing eapabilines and

resources of the Insumt~ of

Transportation Studies at

Berkeley, Davis, Irvine, andLos Amgeles, the Insumte of

Urban and Regmnal Develop-ment a~ Berkeley, and severalacademic departments at ~heBerkeley, Dav~s, Irvine, and

Los Angeles campuses.

FaculD’ and smaenrs on otherUmvemty of Cahfomia

campuses may participate In

Center a~tivi~es. Researchersat other unive~ifies within the

region also have oppotmrdfiesto eollaborat~ with UC facudty

on selected studies

UCTC’s educa~onal andresearch programs arc focused

on strategic planrdng forimproving merxopolimn

accessibility, with emphasison the special conditmns in

Regmn IX. Partmul~r alXenfion

is directed to strategms for

using transportation as aninstrument of ~onomic

development, while also ac-

commodating to the regmn’s

persistent expansmn andwrale maimmnmg and enhanc-ing the qualat7 of Ilfe there

The Center distributes reportson its research in working

papers monographs, and inrepnnts of pubhshed m’ucles.

It also pubhshes Acce.~s, amagazine presenting sum°manes of selected smthes. For

a list of pubhcafions m pnnt,

write to ’,he address below

Univer~ity of CaIEornmTransporta~n Center

108 Naval Amhitccmm BuddingBerkeley. Cahforma 94720Tel 510/643-7378FAX: 510/643-5,~56

DISCLAIMERThe contents of this report reflect the views of the authors, who are

responsible for the facts and the ~ccumey of the inbrrnation presentedherein. This document is disserninated underth~ sponsorship of the

Department of Transportation, Llniversity Transportation Centers Program,in the interest of information exchange. The IJ.S. Government assumes no

|iabitit~ for t~e contents or use thereof.

TI~ cont=n~ of this z~port retie= the vtsws of tl~ author who m z=spenmblefor the fats mad accuracy of the dam t~semed hemm The conmn~,do notnecess~’fly reflect the offieLd mews or pohez~ of the State of C~[fonua or the

U S, Departm=nt of Transpormnon. Thr~ report does no~ conslatum a standard,spe:ffieatx~r~ or mgulauon.

Scienceand Uncertainty in Environmental Regulation: Insights fromthe Evaluation of California’s Smog Check Program

Louise Wells BedsworthEnergy and Resources Group

Umvermty of Cahforma, Berkeley

William E. KastenbergDepartment of Nuclear EngineeringUmversIty of Cahfomm, Berkeley

Reprinted fromSctence and Pubhc Pohcy

Volume 29, No I, pp 13-24 (2002)

UCTC No. 617

The University of California Transportation CenterUniversity of California at Berkeley

Environmental regulation

Science and uncertainty in environmentalregulation: insights from the evaluation ofCalifornia’s Smog Check program

Louise Wells Bedsworth and William E Kastenberg

Environmental decision making is’ a complexprocess confounded by technical uncertalnty~politicai’ pressure, and societal interests. Newcalls fo,~ environmental decision-making frame-works emphasize the need for an holisticapproach that incorporates technical and non-technical expertise, and participation by all in-terested and affected parties. In this paper, weanalyze the evaluation of an environmental regu-latorl, program to characterize the interaction ofscience and policy and the processing of uncer-taint?/using concepts from science and technol-ogy studies. This demonstrates the influence ofinstitutional goals and commitments on the up-take and use of science and the processing ofuncerta’mtF in the regulatory process. We discussthe implications of such analyses on the devel-opment of new environmental decision-makingframeworks.

Lomse Wells Bedsworth m m the Energy and Resources Groupand Wllham E Kastenberg (corresponding author) is m theDepartme nt of Nuclear Engineering, Umverslty of Calffomm atBerkeley 4103 Etzheverry Hall #1730, Berkeley, CA 94720-i730 USA, Tel +1 510 643 0574, Fax +1 510 643 9685, E-marl kas. enbe@nuc berkeley edu

The anthors are grateful to Davld Guston and an anonymousreviewer for helpful comments on earher drafts of tbas paperThey also thank the manerous participants m the Smog Check 1Ievaluatma process who have supported this research Fundingfor this Froject was provided by the EPA STAR Graduate Fel-lowslup program and the Ulnverslty of Cahforma DlssertatlonYear Fellowship

R~guESOLVI~, G COMPLEX envaronmenmtlator3 ~ssues reqmres consideration of

e techaucal, pohtmal, and societal lmphca-t~ons of a declmon Several national studies haverecognized this requirement and have suggested in-tegrated or hohstlc methods for approaclung com-plex enwronmental declslon making Suchframeworks include broader societal and polmcalconslderatlons, pamclpataon of all interested andaffected parties, and maintaining a sound sclentlficbasis (NRC, 1996, Presldentmt/CongresslonalConmussmn on 1Rask Assessment and Rask Man-agement, 1997)

Exastmg declslon-makmg parachgms such as cost-benefit analys~s or unhty theory do not prowde con-ceptual or procedural connectmns between thesecons~demtmns Nonetheless, most of the work onintegrated decmmn making to date has centered onconstructing methods that rely on decision analyticmethods such as multmttnbute uttht~ analysis and itsvariants and derwatwes (Keeney and Rmffa, 1976,Merkhofer and Keeney, 1987, Hong and Aposto-lakls, 1993, Reckhow, 1994, Apostolakls andPickett, 1998)

In thls paper, we explore uncertainty m envaron-mental regulation using methods and theones fromscience and technology st-aches (STS) Tlus analys~sdemonstrates why tt ~s dtfficult, if not maposstble, torepresent enwronmental dec~smns fully using a utah-tartan or cost-benefit methodology We examine aspecific case m regulatory enwrormlental dec~smnmakang, the evaluauon of Cahforma’s motor vehiclemspectmn and maintenance program, and show howinteractions of setence and policy and the processing

Science and Pubhc Pohcy February 2002 0302-3427/02/010013-12 US$08 O0 © Beech Tree Pubhshmg 2002 ] 3

Sczence and uncertam~ tn envzronmental rcgula~on

Louise W Bedsworth is a doctoral candudate m the Energyand Resources Group at UC Berkeley Her dissertation ana-lyzes the Interaot=on of science and poflcy m the design andevalu~on of CaNomla’s Smog Check program

Bill Kastenberg is the Daniel M Tellep Dl~mgu~shed Profes-sor of Engineering, University of Calffomm at Berkeley lieteaches courses and supervises research on risk analysis,nuclear reactor safety, environmental conflict resolution andmulti-stakehOlder decision making, and more recently, onengmeenng ethics

of uncertainty shape the regulatory &scoursethrough argument formation and debate

Further, we argue that an understanding of thescience pohey mterface and the treatment of uncer-tainty m the deczslon-matang process provides astrong conceptual and analylacal foundauon throughw~ch environmental dee]stun making can be ana-lyzed and understood ~rtule exploratory m nature,tt~s paper alms to demonstrate the power of narra-uve or descnpUve approaches, such as STS, mthe study and practice of env~romuental regulator~declsmn makang

Expertise. uncertain~, and decision making

Tradmonal models of the science pohcy mteractmnrepresent sczence as a source of objecUve truth thatgrades the dee~smn-makmg process (the ’truthspeaks to power" model) This model has served toreform and advocate the appheatmn of cost-benefitand uUhtanan models to declsxon making Inmghtsprovaded by researchers m STS and other fields havedemonstrated that tins model of the science poheyrelatmnslnp Is illusive (see, for example, Majone,1989, Jasanoff, 1990. Herrlck and Jamaeson, 1995.Elmnga. 1997, Jager, 1998. Jasanoff and Wynne,t998) Shefla Jasanoff has sho~a how sclentlficknowledge and ’facts’ are contingent and con-structed, resulting m their deeonstrucUon m theregulatory arena (Jasanoff, 1986, 1987, 1990)

These ctmUenges to the authority and legmmacyof science result m attempts to construct boundariesBoundaries and their assocmted ’boundary work" areattempts to classify knowledge, mformatmn, or evenpeople and groups as legntnnate vs dlegltlmate, sel-enufic vs unselenUfic, or resider vs outmder (Cneryn,1983, Jasanoff, 1990) These processes of decon-strueUon, boundary defmmon, and boundary main-tenance often brmg sclenUfic inputs and, especially,uncertam~ rote the center of regulatory debates andchallenges both as the issues of contentmn and asleveragmg tools

Simon Shaclde~ and Brian Wymae have demon-strated how &fferent interpretations of the sametechmcal phenomenon, uncertainty, or fact can se~’eas a means of commumcaUon across these bounda-ries between 6tfferent soeml groups or "worlds"(Shacldey and Wynne, 1996) In tl~s role, facts.phenomena, and uncertainties serve as "’boundary.

objects" or "boundary ordering dewces" (Gmryn,1983, 1999)

In addmon to its traportance at the boundariesbetween socml worlds, uneertam.ty serves as a pow-erful means to promote aCtlor~ mactmn, or delay mthe regulatory arena (Majone, 1989, Funtowlez andRavetz, 1990, van Asselt et al, 1996) Campbellshows how uncertainty can serve as a locus ofdebate among experts and as a strategic tool thatenables an expert to maintain authority m a gwensituation (Campbell, 1985) Susan Lelgh Star b.~sshown ho~ tmcertamty is used to reffy the authorityof science through the transformation of local uncer-tamtms rote globally-accepted certainties that aremanaged and controlled by science, thereby rnam-taming ~ts authority (Star, 1985)

In addauon to the d~scusmon of ~ts role as a rhe-torical and strategic tool, the meamng of the word’uncertainty’ has been d~seussed and defined bymany d~fferent researchers Several defmlUons andclassifications of uncertainty exist m the envaron-mental pohcy hterature As Table I ~llustrates, theserange from definmons that are primarily techmeal tothose that encompass a broader, construcUvast wowof scxentrfic mformaUon and the dec~smn-makmgprocess

The breadth m the defmmon of uncertainty sug-gests ~t ~s more tlmn a staust~cal or probab~hst}c phe-nomenon Rather, uncertainty extends from data andmodels rote supposmons based on that mformaUon,problem deflmUons, and the design of soluuons

Table 1 Uncertainty classzficat~on schemes

Res~’¢her

Morgan and Hennon(~0)Rowe (1994)

Snrader-Frechette(1996)

National ResearchCouncil (I 996)

Wynne (1992)

Classes of uncertainty

Uncertainty ~n empirical quantitiesUncertainty in model form

Temporat uncertaintyStructural uncertaintyMetrical uncertaintyTranslational u~certamty

Framing uncertaintyModehng uncertaintyStatistical uncertaintyDecision-theoretic uncertainty

Aleatory uncertaintyEp=steml¢ uncertaintyIndeterminacy~ignorance

InexactnessUnrehabl[rb]Border with Ignorance

RiskUncertaintyIgnoranceIndeterminacy~

Note There is an important dlstmct~on betweenIndeterminacy as defined by NRC and as defined byWynne NRC states that indeterminacy Is uncertaintyabout which model to use Wynne’s definition poses aquestion that owrtays the entire science-policytnteractmn Does policy direct science or is poItcymodified to justify science?

14 Sc=eace and Pubhc Policy Februa~T 2002

The breadth in the definition ofuncertainty suggests that it is morethan a statistical or probabilisticphenol~aenon: it extends from data andmodels into suppositions based on thatinformation, problem definitions, andthe de,,dgn of solutions

The s~x classtficatmns shown m Table 1 rangefrom those that are predominantly techmcally based,focusing on data and models (Morgan and Hennon,1990), through those that address uncertainty m thepohcy proces~ including commumcaUon and prob-lem frarrang (Rowe, 1994, Shrader-Frechette, 1996),to those that address ep~stemlc hnutatlons to scleneeand the realm of Ignorance (Funtovacz and Ravetz,1990~ 19,92, Wynne~ 1992, NRC, 1996)

Drawing on these defmmons of uncertainty andinsights unto the mteracUon of scmnce and pohcyfrom STS, we examine an environmental regulatoryprocess and the associated uncertainty m detad Ouranalysis demonstrates why expanded defu-utmns ofuncertainty are needed to understand the envaron-mental regulatory process

Further, our analyms of the mterjectmn of scienceand uncertainty into the regulatory chseourse pro-rides an account of how actors m the regulatory de-msmn-makmg process construct arguments throughthe presentation and mterpretatmn of scientific m-formation~ and the mterpretatmn and emphasis onchfferent types and sources of uncertainty Together,these character~stms of the environmental regulatoryprocess suggest that understanding the process ofapplying scientific mformauon to enwronmentalregulatmn ~s an essentml component of understand-mg the arguments, controvermes, and aclaons menvtronraentaI regulatmn

Case study: Smog Check II evaluation

In tins case stud),, we examine the evaluatmn ofCahfomm’s enhanced motor veincle mspectmn andmamten, mce (I/M) program, Smog Check II I/Mprograrrs reqmre registered vebacle owners to havetheir vehmles’ ermssmns and their errassmns controlsystems tested on a regular basis We have chosen toexamine ttus program because ~t contains many ele-ments oornmon to environmental regulatory dec~smnmaking Metrics of success are tmeertam and non-muform, there *s not agreement among the pamc,-patmg and/or interested groups on the appropriatemethods of evaluatmn, and polmcal cornrmtmentsand pubhc accountability rely on the success of theprogram

Sczence and uncertmn~ m envtronmental regulctt~on

In adchtlon, there are Iugh enwronmental and eco-normc stakes assocmted wath a successful motor ve-Incie mspectaon and maintenance program InCahforma, Smog Check II ts responsible for onequarter of the enussmns reductions outlined m theState lmplementatmn Plan (SIP) These reductmnsare needed for the most polluted areas m Cahformaff they are to meet the Nataonal Ambmnt Air QuahtyStandards (NAAQS) 1 If these standards are not met,the state can lose bflhons of dollars m federal bagh-way funds through sanctmns Imposed by the UmtedStates Envtronmental Protectmn Agency (USEPA)

In flus paper, we analyze the evaluatmn of SmogCheck II, and characterize the processing of tmcer-tamty SpeclficaUy. we focus on how pamclpantsand reports present and chseuss evaluataon methodsand results Also we examine what types of tmcer-tamty arise m the evaluatmn process and how thesetranslate into the regulatory chscourse

Background

USEPA states that I/M programs are deslgued to"ensure that vebacles stay clean m actual consumeruse [encourage] proper vetncle maintenance andchscourage tampering w~th errusslon control dewces"(USEPA, i994, page 1) UM programs were eluded m the 1977 Amencknaents to the Clean AirAct as a way for states to attmn the NAAQS, butmore prescnptwe gmdelmes were included m theClean Azr Act Amendments of 1990 These reqmredthe most polluted of the areas designated non-attainment for the carbon monoxide (CO) or ozone(O3) NAAQS to implement enhanced mspectmn andmaintenance programs 2 E~thanced I/M programs aredesigned to measure tadplpe ermssmns of CO andthe precursors of 03 formation, hydrocarbons (HC)and oxades of mtrogen (NO×) In addltmn, the en-hanced I/M test as env~smned by USEPA, includesthe measurement of evaporatwe HC ennss~ons

The Clean Aar Act Amendments of 1990 andUSEPA’s final rule envlsmned enhanced I/M pro-grams as centrahzed, test-only programs In a cen-trahzed program, velucles are tested at a statmn runby a contractor who is respons~ble for all the testingfaelhtxes m the regmn and that testing is conductedat a facdity separate from repmr The final rule alsoincluded a reqmrement that all enussmns tests are tobe conducted usmg a dynamometer, or a treadmdl-1A:e dewce for cars, and are to use a spectfic testmethod called the IM240 3 Inspections m USEPA’senhanced UM program also test evaporatwe erms-s~ons from vehicles, not just tadplpe emlsmons(USEPA, 1992)

Different incarnations of Calfforma’s SmogCheck program have been m place since 1984 Theprogram has undergone a number of changes as aresult of techracal maprovements and regulatory re-qmrements Prior to the 1990 Clean Air ActAmendments, the enttre state was subject to SmogCheck testing e~ther b~enmally or on change of

Science and Pubhc Pohcy February 2002 I5

Sczence and unoertmnty m envtronrnenfat regulal~on

ownerstup of the vetu.cle, depending on the aLr qual-ity m each geographic region wttlun the state Alltesting was performed at privately owned servicestatmns using a two-speed taflp~pe test

The Clean Atr Act Amendments of 1990 reqmredthat the most polluted areas of the state tmplementenhanced I/M as defined by the USEPA EnhancedI/M was designed to reduce the impact of fraudulenttesting, tampering, and cheating on the performanceof I/M programs by separating test and repair and touse hlgh-tech testing eqmpment stated to modemtechnology vehlcles (USEPA, 1995) Currently,there are three dLfferent forms of the Smog Checkprogram m place m different geograptuc regaons mCalff-orma -- basac areas, ehange-of-m~aershapareas, and enhanced areas 4 Smog Check II testing isonly m place m the enhanced areas

The serrate statmn industry m Cahfomm resastedamplementmg USEPA’s enhanced I/M program be-cause of the loss of emassmns-testmg revenue (re-pmrs could stall be performed under a centrahzedprogram) and the 1ugh cost of the IM240 test eqmp-meat Because of these pressures and concerns aboutconsumer convemence, Cahfonua was reluctant tocreate a completely centrahzed program lhke the onethe USEPA reqmred

After the Clean Aar Act Amendments of 1990,Cahforma entered negoUaUon wlth the USEPA andreached an agreement m 1994 to create Smog CheckII, the hybrid program currently m place m thestate’s most senous non-attainment areas Tlus pro-gralTl lS all maportant part of the SIP, the regulatorydocument the state as required to subrmt to USEPAshowing how and when areas of the state that do notmeet the naUonal ambaent mr quahts’ standards willcome into comphance

Smog Check lI contmns some of the elements ofUSEPA’s prototype enhanced program, but mare-tams a network of independent statmns that performboth test and repmr Tins case study focuses on theevaluation of tins hybrid program, whach was re-qmred after two years of program operatmn

Evaluanon process

The USEPA’s final rule on enhanced mspectaon andmaintenance programs reqmres states to evaluatetheir I/M prograras baenmally, beginning two yearsafter the unplementaUon of mspectaons 0LISEPA,1992) In Callforma. the evaluataon process is a po-lmcally contenuous ~ssue for a number of reasonsTo begin with, the program is not achieving theen-assmn reductions that ~t ~s obhgated to m the SIP(ARB. 1994, 2000) A failure to meet the SIP re-qmrements affects the state’s ttmelme to meet fed-eral air quahty standards, whach can, m turn, affectthe state’s reeeapt of federal funding for transporta-Uon projects Smog Check II has also been a pohu-cal ’hot’ issue an Calfforma since the enhancedprogram was proposed following passage of the1990 Clean Aar Act Amendments, attracting the

anterest of clasmc car collectors, serwce stations, andmotonsts

Inst~tutaonally, Smog Cheek 17 evaluatmn as acomplex process as well The Bureau of AutomotaveRepair (BAR), a &wsmn of the Cahfemm Depart-ment of Consumer Asr~asrs is responsible for over-seeing the program Table 2 shows the regulatoryagencaes revolved m Smog Check and thetr respon-s~bdmes Both the Atr Resources Board (ARB) andthe Inspectmn and Maintenance Rewew Comnuttee(IMRC) are responsible for evaluating the program

While the purposes of the two e~’aluatmns are dif-ferent (ARB prepares an evaluatmn to be subrmttedto the legislature and USEPA to demonstrate SIPcomphance, and the IMRC prepares an evaluataon tobe gwen to the state legislature) the pubhc presenta-tmn of the results tughhghts amportant differencesbetween the two organxzatmns These differencesernst m the agencaes’ m~ssmns, conumtments, andaccountability

In addmon to the polmcal and mstltutmnalchallenges, the evaluation process as a techmcallycomplex endeavor because of the difficulty both mobtmmng data and m selecting an evaluataon meth-odologw The vanablhty of the driving cycle, macer-tamty m measurement, and the influence of driverbehawor combine w~th other factors to make ~t very

Table 2 Gover.mental orgamzatlons ~nvolved in SmogCheck

Group

Bureau ofAutomot=ve Repair(BAR)

Av Resources Board -(ARB)

Inspect=on andMaintenance RevtewCommittee (Rv1R C)

Department of Motor -Vehicles (DMV)

US Env=ronmentalProtection Agency(USEPA)

Programmat=c responmbdtty

Dw=mon of the State Department ofConsumer ProtectionImplements Smog CheckCertifies, trams, and monitors stabonsand teohmctansCollects program dataDevelops speceficabons for tesbngequipment and procedures

Diwslon of the CaliforniaEnvironmental Protecbon AgencyPrepares State Implementatmn Plan(s~P)Demonstrates SIP compkance toEPA

Appointed by the governor anti thelegtstature to monttor theperformance of Smog Check,evaluate the program, and suggestprogram tmprovementsEvaluate Smog Check and presentresults to the legislatureHold (regular) public meetings

RegJster veNclesMaintain database of mspecboncertificates for vehEclesSend out Smog Check reminders withregistration renewal nottoes

Wrote and demgned requirements forenhanced I/MOversees Emplementabon of theClean Air Act leg=slst~onOversees state programs andevaluabonsApproves/rejects S~P

16 Science and Pubhc Poh6~r Februar): 2002

problematac to obtain representatave vehicle erms-mons da(a

Given the challenges encountered m obtmmngrepresentatave data, there are often many mformataongaps that must be filled m the evaluataon processWhen m£ormatlon *s lacking, ermsslons models canbe used to t511 m data gaps, but the models remainhighly uncertain These teebaucal challenges providea hvely barns for regulatory debate, particularlysince there is no single, agreed upon method forprogram evaluation

Evaluation as regulatory scwnce

Gaven the context and the regulatory motwatlon forthe evaluation, the evaluation process closely resem-bles regulatory science as defined by Jasanoff (1990,page 80) She defines three mare components ofregulatory scmnce knowledge production, knowl-edge sycthesIs, and prechctaon (t990, page 77) TheSmog Check evaluation process shows evidence ofall three knowledge production m the gathenng ofenusslons data, knowledge synthesis m the analysisof th).s data, and elements of prechcuon are seen how thls mformatmn is brought together and how itis presented as a measure of program effeetweness

Another aspect of the evaluatmn process that re-flects the charaetenstms of the regulator3, science ~sm the ARB’s and the IMRC’s use of "boundar3,work’ Jasanoff describes boundary work as a pro-cess of identff3nng who is m and who Is out, or mwhat realm a pamcular issue lies (1990, page 14) the presentation of evaluatmn results, the ARB andthe IMRC attempted to maintain a boundary be-tween the ’numbers’ and the pohey recommenda-tmns [n 1MRC public meetings, the committeechairperson continually expressed a desire to mare-tam. a separatmn between the "findings of fact", s orthe evaluation results, and policy recommendatmnsbased on these findings At one point the IMRCchmrperson stud

" l want to really emphasize -- it is mapor-rant to separate out the empmcal results fromany ~ecornmendatmns The results smaply tellus what do we know, what do we observeJudgements about pohcy have to take into ac-count such things as -- where do we need to bewith respect to the federal law They need totake into account cons~deratlons about Impactson different umverses of cmzens m the State ofCahfigrma let us not mingle the two ,,6

Through tins boundary constructmn, the committeeattemptx~d to protect the finchngs from polmcal m-fiuences so that they remained legntmmte bases forthe comrmttee’s pohey recommendatmns

In the Ught coupling of pohey and scmnoe m theregulatory science process, uncertainty is often thelocus of debate k is used to bolster and supportoplmons, agendas, and conflicts (Jasanoff, 1990,

Sctence and uncertainty m envtronmental regulatmn

FtmtoWlCZ and Ravetz, 1992, Ozawa, 1996, van As-selt et al, 1996, van Asselt and Rotmans, 1996) Un-certainty is amphfied or ignored, or somettmes acombmataon of the two, depending on the mot~va-tmn of dafferent actors (Wynne, 1987)

In a regulatory decision-making process, un-certainty can be used to advocate precautaon mpursuing potentmlly harmful enwronmental choices(the Precautionary Principle), to advocate cost-effectweness, or to delay regulatory aetaon alto-gether Uncertainty serves as an mfiuentlal rhetoncaland strategic devace m envaronmental regulation anddecision mahng

Uncertainty in the Smog Check II evaluation

Before the completmn of e~ther of the evaluatmns, amember of the IMRC stated m an mtervlew that fftheir evaluation " shows that [Smog Cheekl asfalhng well short of the mark,. I think that you willget people stmply chsputmg the data ,,7

Uncertainty m evaluatwn reports

Several techmcal uncertamt|es affect the evaluatmnprocess As was acknowledged m almost every in-terview conducted, the data that is avatlable on I/Mprogram performance is often contradaetory, chfficultto obtain, and generally scarce In addmon, manydata sources have been used to challenge the vall&tyof UM programs by supporting a eharactenzatmn ofvelucle ermsmons and driver behavaor that eontra-dints the characteristics represented or addressed mcurrent I/M program design and the models that areused to predmt their benefits (Lawson, 1993, 1995,Stedman eta!, 1997, 1998) Given the controversyover data and ,ts use m cnttqnes of I/M programs,data selectmn and avaflablhty is a very n-aportantcomponent of the evaluatmn process

Snnply put, data on veinele enusslons are neededfor the evaluatmn of an I/M program to esumateemasslons before and after a Smog Check II inspec-tion m order to arnve at an esttrnate of programbenefits Several different sources of data wereavmlable m the Smog Check 11 evaluataon First,mflhons of enussions measurements are collectedwhen veineles have thezr scheduled Smog Check ]Iinspection (close to nine rmlhon vehicles are testedper year) These data are contmned m the VebacleIdentffieatmn Database (VID). and are referred to \qD data

Second, roadside inspections have been conductedby BAR on tens of thousands of veincles that wererandomly pulled over In a roadside mspectJon~emmmons are measured using the Smog Check IImspectaon protocol (roadside data)

Finally, some velucle measurements are taken us-mga remote sensing dexaee (RSD) that uses a spec-trophotometer to measure emissions m an exhaustplume as a veincle drives by a specific location

Science ~md Public Pohcv February 2002 17

Sconce and uncertainty ~n enmronmental regulation

Uncertainties that arise in relation tothe data are in measurement,variability, reliability, and the clara’sability to represent actual on-roadvehicle emissions: each of these needsto be managed in the calculation ofprogram benefits

(Bishop and Stednmn, 1996) Such a measurement taken wathout stopping the velucle and w~thout even,necessarily, notifying the driver (RSD data)

A number of uncertamtaes are assocmted wathmeasuring vetncle ermsslons m general and, spemfi-call),, m relating tbas data to evaluating program ef-fectweness (Wenzel et aI, 2000) The uneertmntaesthat arise m relation to the data are generally meas-urement uncertainty, vanabfllty (aleatory uncer-tainty), data rehablhty, and the aNhty of the data torepresent actual on-road vehaete emlssmns Each ofthese needs to be managed m the ealculataon of pro-gram benefits No single evaluation methodologyhas been accepted by the I/M cornmumty (p~amanlyconsastmg of scientists, regulators, and techmcmns),and, therefore, no one data source ~s accepted asbeing the best one for evaluation

In addition to ermsslons measurements, regulator3,agenmes rel) on ermssmn factor models to esttmatethe emIssmn contribution from mobde sources EPAuses the series of MOBILE models Cahforma usesits own series called EMFAC A complete &scus-stun of the critiques and hmttatmns of these modelsis beyond the scope of ttus paper, but numerousstudaes have ~awn attentmn to flaws m them

Crmclsms mclude underestmaatmg on-road vehI-cte enussmns, poor representatmn of tugh ermtters,and poor modeling of evaporatwe emass~ons (for more detailed discussion of enuss~on factor models,see Fuj~ta et al, 1992, GAO, 1997, Harley et al,1997, Pollack et al, 1999~ NRC, 2000. Sawyer et al,2000) Desplte these criticisms, Mates are reqmred touse an approved ermsslon factor model m their SIPsto demonstrate the ermssmn reductmns that ,~U beactueved by an I/M program (USEPA_ 1992)

Both enussmns measurements and enuss~on factormodels were an invaluable source of mformat~onused by the groups m the evaluatmn process Inspec-tmn and rnamtenance program evaluataon is an ac-V.ve area of research and there ts no single acceptedmethod for the evaluatmn of a program hke SmogCheck II (USEPA, 1998, Coordinating ResearchCouncil, 2000) As was mentioned earlier, both theARB and/MRC completed evaluatmns of the SmogCheck II program Table 3 shows elements that werem each of the reports, including data that were used,evalua~aon goals, and the treatment of uncertainty

Each of the evaluatmn reports &scussed the tm-certamtaes and lnmtations m the available data, and,m each, the agency ldenttfied how ,t was planning torespond to, and manage, these tmcertamt~es The twostyles were qmte different The IMRC report de&-eared several pages, and spent a great deal of ttme mpubhc meetings, chscussmg the tmcertamtaes andcomplexaUes m evaluating an I/M program In theface of this uncertainty, IMRC stated that it prowded"a range of benefit esttmates that reflects the tmcer-tamty inherent m estmaatmg tons-per-day emissionreduct,ons’" (IMRC, 2000)

On the other hand, ARB bnefl!¢ discussed uncer-tamDes m the data sources and then selected onetype of data on which to base zts analysts, statingthat at was using the "best available data from a realworld standpoint" (ARB. 2000) From ttus point on,it dad not dascuss tmcertmn~ m the data or theanatys~s, but only an relataon to the IM-RC report

Where data were unavadable, both evaluationsrequtred assumptmns to fill m the gaps ARB usedthe current versmn of Cahforma’s ermssmn factormodel, EMFAC2000, to make esttmates of programperformance (ARB, 2000) The IMRC evaluatorsmade assumptmns and prowded bounding esttrnatesto examine the sensmv~ty of its predactmn of pro-gram benefits to these assumptions (/MRC, 2000)

In splte of the tmcertamtles, each report prowdesan estimate of program benefits m tons per da) ofemassmn reductmns attributable to Smog Check IIAs Table 3 shows, these esUmates do overlap, de-spite dtfferences m the methodoIog~es and data usedNonetheless, the fact that the two reports dad notmatch up exactly threatened the legmmacy of theevaluatmn process

Both the ARI3 and IMRC focused on the differ-ences between the reports and used these dafferencesas a bas~s for crmqumg the other’s report anddefending its own As the two reports were releasedto the same body of interested and affected pames,debates inevitably arose over the vahdaty and appro-priateness of each of the studies Pubhc debate and&scussmn was facflltated by the 1MRC’s regular,pubhc meetings that are reqmred m the committee’sgoverning legaslatmn 9

Each of these reports had a &fferent regulatorymandate and sought to answer a dafferent set ofquestmns Challenges arose over wbach data set wasappropriate -- roadside data versus VID data Ques-tmns were posed about the method of armlysls and,m particular, the approprmte role of models Finally,the questmn arose regarding the tmportar~ce andrelevance of the SIP cormmtment as an evaluatmncriterion These three debates formed the basis of theregulatory dascourse that developed surrounding theevaluaUon process

Uncertainty m the regulatory &scourse

Given the numerous uncertainties m the available dataand models, it is not surpnsmg that tmcertamty was

18 Science and Pubhc Pohcy February 2002

Table 3. Major f~atures of the ARB and IMRC evaluation reports

Science and uncertam(y tn enwronmental regulation

Feature

Regulatory mandate

Questlec(~,) answered evalu#,ion report

Data sources used

Modehng data used

Overall predicted programbenefits (tc,ns per day)

IMRC Report

State law requires reporting the StateLegislature

To what extent =s the Smog Check programreducing the emms~ons of on-road vehicles,and does program effectiveness change overtime?

What underlying causa~ factors make theSmog Check program more or less effective?

What ,s the cost and cost effectNeness ofSmog Check?

How can the effectiveness and costeffectiveness of the Smog Check Programbe improved?

VID data to estimate em=sslon reduct=oos(exhaust and assess durab~IRy of repairs,determine program avoidance (one-cyclebenefits of the program)

Rosdslde data to eshmate emissionreduct)ons (incremental benefits of theprogram over basic Smog Check)

Remote sensing measurements to assessdurabitrty of repairs

None, did ,nctude ARB’s EMFAC200D eshmste forevaporative emission reductions for comparativepurposes

HCb 86COb 1686NO~D 83

ARB report

Federal law requires report to the EPA

State law requ=res report)rig the IMRC

How wefl Is the enhanced element of the state’sI/M program, Smog Check II mectmg therequirements of the SIP’~

Roadside pullover data to est~nate exhaustemission reductions, average fleet emission re-ductions (g/mile)

EMFAC 2000 model used to est=mate tons-per-day emmston reduchon

EMFAC 2000 used to d~ermme the effect=veness ofthe program as compared to the SIP, preclctevaporative ernms~on reductions

HC~ 93 (evaperattve + exhaust)CO~ 785NOx" 27

Estimated ,mcertamty HC (40, 116) NOt reported(high, low) CO (864, 2235)

NO, (59, 93)

Esbmsted 1emission reductions Not reported HC 28In "SIP currency~’° NO, 12(tons per day) CO not reported

These numbers cannot be compared to AREs finaf numbers (in the final row) because the ARB estimate is reported m "SIP currency"which i,, determined using the EMFAC model These estimates of emission reductions are from analysis of the roadside pullover data

b These hgures represent the "best estimate" of emissmons benefits from the Smog Check 11 program, including estimates of pro-

respect=on, pretest repair, and removal of vehicles from the program area They do not include estimstss of evaporatwe emmslonbenefitsThe SIP eshmates o{ program performance were calculated using the EMFAC7F model This model Is now known to underest,mateemlss)cns Therefore, to reduce emissions 1 ten per day m EMFAC7F, or "SIP currency~, ~ is necessary to reduce emissions more than1 ton per day m the "real world" In order to compare their numbers calculated from the roadside analysts to the SIP, ARB had totranslate these values into "SIP currency" (ARB, 2000, page V-l)

nnportant m the regulato~ d~scourse surrounding theevaluat~(m However, m contrast to the evaluation re-ports that focused on uncertainties m data and models,the pubh e d~scuss~on focused on the dafferences m howthese uncertamtaes were addressed The tmcertamt~esthat were revealed and h~ghhghted through the regula-tory chs,zourse lay m methodological and ep~stemo-logical d~fferences m the evaluataon processes and notm the data More spec~fi~Aly, the d~scuss~on focusedon the ~aanagement of the uncertainties assocmtedwzth evaluation m each of the reports The regulatorydxscourse was dominated by fl~ree debates over theselect~on of d~ta, the method of evaluation, and theevaluat~o~ criterion These debates were framed mtechmca[ terms and focused on ~ssues assocmtedwzth uncertamb, m evaluation data and methods, butalso relented to institutional commitments mud agendas

SelectTng data what you don’t know can hurt you

Imme&ately, upon release of the two reports,questions arose over which was the appropriatetype of data to use Each data set has ~ts own ltm-~tataons and advantages ARB argued that theIMRC’s use of the VII) data was not vahd because ~tdoes not account for the effect of pre-mspect~enrnamtenance and repmr Thus, a vehicle could havean unoffieml pro-test, be repaired, and then pass thetest w~thout being registered m the VID Then theemissions benefits achaeved by the pro-test mad re-pmr are not counted ARB argued that tbas ~ssue m-vahdated the VID data as a tool for programevaluation In addataon, VID data are subject to theinfluences of fraud and cheating that occur m theSmog Check II stations ARB argued that these

Science a]~d Pubhc Pohcy February 2002 19

Science and uncertainty m envzronmental regulaaon

issues mean that VID data are not appropnate forevaluation

The IMRC evaluators, on the other Mad, wluleadmlltmg lun/tatmns, argued that the VID dam pro-vided an excellent opportumty for examtmng &verseaspects of the program because of the large numberof measurements Thls allows the VID data to bebroken into smaller groups to look at sub-populations of the vehicle fleet In addmort, the VIDcontains several measurements for some of the vehi-cles because of the change-of-ownerslup requtre-ment~° and allows for a temporal comparisonof Smog Check II testing data before and aftermspeetlons

At the same tame, the IMRC baghhghted that theroadside data used by ARB contains potential sam-phng biases because of soclo-economac dafferencesacross sampling areas Other biases m the data canarise fi’om tmamg of the inspections and the hm~tedpopular, on of vehicles that are captured ARB ar-gued that roadside data are the most accurate repre-sentanon of on-road vehicle ermsslons because youcapture the vebacles m the con&tlon that they aredriven and, ldeaUy, you measure emissions of velu-des m propomon to the amount that they are danvenon the road

EvaluatTon method getting from data to answers

A central questmn raised by each of the groups wasnot only what data dad the other group select, buthow did ~t apply that data Each group crmqued theother’s report vclth respect to the evaluation method-ology Tbas focus on method seemed a natural locusfor cntimsm and critique of the reports When it be-came e~ndent that the ARB study would be releasedat the same tune as the IMRC’s and that they bothprovided estmaates of tons-per-day emission reduc-tions, one ARB staff member sazd

" our real concern became ~ do the numbersmatch up9 And, more important than do thenumbers match up, ff the methodology is &f-ferent, ~s it going to be as cre&ble or accurateas the methodology that was chosen for theARB/BAR report9’’H

Much of the debate focused on the application ofdata and the response to uncertainty The debateover method became, m some ways, a battle of slm-phclty versus complemty. The IMRC critiqued theARB for hiding assumpraons that were made m itsanalysis and oversmaphf3nng the analyms TheIMRC also critiqued the ARB’s use of EM-FAC2000, pointing to the numerous uncertamtmsand [mutations m enussions-factor models ~2 Over-all, IMRC argued hhat the ARB was not clear m re-vealing the uncertainties and assumptions inherent mits analysts

The ARB defended ~ts methodology and data se-lection, stating the methods used data that reqmred

The regulatory discourse wasdominated by three debates: sdectionof data (what you don’t know can hurtyou); method of evaluation (gettingfrom data to answers); and evaluationcriteria (is the target 110 tons per dayemission reduction)

the fewest "’leaps of faith" ~3 At the same tame, theARB pointed to the complexity of the LMRC’smethodology It questioned the IMRC about why itdad not use a mmpler methodology, stating that amrnpler method was relevant because ARB dad itthat way ~4 Further, the ARB argued that the IMRCreport d~d not dascuss the limitations of the VID datam a manner that reflected thetr seriousness

Evaluation crztena 110 zs the magTc number

The tturd debate that dominated the regulatory dos-course was over evaluanon criteria One metric ofsuccess of the Smog Check LI program as the t 10tons per day of em~sslon reductmns attributed toSmog Check IIm the 1994 ozone SIP Given theuncertainty’ an the models used to predact programperformance m the SIP, the relevance of that numberas an evaluation metrm was a point of dasagreementFor ARB, the SIP is a defining document, at outlineshow and when the state will comply with clean mrguldelmes As such, the ARB emphasized the Im-portance of the SIP commitment as an evaluatmnmetric

" [you] can°t separate [Smog Check and theSIP] You have Smog Check because of thefederal Clean Atr Act and the SIP Therefore,that’s the world we lave in ,,~5

For ARB, the SIP and the regulatory mandate that xtcomes create a ~orld in wluch its relevance cannotbe ignored, and, therefore, it defined the ARB’sevaluatmn method

Having a &fferent regulatory mandate, the IMRCdoes not imbue the SIP comrmtment w~th the sameimportance 16 Therefore. the IMRC wowed the SIPcornrmtment as a regulatory artlfact~ and not a meas-ure of program success that was relevant to itsevaluation The IMRC’s goat was focused on com-pleting a ngorous and scientifically defensibleevaluation In one pubhc meeting, the chmrperson ofthe IMRC sXated

’° the novel and path-breaking aspect of thisreport is that it attempts to get out of thatcycle ~acknowledgmg the unportance of the

20 Scmnce and Pubhc Pohcy February 2002

SIP for a legal standpoint_ but also aclmowl-edging that there is a big wide world out there,lots of complex things going on w~.h SmogCheck, and we want to get a grip on those ,,~7

For IMRC, the goal was to achaeve scientific ~dealsand ~gor, further evidenced by ats employment ofcontractors from Lawrence Berkeley Natmnal Labo-ratory, a Inghly respected sc|entific mstatutaon, toperform the evaluataon and the chmrperson’s fierceattempts to maintain a separatmn between anal~calresults ~a~d policy reeomrnendaUons

The clash over these dafferent mterpretatmns ofthe SIP comnutment remained firmly entrenchedthroughout the evaluatmn process In its final report,ARB wrote that, ’~here are still s~gmficant chs-agreements between the UM Revaew Cornrmttee and[the] ARB regardmg the need and trnportance ofconsidering the SIP targets, and program perform-ante m relation to those targets" (ARB, 2000, pageES-5) Likewise, m its final report, the IMRC statesthat the "[r]esponsxbthty for evaluating Cahforma’sSIP comphance rests solely w~th the ARB" (IMRC,2000, page ES-6)

Uncertainty m evaluatton reports and dtscourse

In a sense, the concept of uncertainty changes as welook at the evaluaUon reports and the regulatory d~s-course surroanCmg the evaluation process In thereports° uncertamt-y as chscussed m terms of meas-urement msues, lmutaUons m data, and complexitiesm models However~ m the regulatory debates andchscourse, uncertmntaes were &scussed m terms ofselectmn of data, methods of evaluauon, and selec-laon of evaluation criteria For example, the debateover dalxt selectmn focused on what the data can andcannot l~ell you, not how uncertain a gwen measure-ment is

These debates h~ghhght the influence of msutu-honal comrmtments and responsabflmes on the useof sc~enttfic mformat~on m the regulatory processEven though the ~ssues that formed the basas of thedebates were framed m technical terms (modeling vsempmcal data, roadside pullover data vs VII) data)these issues were very. closely t~ed to mststutlonalframes and eommmnents ARB defended its meth-odology and data selectmn hlghhghtmg the tmpor-tance of the SIP comn~tment and Its relevance toARB’s methods and evaluaUon For ARB. this cre-ams a snnphclty m Its rmsslon and methods that ~sreflected m its arguments m the debate For ARB,ttus analysis was sunpl3, meant to "check off abox" i~

The I MRC, on the other hand, defended ~ts repre-sentaUon of the uncertamues and its response to theuneertmntles as being more robust Further, m ~tsfinal report, the IMRC stated that the SIP was not itsresponstbfla~v For the IMRC, the SIP does not de-flue ~ts mstltuUonal rmssmn or respons~blhty IMRCvalued the sc~entffic cre&bfllty of ~ts report, winch ~s

S~ence and uncertainty tn envtronmental regulation

clear m its focus on the robusmess of its analysis aswell as mats focus on the complemty of the evalua-tion process Tins ~mportance is evident m theIMRC’s chairperson’s desenptmn of the report as"novel and path-breakang’ 19

The uncertainties that dominated the regulatowchscourse were not only related to the uncertainty mthe data and models, but to how that relates to themstatutmn’s goals and respov~btlmes In one sense,tins creates a new layer of uncertainty introduced bythe uptake and apphcat~on of uncertain mfonnaUoninto evaluatmn methods and results Therefore, un-derstanding uncertainty, and ~ts tmpaet on regulatorydec~smn making is incomplete ff uncertainties m thescientific methods and data are consadered aloneBecause of the relauonstup between evalualaonmethods and regulator), goals, the dominant uncer-tainty in the discourse ~s no longer one that can bedescribed through statistical or analyucal methods 20One way to better understand these tmcertamt~es, asis shown by tbas case. ~s through descnpuve or nar-ratwe anaIys~s

Discuss~o~ and implications

The Smog Check ]I evaluation process as a tughlyuncertain and contentmus regulatory process thatdemonstrates the influence that science and uncer-tainty have on the construction of arguments Theavadable data and models are amportant to the pro-cess, but even more slgmficant Is how pames selectamongst the mformatmn, mter-pret and respond touncertainties, and set criteria for evalual~on

The two evaluation reports were never reconciledand neaher the ARB nor the IMRC ever fully ac-cepted the other’s methods and results The reasonfor the lack of resolution hes m the differences m thegroups’ mstltutmnal goals and commitments, andthe ways in which the two groups selected data,analyzed that data, and set regulator3, performancestandards that reflect these goals

These dtftbrences not only created the frameworkv~tinn which each group conducted ~ts evaluation,but they also shaped the regulatory debates and dis-course In the reports, uncertainties were chscussedand analyzed wth respect to the evaluation method-otogy Then, m the debates and discourse, tins dis-cussion extended to the mst~tutaonai goals andcommitments that Influenced d~scusslon of data andmethods m the report Wath tins shaft, the uncenam-Ues that became the focus of the debate were notonly related to uncermmUes m the data, but to howthose uncertamUes were treated and responded to inthe two reports

The process of constructing a regulatory argu-ment, m tins case, about how Smog Check II isperforming, ~s complex W~thout analyzing and un-derstanding how the ARB and the IMRC each se-lected data and methods to perform their evaluation,the source of d~sagreement ~s eluswe D~sagreement

Science ~nd Pubhc Pohcy February 2002 21

Sc~enee and uncertainty m envzronraentaI regula~7on

over the value of data sources and the appropriateevaluation methodology are not hnked m purelytechmeal or scientific sources, but to m~tuUonalgoals and comnuUuents, or to the context m wbaehthe evaluation is being conducted

Understanding the debates and uncertamtms ob-served m the regulatory discourse reqmres an analy-sis of the sclenUfic, msntuuonal, and polmealcomponents of the regulatory process as a whole andnot as separate parts Other researchers have ob-served and noted the influence of interests and goalson the constmcUon of arguments m pohcy debates(see, for example, Hernck and Jarmeson, 1995 onacld ram. Soneryd and Uggla. 2000 on planning de-clsaons m Sweden, and Majone. 1989, Roe, 1994,Schon and Rein, 1994 for more general &seussmn)

Because of the trnportanec of how arguments areconstructed m the regulatory d~scourse, the SmogCheek II evaluation process demonstrates a crucialweakness m utlhtanan decision analytic techmquesfor use m understanding and de, eloping environ-mental decls~on-makang frameworks Ut:htananmethods such as cost-benefit anatysls are ’ends-oriented’, focusing sole|y on decls|on outcomesThese methods cannot account for the context ofthe env:ronmental dec~smn making and regula-tor~ processes In other words, the3, do not incor-porate reformation on the formauon of values or ar-guments

For example, ApostoIakls and Pickett (1998) at-tempted to tmplement an analytic-dehberatwe pro-cess, lake that enwsmned by the National ResearchCouncil (NRC. 1996), for a remediation projectThey used the analyt~c-hlerarehy process to assessthe priorities of stakeholders as input to a mulUat-tribute utfllty anaIys~s Despite several surveys andchscussmns vath the participants, the decision ana-l~m outcome based on the multmtmbute ut~htyanalys~s was, m the end, unacceptable to thestakeholders (Apostolakas and Pickett, 1998) z’

The reason for the failure to find an acceptablesolut:on lay m the fact that the declslon anatyacmethods could not allununate the stakeholders’ un-derlying, long-term, and rather negative feehngsabout the mst~tuuon responsible for cleaning up thes~te These feelings created the context w~thua wluchthe decision was to be made and they were not in-cluded m the analyUc-dehberaUve process

The Apostolakas and Pickett ease illustrates theweakness m uUhtanan methods as apphed to envi-ronmental decisions Combined with the evidence ofthe influence of mstltuUonal goals and comrrnunentson the uptake and use of sclence and the processingof uncertainty na envarormaentat regulatory processfrom our analysis, these cases demonstrate that analtematave framework is needed to understand envi-ronmental regulatory deelsmn making Our analystsdemonstrates the value of a narrauve and descnpuveapproach for unders’tandmg the influence of tnstltu-Uonal goals and comnutrnents on the processing ofscaence and mnoertamty and the construction of

arguments and debates m the envaronmental regula-tory process

As our analysis of the Smog Check II evaluaUonprocess reveals, the uptake and use of scmnttficmformaUon and uncertainty is influenced by themsUtutmnal context of the declslon making To un-derstand controversy and debate m the envaron-mental regulatory process, tbas influence needs to berecogmzed m the methods used for the developmentof frameworks for envaromaaental declsmn makingOur ease study demonstrates the usefuhaess of narra-rave and descriptive analysis of the processing ofuncertainty m the regulatory process for acluevangtins goal

Other methodolog:es that incorporate narmtaveand descnpuve analysis of the enw, ronmentat regula-tory process include frame analysis as described bySehon and Rein (Sehon and Rem, 1994, Rein andSchon, 1996), the STS, boundary approach em-ployed by Jasanoff (1990), or narratzve pohcy analy-sls as descnbed by Roe (1994) Each of thesemethods has the advantage of examining the con-stmctaon of arguments and their mteracUon m theregulatory process LLke the method employed mfins analysis, these approaches to understanding andevaluating envaronmentaI decmmn making allow forthe consideration of the context of decls~on making,whmh is erucml for the development of effeetweenwronmental dec~smn making frarnewor~

Notes

1 The NAAQS are set by the EPA for s~x cr~tena pollutantscarbon monoxide, ozone, nitrogen dlo×~de, sulfur d~ox~de,particulate matter, and lead

2 The term "attainment" refers to whether or not an area of thecountry meets the national ambmnt e=r qual=ty standard for ag~ven pollutant Areas are class=fled by the EPA according totheir degree of non--atta~nment extreme, senous, severe,and worse States w~th areas that are out of attainment arerequired to submit a plan, known as a State ImptementabonPlan, to EPA, demonstrating how and when the non-attainment areas will come Into complmnce w~th the atr qual-=ty standards Areas wth more severe atr pollutmn area~towed more t=me to come trite attainment

3 The IM240 =s a 240-seccnd dynamometer test that =s a sub-set cf the Federal Test Procedure, the, test used to certifythat new veh=cles meet emission standards and fueleffic=ency requirements

4 The areas of the stats subject to Sine9 Check II are theurban=ed portions of Los Angeles, Ventura, and San D~ego,the metropolitan area of Sacramento, the Southeast Desert,and the San Jcaquln Valley

5 Excerpt from an IMRC publ=c meeting, 3 May 2000 m SanFrancisco, California

6 Excerpt from an IMRC pubhc meeting, 19 June 2000 m Sac-ramento, Calffomm

7 IMRC Member, ~nterv~ewed by Loome Welts Bedsworth,Berkeley, Cahfom~a, ~ 8 November 1999

8 The dnver may not=co the set-up en the s~de of the road, and~n some cases t ~s used to reform drivers of their vehicle’sem~ss=ons (see, for example, B=shop st el, 2000), bu~, for themost part, RSD can be =mpiemented and used w=thcut d~s-rupt, n9 tra~e flow

9 Cahferma Health snd Safety Code 44021, 44021 (a)(4) cifically d,scusses pubhc meetings

10 The change-of-ownership requtrement =s designed to protectvehmle buyers by requ~nn9 that veh=cfe sellers obtain a

22 Scmnce and Pubhc Pehc5 February 2002

Smog Check cer’afieate for that vehicle pnor to the sale’~ 1 Inter/law st ARB, Sacramento, California, 5 June 200112 To illustrate the contentious nature of the use of em=eslor~

factor models, It is worth noting that the BAR did not agreewith the ARB s use of the model eK’her The ARB and BARbegan work on the Smog Check II evaluation together Infact, analysts from BAR completed most, If not all, of theroadside data analy~s When ARB decided to use the EM-FAC2000 model in the evalust=on report, BAR asked to haveit~ name removed from the report From that point on, theARB took the lead on the evalustion report In pubhc meet-Ings, representa~ee of BAR wout~ answer specific quee-bons relating to data or the roadside analysis, but essentiallyail the other Input was prowded by ARB representatives

13 IMRC meeting, 3 May 2000 San Francisco, Cahfomla14 IMRC mesbng, 3 May 2000 San Francisco, California15 IMRC mestlr~g, 19 June 2000 Redondo Beach, Cahforn~a16 it as mportant to note here that the IMRC =s made up of rap-

reeer~tatwes from all roterested and affected parties In thisdebale over the retewnoe of the SiP, there was disagree-ment within the committee The member who represents ana=r-qual~ management district also Viewed the SIP com-mitment as a number that carried great importance m hisarea, a major non-attainment area of the state tn fact, thisdebale and the fact that the IMRC would not include ad~cusslon of the SIP was a major factor Jn his decision notto vote to accept the IMRC evaluation report

17 IMR¢, meebng, 19 June 2000 Redondo Beach, CaEtfomla18 Interview at ARB, Sacramento, California, 5 June 200119 IMRC, meeting, 19 June 2000, Redondo Beach, Cailfomla20 This influence of interests and values on the processing of

uncel~amty In regulatory debates has been noted by StldmgIn hi=~ analysis of precaution =n risk management processes(Shrhng, 1999) They term the combmahon of nsk, uncer-famty, ignorance and ambiguity as "mcartr~ude" to dehneate ~tfrom the "traditmnal" defin[tmn of uncertainty In their casework, they note the tendency to try to treat ignorance and In-tractz=ble uncertainty using propablilstJc methods, despitetheir inadequacy

21 A senes of papers discuss this attempt to implement theanalytlc-dehberstlve process using uhhtanan declslon-analyL~c tools The one cEted m the text as an overview, othersinclude (Accorsl et el, 1999a, 1999b)

Referen=ces

Accorsl, Roberto, George Apostoiak~s and Ennco Z~o (~999a),"Pnont=zmg stakeholder concerns an environmental risk man-egement", Journal of Risk Research, 2(1), pages 11-29

Accorel, I:; E Z=o and G E Apostolales (1999b), "Developing utd~functions for env~ronmentaf decision mal~ng", Progress In Nu-clear Energy, 34(4), pages 387-411

Apostolakls George E and Susan E P~ckett (1998), "Dehberatlonmtegrahng analytical results into environmental daces=on mak-ing revolving mu~hpts stakeholders", R~ak Analysis, 18(5),pages 621-634

ARB, Air Resources Board (1994), California State Implements-#on Plan, Volumes I-IV (vwvw arb ca gov/sip/s~p htm)

ARB, Air Resources Board (2000), Evaluation of California’s En-hanced Inspechon and Maintenance Program (Smog CheckII), 12 July 2000, Sacramento

~=shop, G A and D H Stedman (1996), "Meesunng the emissionsof passing cars", Accounts of Chemical Research, 29(10),pages 469-495

Bishop, Gary A, et al (2000), "Dnve-by motor vehicle emissionsimmediate feedback in reducing air pol~ubon", EnvironmentalScience and Technology, 34(6), pages 1110-1116

Campbell, Brian L (1985), "Uncertainty as symbolic action in dis-putes among experts", Social Studies of Science, 15, pages429--.4~,3

Coordmahng Research Council (2000), "Session IX evaluatingI/M prcgrams", presented at 10th On-Road Vehicle EmissionsWorkshop, San Diego, CA

Elzmga, A (1997), "From Arrhen=us to megasc~ence interplaybetweet~ science and pubhc decision making", Amble, 26(1),pages ~2--80

;uj=ta, Eric M, eta/ (1992), "Comparison of emission inventoryand ambient concentrahon rabos of CO, NMOG, and NOx in

Sctanc¢ and uncertmnty zn enwronmenta] regulatzon

California’s South Coast Air Basra", Journal of the Air andWaste Management Assoc~abon, 42(3), pages 264-276

Funtow=cz, SIIv~o, and Jerry Ravetz (1990), Uncertainty and Qua/-try in Science for Public Po#cy (Kluwe. Academic Publishers,Dordrecht)

Funtowcz, S O and J R Ravetz (1992), ’3"brae types of nskassessment and the emergence of post-normal science", mSheldon Knmsky and Dominic Gelding, (editors), Social Theo-ries of Risk (Praeger, Westport)

GAO (1997), A~r Pollution Ltmrtabans of EPA’s Motor VehicleEmlsstons Model and Plans to Address Them (GovernmentAccounting Office, Washington DC, GAO/RCED-97o210)

G~eryn, Thomas F (1983), "Boundary-work and the demarcationcf science from non-s~ence strains and interests m profes-sional ideologies of scientists~, American Sociological Rewew,48(December), pages 781-795

G~eryn, Thomas F (1999), Cultural Boundanos of Science Credl-b#tty on the Line (The Unwersity of Chicago Press, Chicago)

Harley, Robert A, Robert F Sawyer and Jana B Mdford (1997),"Updated photeehem=cal modeling for California’s South CoastAir Basra comparison of chemical mechanisms and motorveh~ote emission inventories", Environmental Science andTechnology, 31 (10), pages 282.9-2839

Flemck, Charles, and Dale Jam~eeon (1995), "The social con-struction of acid ram some lmphcahons for science/policy as-ssssment", Global Environmental Change, 5(2), pages 105-112

Hong, Yuan and George Apestolal~s (1993), "Conditional Influ-ence diagrams =n risk management", R~sk Analysts, 13(6),pages 625--536

IMRC (2000), Smog Check U Evaluat/on, 19 June 2000, Sacra-mento

Jager, Jill (1998), "Current thinking on using scientific findings environmental policy making", En~rcnmentai Mcde~ng andAssessment, 3, pages 14.3-153

Jasanoff, She~la (1986), Risk Management and Political Culture(Russell Sage Foundation, New York)

Jasanoff, Shella (1987), "Contested boundaries m pohcy-relevantscience", Social Studies of Science, 17, pages 195-/’30

Jasanoff, Shelta (1990), The Frith Branch S~ence Adwsers aaPol/cymakers (Harvard University Press, Cambridge)

Jesanoff, Shells and Bnan Wynne (1998), "Science and decisionmalong", in Steven Rayner and Elizabeth L Malone (e~tors),Human Choice and Climate Change, Volume I The SocietalFramework (Battelle Press, Columbus)

Keeney, Ralph L, and Howard Ralffa (1976), Doorstone with Mu/-~le Objecf~vss Preferences end Value Tradeoffs (John Wileyand Sons, New York)

Lawson, Dougtas R (~993), "’Passing the test’ -- human behaviorand Catfforma’s Smog Check Program", Journal of the Air andWaste Management Asso~at~on, 43, pages 1567-1575

Lawson, Douglas R (1995), "The costs ~ ’M’ m I/ M ~ refiechonson tnspecbon/Mamtenance Programs", Journal of the A~r andWaste Management Assoc~ahon, 45, pages 465-476

Majone. Gisndomentco (1989), Evidence, Argument, and Per-suasion In the Po#cy Process (Yale Universe, Press, NewHaven)

Merkhofer, Miley W, and Ralph L Kesney (1987), "A mulhattnbuteutiht3~ analys~s of altemabve sites for the disposal of nuclearwaste", Risk Analysts, 7(2), pages 173-194

Morgan M Granger and Max Hennon (1990), Uncertainty AGuide to Deahng wrth Uncertainty In Quantffatwe Risk and Po/-;cy Analysts (Cambridge University Press, Cambndge)

NRC (1996), Understanding Risk Informing Decisions In a De-mocrabc Society (National Academy of Sciences, WashingtonDC)

NRC (2000), Modetlng Motor Vehicle Emissions (National Acad-emy Press, Washington DC)

Ozswa, Genres (1996), "Smence In enwronmental conflicts", So-ciological Perspect/ves, 39(2), pages 219--230

Pollack, Alison K, et al (1999), Invesbgatlon of Emission Factorsin the California EMFAC7G Model (Coordinating ResearchCouncil, Atlanta Georgia)

Prestdential/Congress=ona~ Comm~es~on on Risk Assessment andRisk Management (I997), Risk Assessment and R~sk Man-agement in Regulatory Decision-Making (Final Report VolumeIi) (Presidential/Congressional Commission on Risk Assess=ment and Risk Management, Washington DC)

Reckhow, Kenneth H (1994), "A decision analytic framework forenwronmental analysis and simulation modeling", Environ-mental Toxicology and Chemistry, t3(~ 2), pages 1901-1906

Science a ~d Fubhc ])ohc-T FebruaD, 2002 23

Sezence and uncertam~ zn enwronmental regulation

Rein, Martin, and Donald Sch6n (1995), "Refraining poh~y dLs-~urse’, tn Frank Fischer end John Forester (editors), TheArgumentative Turn m Policy Analysts and Planning (DukeUn=versK~/Press, Durham)

Roe Emery (I994), Narrabve Pol/cy Analysis Theory and Prac-bce (Duke University Press, Durham)

Rowe, Wdham D (1994), "Understanding uncerta=nty~, R/sk Analy-sin, 14(6), pages 743-750

Sawyer, R F, et ai (2000), =Mobile source critical Review 1998NARSTO assessment", Atrnosphenc Enwronment, 34, pages2161-2181

Schon, Donald A and Martin Rein (1994), Frame Reflecbon To-ward the Resolution of Intractable Pohcy Controversies (BasleBooks, New York)

Shackley, Simon and 8nan Wynne (1996), "Representing uncer-tainty =n global climate change scmnce and pohcy boundary-ordenng dewces and authority", S~enoe, Technology, andHuman Values, 21 (3), pages 275--302

Shrader-Frechette, Krist=n (1996), =Methodological rules for fourclasses of uncedamty", =n John Lemons (editor), Sclenbfic Un-certainty and Enwronmental Problem So/wng (BlackwellScience, Cambndge)

Soneryd, Lmda, and Ylva Uggla (2000), "Poht=cs as a struggteover definrhon -- two case studies", Enwronmental Scienceand Policy, 3, pages 277-285

Star, Susan Lelgh (1985), "Scientific work and uncertainty", SocialStudies of Science, 15, pages 391-427

Stedman, Donald H, et at (1997), "On-road evaluation of automobile emission test program", Enwronmental Soenceend Technology, 31 (3), pages 927-931

Stedman, Donald H, Gary A B=shop and Robert S Slott (1998),"Repa=r avoidance and evaluating mspechon and maintenanceprograms", Environmental Science and Technology, 32(10),

pages 1544-I 545Sttdlng, Andrew (1999), On Science and Precaution m the Man-

agement of Technological Risk (Final Report of a Project forthe EC Forward Studies Liner) (ESRC Global EnwrenmentalChange Programme, Sussex)

USEPA (’[992) =lnspechon and maintenance program require-ments final rule", Federal Register, 40 CFR Part 51, SubpartS (5 November 1992)

USEPA (1994}, Clean Cars for Clean AIF lnspechon and Mainte-nance Program (Office of Mobile Sources, Ann Arbor, EPA400-F-92-016)

USEPA (1995),//M Briefing Book Everything You Ever Wanted toKnow About Inspecbon and Maintenance (Office of Aw andRad=at~on, Washington DC)

USEPA (1998), Inspecbon and Maintenance (I/?A) Program Effec-t;veness Methodologies (Office of Mobde Sources, Ann Arbor,EPA420-S-98-015)

van Asselt, Maqolem B A, Arthur H W Beusen and Honk B MH~idennk (1996), "Uncertainty =n integrated assessment a so-cial scientific perspectwe’, Enwronmentef Mode~ng and As-sessment, 1 pages 71-90

van Asselt Mar~e[eEn B A, and Jan Rotmans (1996), "Uncertaintyin perspective", Global Environmental Change, 6(2), pages121-157

Wenze[, Tom, Brett C Singer and Robert Slott (2000), "Some~ssues In the statFst~cal analysts of vehicle em~ss=ons", Journalof Transportation Stabstlcs, 3(2), pages ’~-14

Wynne, Brian (198"0, "Uncertainty ~ teohmcs~ and social", inHarvey Brooks and Chester L Cooper (edrtors), Science forPubho Policy (Pergamon Press, Oxford)

Wynne, Bnan (1992), "Uncertainty and emqronmental learningraconcelv=ng so=once and policy Jn the preventat=ve parad¢gm",Global Enwronmenta/Change, 2 June, pages 111-127

24 Sctence and Public Poh~, February 2002

Volume 29 Number 1 February 2002

Science andPublic Policy

Of being seen to do the right thing: provisional findings from the firstAustralian consensus conference on Gene Technology in the FoodChainAhson Mohr

Science and uncertainty in environmental regulation: insights from theevaluation of California’s Smog Check programLomse Wells Bedsworth and Wflham E Kastenberg

Scuppering the waves: how they tried to repel dean energyDavid Ross

A theoretlcal review of co-operative relationships between firms andumvers=tiesEva Maria Mora Valentin

Defining a safe genetically modified organism: boundaries of scientificrisk assessmentKatherine Barrett and Ehsabeth Abergel

Evaluation of governments’ scientific output: a bibliometric profile ofCanadaJ-P Robltaille and B Godm

Ultra-left science policy and anti-modermzation in Argentina: OscarVarsavskyMaunclo Schmjet

Plus

Book reviews on pohey analysls studies, Science Wars, and Canada andEurope m cyberspace

Amcle summaries, aufl~or affiliations back coverISSN 0302-3427

Pubhshed from Great Bntam by Beech Tree Pubhshmg

Volume 29 Number 1 February 2002

Science and Public Policy

Main art~c|es in this issue Full summaries: see individual arncles

Of being seen to do the right thingprowstonal findings from the firstAustrahan consensus conference on GeneTechnology m the Food ChainAlison Mohr (Nathan Campus Ga-fffithb m versl~:, Austraha)Pages 2-12

Science and uncertamt3’ In environmentalregulation insights from the evaluation ofCahforma’s Smog Check programLomse Wells Bedsworth and Wtlham EKastenberg (lomverstty of Cahforma atBerkeley, USA)Pages I3-24

Scuppenng the waves how they tried torepel dean energyDavid Ros~ (Journalist, LrK)Pages 25-35

A theoretical review of co-operatwerelationships between firms anduniversitiesEva Maria Morn Valentm (Umversidad KeyJuan Carlos, Spare)Pages 37-46

Defining a safe geneticaUy modifiedorganism boundaries of scientific riskassessmentKatherine Barrett (Umverstty of Vietona,Canada) and Ehsabeth Abergel (YorkUmversIty, Canada)Pages 47-58

Evaluation of governments’ sclentaticoutput a bibliometrtc profile of CanadaJ-P Rob~tailie and B Godm (OST/IN’RS,Canada)Pages 59-68

Ultra-left science poh~’ and anti-modernization m Arg, entma OscarVarsavskyMauncm Schoijet (Umversldad AutonomaMetropoh tana-X ochtmflco, Mexico)Pages 69-75

Ttus is an mmal evaluation of the first Australian consensus conference [t illustrates lessonsle2unt from staging this method of participatory techrtobg~ assessment (pTA) by applyingan analytical framework of three mmensmns socml context, mstltutmnal context, and p’I Aarremgement While Australia stands to benefit from ~ls style of decision-making there arehurdles to be overcome When transplanting the consensus conference model into a newsocial context, a period of m~tic~pato,~/sociahsation is needed so that orgarasers ann pamcl-pants are clear about what can and cartrtot be achieved

New calls for envlrom~ental decismn-makmg frarne~orks emphasxze the need for an hohs-tic approach that incorporates technical and non-tectuucal expertise, and participation by allinterested and affected parties We analyze the evaluatmn of an environmental regulatoryprogram to characterize the interaction of sctence and pohcy and the processmg of uncer-tainty using concepts from S&T studms This demonstrates the influence of msUtuttonalgoals and commitments on the uptake and use of smen~e and the processing of urmertamtyin the regulator./process We discuss the tmphcahons of such analyses on the developmentof new envlromnental declmon-makmg frameworks

This paper zs a study of the measures used by what the author calls "the energy estabhsh-re.ant" in the UK to hinder the development of wave energy fron, the time of the inventionm the mld-70s of se~,eral devines designed to convert sea waves into elecmmty and turn stinto a central resource fo~ the natron and for other countries The paper anks the hostlhty ofthe Department of Energ,, and its qatelhte bodies to the Brmsh Govermnent’s plan to con-struct ten pressunsed water reactors aad give a wmor role to nuclear power, plus tiae influ-ence of the oil and gas lobbies

The goal of this paper is to present a theoretical vleu of the co-operatwe relatmnstups be-tween firms mid umversmes We have revised the main subjects and key topics that areanalysed m the literature tbcus on umversity-mdustry tmKages As we have identifiedScience and Pubfic Pot,cy as a lournal that has pubhshed severat articles about umverstt,~-industry collaboratmn, m the second part of the paper we present a selected and annotatedblbhograph~ of appropriate articles that were published between 1990 and 2000 by thejournal

The development and commerctahsation of genetically modified (GM) crops conunuesdespite persisung uncertamtms regarding envtronmenta[ ~mpacts Regulators m Canadahavelatrned that ex:stmg federal policies for assessing en¢lronmental hazards are sc:en~e-based’ and sutTlciently precautionary. We c)mIlenge flus by examining the smenttfic dataused to approve one variety of GM canota for environmental release We argue that thelegitimacy and plausibility of the regulatory decision rests significantly on boundaries con-strueted around the definmon of a ’science-based risk assessment’ We advocate a strongerrole for the precautmnar) pnnmple

Over the last 15 years, budgetar~ restrictions on government departments have, according tosome, compronnsed the scientific production of public R&D mboratones Ttus artzcle usesb~bhometnc data to took at the scientific production of Canadian Federal intramural R&DThe data show the major amportartce of the Fedora! Government’s contnbutmn to the ad-vancement of Canadmn scmnce -- over a third of Canadian pubhcatlons m several disc~ph-nary specmhties In the disciplines in which they have dzstmgmshed tnemselves the most,federal researchers have, m terms of the quality of pubhcatmns, no cause to be envmus o1Canadian researchers in general

The anltc~ent~ficJsmo trend that started m Argentina in 1962 was a resistance to moderniza-tion Oscar Varsavsky’s best known work of 1969 combined elements of an ultra-lcRtstcritique of scmnce wath a critique of the way m which Argentine science was developingHe had a very important ideological influence m the l )70s m much of Latin America mem~ techmcal and scientific groups His work was used by obscm-ant~st elements lotrepressive pohcms

Pubhshed from Cereal Britain by Beech Tree |htbh~hmg. 10 Warlord Ctose (,uddford, Surrey GU I 2FP

Science and Public Policy

University lecturers and researchers, ministries,research councils, consultants and others fromthese 66 countries get it:

~-mtand, Sweden, Norway, Denmark, Iceland UK,I, mand Belgmm, The Netherlands, France, German3Luxembourg, Switzerland, Italy Spain, PortugaIAustria, Greece~ Cyprus, Malta, Estoma, RussiaPoland, Czech Repubhc, SlovaMa, Bulgaria, Romama.Hungary., Slovema Turke~ Israel, UN ESCWA -Iraq Bahrain. Pakistan In&a Bangladesh, Srl Lanka

Thailand, Malaysia, Singapore, Indonesia, SouthKorea. Laos, China, Talwan Japan, Tumsla, Ntgena,Kenya Botswana, South Africa, Canada, USAMexico, Cuba, Trinidad and Tobago, Venezuela.Colombia, Brazil, Peru Uruguay, Argentina, Australiaand New Zealand amongst others

They probably include people with interestssimilar to yours. It’s easy to join them.

Ihe ~apers m Sc ,en~e (,nd Pubhc Pollc~ are originalretereed, mostly research-based, and are read across theworld If you subscribe they will bring you theexperience ofomers m studying analysing ane~mplemennng scmnce and technotog3 policies

I-he papers cover old andnew topic’, h~gh and lowtt.chnology, rich and poorareas and the NICs SPPIs pubhsh ng work by thebl~ name~ and by the up-and-coming See nextpage for recent examptesI1 ~lll also give you bookreviews, viewpoints, somelclle~s and other moreoccasmnal features

Science an~PUDIIC POliCy

EclitorsKleron F|anagan Pohcy Research m EngineeringScmnce and Technology Manchester Umversw UKDavid Guston Rutgers Umver~t2, New Jersey USASusan Cozzens Georgia Institute ofTechnolog,,Atlanta USA (Consulting editor)

AdvisorsDesmond All Mlllenmunl3 Corn, ailing ] rlnload ann 1 (~b,lul~

Jennifer Bond NSF USAB Bowonder Administrative btafl ( ollcg~, cd indiaJean-Jacques ( hanaron tJmvcrstl3 ol (~renoble t rat~c~.Daryl Chubm Nauonat Scmncc Board I;SgPeter Heale~ Scmnce Pohcy Support Group lxmdon [’kI,~rs tngelstam t ech and bovla~ Change [ mkopmg S’acdt_r,Ron John,ston ALIIC I./m~crsllx ot Svdnc3 Auslraha( alesmus )urea KSG Harvard l.Jm~crsn.’, t hAHwan-Suk Kim KAISI" KoreaPatncm MeLaughlandeArregmGRAD[ l,ma I’cruFujm Nrwa NISTEP S&T Agenc,~ MpanJacques Richardson Consuhant FranceMichael~ Smith Commonwealth Science CouncilJohannes Weyer lJn;vers]~y of Bleteleld German)Dorothy Zmberg STPP Harvard [mlverslly USABook review edJlror Paul Rosen %A I S1.J York 11n,~ 1 ,k

For authorsWe aim to prowde a good service, including ~astrefereeing (when possible) and publication often v,~hlns~x months of acceptance We supp}y offprints -- andwill also mall them ou{ to people authors suggest flee

Some recent articles from Science and Public Policy

The American research unn-e~Mtv m America’s de.factotechnology pohcy, MichaelC row (Provost (Research).Cokunbia IJmverslty. USA) et atFebruary 2001

Academic research m EuropeKelth Pavia1 (SPRU, UK) Dec2000

Are government investments inR&D and market environmentneeded for indigenous privateR&D m LDCs? Evidence fromKorea. Sun (3 Klm ( Faetonktntversity. Korea). Feb 2000

Technology. pohey m the USand EU: shlftmg orientationtowards technology users,Nlcllolas S Vonortas (GeorgeVvashm~on Umversltx: USA).April 2000

DwersiW and tdent~t 3 themerger of five researchcouncils m Norwaj, Hans Skoie(NIF’I.J Nor\~ay) April 2000

Patents m a world of complextechnologies Don Kash (GeorgeMason Umversll) [/~.A) e:a/February 2001

Changing S&T pohcv culture~phase~ and ~rands m S&T mtndm V V Knshna (3 NehruUnJvers~tj. india) June 2001

Services and the search torrenovation mdmators a reviewof no/renal and internationalsurveys, Falz Galk)uj (LJnlversRxof Lille. France) el a/ Aug ]999

Research excellence andpmented innovations, Dmnat ticks Francis Norm et al (CHtResearch USA), October 2000

Techmcal roles and success of[IS federal laboratory-industrypartnerships. Barry Bozeman(GeTech, USA) et al. June 2001

Technology transfer or mcub-arran? Technology businessincubators and S&T parks inthe Phflkppmes, S Macdonald(Sheffield Umverslty, UK) el al,October 2001

The pohtms of export adwce:lessons from the early historyol the BSE saga. Erlk Millstoneet a/(SPRU, UK). April 2001

Anal?sis of MITUs ’Visions’and challenges for Japan’stechnology pohcy-makmgmechanisms Chih-iro Watanabeet a/(Tokyo Institute of-I ech-nolng~) February ] 999

Science and technology tssuesfor |egtslators Gary Kass (Paris-amentar~ Office of S& f. UK),( )~tober 2000

Promotmn of co-operativeresearch a Spanish experience,luan Acosta Ballesteros (Umv-ersn\ of Laguna, Spare) el al,()ctober 2000

Contnbutmn of b~sle researchto the Irmsh renovation system,t.rik Arnold et al (TechnopohsconsuItMg, Brighton. UK), April2001

SOME SPECIAL ISSUES

Democracy m S&T policyadvice in Europe, guest e&tedby M de Jong and M Mentzel(NetherIands), December 2001

Sense and nonsense in S&Tproductivity indicators, Ren-uBurr6 (Observatolre des S&T,France) 1 of g papers on Bench-marking RTD policies, gueste&ted by N~kos Kastrmos (EC,Brussels), August 2001

Interac~ve social science, guestedlted by Clans CaswaI1 (ESRC,UK) and Ehzabeth Shove (Lan-caster, UK), Mike Gibbons,Steve Woolgar, Ane Rap, etcJune 2000 See page 1 t

Technology transfer and East-ern Europe. guest edated byHenry. Etz-kow~tz, August 2000

Consensus conferences, focusgroups, citizens’ panels, etc,guest echted by Simon Joss (UrnofWestrnmster, UK), Oct 1999

Changes in Europe’s pubhcsector research, guest edated byJacky Senker (SPRU). Dec 2000

Sclennfic expemse and polmcalaccountabfllty paradoxes of sciencem pohucs, Peter Wmngan(Blelefeld Germany), 1 of 7 relatedpapers Jlme 1999

A|| 30 issues, q996-00:£100, U$$qBO or g’!64SornL max b~ photocopies

Subscription information...Published bimonthly (February, April, June, elc) A4 size,488 pages in 2001 464 m 2000 452 in 1999, 424 in1998 on acid-free paper t roe online access Co" hard copysubscribers at www catchword tom Back issues avail-able, start with any issue mcIuded b,, onhne aggregatorsarid abstracting and documenl dehveD services Index inDecember issues 1SSN 0302-3427 (print) and 147i-5430(electronm) Multi-year ~uhscrlplmns include a 20%discount 2002 lssue~ fo,m ~olume 29 I or prices seefollow-up (new column)

... and follow-uP througl www scpol demon co uk

-e44 t483 824871

,~ +44 1483 567497

b~4 Bill Page, Beech Free Publ,shmg

I0 Watford Close. GufidtbrdSurrey GU1 2EP. t~K