SOUP TO NUTS ON NON-ENERGY BENEFITS / NEISeedal2017.uci.edu/wp-content/uploads/Wednesday-07-Skumatz-smalle… · SOUP TO NUTS ON NON-ENERGY BENEFITS / NEIS Methods, ... Com’l labor,

Post on 21-Aug-2018

212 Views

Category:

Documents

0 Downloads

Preview:

Click to see full reader

Transcript

SOUP TO NUTS ON NON-ENERGY

BENEFITS NEIS

Methods Results andApplication at the Utility amp

Regulatory LevelEEDAL Conference Irvine

September 2017

Lisa A Skumatz PhD Skumatz Economic Research Associates Inc

762 Eldorado Drive Superior CO 80027303494-1178 email skumatzserainccom

copy 2017 SERA all rights reserved may be used with permission of author

SERA

TOPICS Background Measurement of NEBs Risk NEBs in State BC Risk Gaps and Conclusions

SERA

BACKGROUND NEBS Attention growing on NEBs after 20 years BC to assess investments (programs portfolios)

BC=f[PV(attributable benefits stream)] [PV(attributable cost stream)]

(With variations based on perspectivehellip) NEBs are net enhancement to ldquoBrdquo - incomplete

Biased investments decisions because all costs not all benefits

Omission - ldquo0rdquo is the wrong number High value from multiple quantitative studies

Evaluationrsquos purpose ndash to inform decision-making

SERA

20+ YEARS OF NEBS PROGRESShellip

But there still isnrsquot agreement on name - NEB OPI NNEB MB co-benefitshellipSource SERA all rights reserved

1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016

Perspectives +-Basic measurementLI Res amp Comrsquol

Explore BC (LIPPT)Expanding RampC Ests Initial mktg applics

Expanding estimates sectors studies methods Wider use in marketingInitial applications in planning

Re-explore BCIntroduction in states growthExpanding literature

Lists

SERA

5

NEB PERSPECTIVES CATEGORIES BEST PRACTICES

Utility Society Participant (ResampICI)bullCarrying cost on arrearages bullBad debt written offbullShutoffs ReconnectsbullNotices calls collection costsbullEmergency gas service calls (for gas flex connector and other programs)bullInsurance savingsbullTransmission and distribution savings (usually distribution)bullFewer substations etcbullPower quality reliabilitybullReduced subsidy payments (low income)bullOther

bullEconomic development benefits ndash direct and indirect multipliersbullTax effectsbullEmissions environmental (trading values andor health hazard benefits)bullHealth and safety equipmentbullWater and waste water treatment or supply plantsbullFish wildlife mitigationbullNational securitybullHealth carebullOther

bullWater wastewater bill savingsbullOperating costs (non-energy) bullEquipment maintenancebullEquipment performance (push air better etc)bullEquipment lifetimebullShutoffs ReconnectsbullProperty value benefits sellingbull(Bill-related) calls to utilitybullComfortbullAesthetics appearancebullFires insurance damage (gas)bullLighting quality of light bullNoisebullSafety

bullControl over billbullUnderstanding knowledgebullldquoCarerdquo or ldquohardshiprdquo (low income)bullIndoor air qualitybullHealth lost days at work or schoolbullFewer movesbullDoing good for environmentbullSavings in other fuels or services (as relevant)bullGHG and environmental effectsbullNegatives

Source (SkumatzSERA1996 on)

Net ThreeNet Positive amp negativeNet beyond standard efficiencyNet to gross

Non-OverlappingConsistent Units

SERA

KEY APPLICATIONS OF NEBS

MARKETING amp ROI ndashSell whatrsquos valuable to customers link to

peers

PROGRAM REFINEMENT ndashPositive amp Negative NEBs for

measures barriers incentives and targeting

TRAIN THE CHAIN ndashAlign Educate Actors on NEB

priorities

POLICY GOALS Quantifies Non-energy goals (eg

Low income jobs etc)

BC TESTS ndashRefined CE for program amp portfolio reduce bias in

investment

6

Source SERA all rights reserved

SERA

USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS

NegativeNEBs

Solar WH

Appearance -$14 NZ

Maintenance -$9 NZ

Source Skumatz Economic Research Associates research ndash 1998-2005

-005

0

005

01

015

02

Maint

Productiv Perf Life

Op CostTenant S

atComfor

t

Lite Safe SellEnvir

oOther

AampE Owner

Commercial Example

Residential Example

SERA

NEB MEASUREMENT

8

SERA

NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES

Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip

9

Monetized NEBs

Source Skumatz SERA research

NEB

s

Direct

Secondary

Model

Survey Story of a ferryhellip then it is academic

Corp Records Utility data

Change x valueFinancial Calcs

Third party jobsAnd emissions

Specialized academic Best for some NEBs

SERA

MEASURING PARTICIPANTNEBs Best practices

WTP not fruitful 7+ better options

Net Three Net positive amp negative Net beyond standard

efficiency Net to Gross

Non-overlapping Consistent units Recommend in ALL

process (impact) surveys - barriers

HTM=Hard to measure HTA=Hard to answer

Direct calc Accurate Small ldquonrdquo

WTPWTA Accepted Volatile HTA

Relative Fast strong HTA

LMSFast

strong clear ETA

Unfamiliar

Logit Conjoint

rankStrong Slow

complex

Regression Defensible Limited data cost

Market value Strong Data

Other Exploring Cost tradeoffs

Source SERA Research

SERA

HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)

11

Monetized NEBs

Direct Secondary Model Survey

Attributable Change (study)

Value or Financial

Calc

X

Total Attrib Stated Relative

Effect

Savings (or translation)(ldquoNormrdquo)

Individual NEB Shares

X

X

SERA

PARTICIPANT MEASUREMENT METHODS COMPARISON

005

115

225

335

445

MW

x

NE

Wx

CA

Wx

CA

Wx

SF-N

C

MF-

NC

C-

Lite

C-R

eb

C-R

eb

multLMS VerbalWTP

Source SERA Research

Chart1

mult
LMS Verbal
WTP
122
156
42
08
1
2
098
4
06
13
12
075
067
104
115
27
075
055
32
084
091

Sheet1

Sheet1

mult
LMS Verbal
WTP

Sheet2

Sheet3

SERA

PROS AND CONS OF NEBS MEASUREMENT METHODS

13

Source Skumatz SERA research

Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

Long history easy secondary sources Credible to reviewers vetted inputs

Not available for all NEBs

Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

Third party peer-vetted models available for economics jobs and emissions

Not available for all NEBs

Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

SERA

PROGRESS TRANSFERABILITY OF NEBS

14

Source Skumatz SERA research

Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

Program measure dependent

bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

Residential Target dependent (low income or MF vs SF)

bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

Cost savings consistency if transferable vs risk

SERA

RANGES TYPICAL NEB VALUES FOR WxRETROFI

15

SERA

PROGRESS AND GAPS Consistency Transferability

Some programs types others less studied Transferability research

Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

measures until then Across the board Savings share Regression

SERA

PROGRESS AND GAPS Gas vs Electric

Some researchndashusually program-wide not broken down International research found participant NEBs have

similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

MF Less-commonly-studied complicated by poor response

and complexity of sector (decision-maker some measures in home some central) separate from low income not common

Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

Commercial gaps in some measures

SERA

USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

18

0

005

01

015

02

025

Environment

Quality

Quantity

PgmAvailFlicker

Mainten

ance

Lifetime

SalesP

roductivity

Safety

Noise

Control

Illness

Total NEBs Multiplier=X

$

Savings value=$X

Chart3

023
016
014
01
008
006
006
006
005
005
003
0

Chart1

Vendor
NP
Partic
028
02
02
028
013
017
031
022
016
006
008
014
005
006
014
004
009
007
004
003
007
-006
018
006

Sheet1

Sheet1

Sheet2

Sheet3

SERA

NEB VALUES

19

SERA

ARE NEBS HIGH VALUE Energy savings are often the minority of

benefits from program measuresndash Donrsquot ignore that

20

0 05 1 15 2 25 3

Energy Sav

NEBs

NEB vs Energy Savings ValueIncluding all NEBs

Source (SkumatzSERA 2010 amp others)

Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

SERA

RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

MF Many others

Thousands of surveys results By measures By program types By sectors By stakeholders By geography

Variety of end uses

SERA

TOP NEBS FOR WEATHERIZATION PROGRAM

Weatherization program multipliergt1

Source SERA Research

SERA

RESULTS FOR ENERGY STARreg APPLIANCES

0002004006008

01012014016018

Maint

Appear

Perf LifeNoise Satis

Comfort

LiteSafe

ty SellMove

Water

Enviro

Refrig DW CW

Source SERA Research

SERA

INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

5 increments to 50

Source SERA research

Chart5

EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033

cx

cx

nz

Sheet3

Sheet3

Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)

SERA

ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

0

002

004

006

008

01

012

Maint

Perf LifeSatis

f

ComfortAesth

NoiseSafe

ty

LightingSelli

ng

Stay in home

Enviro

Sick Days

Calls

ES Homes HP wES Energy savings multiplier=1

Source SERA Research

SERA

CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

DG CHP HVAC Equipment rebate Other

Building codes incentives by cities

Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

Variety of end uses

SERA

AUDIT FINANCIAL PROGRAM

0

01

02

03

04

05

06

07

Variety of measureshellip disaggregatedResults by measure type businessType etchellip

Source SERA Research

SERA

CampI NEW CONSTRUCTION

-005

0

005

01

015

02

AampE Owner(Pct of Total Partic NEBs)

Source SERA Research

SERA

CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

0 2 4 6 8 10 12

m Product losses

l Ease of selling leasing building

n Labor requirements

h Aesthetics appearance

j Noise

b Equipment maintenance

f Tenant satisfaction

d Equipment lifetimes

e Productivity

k Building safety[2]

g Comfort

i Lighting Quality of Light

c Equipment performance[1]

a Operating costs

o Doing good for the Environment[3]

Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

vcv

Source SERA Research

Chart1

CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011

Sheet1

Sheet1

CampI Technical Assistance Program NEBs (Participant)

Sheet2

Sheet3

SERA

EXPRESSING NEBS VALUEndashCxe

Strong value from RetroCxSource SERA Research

SERA

SOCIETAL NEB EXAMPLES

31

SERA

JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

32

0

02

04

06

08

1

12

HPWxRetrof Appliance

CA

WI

Natl

(Source Skumatz SERAECEEE 2007 ACEEE 2006)

Jobs Economic

SERA

EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

33

SERA

UTILITY NEBSUtility NEBs for Template Program

Debt WriteOff (util)13

Rate Subsidy(util)61

HealthSafety(util)0

Colln Costs (util)0

Gas Calls (util)0

Calls to CSRs(util)2

TampD (util)16

Arrears (util)0

Reconnects (util)0

Notices (util)7

Shutoffs (util)1

Rate subsidy TampD

Payment-related

Source Skumatz Economic Research Associates research

Example Low Income Weatherization

SERA

ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

Value of 100 HALVES Payback doubles ROI

Source Skumatz Economics (SERA)

35

( for NEBs Multiplier of 125)

SERA

NEBS IN COST-EFFECTIVENESS APPLICATIONS

36

hellip2001 ON

SERA

37

NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

For true representation of B amp C elements of NEBs address missing factor bias

Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

BC applications were considered early on then pulled back awaiting more quantitative evidence

SERA

CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

38

Test Benefits Costs States Using Traditionally

Improved treat-ment with NEBs

Utility Cost (or Program Administrator Test) (UCT or PAC)

bull Avoided supply costs for transmission distribution and generation (TDampG)

bull Avoided gas and water supply costs

bull Program administration

bull Participant incentives

bull Increased supply cost

CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

Use cost only paid by the utility

Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

Same as above plus bull increased revenue

Same as above plus bull Decreased revenue

AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

Participant cost bull Utility bill reductions bull Participant incentives

bull Participant direct costs

AR CA FL HI IA IN MI MN NY VA

Participant NEBs

Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

SERA

APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

Expansion Concern with risk calculation accuracy

39

Source Skumatz SERA research

SERA

40

METHODS TO INCLUDE NEBsIN REGULATORY TESTS

Maximize DSM opportunities amp feedback Accuracy tailoring

Minimize Regulatory amp Implementer Risk

Minimize Evaluation Cost

Adder

Readily MeasurableHybrid

All NEBsSource SERA Research

SERA

41

STATE TREATMENT OF NEBS NEBsRegulatory

Screening Application Utilities regions

Program Marketing Fairly widespread use in utilities states across the country

MO

RE AGGRESSIVE ==gt

Test Pgm Screen - adder

IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

Test Pgm Screen - readily measured

MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

Test Hybrid (potential adder amp measured)

CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

Test Pgm screen - Broad

With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

SERA

42

Key

States with NEBs in CE tests for At least one type of program

copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

STATES WITH NEBS IN CE TESTS

NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

SERA

SELECTED STATE PROGRESS CA

2001 ndash LIPPT model NY

Mid-2000s measurement amp scenarios not included CO

Adders (10 electric 25 LI 5 gas) VT

15 adder helped by previous research DC

10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

43

Source Skumatz SERA research

SERA

SELECTED STATE PROGRESS Dominos ongoing

Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

44

Source Skumatz SERA research

SERA

APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

Expansion Concern with risk calculation accuracy

45

Source Skumatz SERA research

SERA

46

NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

SourceSkumatz SERA Independent research

NTG ndash accuracy measurement incomplete

Savings Impact repeatedly amp expensivelymeasured little variation $100K+

NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

Cost Complicated expensive local changes

RELATIVE Risk from NEBs

Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

NEBs as accurate as many inputs (all could improve)

SERA

DISCOUNT RATE MATTERS

47

Source Skumatz SERA Research

0

20

40

60

80

100

120

1 2 3 4 5 6 7 8 9 10

PV Ratios vs 1 Discount Rate

10 yr PV 20 yr PV

Discount rate - percent

WACCSocialLTT

Source Skumatz Electricity Journal IEPEC paper independent research

SERA

EUL VARIATIONS-ADOPTED VALUES

Source Skumatz Economic Research Associates SERA independent research

0

50

100

150

200

250

300

350

400

450

0 10 20 30 40 50 60 70

EUL Variation by Measure

67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

Poorly supportedOldTechnology change

SERA

KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

change the BC conclusion

On the cusphellip

If NO Yoursquore doneAnd bias addressedsufficiently

IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

Source SERA all rights reserved

SERArsquos lsquoNEB-ItrdquoModel ndash

SERA

IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

Utility Soc Part Conserv Recrsquom

Rationale

Base Percent X X X Program-invariant

Low Income X X X X Multiple sources

Weatherization X X X Substantial Participantimpacts

Measure Program-specific

X Varies by measure sector

Other Recomrsquos Local Research

50

Source Skumatz SERA

Developing values for multiple states amp utilities

SERA

TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

Other sources of bias to research (EUL NTG etc) bring high variation

5) States are incorporating NEBs dominos input in deliberations in multiple states

6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

THANK YOU

Questions

Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

SERA

POWER OF NEBS ndashBARRIERS DISCONNECTS

53

SERA

NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

Negative NEB values cost of barrier

Solar Water Heat NZ$ Euros

Solar Design NZ$ Euros

Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

-23 -12 (079)

-11 -6 (06)

Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

Source Skumatz Economic Research Associates research

SERA

CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

-5-4-3-2-1012345

Op Co

st

Init C

ost

Maint

Perfo

rm

Prod

uctiv

Tena

nt Sa

t

Comf

ort

Appe

ar

Light

Qual

SellL

ease

Devp Ownr AampE

Note AgtE (~4 vs ~2) for these NEBs

Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

Source Skumatz Economic Research Associates Inc 2002

Owners more positive ndashUnderinvestment train

  • Soup to nuts on non-energy benefits NEIs
  • TOPICS
  • BACKGROUND NEBS
  • 20+ YEARS OF NEBS PROGRESShellip
  • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
  • KEY APPLICATIONS OF NEBS
  • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
  • Neb measurement
  • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
  • MEASURING PARTICIPANTNEBs
  • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
  • PARTICIPANT MEASUREMENT METHODS COMPARISON
  • PROS AND CONS OF NEBS MEASUREMENT METHODS
  • PROGRESS TRANSFERABILITY OF NEBS
  • RANGES TYPICAL NEB VALUES FOR WxRETROFI
  • PROGRESS AND GAPS
  • PROGRESS AND GAPS
  • USING NEB NEI VALUES
  • Slide Number 19
  • ARE NEBS HIGH VALUE
  • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
  • TOP NEBS FOR WEATHERIZATION PROGRAM
  • RESULTS FOR ENERGY STARreg APPLIANCES
  • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
  • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
  • CampI PROGRAMS ldquoNEBBEDrdquo
  • AUDIT FINANCIAL PROGRAM
  • CampI NEW CONSTRUCTION
  • CampI TECHNICAL ASSISTANCE
  • EXPRESSING NEBS VALUEndashCxe
  • Slide Number 31
  • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
  • EMISSIONS (SOCIETAL NEBS)
  • UTILITY NEBS
  • ADJUSTED PAYBACKS
  • NEBS IN COST-EFFECTIVENESS APPLICATIONS
  • NEBS IN BC
  • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
  • APPLICATION TO BC
  • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
  • STATE TREATMENT OF NEBS NEBs
  • Slide Number 42
  • SELECTED STATE PROGRESS
  • SELECTED STATE PROGRESS
  • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
  • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
  • DISCOUNT RATE MATTERS
  • EUL VARIATIONS- ADOPTED VALUES
  • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
  • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
  • TAKEAWAYSON NEBSNEIS
  • THANK YOUQuestions
  • Slide Number 53
  • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
  • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
    Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
    m Product losses20
    l Ease of selling leasing building33
    n Labor requirements45
    h Aesthetics appearance59
    j Noise50
    b Equipment maintenance660
    f Tenant satisfaction60
    d Equipment lifetimes718
    e Productivity724
    k Building safety[2]73
    g Comfort933
    i Lighting Quality of Light916
    c Equipment performance[1]1019
    a Operating costs1165
    o Doing good for the Environment[3]1132
    Other03
    [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
    [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
    [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
    m Product losses
    l Ease of selling leasing building
    n Labor requirements
    h Aesthetics appearance
    j Noise
    b Equipment maintenance
    f Tenant satisfaction
    d Equipment lifetimes
    e Productivity
    k Building safety[2]
    g Comfort
    i Lighting Quality of Light
    c Equipment performance[1]
    a Operating costs
    o Doing good for the Environment[3]
    Share of ZALEH NEBS from various measures
    Each presented as multiple of the (overall) energy savings
    Super Insulation037
    Double Glazing026
    Solar Design025
    Solar Water Heating005
    EnergySmart (insul Retrofit)Dunedin (insul)
    Other124
    Noise24
    Budgetfinances25
    Environmental65
    Health78
    Reduced moves86
    Ease of sellingrenting130
    DampnessMold1316
    Comfort-related4933
    101101
    Operational NEBs44
    DesignConstruction NEBs31
    Occupant NEBs25
    1
    OtherOther
    NoiseNoise
    BudgetfinancesBudgetfinances
    EnvironmentalEnvironmental
    HealthHealth
    Reduced movesReduced moves
    Ease of sellingrentingEase of sellingrenting
    DampnessMoldDampnessMold
    Comfort-relatedComfort-related
    Boiler
    VendorNPPartic
    Perform0280202
    Footprint028013017
    ControlFeatures031022016
    Tenant compl006008014
    Noise005006014
    Maint004009007
    Other004003007
    Lifetime-006018006
    MF New construction energy star
    Operating cost014
    Environment012
    Maintenance01
    Lifetime01
    Performance009
    Occupant Satisf009
    Comfort008
    Noise006
    Sellingleasing006
    Aesthetics005
    Light Quality005
    Power QualReliab004
    Safety002
    RIGHTLIGHTS
    Environment023
    Quality016
    Quantity014
    PgmAvail01
    Flicker008
    Maintenance006
    Lifetime006
    SalesProductivity006
    Safety005
    Noise005
    Control003
    Illness0
    PerformPerformPerform
    FootprintFootprintFootprint
    ControlFeaturesControlFeaturesControlFeatures
    Tenant complTenant complTenant compl
    NoiseNoiseNoise
    MaintMaintMaint
    OtherOtherOther
    LifetimeLifetimeLifetime
    Environment
    Quality
    Quantity
    PgmAvail
    Flicker
    Maintenance
    Lifetime
    SalesProductivity
    Safety
    Noise
    Control
    Illness

    Variability

    Relevant NEB Categories

    Program measure invariant (suitable for ldquoadderrdquo)

    middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

    Program measure dependent

    middot Economic ndash societal (depends on measures and local manufacture installation)

    middot Health and safety health care illnesses ndash societal and participant (measure)

    middot Water wastewater infrastructure and water bill savings ndash societal and participant

    middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

    Climate dependent

    middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

    Residential Target dependent (low income or MF vs SF)

    middot Payment related ndash utility (arrearages etc stronger for low income targets)

    middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

    middot Participant benefits related to hardship and payments

    middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

    Used for

    Major Advantages

    Major Disadvantages

    Direct measurement (specific studies of changes on-site)

    Comrsquol labor productivity etc

    Direct precise attributable

    Small estimation sample sizes specialized cases poor transferability expensive

    Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

    Insurance water health others

    Long history easy secondary sources

    Credible to reviewers vetted inputs

    Not available for all NEBs

    Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

    Emissions economics

    Third party peer-vetted models available for economics jobs and emissions

    Not available for all NEBs

    Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

    Wide variety of Participant NEBs

    Large sample sizes amp statistical properties

    Affordable

    Multiple estimates leading to similar ranges

    Direct method of measuring some key NEBs

    Concerns about surveys as a source of quantitative values amp reliability

    Recall from survey respondents

    Proper attribution to programs measures

    MWxNEWxCA WxCA WxSF-NCMF-NCC- LiteC-RebC-Reb
    mult1220813075104075084
    LMS Verbal15610980612067115055091
    WTP42242732
    MWxMWxMWx
    NEWxNEWxNEWx
    CA WxCA WxCA Wx
    CA WxCA WxCA Wx
    SF-NCSF-NCSF-NC
    MF-NCMF-NCMF-NC
    C- LiteC- LiteC- Lite
    C-RebC-RebC-Reb
    C-RebC-RebC-Reb

    SERA

    TOPICS Background Measurement of NEBs Risk NEBs in State BC Risk Gaps and Conclusions

    SERA

    BACKGROUND NEBS Attention growing on NEBs after 20 years BC to assess investments (programs portfolios)

    BC=f[PV(attributable benefits stream)] [PV(attributable cost stream)]

    (With variations based on perspectivehellip) NEBs are net enhancement to ldquoBrdquo - incomplete

    Biased investments decisions because all costs not all benefits

    Omission - ldquo0rdquo is the wrong number High value from multiple quantitative studies

    Evaluationrsquos purpose ndash to inform decision-making

    SERA

    20+ YEARS OF NEBS PROGRESShellip

    But there still isnrsquot agreement on name - NEB OPI NNEB MB co-benefitshellipSource SERA all rights reserved

    1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016

    Perspectives +-Basic measurementLI Res amp Comrsquol

    Explore BC (LIPPT)Expanding RampC Ests Initial mktg applics

    Expanding estimates sectors studies methods Wider use in marketingInitial applications in planning

    Re-explore BCIntroduction in states growthExpanding literature

    Lists

    SERA

    5

    NEB PERSPECTIVES CATEGORIES BEST PRACTICES

    Utility Society Participant (ResampICI)bullCarrying cost on arrearages bullBad debt written offbullShutoffs ReconnectsbullNotices calls collection costsbullEmergency gas service calls (for gas flex connector and other programs)bullInsurance savingsbullTransmission and distribution savings (usually distribution)bullFewer substations etcbullPower quality reliabilitybullReduced subsidy payments (low income)bullOther

    bullEconomic development benefits ndash direct and indirect multipliersbullTax effectsbullEmissions environmental (trading values andor health hazard benefits)bullHealth and safety equipmentbullWater and waste water treatment or supply plantsbullFish wildlife mitigationbullNational securitybullHealth carebullOther

    bullWater wastewater bill savingsbullOperating costs (non-energy) bullEquipment maintenancebullEquipment performance (push air better etc)bullEquipment lifetimebullShutoffs ReconnectsbullProperty value benefits sellingbull(Bill-related) calls to utilitybullComfortbullAesthetics appearancebullFires insurance damage (gas)bullLighting quality of light bullNoisebullSafety

    bullControl over billbullUnderstanding knowledgebullldquoCarerdquo or ldquohardshiprdquo (low income)bullIndoor air qualitybullHealth lost days at work or schoolbullFewer movesbullDoing good for environmentbullSavings in other fuels or services (as relevant)bullGHG and environmental effectsbullNegatives

    Source (SkumatzSERA1996 on)

    Net ThreeNet Positive amp negativeNet beyond standard efficiencyNet to gross

    Non-OverlappingConsistent Units

    SERA

    KEY APPLICATIONS OF NEBS

    MARKETING amp ROI ndashSell whatrsquos valuable to customers link to

    peers

    PROGRAM REFINEMENT ndashPositive amp Negative NEBs for

    measures barriers incentives and targeting

    TRAIN THE CHAIN ndashAlign Educate Actors on NEB

    priorities

    POLICY GOALS Quantifies Non-energy goals (eg

    Low income jobs etc)

    BC TESTS ndashRefined CE for program amp portfolio reduce bias in

    investment

    6

    Source SERA all rights reserved

    SERA

    USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS

    NegativeNEBs

    Solar WH

    Appearance -$14 NZ

    Maintenance -$9 NZ

    Source Skumatz Economic Research Associates research ndash 1998-2005

    -005

    0

    005

    01

    015

    02

    Maint

    Productiv Perf Life

    Op CostTenant S

    atComfor

    t

    Lite Safe SellEnvir

    oOther

    AampE Owner

    Commercial Example

    Residential Example

    SERA

    NEB MEASUREMENT

    8

    SERA

    NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES

    Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip

    9

    Monetized NEBs

    Source Skumatz SERA research

    NEB

    s

    Direct

    Secondary

    Model

    Survey Story of a ferryhellip then it is academic

    Corp Records Utility data

    Change x valueFinancial Calcs

    Third party jobsAnd emissions

    Specialized academic Best for some NEBs

    SERA

    MEASURING PARTICIPANTNEBs Best practices

    WTP not fruitful 7+ better options

    Net Three Net positive amp negative Net beyond standard

    efficiency Net to Gross

    Non-overlapping Consistent units Recommend in ALL

    process (impact) surveys - barriers

    HTM=Hard to measure HTA=Hard to answer

    Direct calc Accurate Small ldquonrdquo

    WTPWTA Accepted Volatile HTA

    Relative Fast strong HTA

    LMSFast

    strong clear ETA

    Unfamiliar

    Logit Conjoint

    rankStrong Slow

    complex

    Regression Defensible Limited data cost

    Market value Strong Data

    Other Exploring Cost tradeoffs

    Source SERA Research

    SERA

    HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)

    11

    Monetized NEBs

    Direct Secondary Model Survey

    Attributable Change (study)

    Value or Financial

    Calc

    X

    Total Attrib Stated Relative

    Effect

    Savings (or translation)(ldquoNormrdquo)

    Individual NEB Shares

    X

    X

    SERA

    PARTICIPANT MEASUREMENT METHODS COMPARISON

    005

    115

    225

    335

    445

    MW

    x

    NE

    Wx

    CA

    Wx

    CA

    Wx

    SF-N

    C

    MF-

    NC

    C-

    Lite

    C-R

    eb

    C-R

    eb

    multLMS VerbalWTP

    Source SERA Research

    Chart1

    mult
    LMS Verbal
    WTP
    122
    156
    42
    08
    1
    2
    098
    4
    06
    13
    12
    075
    067
    104
    115
    27
    075
    055
    32
    084
    091

    Sheet1

    Sheet1

    mult
    LMS Verbal
    WTP

    Sheet2

    Sheet3

    SERA

    PROS AND CONS OF NEBS MEASUREMENT METHODS

    13

    Source Skumatz SERA research

    Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

    Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

    Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

    Long history easy secondary sources Credible to reviewers vetted inputs

    Not available for all NEBs

    Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

    Third party peer-vetted models available for economics jobs and emissions

    Not available for all NEBs

    Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

    Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

    Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

    SERA

    PROGRESS TRANSFERABILITY OF NEBS

    14

    Source Skumatz SERA research

    Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

    bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

    Program measure dependent

    bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

    education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

    Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

    Residential Target dependent (low income or MF vs SF)

    bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

    vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

    Cost savings consistency if transferable vs risk

    SERA

    RANGES TYPICAL NEB VALUES FOR WxRETROFI

    15

    SERA

    PROGRESS AND GAPS Consistency Transferability

    Some programs types others less studied Transferability research

    Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

    measures until then Across the board Savings share Regression

    SERA

    PROGRESS AND GAPS Gas vs Electric

    Some researchndashusually program-wide not broken down International research found participant NEBs have

    similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

    MF Less-commonly-studied complicated by poor response

    and complexity of sector (decision-maker some measures in home some central) separate from low income not common

    Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

    Commercial gaps in some measures

    SERA

    USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

    Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

    18

    0

    005

    01

    015

    02

    025

    Environment

    Quality

    Quantity

    PgmAvailFlicker

    Mainten

    ance

    Lifetime

    SalesP

    roductivity

    Safety

    Noise

    Control

    Illness

    Total NEBs Multiplier=X

    $

    Savings value=$X

    Chart3

    023
    016
    014
    01
    008
    006
    006
    006
    005
    005
    003
    0

    Chart1

    Vendor
    NP
    Partic
    028
    02
    02
    028
    013
    017
    031
    022
    016
    006
    008
    014
    005
    006
    014
    004
    009
    007
    004
    003
    007
    -006
    018
    006

    Sheet1

    Sheet1

    Sheet2

    Sheet3

    SERA

    NEB VALUES

    19

    SERA

    ARE NEBS HIGH VALUE Energy savings are often the minority of

    benefits from program measuresndash Donrsquot ignore that

    20

    0 05 1 15 2 25 3

    Energy Sav

    NEBs

    NEB vs Energy Savings ValueIncluding all NEBs

    Source (SkumatzSERA 2010 amp others)

    Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

    SERA

    RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

    MF Many others

    Thousands of surveys results By measures By program types By sectors By stakeholders By geography

    Variety of end uses

    SERA

    TOP NEBS FOR WEATHERIZATION PROGRAM

    Weatherization program multipliergt1

    Source SERA Research

    SERA

    RESULTS FOR ENERGY STARreg APPLIANCES

    0002004006008

    01012014016018

    Maint

    Appear

    Perf LifeNoise Satis

    Comfort

    LiteSafe

    ty SellMove

    Water

    Enviro

    Refrig DW CW

    Source SERA Research

    SERA

    INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

    5 increments to 50

    Source SERA research

    Chart5

    EnergySmart (insul Retrofit)
    Dunedin (insul)
    001
    024
    002
    004
    002
    005
    006
    005
    007
    008
    008
    006
    013
    0
    013
    016
    049
    033

    cx

    cx

    nz

    Sheet3

    Sheet3

    Super Insulation
    Double Glazing
    Solar Design
    Solar Water Heating
    Super Insulation
    Double Glazing
    Solar Design
    Solar Water Heating
    EnergySmart (insul Retrofit)
    Dunedin (insul)

    SERA

    ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

    0

    002

    004

    006

    008

    01

    012

    Maint

    Perf LifeSatis

    f

    ComfortAesth

    NoiseSafe

    ty

    LightingSelli

    ng

    Stay in home

    Enviro

    Sick Days

    Calls

    ES Homes HP wES Energy savings multiplier=1

    Source SERA Research

    SERA

    CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

    DG CHP HVAC Equipment rebate Other

    Building codes incentives by cities

    Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

    Variety of end uses

    SERA

    AUDIT FINANCIAL PROGRAM

    0

    01

    02

    03

    04

    05

    06

    07

    Variety of measureshellip disaggregatedResults by measure type businessType etchellip

    Source SERA Research

    SERA

    CampI NEW CONSTRUCTION

    -005

    0

    005

    01

    015

    02

    AampE Owner(Pct of Total Partic NEBs)

    Source SERA Research

    SERA

    CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

    0 2 4 6 8 10 12

    m Product losses

    l Ease of selling leasing building

    n Labor requirements

    h Aesthetics appearance

    j Noise

    b Equipment maintenance

    f Tenant satisfaction

    d Equipment lifetimes

    e Productivity

    k Building safety[2]

    g Comfort

    i Lighting Quality of Light

    c Equipment performance[1]

    a Operating costs

    o Doing good for the Environment[3]

    Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

    vcv

    Source SERA Research

    Chart1

    CampI Technical Assistance NEBs
    002
    003
    004
    005
    005
    006
    006
    007
    007
    007
    009
    009
    01
    011
    011

    Sheet1

    Sheet1

    CampI Technical Assistance Program NEBs (Participant)

    Sheet2

    Sheet3

    SERA

    EXPRESSING NEBS VALUEndashCxe

    Strong value from RetroCxSource SERA Research

    SERA

    SOCIETAL NEB EXAMPLES

    31

    SERA

    JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

    32

    0

    02

    04

    06

    08

    1

    12

    HPWxRetrof Appliance

    CA

    WI

    Natl

    (Source Skumatz SERAECEEE 2007 ACEEE 2006)

    Jobs Economic

    SERA

    EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

    33

    SERA

    UTILITY NEBSUtility NEBs for Template Program

    Debt WriteOff (util)13

    Rate Subsidy(util)61

    HealthSafety(util)0

    Colln Costs (util)0

    Gas Calls (util)0

    Calls to CSRs(util)2

    TampD (util)16

    Arrears (util)0

    Reconnects (util)0

    Notices (util)7

    Shutoffs (util)1

    Rate subsidy TampD

    Payment-related

    Source Skumatz Economic Research Associates research

    Example Low Income Weatherization

    SERA

    ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

    be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

    Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

    Value of 100 HALVES Payback doubles ROI

    Source Skumatz Economics (SERA)

    35

    ( for NEBs Multiplier of 125)

    SERA

    NEBS IN COST-EFFECTIVENESS APPLICATIONS

    36

    hellip2001 ON

    SERA

    37

    NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

    For true representation of B amp C elements of NEBs address missing factor bias

    Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

    BC applications were considered early on then pulled back awaiting more quantitative evidence

    SERA

    CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

    38

    Test Benefits Costs States Using Traditionally

    Improved treat-ment with NEBs

    Utility Cost (or Program Administrator Test) (UCT or PAC)

    bull Avoided supply costs for transmission distribution and generation (TDampG)

    bull Avoided gas and water supply costs

    bull Program administration

    bull Participant incentives

    bull Increased supply cost

    CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

    Use cost only paid by the utility

    Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

    Same as above plus bull increased revenue

    Same as above plus bull Decreased revenue

    AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

    Participant cost bull Utility bill reductions bull Participant incentives

    bull Participant direct costs

    AR CA FL HI IA IN MI MN NY VA

    Participant NEBs

    Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

    Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

    SERA

    APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

    Expansion Concern with risk calculation accuracy

    39

    Source Skumatz SERA research

    SERA

    40

    METHODS TO INCLUDE NEBsIN REGULATORY TESTS

    Maximize DSM opportunities amp feedback Accuracy tailoring

    Minimize Regulatory amp Implementer Risk

    Minimize Evaluation Cost

    Adder

    Readily MeasurableHybrid

    All NEBsSource SERA Research

    SERA

    41

    STATE TREATMENT OF NEBS NEBsRegulatory

    Screening Application Utilities regions

    Program Marketing Fairly widespread use in utilities states across the country

    MO

    RE AGGRESSIVE ==gt

    Test Pgm Screen - adder

    IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

    Test Pgm Screen - readily measured

    MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

    Test Hybrid (potential adder amp measured)

    CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

    Test Pgm screen - Broad

    With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

    Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

    SERA

    42

    Key

    States with NEBs in CE tests for At least one type of program

    copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

    STATES WITH NEBS IN CE TESTS

    NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

    groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

    SERA

    SELECTED STATE PROGRESS CA

    2001 ndash LIPPT model NY

    Mid-2000s measurement amp scenarios not included CO

    Adders (10 electric 25 LI 5 gas) VT

    15 adder helped by previous research DC

    10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

    43

    Source Skumatz SERA research

    SERA

    SELECTED STATE PROGRESS Dominos ongoing

    Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

    Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

    Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

    Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

    44

    Source Skumatz SERA research

    SERA

    APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

    Expansion Concern with risk calculation accuracy

    45

    Source Skumatz SERA research

    SERA

    46

    NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

    [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

    SourceSkumatz SERA Independent research

    NTG ndash accuracy measurement incomplete

    Savings Impact repeatedly amp expensivelymeasured little variation $100K+

    NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

    EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

    Cost Complicated expensive local changes

    RELATIVE Risk from NEBs

    Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

    Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

    NEBs as accurate as many inputs (all could improve)

    SERA

    DISCOUNT RATE MATTERS

    47

    Source Skumatz SERA Research

    0

    20

    40

    60

    80

    100

    120

    1 2 3 4 5 6 7 8 9 10

    PV Ratios vs 1 Discount Rate

    10 yr PV 20 yr PV

    Discount rate - percent

    WACCSocialLTT

    Source Skumatz Electricity Journal IEPEC paper independent research

    SERA

    EUL VARIATIONS-ADOPTED VALUES

    Source Skumatz Economic Research Associates SERA independent research

    0

    50

    100

    150

    200

    250

    300

    350

    400

    450

    0 10 20 30 40 50 60 70

    EUL Variation by Measure

    67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

    Poorly supportedOldTechnology change

    SERA

    KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

    Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

    measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

    change the BC conclusion

    On the cusphellip

    If NO Yoursquore doneAnd bias addressedsufficiently

    IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

    Source SERA all rights reserved

    SERArsquos lsquoNEB-ItrdquoModel ndash

    SERA

    IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

    Utility Soc Part Conserv Recrsquom

    Rationale

    Base Percent X X X Program-invariant

    Low Income X X X X Multiple sources

    Weatherization X X X Substantial Participantimpacts

    Measure Program-specific

    X Varies by measure sector

    Other Recomrsquos Local Research

    50

    Source Skumatz SERA

    Developing values for multiple states amp utilities

    SERA

    TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

    Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

    Other sources of bias to research (EUL NTG etc) bring high variation

    5) States are incorporating NEBs dominos input in deliberations in multiple states

    6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

    THANK YOU

    Questions

    Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

    SERA

    POWER OF NEBS ndashBARRIERS DISCONNECTS

    53

    SERA

    NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

    Negative NEB values cost of barrier

    Solar Water Heat NZ$ Euros

    Solar Design NZ$ Euros

    Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

    -23 -12 (079)

    -11 -6 (06)

    Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

    Source Skumatz Economic Research Associates research

    SERA

    CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

    NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

    -5-4-3-2-1012345

    Op Co

    st

    Init C

    ost

    Maint

    Perfo

    rm

    Prod

    uctiv

    Tena

    nt Sa

    t

    Comf

    ort

    Appe

    ar

    Light

    Qual

    SellL

    ease

    Devp Ownr AampE

    Note AgtE (~4 vs ~2) for these NEBs

    Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

    Source Skumatz Economic Research Associates Inc 2002

    Owners more positive ndashUnderinvestment train

    • Soup to nuts on non-energy benefits NEIs
    • TOPICS
    • BACKGROUND NEBS
    • 20+ YEARS OF NEBS PROGRESShellip
    • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
    • KEY APPLICATIONS OF NEBS
    • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
    • Neb measurement
    • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
    • MEASURING PARTICIPANTNEBs
    • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
    • PARTICIPANT MEASUREMENT METHODS COMPARISON
    • PROS AND CONS OF NEBS MEASUREMENT METHODS
    • PROGRESS TRANSFERABILITY OF NEBS
    • RANGES TYPICAL NEB VALUES FOR WxRETROFI
    • PROGRESS AND GAPS
    • PROGRESS AND GAPS
    • USING NEB NEI VALUES
    • Slide Number 19
    • ARE NEBS HIGH VALUE
    • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
    • TOP NEBS FOR WEATHERIZATION PROGRAM
    • RESULTS FOR ENERGY STARreg APPLIANCES
    • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
    • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
    • CampI PROGRAMS ldquoNEBBEDrdquo
    • AUDIT FINANCIAL PROGRAM
    • CampI NEW CONSTRUCTION
    • CampI TECHNICAL ASSISTANCE
    • EXPRESSING NEBS VALUEndashCxe
    • Slide Number 31
    • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
    • EMISSIONS (SOCIETAL NEBS)
    • UTILITY NEBS
    • ADJUSTED PAYBACKS
    • NEBS IN COST-EFFECTIVENESS APPLICATIONS
    • NEBS IN BC
    • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
    • APPLICATION TO BC
    • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
    • STATE TREATMENT OF NEBS NEBs
    • Slide Number 42
    • SELECTED STATE PROGRESS
    • SELECTED STATE PROGRESS
    • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
    • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
    • DISCOUNT RATE MATTERS
    • EUL VARIATIONS- ADOPTED VALUES
    • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
    • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
    • TAKEAWAYSON NEBSNEIS
    • THANK YOUQuestions
    • Slide Number 53
    • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
    • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
      Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
      m Product losses20
      l Ease of selling leasing building33
      n Labor requirements45
      h Aesthetics appearance59
      j Noise50
      b Equipment maintenance660
      f Tenant satisfaction60
      d Equipment lifetimes718
      e Productivity724
      k Building safety[2]73
      g Comfort933
      i Lighting Quality of Light916
      c Equipment performance[1]1019
      a Operating costs1165
      o Doing good for the Environment[3]1132
      Other03
      [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
      [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
      [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
      m Product losses
      l Ease of selling leasing building
      n Labor requirements
      h Aesthetics appearance
      j Noise
      b Equipment maintenance
      f Tenant satisfaction
      d Equipment lifetimes
      e Productivity
      k Building safety[2]
      g Comfort
      i Lighting Quality of Light
      c Equipment performance[1]
      a Operating costs
      o Doing good for the Environment[3]
      Share of ZALEH NEBS from various measures
      Each presented as multiple of the (overall) energy savings
      Super Insulation037
      Double Glazing026
      Solar Design025
      Solar Water Heating005
      EnergySmart (insul Retrofit)Dunedin (insul)
      Other124
      Noise24
      Budgetfinances25
      Environmental65
      Health78
      Reduced moves86
      Ease of sellingrenting130
      DampnessMold1316
      Comfort-related4933
      101101
      Operational NEBs44
      DesignConstruction NEBs31
      Occupant NEBs25
      1
      OtherOther
      NoiseNoise
      BudgetfinancesBudgetfinances
      EnvironmentalEnvironmental
      HealthHealth
      Reduced movesReduced moves
      Ease of sellingrentingEase of sellingrenting
      DampnessMoldDampnessMold
      Comfort-relatedComfort-related
      Boiler
      VendorNPPartic
      Perform0280202
      Footprint028013017
      ControlFeatures031022016
      Tenant compl006008014
      Noise005006014
      Maint004009007
      Other004003007
      Lifetime-006018006
      MF New construction energy star
      Operating cost014
      Environment012
      Maintenance01
      Lifetime01
      Performance009
      Occupant Satisf009
      Comfort008
      Noise006
      Sellingleasing006
      Aesthetics005
      Light Quality005
      Power QualReliab004
      Safety002
      RIGHTLIGHTS
      Environment023
      Quality016
      Quantity014
      PgmAvail01
      Flicker008
      Maintenance006
      Lifetime006
      SalesProductivity006
      Safety005
      Noise005
      Control003
      Illness0
      PerformPerformPerform
      FootprintFootprintFootprint
      ControlFeaturesControlFeaturesControlFeatures
      Tenant complTenant complTenant compl
      NoiseNoiseNoise
      MaintMaintMaint
      OtherOtherOther
      LifetimeLifetimeLifetime
      Environment
      Quality
      Quantity
      PgmAvail
      Flicker
      Maintenance
      Lifetime
      SalesProductivity
      Safety
      Noise
      Control
      Illness

      Variability

      Relevant NEB Categories

      Program measure invariant (suitable for ldquoadderrdquo)

      middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

      Program measure dependent

      middot Economic ndash societal (depends on measures and local manufacture installation)

      middot Health and safety health care illnesses ndash societal and participant (measure)

      middot Water wastewater infrastructure and water bill savings ndash societal and participant

      middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

      Climate dependent

      middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

      Residential Target dependent (low income or MF vs SF)

      middot Payment related ndash utility (arrearages etc stronger for low income targets)

      middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

      middot Participant benefits related to hardship and payments

      middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

      Used for

      Major Advantages

      Major Disadvantages

      Direct measurement (specific studies of changes on-site)

      Comrsquol labor productivity etc

      Direct precise attributable

      Small estimation sample sizes specialized cases poor transferability expensive

      Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

      Insurance water health others

      Long history easy secondary sources

      Credible to reviewers vetted inputs

      Not available for all NEBs

      Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

      Emissions economics

      Third party peer-vetted models available for economics jobs and emissions

      Not available for all NEBs

      Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

      Wide variety of Participant NEBs

      Large sample sizes amp statistical properties

      Affordable

      Multiple estimates leading to similar ranges

      Direct method of measuring some key NEBs

      Concerns about surveys as a source of quantitative values amp reliability

      Recall from survey respondents

      Proper attribution to programs measures

      MWxNEWxCA WxCA WxSF-NCMF-NCC- LiteC-RebC-Reb
      mult1220813075104075084
      LMS Verbal15610980612067115055091
      WTP42242732
      MWxMWxMWx
      NEWxNEWxNEWx
      CA WxCA WxCA Wx
      CA WxCA WxCA Wx
      SF-NCSF-NCSF-NC
      MF-NCMF-NCMF-NC
      C- LiteC- LiteC- Lite
      C-RebC-RebC-Reb
      C-RebC-RebC-Reb

      SERA

      BACKGROUND NEBS Attention growing on NEBs after 20 years BC to assess investments (programs portfolios)

      BC=f[PV(attributable benefits stream)] [PV(attributable cost stream)]

      (With variations based on perspectivehellip) NEBs are net enhancement to ldquoBrdquo - incomplete

      Biased investments decisions because all costs not all benefits

      Omission - ldquo0rdquo is the wrong number High value from multiple quantitative studies

      Evaluationrsquos purpose ndash to inform decision-making

      SERA

      20+ YEARS OF NEBS PROGRESShellip

      But there still isnrsquot agreement on name - NEB OPI NNEB MB co-benefitshellipSource SERA all rights reserved

      1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016

      Perspectives +-Basic measurementLI Res amp Comrsquol

      Explore BC (LIPPT)Expanding RampC Ests Initial mktg applics

      Expanding estimates sectors studies methods Wider use in marketingInitial applications in planning

      Re-explore BCIntroduction in states growthExpanding literature

      Lists

      SERA

      5

      NEB PERSPECTIVES CATEGORIES BEST PRACTICES

      Utility Society Participant (ResampICI)bullCarrying cost on arrearages bullBad debt written offbullShutoffs ReconnectsbullNotices calls collection costsbullEmergency gas service calls (for gas flex connector and other programs)bullInsurance savingsbullTransmission and distribution savings (usually distribution)bullFewer substations etcbullPower quality reliabilitybullReduced subsidy payments (low income)bullOther

      bullEconomic development benefits ndash direct and indirect multipliersbullTax effectsbullEmissions environmental (trading values andor health hazard benefits)bullHealth and safety equipmentbullWater and waste water treatment or supply plantsbullFish wildlife mitigationbullNational securitybullHealth carebullOther

      bullWater wastewater bill savingsbullOperating costs (non-energy) bullEquipment maintenancebullEquipment performance (push air better etc)bullEquipment lifetimebullShutoffs ReconnectsbullProperty value benefits sellingbull(Bill-related) calls to utilitybullComfortbullAesthetics appearancebullFires insurance damage (gas)bullLighting quality of light bullNoisebullSafety

      bullControl over billbullUnderstanding knowledgebullldquoCarerdquo or ldquohardshiprdquo (low income)bullIndoor air qualitybullHealth lost days at work or schoolbullFewer movesbullDoing good for environmentbullSavings in other fuels or services (as relevant)bullGHG and environmental effectsbullNegatives

      Source (SkumatzSERA1996 on)

      Net ThreeNet Positive amp negativeNet beyond standard efficiencyNet to gross

      Non-OverlappingConsistent Units

      SERA

      KEY APPLICATIONS OF NEBS

      MARKETING amp ROI ndashSell whatrsquos valuable to customers link to

      peers

      PROGRAM REFINEMENT ndashPositive amp Negative NEBs for

      measures barriers incentives and targeting

      TRAIN THE CHAIN ndashAlign Educate Actors on NEB

      priorities

      POLICY GOALS Quantifies Non-energy goals (eg

      Low income jobs etc)

      BC TESTS ndashRefined CE for program amp portfolio reduce bias in

      investment

      6

      Source SERA all rights reserved

      SERA

      USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS

      NegativeNEBs

      Solar WH

      Appearance -$14 NZ

      Maintenance -$9 NZ

      Source Skumatz Economic Research Associates research ndash 1998-2005

      -005

      0

      005

      01

      015

      02

      Maint

      Productiv Perf Life

      Op CostTenant S

      atComfor

      t

      Lite Safe SellEnvir

      oOther

      AampE Owner

      Commercial Example

      Residential Example

      SERA

      NEB MEASUREMENT

      8

      SERA

      NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES

      Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip

      9

      Monetized NEBs

      Source Skumatz SERA research

      NEB

      s

      Direct

      Secondary

      Model

      Survey Story of a ferryhellip then it is academic

      Corp Records Utility data

      Change x valueFinancial Calcs

      Third party jobsAnd emissions

      Specialized academic Best for some NEBs

      SERA

      MEASURING PARTICIPANTNEBs Best practices

      WTP not fruitful 7+ better options

      Net Three Net positive amp negative Net beyond standard

      efficiency Net to Gross

      Non-overlapping Consistent units Recommend in ALL

      process (impact) surveys - barriers

      HTM=Hard to measure HTA=Hard to answer

      Direct calc Accurate Small ldquonrdquo

      WTPWTA Accepted Volatile HTA

      Relative Fast strong HTA

      LMSFast

      strong clear ETA

      Unfamiliar

      Logit Conjoint

      rankStrong Slow

      complex

      Regression Defensible Limited data cost

      Market value Strong Data

      Other Exploring Cost tradeoffs

      Source SERA Research

      SERA

      HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)

      11

      Monetized NEBs

      Direct Secondary Model Survey

      Attributable Change (study)

      Value or Financial

      Calc

      X

      Total Attrib Stated Relative

      Effect

      Savings (or translation)(ldquoNormrdquo)

      Individual NEB Shares

      X

      X

      SERA

      PARTICIPANT MEASUREMENT METHODS COMPARISON

      005

      115

      225

      335

      445

      MW

      x

      NE

      Wx

      CA

      Wx

      CA

      Wx

      SF-N

      C

      MF-

      NC

      C-

      Lite

      C-R

      eb

      C-R

      eb

      multLMS VerbalWTP

      Source SERA Research

      Chart1

      mult
      LMS Verbal
      WTP
      122
      156
      42
      08
      1
      2
      098
      4
      06
      13
      12
      075
      067
      104
      115
      27
      075
      055
      32
      084
      091

      Sheet1

      Sheet1

      mult
      LMS Verbal
      WTP

      Sheet2

      Sheet3

      SERA

      PROS AND CONS OF NEBS MEASUREMENT METHODS

      13

      Source Skumatz SERA research

      Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

      Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

      Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

      Long history easy secondary sources Credible to reviewers vetted inputs

      Not available for all NEBs

      Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

      Third party peer-vetted models available for economics jobs and emissions

      Not available for all NEBs

      Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

      Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

      Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

      SERA

      PROGRESS TRANSFERABILITY OF NEBS

      14

      Source Skumatz SERA research

      Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

      bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

      Program measure dependent

      bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

      education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

      Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

      Residential Target dependent (low income or MF vs SF)

      bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

      vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

      Cost savings consistency if transferable vs risk

      SERA

      RANGES TYPICAL NEB VALUES FOR WxRETROFI

      15

      SERA

      PROGRESS AND GAPS Consistency Transferability

      Some programs types others less studied Transferability research

      Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

      measures until then Across the board Savings share Regression

      SERA

      PROGRESS AND GAPS Gas vs Electric

      Some researchndashusually program-wide not broken down International research found participant NEBs have

      similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

      MF Less-commonly-studied complicated by poor response

      and complexity of sector (decision-maker some measures in home some central) separate from low income not common

      Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

      Commercial gaps in some measures

      SERA

      USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

      Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

      18

      0

      005

      01

      015

      02

      025

      Environment

      Quality

      Quantity

      PgmAvailFlicker

      Mainten

      ance

      Lifetime

      SalesP

      roductivity

      Safety

      Noise

      Control

      Illness

      Total NEBs Multiplier=X

      $

      Savings value=$X

      Chart3

      023
      016
      014
      01
      008
      006
      006
      006
      005
      005
      003
      0

      Chart1

      Vendor
      NP
      Partic
      028
      02
      02
      028
      013
      017
      031
      022
      016
      006
      008
      014
      005
      006
      014
      004
      009
      007
      004
      003
      007
      -006
      018
      006

      Sheet1

      Sheet1

      Sheet2

      Sheet3

      SERA

      NEB VALUES

      19

      SERA

      ARE NEBS HIGH VALUE Energy savings are often the minority of

      benefits from program measuresndash Donrsquot ignore that

      20

      0 05 1 15 2 25 3

      Energy Sav

      NEBs

      NEB vs Energy Savings ValueIncluding all NEBs

      Source (SkumatzSERA 2010 amp others)

      Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

      SERA

      RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

      MF Many others

      Thousands of surveys results By measures By program types By sectors By stakeholders By geography

      Variety of end uses

      SERA

      TOP NEBS FOR WEATHERIZATION PROGRAM

      Weatherization program multipliergt1

      Source SERA Research

      SERA

      RESULTS FOR ENERGY STARreg APPLIANCES

      0002004006008

      01012014016018

      Maint

      Appear

      Perf LifeNoise Satis

      Comfort

      LiteSafe

      ty SellMove

      Water

      Enviro

      Refrig DW CW

      Source SERA Research

      SERA

      INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

      5 increments to 50

      Source SERA research

      Chart5

      EnergySmart (insul Retrofit)
      Dunedin (insul)
      001
      024
      002
      004
      002
      005
      006
      005
      007
      008
      008
      006
      013
      0
      013
      016
      049
      033

      cx

      cx

      nz

      Sheet3

      Sheet3

      Super Insulation
      Double Glazing
      Solar Design
      Solar Water Heating
      Super Insulation
      Double Glazing
      Solar Design
      Solar Water Heating
      EnergySmart (insul Retrofit)
      Dunedin (insul)

      SERA

      ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

      0

      002

      004

      006

      008

      01

      012

      Maint

      Perf LifeSatis

      f

      ComfortAesth

      NoiseSafe

      ty

      LightingSelli

      ng

      Stay in home

      Enviro

      Sick Days

      Calls

      ES Homes HP wES Energy savings multiplier=1

      Source SERA Research

      SERA

      CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

      DG CHP HVAC Equipment rebate Other

      Building codes incentives by cities

      Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

      Variety of end uses

      SERA

      AUDIT FINANCIAL PROGRAM

      0

      01

      02

      03

      04

      05

      06

      07

      Variety of measureshellip disaggregatedResults by measure type businessType etchellip

      Source SERA Research

      SERA

      CampI NEW CONSTRUCTION

      -005

      0

      005

      01

      015

      02

      AampE Owner(Pct of Total Partic NEBs)

      Source SERA Research

      SERA

      CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

      0 2 4 6 8 10 12

      m Product losses

      l Ease of selling leasing building

      n Labor requirements

      h Aesthetics appearance

      j Noise

      b Equipment maintenance

      f Tenant satisfaction

      d Equipment lifetimes

      e Productivity

      k Building safety[2]

      g Comfort

      i Lighting Quality of Light

      c Equipment performance[1]

      a Operating costs

      o Doing good for the Environment[3]

      Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

      vcv

      Source SERA Research

      Chart1

      CampI Technical Assistance NEBs
      002
      003
      004
      005
      005
      006
      006
      007
      007
      007
      009
      009
      01
      011
      011

      Sheet1

      Sheet1

      CampI Technical Assistance Program NEBs (Participant)

      Sheet2

      Sheet3

      SERA

      EXPRESSING NEBS VALUEndashCxe

      Strong value from RetroCxSource SERA Research

      SERA

      SOCIETAL NEB EXAMPLES

      31

      SERA

      JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

      32

      0

      02

      04

      06

      08

      1

      12

      HPWxRetrof Appliance

      CA

      WI

      Natl

      (Source Skumatz SERAECEEE 2007 ACEEE 2006)

      Jobs Economic

      SERA

      EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

      33

      SERA

      UTILITY NEBSUtility NEBs for Template Program

      Debt WriteOff (util)13

      Rate Subsidy(util)61

      HealthSafety(util)0

      Colln Costs (util)0

      Gas Calls (util)0

      Calls to CSRs(util)2

      TampD (util)16

      Arrears (util)0

      Reconnects (util)0

      Notices (util)7

      Shutoffs (util)1

      Rate subsidy TampD

      Payment-related

      Source Skumatz Economic Research Associates research

      Example Low Income Weatherization

      SERA

      ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

      be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

      Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

      Value of 100 HALVES Payback doubles ROI

      Source Skumatz Economics (SERA)

      35

      ( for NEBs Multiplier of 125)

      SERA

      NEBS IN COST-EFFECTIVENESS APPLICATIONS

      36

      hellip2001 ON

      SERA

      37

      NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

      For true representation of B amp C elements of NEBs address missing factor bias

      Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

      BC applications were considered early on then pulled back awaiting more quantitative evidence

      SERA

      CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

      38

      Test Benefits Costs States Using Traditionally

      Improved treat-ment with NEBs

      Utility Cost (or Program Administrator Test) (UCT or PAC)

      bull Avoided supply costs for transmission distribution and generation (TDampG)

      bull Avoided gas and water supply costs

      bull Program administration

      bull Participant incentives

      bull Increased supply cost

      CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

      Use cost only paid by the utility

      Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

      Same as above plus bull increased revenue

      Same as above plus bull Decreased revenue

      AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

      Participant cost bull Utility bill reductions bull Participant incentives

      bull Participant direct costs

      AR CA FL HI IA IN MI MN NY VA

      Participant NEBs

      Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

      Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

      SERA

      APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

      Expansion Concern with risk calculation accuracy

      39

      Source Skumatz SERA research

      SERA

      40

      METHODS TO INCLUDE NEBsIN REGULATORY TESTS

      Maximize DSM opportunities amp feedback Accuracy tailoring

      Minimize Regulatory amp Implementer Risk

      Minimize Evaluation Cost

      Adder

      Readily MeasurableHybrid

      All NEBsSource SERA Research

      SERA

      41

      STATE TREATMENT OF NEBS NEBsRegulatory

      Screening Application Utilities regions

      Program Marketing Fairly widespread use in utilities states across the country

      MO

      RE AGGRESSIVE ==gt

      Test Pgm Screen - adder

      IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

      Test Pgm Screen - readily measured

      MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

      Test Hybrid (potential adder amp measured)

      CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

      Test Pgm screen - Broad

      With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

      Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

      SERA

      42

      Key

      States with NEBs in CE tests for At least one type of program

      copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

      STATES WITH NEBS IN CE TESTS

      NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

      groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

      SERA

      SELECTED STATE PROGRESS CA

      2001 ndash LIPPT model NY

      Mid-2000s measurement amp scenarios not included CO

      Adders (10 electric 25 LI 5 gas) VT

      15 adder helped by previous research DC

      10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

      43

      Source Skumatz SERA research

      SERA

      SELECTED STATE PROGRESS Dominos ongoing

      Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

      Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

      Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

      Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

      44

      Source Skumatz SERA research

      SERA

      APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

      Expansion Concern with risk calculation accuracy

      45

      Source Skumatz SERA research

      SERA

      46

      NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

      [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

      SourceSkumatz SERA Independent research

      NTG ndash accuracy measurement incomplete

      Savings Impact repeatedly amp expensivelymeasured little variation $100K+

      NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

      EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

      Cost Complicated expensive local changes

      RELATIVE Risk from NEBs

      Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

      Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

      NEBs as accurate as many inputs (all could improve)

      SERA

      DISCOUNT RATE MATTERS

      47

      Source Skumatz SERA Research

      0

      20

      40

      60

      80

      100

      120

      1 2 3 4 5 6 7 8 9 10

      PV Ratios vs 1 Discount Rate

      10 yr PV 20 yr PV

      Discount rate - percent

      WACCSocialLTT

      Source Skumatz Electricity Journal IEPEC paper independent research

      SERA

      EUL VARIATIONS-ADOPTED VALUES

      Source Skumatz Economic Research Associates SERA independent research

      0

      50

      100

      150

      200

      250

      300

      350

      400

      450

      0 10 20 30 40 50 60 70

      EUL Variation by Measure

      67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

      Poorly supportedOldTechnology change

      SERA

      KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

      Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

      measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

      change the BC conclusion

      On the cusphellip

      If NO Yoursquore doneAnd bias addressedsufficiently

      IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

      Source SERA all rights reserved

      SERArsquos lsquoNEB-ItrdquoModel ndash

      SERA

      IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

      Utility Soc Part Conserv Recrsquom

      Rationale

      Base Percent X X X Program-invariant

      Low Income X X X X Multiple sources

      Weatherization X X X Substantial Participantimpacts

      Measure Program-specific

      X Varies by measure sector

      Other Recomrsquos Local Research

      50

      Source Skumatz SERA

      Developing values for multiple states amp utilities

      SERA

      TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

      Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

      Other sources of bias to research (EUL NTG etc) bring high variation

      5) States are incorporating NEBs dominos input in deliberations in multiple states

      6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

      THANK YOU

      Questions

      Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

      SERA

      POWER OF NEBS ndashBARRIERS DISCONNECTS

      53

      SERA

      NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

      Negative NEB values cost of barrier

      Solar Water Heat NZ$ Euros

      Solar Design NZ$ Euros

      Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

      -23 -12 (079)

      -11 -6 (06)

      Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

      Source Skumatz Economic Research Associates research

      SERA

      CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

      NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

      -5-4-3-2-1012345

      Op Co

      st

      Init C

      ost

      Maint

      Perfo

      rm

      Prod

      uctiv

      Tena

      nt Sa

      t

      Comf

      ort

      Appe

      ar

      Light

      Qual

      SellL

      ease

      Devp Ownr AampE

      Note AgtE (~4 vs ~2) for these NEBs

      Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

      Source Skumatz Economic Research Associates Inc 2002

      Owners more positive ndashUnderinvestment train

      • Soup to nuts on non-energy benefits NEIs
      • TOPICS
      • BACKGROUND NEBS
      • 20+ YEARS OF NEBS PROGRESShellip
      • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
      • KEY APPLICATIONS OF NEBS
      • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
      • Neb measurement
      • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
      • MEASURING PARTICIPANTNEBs
      • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
      • PARTICIPANT MEASUREMENT METHODS COMPARISON
      • PROS AND CONS OF NEBS MEASUREMENT METHODS
      • PROGRESS TRANSFERABILITY OF NEBS
      • RANGES TYPICAL NEB VALUES FOR WxRETROFI
      • PROGRESS AND GAPS
      • PROGRESS AND GAPS
      • USING NEB NEI VALUES
      • Slide Number 19
      • ARE NEBS HIGH VALUE
      • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
      • TOP NEBS FOR WEATHERIZATION PROGRAM
      • RESULTS FOR ENERGY STARreg APPLIANCES
      • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
      • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
      • CampI PROGRAMS ldquoNEBBEDrdquo
      • AUDIT FINANCIAL PROGRAM
      • CampI NEW CONSTRUCTION
      • CampI TECHNICAL ASSISTANCE
      • EXPRESSING NEBS VALUEndashCxe
      • Slide Number 31
      • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
      • EMISSIONS (SOCIETAL NEBS)
      • UTILITY NEBS
      • ADJUSTED PAYBACKS
      • NEBS IN COST-EFFECTIVENESS APPLICATIONS
      • NEBS IN BC
      • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
      • APPLICATION TO BC
      • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
      • STATE TREATMENT OF NEBS NEBs
      • Slide Number 42
      • SELECTED STATE PROGRESS
      • SELECTED STATE PROGRESS
      • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
      • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
      • DISCOUNT RATE MATTERS
      • EUL VARIATIONS- ADOPTED VALUES
      • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
      • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
      • TAKEAWAYSON NEBSNEIS
      • THANK YOUQuestions
      • Slide Number 53
      • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
      • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
        Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
        m Product losses20
        l Ease of selling leasing building33
        n Labor requirements45
        h Aesthetics appearance59
        j Noise50
        b Equipment maintenance660
        f Tenant satisfaction60
        d Equipment lifetimes718
        e Productivity724
        k Building safety[2]73
        g Comfort933
        i Lighting Quality of Light916
        c Equipment performance[1]1019
        a Operating costs1165
        o Doing good for the Environment[3]1132
        Other03
        [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
        [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
        [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
        m Product losses
        l Ease of selling leasing building
        n Labor requirements
        h Aesthetics appearance
        j Noise
        b Equipment maintenance
        f Tenant satisfaction
        d Equipment lifetimes
        e Productivity
        k Building safety[2]
        g Comfort
        i Lighting Quality of Light
        c Equipment performance[1]
        a Operating costs
        o Doing good for the Environment[3]
        Share of ZALEH NEBS from various measures
        Each presented as multiple of the (overall) energy savings
        Super Insulation037
        Double Glazing026
        Solar Design025
        Solar Water Heating005
        EnergySmart (insul Retrofit)Dunedin (insul)
        Other124
        Noise24
        Budgetfinances25
        Environmental65
        Health78
        Reduced moves86
        Ease of sellingrenting130
        DampnessMold1316
        Comfort-related4933
        101101
        Operational NEBs44
        DesignConstruction NEBs31
        Occupant NEBs25
        1
        OtherOther
        NoiseNoise
        BudgetfinancesBudgetfinances
        EnvironmentalEnvironmental
        HealthHealth
        Reduced movesReduced moves
        Ease of sellingrentingEase of sellingrenting
        DampnessMoldDampnessMold
        Comfort-relatedComfort-related
        Boiler
        VendorNPPartic
        Perform0280202
        Footprint028013017
        ControlFeatures031022016
        Tenant compl006008014
        Noise005006014
        Maint004009007
        Other004003007
        Lifetime-006018006
        MF New construction energy star
        Operating cost014
        Environment012
        Maintenance01
        Lifetime01
        Performance009
        Occupant Satisf009
        Comfort008
        Noise006
        Sellingleasing006
        Aesthetics005
        Light Quality005
        Power QualReliab004
        Safety002
        RIGHTLIGHTS
        Environment023
        Quality016
        Quantity014
        PgmAvail01
        Flicker008
        Maintenance006
        Lifetime006
        SalesProductivity006
        Safety005
        Noise005
        Control003
        Illness0
        PerformPerformPerform
        FootprintFootprintFootprint
        ControlFeaturesControlFeaturesControlFeatures
        Tenant complTenant complTenant compl
        NoiseNoiseNoise
        MaintMaintMaint
        OtherOtherOther
        LifetimeLifetimeLifetime
        Environment
        Quality
        Quantity
        PgmAvail
        Flicker
        Maintenance
        Lifetime
        SalesProductivity
        Safety
        Noise
        Control
        Illness

        Variability

        Relevant NEB Categories

        Program measure invariant (suitable for ldquoadderrdquo)

        middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

        Program measure dependent

        middot Economic ndash societal (depends on measures and local manufacture installation)

        middot Health and safety health care illnesses ndash societal and participant (measure)

        middot Water wastewater infrastructure and water bill savings ndash societal and participant

        middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

        Climate dependent

        middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

        Residential Target dependent (low income or MF vs SF)

        middot Payment related ndash utility (arrearages etc stronger for low income targets)

        middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

        middot Participant benefits related to hardship and payments

        middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

        Used for

        Major Advantages

        Major Disadvantages

        Direct measurement (specific studies of changes on-site)

        Comrsquol labor productivity etc

        Direct precise attributable

        Small estimation sample sizes specialized cases poor transferability expensive

        Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

        Insurance water health others

        Long history easy secondary sources

        Credible to reviewers vetted inputs

        Not available for all NEBs

        Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

        Emissions economics

        Third party peer-vetted models available for economics jobs and emissions

        Not available for all NEBs

        Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

        Wide variety of Participant NEBs

        Large sample sizes amp statistical properties

        Affordable

        Multiple estimates leading to similar ranges

        Direct method of measuring some key NEBs

        Concerns about surveys as a source of quantitative values amp reliability

        Recall from survey respondents

        Proper attribution to programs measures

        MWxNEWxCA WxCA WxSF-NCMF-NCC- LiteC-RebC-Reb
        mult1220813075104075084
        LMS Verbal15610980612067115055091
        WTP42242732
        MWxMWxMWx
        NEWxNEWxNEWx
        CA WxCA WxCA Wx
        CA WxCA WxCA Wx
        SF-NCSF-NCSF-NC
        MF-NCMF-NCMF-NC
        C- LiteC- LiteC- Lite
        C-RebC-RebC-Reb
        C-RebC-RebC-Reb

        SERA

        20+ YEARS OF NEBS PROGRESShellip

        But there still isnrsquot agreement on name - NEB OPI NNEB MB co-benefitshellipSource SERA all rights reserved

        1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016

        Perspectives +-Basic measurementLI Res amp Comrsquol

        Explore BC (LIPPT)Expanding RampC Ests Initial mktg applics

        Expanding estimates sectors studies methods Wider use in marketingInitial applications in planning

        Re-explore BCIntroduction in states growthExpanding literature

        Lists

        SERA

        5

        NEB PERSPECTIVES CATEGORIES BEST PRACTICES

        Utility Society Participant (ResampICI)bullCarrying cost on arrearages bullBad debt written offbullShutoffs ReconnectsbullNotices calls collection costsbullEmergency gas service calls (for gas flex connector and other programs)bullInsurance savingsbullTransmission and distribution savings (usually distribution)bullFewer substations etcbullPower quality reliabilitybullReduced subsidy payments (low income)bullOther

        bullEconomic development benefits ndash direct and indirect multipliersbullTax effectsbullEmissions environmental (trading values andor health hazard benefits)bullHealth and safety equipmentbullWater and waste water treatment or supply plantsbullFish wildlife mitigationbullNational securitybullHealth carebullOther

        bullWater wastewater bill savingsbullOperating costs (non-energy) bullEquipment maintenancebullEquipment performance (push air better etc)bullEquipment lifetimebullShutoffs ReconnectsbullProperty value benefits sellingbull(Bill-related) calls to utilitybullComfortbullAesthetics appearancebullFires insurance damage (gas)bullLighting quality of light bullNoisebullSafety

        bullControl over billbullUnderstanding knowledgebullldquoCarerdquo or ldquohardshiprdquo (low income)bullIndoor air qualitybullHealth lost days at work or schoolbullFewer movesbullDoing good for environmentbullSavings in other fuels or services (as relevant)bullGHG and environmental effectsbullNegatives

        Source (SkumatzSERA1996 on)

        Net ThreeNet Positive amp negativeNet beyond standard efficiencyNet to gross

        Non-OverlappingConsistent Units

        SERA

        KEY APPLICATIONS OF NEBS

        MARKETING amp ROI ndashSell whatrsquos valuable to customers link to

        peers

        PROGRAM REFINEMENT ndashPositive amp Negative NEBs for

        measures barriers incentives and targeting

        TRAIN THE CHAIN ndashAlign Educate Actors on NEB

        priorities

        POLICY GOALS Quantifies Non-energy goals (eg

        Low income jobs etc)

        BC TESTS ndashRefined CE for program amp portfolio reduce bias in

        investment

        6

        Source SERA all rights reserved

        SERA

        USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS

        NegativeNEBs

        Solar WH

        Appearance -$14 NZ

        Maintenance -$9 NZ

        Source Skumatz Economic Research Associates research ndash 1998-2005

        -005

        0

        005

        01

        015

        02

        Maint

        Productiv Perf Life

        Op CostTenant S

        atComfor

        t

        Lite Safe SellEnvir

        oOther

        AampE Owner

        Commercial Example

        Residential Example

        SERA

        NEB MEASUREMENT

        8

        SERA

        NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES

        Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip

        9

        Monetized NEBs

        Source Skumatz SERA research

        NEB

        s

        Direct

        Secondary

        Model

        Survey Story of a ferryhellip then it is academic

        Corp Records Utility data

        Change x valueFinancial Calcs

        Third party jobsAnd emissions

        Specialized academic Best for some NEBs

        SERA

        MEASURING PARTICIPANTNEBs Best practices

        WTP not fruitful 7+ better options

        Net Three Net positive amp negative Net beyond standard

        efficiency Net to Gross

        Non-overlapping Consistent units Recommend in ALL

        process (impact) surveys - barriers

        HTM=Hard to measure HTA=Hard to answer

        Direct calc Accurate Small ldquonrdquo

        WTPWTA Accepted Volatile HTA

        Relative Fast strong HTA

        LMSFast

        strong clear ETA

        Unfamiliar

        Logit Conjoint

        rankStrong Slow

        complex

        Regression Defensible Limited data cost

        Market value Strong Data

        Other Exploring Cost tradeoffs

        Source SERA Research

        SERA

        HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)

        11

        Monetized NEBs

        Direct Secondary Model Survey

        Attributable Change (study)

        Value or Financial

        Calc

        X

        Total Attrib Stated Relative

        Effect

        Savings (or translation)(ldquoNormrdquo)

        Individual NEB Shares

        X

        X

        SERA

        PARTICIPANT MEASUREMENT METHODS COMPARISON

        005

        115

        225

        335

        445

        MW

        x

        NE

        Wx

        CA

        Wx

        CA

        Wx

        SF-N

        C

        MF-

        NC

        C-

        Lite

        C-R

        eb

        C-R

        eb

        multLMS VerbalWTP

        Source SERA Research

        Chart1

        mult
        LMS Verbal
        WTP
        122
        156
        42
        08
        1
        2
        098
        4
        06
        13
        12
        075
        067
        104
        115
        27
        075
        055
        32
        084
        091

        Sheet1

        Sheet1

        mult
        LMS Verbal
        WTP

        Sheet2

        Sheet3

        SERA

        PROS AND CONS OF NEBS MEASUREMENT METHODS

        13

        Source Skumatz SERA research

        Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

        Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

        Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

        Long history easy secondary sources Credible to reviewers vetted inputs

        Not available for all NEBs

        Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

        Third party peer-vetted models available for economics jobs and emissions

        Not available for all NEBs

        Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

        Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

        Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

        SERA

        PROGRESS TRANSFERABILITY OF NEBS

        14

        Source Skumatz SERA research

        Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

        bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

        Program measure dependent

        bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

        education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

        Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

        Residential Target dependent (low income or MF vs SF)

        bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

        vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

        Cost savings consistency if transferable vs risk

        SERA

        RANGES TYPICAL NEB VALUES FOR WxRETROFI

        15

        SERA

        PROGRESS AND GAPS Consistency Transferability

        Some programs types others less studied Transferability research

        Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

        measures until then Across the board Savings share Regression

        SERA

        PROGRESS AND GAPS Gas vs Electric

        Some researchndashusually program-wide not broken down International research found participant NEBs have

        similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

        MF Less-commonly-studied complicated by poor response

        and complexity of sector (decision-maker some measures in home some central) separate from low income not common

        Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

        Commercial gaps in some measures

        SERA

        USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

        Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

        18

        0

        005

        01

        015

        02

        025

        Environment

        Quality

        Quantity

        PgmAvailFlicker

        Mainten

        ance

        Lifetime

        SalesP

        roductivity

        Safety

        Noise

        Control

        Illness

        Total NEBs Multiplier=X

        $

        Savings value=$X

        Chart3

        023
        016
        014
        01
        008
        006
        006
        006
        005
        005
        003
        0

        Chart1

        Vendor
        NP
        Partic
        028
        02
        02
        028
        013
        017
        031
        022
        016
        006
        008
        014
        005
        006
        014
        004
        009
        007
        004
        003
        007
        -006
        018
        006

        Sheet1

        Sheet1

        Sheet2

        Sheet3

        SERA

        NEB VALUES

        19

        SERA

        ARE NEBS HIGH VALUE Energy savings are often the minority of

        benefits from program measuresndash Donrsquot ignore that

        20

        0 05 1 15 2 25 3

        Energy Sav

        NEBs

        NEB vs Energy Savings ValueIncluding all NEBs

        Source (SkumatzSERA 2010 amp others)

        Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

        SERA

        RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

        MF Many others

        Thousands of surveys results By measures By program types By sectors By stakeholders By geography

        Variety of end uses

        SERA

        TOP NEBS FOR WEATHERIZATION PROGRAM

        Weatherization program multipliergt1

        Source SERA Research

        SERA

        RESULTS FOR ENERGY STARreg APPLIANCES

        0002004006008

        01012014016018

        Maint

        Appear

        Perf LifeNoise Satis

        Comfort

        LiteSafe

        ty SellMove

        Water

        Enviro

        Refrig DW CW

        Source SERA Research

        SERA

        INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

        5 increments to 50

        Source SERA research

        Chart5

        EnergySmart (insul Retrofit)
        Dunedin (insul)
        001
        024
        002
        004
        002
        005
        006
        005
        007
        008
        008
        006
        013
        0
        013
        016
        049
        033

        cx

        cx

        nz

        Sheet3

        Sheet3

        Super Insulation
        Double Glazing
        Solar Design
        Solar Water Heating
        Super Insulation
        Double Glazing
        Solar Design
        Solar Water Heating
        EnergySmart (insul Retrofit)
        Dunedin (insul)

        SERA

        ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

        0

        002

        004

        006

        008

        01

        012

        Maint

        Perf LifeSatis

        f

        ComfortAesth

        NoiseSafe

        ty

        LightingSelli

        ng

        Stay in home

        Enviro

        Sick Days

        Calls

        ES Homes HP wES Energy savings multiplier=1

        Source SERA Research

        SERA

        CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

        DG CHP HVAC Equipment rebate Other

        Building codes incentives by cities

        Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

        Variety of end uses

        SERA

        AUDIT FINANCIAL PROGRAM

        0

        01

        02

        03

        04

        05

        06

        07

        Variety of measureshellip disaggregatedResults by measure type businessType etchellip

        Source SERA Research

        SERA

        CampI NEW CONSTRUCTION

        -005

        0

        005

        01

        015

        02

        AampE Owner(Pct of Total Partic NEBs)

        Source SERA Research

        SERA

        CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

        0 2 4 6 8 10 12

        m Product losses

        l Ease of selling leasing building

        n Labor requirements

        h Aesthetics appearance

        j Noise

        b Equipment maintenance

        f Tenant satisfaction

        d Equipment lifetimes

        e Productivity

        k Building safety[2]

        g Comfort

        i Lighting Quality of Light

        c Equipment performance[1]

        a Operating costs

        o Doing good for the Environment[3]

        Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

        vcv

        Source SERA Research

        Chart1

        CampI Technical Assistance NEBs
        002
        003
        004
        005
        005
        006
        006
        007
        007
        007
        009
        009
        01
        011
        011

        Sheet1

        Sheet1

        CampI Technical Assistance Program NEBs (Participant)

        Sheet2

        Sheet3

        SERA

        EXPRESSING NEBS VALUEndashCxe

        Strong value from RetroCxSource SERA Research

        SERA

        SOCIETAL NEB EXAMPLES

        31

        SERA

        JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

        32

        0

        02

        04

        06

        08

        1

        12

        HPWxRetrof Appliance

        CA

        WI

        Natl

        (Source Skumatz SERAECEEE 2007 ACEEE 2006)

        Jobs Economic

        SERA

        EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

        33

        SERA

        UTILITY NEBSUtility NEBs for Template Program

        Debt WriteOff (util)13

        Rate Subsidy(util)61

        HealthSafety(util)0

        Colln Costs (util)0

        Gas Calls (util)0

        Calls to CSRs(util)2

        TampD (util)16

        Arrears (util)0

        Reconnects (util)0

        Notices (util)7

        Shutoffs (util)1

        Rate subsidy TampD

        Payment-related

        Source Skumatz Economic Research Associates research

        Example Low Income Weatherization

        SERA

        ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

        be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

        Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

        Value of 100 HALVES Payback doubles ROI

        Source Skumatz Economics (SERA)

        35

        ( for NEBs Multiplier of 125)

        SERA

        NEBS IN COST-EFFECTIVENESS APPLICATIONS

        36

        hellip2001 ON

        SERA

        37

        NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

        For true representation of B amp C elements of NEBs address missing factor bias

        Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

        BC applications were considered early on then pulled back awaiting more quantitative evidence

        SERA

        CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

        38

        Test Benefits Costs States Using Traditionally

        Improved treat-ment with NEBs

        Utility Cost (or Program Administrator Test) (UCT or PAC)

        bull Avoided supply costs for transmission distribution and generation (TDampG)

        bull Avoided gas and water supply costs

        bull Program administration

        bull Participant incentives

        bull Increased supply cost

        CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

        Use cost only paid by the utility

        Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

        Same as above plus bull increased revenue

        Same as above plus bull Decreased revenue

        AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

        Participant cost bull Utility bill reductions bull Participant incentives

        bull Participant direct costs

        AR CA FL HI IA IN MI MN NY VA

        Participant NEBs

        Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

        Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

        SERA

        APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

        Expansion Concern with risk calculation accuracy

        39

        Source Skumatz SERA research

        SERA

        40

        METHODS TO INCLUDE NEBsIN REGULATORY TESTS

        Maximize DSM opportunities amp feedback Accuracy tailoring

        Minimize Regulatory amp Implementer Risk

        Minimize Evaluation Cost

        Adder

        Readily MeasurableHybrid

        All NEBsSource SERA Research

        SERA

        41

        STATE TREATMENT OF NEBS NEBsRegulatory

        Screening Application Utilities regions

        Program Marketing Fairly widespread use in utilities states across the country

        MO

        RE AGGRESSIVE ==gt

        Test Pgm Screen - adder

        IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

        Test Pgm Screen - readily measured

        MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

        Test Hybrid (potential adder amp measured)

        CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

        Test Pgm screen - Broad

        With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

        Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

        SERA

        42

        Key

        States with NEBs in CE tests for At least one type of program

        copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

        STATES WITH NEBS IN CE TESTS

        NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

        groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

        SERA

        SELECTED STATE PROGRESS CA

        2001 ndash LIPPT model NY

        Mid-2000s measurement amp scenarios not included CO

        Adders (10 electric 25 LI 5 gas) VT

        15 adder helped by previous research DC

        10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

        43

        Source Skumatz SERA research

        SERA

        SELECTED STATE PROGRESS Dominos ongoing

        Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

        Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

        Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

        Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

        44

        Source Skumatz SERA research

        SERA

        APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

        Expansion Concern with risk calculation accuracy

        45

        Source Skumatz SERA research

        SERA

        46

        NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

        [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

        SourceSkumatz SERA Independent research

        NTG ndash accuracy measurement incomplete

        Savings Impact repeatedly amp expensivelymeasured little variation $100K+

        NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

        EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

        Cost Complicated expensive local changes

        RELATIVE Risk from NEBs

        Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

        Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

        NEBs as accurate as many inputs (all could improve)

        SERA

        DISCOUNT RATE MATTERS

        47

        Source Skumatz SERA Research

        0

        20

        40

        60

        80

        100

        120

        1 2 3 4 5 6 7 8 9 10

        PV Ratios vs 1 Discount Rate

        10 yr PV 20 yr PV

        Discount rate - percent

        WACCSocialLTT

        Source Skumatz Electricity Journal IEPEC paper independent research

        SERA

        EUL VARIATIONS-ADOPTED VALUES

        Source Skumatz Economic Research Associates SERA independent research

        0

        50

        100

        150

        200

        250

        300

        350

        400

        450

        0 10 20 30 40 50 60 70

        EUL Variation by Measure

        67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

        Poorly supportedOldTechnology change

        SERA

        KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

        Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

        measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

        change the BC conclusion

        On the cusphellip

        If NO Yoursquore doneAnd bias addressedsufficiently

        IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

        Source SERA all rights reserved

        SERArsquos lsquoNEB-ItrdquoModel ndash

        SERA

        IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

        Utility Soc Part Conserv Recrsquom

        Rationale

        Base Percent X X X Program-invariant

        Low Income X X X X Multiple sources

        Weatherization X X X Substantial Participantimpacts

        Measure Program-specific

        X Varies by measure sector

        Other Recomrsquos Local Research

        50

        Source Skumatz SERA

        Developing values for multiple states amp utilities

        SERA

        TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

        Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

        Other sources of bias to research (EUL NTG etc) bring high variation

        5) States are incorporating NEBs dominos input in deliberations in multiple states

        6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

        THANK YOU

        Questions

        Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

        SERA

        POWER OF NEBS ndashBARRIERS DISCONNECTS

        53

        SERA

        NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

        Negative NEB values cost of barrier

        Solar Water Heat NZ$ Euros

        Solar Design NZ$ Euros

        Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

        -23 -12 (079)

        -11 -6 (06)

        Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

        Source Skumatz Economic Research Associates research

        SERA

        CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

        NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

        -5-4-3-2-1012345

        Op Co

        st

        Init C

        ost

        Maint

        Perfo

        rm

        Prod

        uctiv

        Tena

        nt Sa

        t

        Comf

        ort

        Appe

        ar

        Light

        Qual

        SellL

        ease

        Devp Ownr AampE

        Note AgtE (~4 vs ~2) for these NEBs

        Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

        Source Skumatz Economic Research Associates Inc 2002

        Owners more positive ndashUnderinvestment train

        • Soup to nuts on non-energy benefits NEIs
        • TOPICS
        • BACKGROUND NEBS
        • 20+ YEARS OF NEBS PROGRESShellip
        • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
        • KEY APPLICATIONS OF NEBS
        • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
        • Neb measurement
        • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
        • MEASURING PARTICIPANTNEBs
        • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
        • PARTICIPANT MEASUREMENT METHODS COMPARISON
        • PROS AND CONS OF NEBS MEASUREMENT METHODS
        • PROGRESS TRANSFERABILITY OF NEBS
        • RANGES TYPICAL NEB VALUES FOR WxRETROFI
        • PROGRESS AND GAPS
        • PROGRESS AND GAPS
        • USING NEB NEI VALUES
        • Slide Number 19
        • ARE NEBS HIGH VALUE
        • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
        • TOP NEBS FOR WEATHERIZATION PROGRAM
        • RESULTS FOR ENERGY STARreg APPLIANCES
        • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
        • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
        • CampI PROGRAMS ldquoNEBBEDrdquo
        • AUDIT FINANCIAL PROGRAM
        • CampI NEW CONSTRUCTION
        • CampI TECHNICAL ASSISTANCE
        • EXPRESSING NEBS VALUEndashCxe
        • Slide Number 31
        • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
        • EMISSIONS (SOCIETAL NEBS)
        • UTILITY NEBS
        • ADJUSTED PAYBACKS
        • NEBS IN COST-EFFECTIVENESS APPLICATIONS
        • NEBS IN BC
        • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
        • APPLICATION TO BC
        • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
        • STATE TREATMENT OF NEBS NEBs
        • Slide Number 42
        • SELECTED STATE PROGRESS
        • SELECTED STATE PROGRESS
        • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
        • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
        • DISCOUNT RATE MATTERS
        • EUL VARIATIONS- ADOPTED VALUES
        • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
        • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
        • TAKEAWAYSON NEBSNEIS
        • THANK YOUQuestions
        • Slide Number 53
        • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
        • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
          Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
          m Product losses20
          l Ease of selling leasing building33
          n Labor requirements45
          h Aesthetics appearance59
          j Noise50
          b Equipment maintenance660
          f Tenant satisfaction60
          d Equipment lifetimes718
          e Productivity724
          k Building safety[2]73
          g Comfort933
          i Lighting Quality of Light916
          c Equipment performance[1]1019
          a Operating costs1165
          o Doing good for the Environment[3]1132
          Other03
          [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
          [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
          [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
          m Product losses
          l Ease of selling leasing building
          n Labor requirements
          h Aesthetics appearance
          j Noise
          b Equipment maintenance
          f Tenant satisfaction
          d Equipment lifetimes
          e Productivity
          k Building safety[2]
          g Comfort
          i Lighting Quality of Light
          c Equipment performance[1]
          a Operating costs
          o Doing good for the Environment[3]
          Share of ZALEH NEBS from various measures
          Each presented as multiple of the (overall) energy savings
          Super Insulation037
          Double Glazing026
          Solar Design025
          Solar Water Heating005
          EnergySmart (insul Retrofit)Dunedin (insul)
          Other124
          Noise24
          Budgetfinances25
          Environmental65
          Health78
          Reduced moves86
          Ease of sellingrenting130
          DampnessMold1316
          Comfort-related4933
          101101
          Operational NEBs44
          DesignConstruction NEBs31
          Occupant NEBs25
          1
          OtherOther
          NoiseNoise
          BudgetfinancesBudgetfinances
          EnvironmentalEnvironmental
          HealthHealth
          Reduced movesReduced moves
          Ease of sellingrentingEase of sellingrenting
          DampnessMoldDampnessMold
          Comfort-relatedComfort-related
          Boiler
          VendorNPPartic
          Perform0280202
          Footprint028013017
          ControlFeatures031022016
          Tenant compl006008014
          Noise005006014
          Maint004009007
          Other004003007
          Lifetime-006018006
          MF New construction energy star
          Operating cost014
          Environment012
          Maintenance01
          Lifetime01
          Performance009
          Occupant Satisf009
          Comfort008
          Noise006
          Sellingleasing006
          Aesthetics005
          Light Quality005
          Power QualReliab004
          Safety002
          RIGHTLIGHTS
          Environment023
          Quality016
          Quantity014
          PgmAvail01
          Flicker008
          Maintenance006
          Lifetime006
          SalesProductivity006
          Safety005
          Noise005
          Control003
          Illness0
          PerformPerformPerform
          FootprintFootprintFootprint
          ControlFeaturesControlFeaturesControlFeatures
          Tenant complTenant complTenant compl
          NoiseNoiseNoise
          MaintMaintMaint
          OtherOtherOther
          LifetimeLifetimeLifetime
          Environment
          Quality
          Quantity
          PgmAvail
          Flicker
          Maintenance
          Lifetime
          SalesProductivity
          Safety
          Noise
          Control
          Illness

          Variability

          Relevant NEB Categories

          Program measure invariant (suitable for ldquoadderrdquo)

          middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

          Program measure dependent

          middot Economic ndash societal (depends on measures and local manufacture installation)

          middot Health and safety health care illnesses ndash societal and participant (measure)

          middot Water wastewater infrastructure and water bill savings ndash societal and participant

          middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

          Climate dependent

          middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

          Residential Target dependent (low income or MF vs SF)

          middot Payment related ndash utility (arrearages etc stronger for low income targets)

          middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

          middot Participant benefits related to hardship and payments

          middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

          Used for

          Major Advantages

          Major Disadvantages

          Direct measurement (specific studies of changes on-site)

          Comrsquol labor productivity etc

          Direct precise attributable

          Small estimation sample sizes specialized cases poor transferability expensive

          Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

          Insurance water health others

          Long history easy secondary sources

          Credible to reviewers vetted inputs

          Not available for all NEBs

          Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

          Emissions economics

          Third party peer-vetted models available for economics jobs and emissions

          Not available for all NEBs

          Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

          Wide variety of Participant NEBs

          Large sample sizes amp statistical properties

          Affordable

          Multiple estimates leading to similar ranges

          Direct method of measuring some key NEBs

          Concerns about surveys as a source of quantitative values amp reliability

          Recall from survey respondents

          Proper attribution to programs measures

          MWxNEWxCA WxCA WxSF-NCMF-NCC- LiteC-RebC-Reb
          mult1220813075104075084
          LMS Verbal15610980612067115055091
          WTP42242732
          MWxMWxMWx
          NEWxNEWxNEWx
          CA WxCA WxCA Wx
          CA WxCA WxCA Wx
          SF-NCSF-NCSF-NC
          MF-NCMF-NCMF-NC
          C- LiteC- LiteC- Lite
          C-RebC-RebC-Reb
          C-RebC-RebC-Reb

          SERA

          5

          NEB PERSPECTIVES CATEGORIES BEST PRACTICES

          Utility Society Participant (ResampICI)bullCarrying cost on arrearages bullBad debt written offbullShutoffs ReconnectsbullNotices calls collection costsbullEmergency gas service calls (for gas flex connector and other programs)bullInsurance savingsbullTransmission and distribution savings (usually distribution)bullFewer substations etcbullPower quality reliabilitybullReduced subsidy payments (low income)bullOther

          bullEconomic development benefits ndash direct and indirect multipliersbullTax effectsbullEmissions environmental (trading values andor health hazard benefits)bullHealth and safety equipmentbullWater and waste water treatment or supply plantsbullFish wildlife mitigationbullNational securitybullHealth carebullOther

          bullWater wastewater bill savingsbullOperating costs (non-energy) bullEquipment maintenancebullEquipment performance (push air better etc)bullEquipment lifetimebullShutoffs ReconnectsbullProperty value benefits sellingbull(Bill-related) calls to utilitybullComfortbullAesthetics appearancebullFires insurance damage (gas)bullLighting quality of light bullNoisebullSafety

          bullControl over billbullUnderstanding knowledgebullldquoCarerdquo or ldquohardshiprdquo (low income)bullIndoor air qualitybullHealth lost days at work or schoolbullFewer movesbullDoing good for environmentbullSavings in other fuels or services (as relevant)bullGHG and environmental effectsbullNegatives

          Source (SkumatzSERA1996 on)

          Net ThreeNet Positive amp negativeNet beyond standard efficiencyNet to gross

          Non-OverlappingConsistent Units

          SERA

          KEY APPLICATIONS OF NEBS

          MARKETING amp ROI ndashSell whatrsquos valuable to customers link to

          peers

          PROGRAM REFINEMENT ndashPositive amp Negative NEBs for

          measures barriers incentives and targeting

          TRAIN THE CHAIN ndashAlign Educate Actors on NEB

          priorities

          POLICY GOALS Quantifies Non-energy goals (eg

          Low income jobs etc)

          BC TESTS ndashRefined CE for program amp portfolio reduce bias in

          investment

          6

          Source SERA all rights reserved

          SERA

          USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS

          NegativeNEBs

          Solar WH

          Appearance -$14 NZ

          Maintenance -$9 NZ

          Source Skumatz Economic Research Associates research ndash 1998-2005

          -005

          0

          005

          01

          015

          02

          Maint

          Productiv Perf Life

          Op CostTenant S

          atComfor

          t

          Lite Safe SellEnvir

          oOther

          AampE Owner

          Commercial Example

          Residential Example

          SERA

          NEB MEASUREMENT

          8

          SERA

          NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES

          Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip

          9

          Monetized NEBs

          Source Skumatz SERA research

          NEB

          s

          Direct

          Secondary

          Model

          Survey Story of a ferryhellip then it is academic

          Corp Records Utility data

          Change x valueFinancial Calcs

          Third party jobsAnd emissions

          Specialized academic Best for some NEBs

          SERA

          MEASURING PARTICIPANTNEBs Best practices

          WTP not fruitful 7+ better options

          Net Three Net positive amp negative Net beyond standard

          efficiency Net to Gross

          Non-overlapping Consistent units Recommend in ALL

          process (impact) surveys - barriers

          HTM=Hard to measure HTA=Hard to answer

          Direct calc Accurate Small ldquonrdquo

          WTPWTA Accepted Volatile HTA

          Relative Fast strong HTA

          LMSFast

          strong clear ETA

          Unfamiliar

          Logit Conjoint

          rankStrong Slow

          complex

          Regression Defensible Limited data cost

          Market value Strong Data

          Other Exploring Cost tradeoffs

          Source SERA Research

          SERA

          HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)

          11

          Monetized NEBs

          Direct Secondary Model Survey

          Attributable Change (study)

          Value or Financial

          Calc

          X

          Total Attrib Stated Relative

          Effect

          Savings (or translation)(ldquoNormrdquo)

          Individual NEB Shares

          X

          X

          SERA

          PARTICIPANT MEASUREMENT METHODS COMPARISON

          005

          115

          225

          335

          445

          MW

          x

          NE

          Wx

          CA

          Wx

          CA

          Wx

          SF-N

          C

          MF-

          NC

          C-

          Lite

          C-R

          eb

          C-R

          eb

          multLMS VerbalWTP

          Source SERA Research

          Chart1

          mult
          LMS Verbal
          WTP
          122
          156
          42
          08
          1
          2
          098
          4
          06
          13
          12
          075
          067
          104
          115
          27
          075
          055
          32
          084
          091

          Sheet1

          Sheet1

          mult
          LMS Verbal
          WTP

          Sheet2

          Sheet3

          SERA

          PROS AND CONS OF NEBS MEASUREMENT METHODS

          13

          Source Skumatz SERA research

          Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

          Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

          Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

          Long history easy secondary sources Credible to reviewers vetted inputs

          Not available for all NEBs

          Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

          Third party peer-vetted models available for economics jobs and emissions

          Not available for all NEBs

          Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

          Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

          Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

          SERA

          PROGRESS TRANSFERABILITY OF NEBS

          14

          Source Skumatz SERA research

          Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

          bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

          Program measure dependent

          bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

          education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

          Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

          Residential Target dependent (low income or MF vs SF)

          bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

          vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

          Cost savings consistency if transferable vs risk

          SERA

          RANGES TYPICAL NEB VALUES FOR WxRETROFI

          15

          SERA

          PROGRESS AND GAPS Consistency Transferability

          Some programs types others less studied Transferability research

          Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

          measures until then Across the board Savings share Regression

          SERA

          PROGRESS AND GAPS Gas vs Electric

          Some researchndashusually program-wide not broken down International research found participant NEBs have

          similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

          MF Less-commonly-studied complicated by poor response

          and complexity of sector (decision-maker some measures in home some central) separate from low income not common

          Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

          Commercial gaps in some measures

          SERA

          USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

          Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

          18

          0

          005

          01

          015

          02

          025

          Environment

          Quality

          Quantity

          PgmAvailFlicker

          Mainten

          ance

          Lifetime

          SalesP

          roductivity

          Safety

          Noise

          Control

          Illness

          Total NEBs Multiplier=X

          $

          Savings value=$X

          Chart3

          023
          016
          014
          01
          008
          006
          006
          006
          005
          005
          003
          0

          Chart1

          Vendor
          NP
          Partic
          028
          02
          02
          028
          013
          017
          031
          022
          016
          006
          008
          014
          005
          006
          014
          004
          009
          007
          004
          003
          007
          -006
          018
          006

          Sheet1

          Sheet1

          Sheet2

          Sheet3

          SERA

          NEB VALUES

          19

          SERA

          ARE NEBS HIGH VALUE Energy savings are often the minority of

          benefits from program measuresndash Donrsquot ignore that

          20

          0 05 1 15 2 25 3

          Energy Sav

          NEBs

          NEB vs Energy Savings ValueIncluding all NEBs

          Source (SkumatzSERA 2010 amp others)

          Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

          SERA

          RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

          MF Many others

          Thousands of surveys results By measures By program types By sectors By stakeholders By geography

          Variety of end uses

          SERA

          TOP NEBS FOR WEATHERIZATION PROGRAM

          Weatherization program multipliergt1

          Source SERA Research

          SERA

          RESULTS FOR ENERGY STARreg APPLIANCES

          0002004006008

          01012014016018

          Maint

          Appear

          Perf LifeNoise Satis

          Comfort

          LiteSafe

          ty SellMove

          Water

          Enviro

          Refrig DW CW

          Source SERA Research

          SERA

          INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

          5 increments to 50

          Source SERA research

          Chart5

          EnergySmart (insul Retrofit)
          Dunedin (insul)
          001
          024
          002
          004
          002
          005
          006
          005
          007
          008
          008
          006
          013
          0
          013
          016
          049
          033

          cx

          cx

          nz

          Sheet3

          Sheet3

          Super Insulation
          Double Glazing
          Solar Design
          Solar Water Heating
          Super Insulation
          Double Glazing
          Solar Design
          Solar Water Heating
          EnergySmart (insul Retrofit)
          Dunedin (insul)

          SERA

          ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

          0

          002

          004

          006

          008

          01

          012

          Maint

          Perf LifeSatis

          f

          ComfortAesth

          NoiseSafe

          ty

          LightingSelli

          ng

          Stay in home

          Enviro

          Sick Days

          Calls

          ES Homes HP wES Energy savings multiplier=1

          Source SERA Research

          SERA

          CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

          DG CHP HVAC Equipment rebate Other

          Building codes incentives by cities

          Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

          Variety of end uses

          SERA

          AUDIT FINANCIAL PROGRAM

          0

          01

          02

          03

          04

          05

          06

          07

          Variety of measureshellip disaggregatedResults by measure type businessType etchellip

          Source SERA Research

          SERA

          CampI NEW CONSTRUCTION

          -005

          0

          005

          01

          015

          02

          AampE Owner(Pct of Total Partic NEBs)

          Source SERA Research

          SERA

          CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

          0 2 4 6 8 10 12

          m Product losses

          l Ease of selling leasing building

          n Labor requirements

          h Aesthetics appearance

          j Noise

          b Equipment maintenance

          f Tenant satisfaction

          d Equipment lifetimes

          e Productivity

          k Building safety[2]

          g Comfort

          i Lighting Quality of Light

          c Equipment performance[1]

          a Operating costs

          o Doing good for the Environment[3]

          Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

          vcv

          Source SERA Research

          Chart1

          CampI Technical Assistance NEBs
          002
          003
          004
          005
          005
          006
          006
          007
          007
          007
          009
          009
          01
          011
          011

          Sheet1

          Sheet1

          CampI Technical Assistance Program NEBs (Participant)

          Sheet2

          Sheet3

          SERA

          EXPRESSING NEBS VALUEndashCxe

          Strong value from RetroCxSource SERA Research

          SERA

          SOCIETAL NEB EXAMPLES

          31

          SERA

          JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

          32

          0

          02

          04

          06

          08

          1

          12

          HPWxRetrof Appliance

          CA

          WI

          Natl

          (Source Skumatz SERAECEEE 2007 ACEEE 2006)

          Jobs Economic

          SERA

          EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

          33

          SERA

          UTILITY NEBSUtility NEBs for Template Program

          Debt WriteOff (util)13

          Rate Subsidy(util)61

          HealthSafety(util)0

          Colln Costs (util)0

          Gas Calls (util)0

          Calls to CSRs(util)2

          TampD (util)16

          Arrears (util)0

          Reconnects (util)0

          Notices (util)7

          Shutoffs (util)1

          Rate subsidy TampD

          Payment-related

          Source Skumatz Economic Research Associates research

          Example Low Income Weatherization

          SERA

          ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

          be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

          Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

          Value of 100 HALVES Payback doubles ROI

          Source Skumatz Economics (SERA)

          35

          ( for NEBs Multiplier of 125)

          SERA

          NEBS IN COST-EFFECTIVENESS APPLICATIONS

          36

          hellip2001 ON

          SERA

          37

          NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

          For true representation of B amp C elements of NEBs address missing factor bias

          Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

          BC applications were considered early on then pulled back awaiting more quantitative evidence

          SERA

          CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

          38

          Test Benefits Costs States Using Traditionally

          Improved treat-ment with NEBs

          Utility Cost (or Program Administrator Test) (UCT or PAC)

          bull Avoided supply costs for transmission distribution and generation (TDampG)

          bull Avoided gas and water supply costs

          bull Program administration

          bull Participant incentives

          bull Increased supply cost

          CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

          Use cost only paid by the utility

          Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

          Same as above plus bull increased revenue

          Same as above plus bull Decreased revenue

          AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

          Participant cost bull Utility bill reductions bull Participant incentives

          bull Participant direct costs

          AR CA FL HI IA IN MI MN NY VA

          Participant NEBs

          Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

          Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

          SERA

          APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

          Expansion Concern with risk calculation accuracy

          39

          Source Skumatz SERA research

          SERA

          40

          METHODS TO INCLUDE NEBsIN REGULATORY TESTS

          Maximize DSM opportunities amp feedback Accuracy tailoring

          Minimize Regulatory amp Implementer Risk

          Minimize Evaluation Cost

          Adder

          Readily MeasurableHybrid

          All NEBsSource SERA Research

          SERA

          41

          STATE TREATMENT OF NEBS NEBsRegulatory

          Screening Application Utilities regions

          Program Marketing Fairly widespread use in utilities states across the country

          MO

          RE AGGRESSIVE ==gt

          Test Pgm Screen - adder

          IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

          Test Pgm Screen - readily measured

          MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

          Test Hybrid (potential adder amp measured)

          CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

          Test Pgm screen - Broad

          With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

          Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

          SERA

          42

          Key

          States with NEBs in CE tests for At least one type of program

          copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

          STATES WITH NEBS IN CE TESTS

          NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

          groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

          SERA

          SELECTED STATE PROGRESS CA

          2001 ndash LIPPT model NY

          Mid-2000s measurement amp scenarios not included CO

          Adders (10 electric 25 LI 5 gas) VT

          15 adder helped by previous research DC

          10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

          43

          Source Skumatz SERA research

          SERA

          SELECTED STATE PROGRESS Dominos ongoing

          Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

          Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

          Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

          Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

          44

          Source Skumatz SERA research

          SERA

          APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

          Expansion Concern with risk calculation accuracy

          45

          Source Skumatz SERA research

          SERA

          46

          NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

          [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

          SourceSkumatz SERA Independent research

          NTG ndash accuracy measurement incomplete

          Savings Impact repeatedly amp expensivelymeasured little variation $100K+

          NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

          EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

          Cost Complicated expensive local changes

          RELATIVE Risk from NEBs

          Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

          Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

          NEBs as accurate as many inputs (all could improve)

          SERA

          DISCOUNT RATE MATTERS

          47

          Source Skumatz SERA Research

          0

          20

          40

          60

          80

          100

          120

          1 2 3 4 5 6 7 8 9 10

          PV Ratios vs 1 Discount Rate

          10 yr PV 20 yr PV

          Discount rate - percent

          WACCSocialLTT

          Source Skumatz Electricity Journal IEPEC paper independent research

          SERA

          EUL VARIATIONS-ADOPTED VALUES

          Source Skumatz Economic Research Associates SERA independent research

          0

          50

          100

          150

          200

          250

          300

          350

          400

          450

          0 10 20 30 40 50 60 70

          EUL Variation by Measure

          67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

          Poorly supportedOldTechnology change

          SERA

          KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

          Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

          measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

          change the BC conclusion

          On the cusphellip

          If NO Yoursquore doneAnd bias addressedsufficiently

          IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

          Source SERA all rights reserved

          SERArsquos lsquoNEB-ItrdquoModel ndash

          SERA

          IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

          Utility Soc Part Conserv Recrsquom

          Rationale

          Base Percent X X X Program-invariant

          Low Income X X X X Multiple sources

          Weatherization X X X Substantial Participantimpacts

          Measure Program-specific

          X Varies by measure sector

          Other Recomrsquos Local Research

          50

          Source Skumatz SERA

          Developing values for multiple states amp utilities

          SERA

          TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

          Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

          Other sources of bias to research (EUL NTG etc) bring high variation

          5) States are incorporating NEBs dominos input in deliberations in multiple states

          6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

          THANK YOU

          Questions

          Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

          SERA

          POWER OF NEBS ndashBARRIERS DISCONNECTS

          53

          SERA

          NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

          Negative NEB values cost of barrier

          Solar Water Heat NZ$ Euros

          Solar Design NZ$ Euros

          Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

          -23 -12 (079)

          -11 -6 (06)

          Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

          Source Skumatz Economic Research Associates research

          SERA

          CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

          NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

          -5-4-3-2-1012345

          Op Co

          st

          Init C

          ost

          Maint

          Perfo

          rm

          Prod

          uctiv

          Tena

          nt Sa

          t

          Comf

          ort

          Appe

          ar

          Light

          Qual

          SellL

          ease

          Devp Ownr AampE

          Note AgtE (~4 vs ~2) for these NEBs

          Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

          Source Skumatz Economic Research Associates Inc 2002

          Owners more positive ndashUnderinvestment train

          • Soup to nuts on non-energy benefits NEIs
          • TOPICS
          • BACKGROUND NEBS
          • 20+ YEARS OF NEBS PROGRESShellip
          • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
          • KEY APPLICATIONS OF NEBS
          • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
          • Neb measurement
          • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
          • MEASURING PARTICIPANTNEBs
          • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
          • PARTICIPANT MEASUREMENT METHODS COMPARISON
          • PROS AND CONS OF NEBS MEASUREMENT METHODS
          • PROGRESS TRANSFERABILITY OF NEBS
          • RANGES TYPICAL NEB VALUES FOR WxRETROFI
          • PROGRESS AND GAPS
          • PROGRESS AND GAPS
          • USING NEB NEI VALUES
          • Slide Number 19
          • ARE NEBS HIGH VALUE
          • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
          • TOP NEBS FOR WEATHERIZATION PROGRAM
          • RESULTS FOR ENERGY STARreg APPLIANCES
          • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
          • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
          • CampI PROGRAMS ldquoNEBBEDrdquo
          • AUDIT FINANCIAL PROGRAM
          • CampI NEW CONSTRUCTION
          • CampI TECHNICAL ASSISTANCE
          • EXPRESSING NEBS VALUEndashCxe
          • Slide Number 31
          • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
          • EMISSIONS (SOCIETAL NEBS)
          • UTILITY NEBS
          • ADJUSTED PAYBACKS
          • NEBS IN COST-EFFECTIVENESS APPLICATIONS
          • NEBS IN BC
          • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
          • APPLICATION TO BC
          • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
          • STATE TREATMENT OF NEBS NEBs
          • Slide Number 42
          • SELECTED STATE PROGRESS
          • SELECTED STATE PROGRESS
          • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
          • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
          • DISCOUNT RATE MATTERS
          • EUL VARIATIONS- ADOPTED VALUES
          • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
          • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
          • TAKEAWAYSON NEBSNEIS
          • THANK YOUQuestions
          • Slide Number 53
          • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
          • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
            Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
            m Product losses20
            l Ease of selling leasing building33
            n Labor requirements45
            h Aesthetics appearance59
            j Noise50
            b Equipment maintenance660
            f Tenant satisfaction60
            d Equipment lifetimes718
            e Productivity724
            k Building safety[2]73
            g Comfort933
            i Lighting Quality of Light916
            c Equipment performance[1]1019
            a Operating costs1165
            o Doing good for the Environment[3]1132
            Other03
            [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
            [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
            [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
            m Product losses
            l Ease of selling leasing building
            n Labor requirements
            h Aesthetics appearance
            j Noise
            b Equipment maintenance
            f Tenant satisfaction
            d Equipment lifetimes
            e Productivity
            k Building safety[2]
            g Comfort
            i Lighting Quality of Light
            c Equipment performance[1]
            a Operating costs
            o Doing good for the Environment[3]
            Share of ZALEH NEBS from various measures
            Each presented as multiple of the (overall) energy savings
            Super Insulation037
            Double Glazing026
            Solar Design025
            Solar Water Heating005
            EnergySmart (insul Retrofit)Dunedin (insul)
            Other124
            Noise24
            Budgetfinances25
            Environmental65
            Health78
            Reduced moves86
            Ease of sellingrenting130
            DampnessMold1316
            Comfort-related4933
            101101
            Operational NEBs44
            DesignConstruction NEBs31
            Occupant NEBs25
            1
            OtherOther
            NoiseNoise
            BudgetfinancesBudgetfinances
            EnvironmentalEnvironmental
            HealthHealth
            Reduced movesReduced moves
            Ease of sellingrentingEase of sellingrenting
            DampnessMoldDampnessMold
            Comfort-relatedComfort-related
            Boiler
            VendorNPPartic
            Perform0280202
            Footprint028013017
            ControlFeatures031022016
            Tenant compl006008014
            Noise005006014
            Maint004009007
            Other004003007
            Lifetime-006018006
            MF New construction energy star
            Operating cost014
            Environment012
            Maintenance01
            Lifetime01
            Performance009
            Occupant Satisf009
            Comfort008
            Noise006
            Sellingleasing006
            Aesthetics005
            Light Quality005
            Power QualReliab004
            Safety002
            RIGHTLIGHTS
            Environment023
            Quality016
            Quantity014
            PgmAvail01
            Flicker008
            Maintenance006
            Lifetime006
            SalesProductivity006
            Safety005
            Noise005
            Control003
            Illness0
            PerformPerformPerform
            FootprintFootprintFootprint
            ControlFeaturesControlFeaturesControlFeatures
            Tenant complTenant complTenant compl
            NoiseNoiseNoise
            MaintMaintMaint
            OtherOtherOther
            LifetimeLifetimeLifetime
            Environment
            Quality
            Quantity
            PgmAvail
            Flicker
            Maintenance
            Lifetime
            SalesProductivity
            Safety
            Noise
            Control
            Illness

            Variability

            Relevant NEB Categories

            Program measure invariant (suitable for ldquoadderrdquo)

            middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

            Program measure dependent

            middot Economic ndash societal (depends on measures and local manufacture installation)

            middot Health and safety health care illnesses ndash societal and participant (measure)

            middot Water wastewater infrastructure and water bill savings ndash societal and participant

            middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

            Climate dependent

            middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

            Residential Target dependent (low income or MF vs SF)

            middot Payment related ndash utility (arrearages etc stronger for low income targets)

            middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

            middot Participant benefits related to hardship and payments

            middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

            Used for

            Major Advantages

            Major Disadvantages

            Direct measurement (specific studies of changes on-site)

            Comrsquol labor productivity etc

            Direct precise attributable

            Small estimation sample sizes specialized cases poor transferability expensive

            Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

            Insurance water health others

            Long history easy secondary sources

            Credible to reviewers vetted inputs

            Not available for all NEBs

            Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

            Emissions economics

            Third party peer-vetted models available for economics jobs and emissions

            Not available for all NEBs

            Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

            Wide variety of Participant NEBs

            Large sample sizes amp statistical properties

            Affordable

            Multiple estimates leading to similar ranges

            Direct method of measuring some key NEBs

            Concerns about surveys as a source of quantitative values amp reliability

            Recall from survey respondents

            Proper attribution to programs measures

            MWxNEWxCA WxCA WxSF-NCMF-NCC- LiteC-RebC-Reb
            mult1220813075104075084
            LMS Verbal15610980612067115055091
            WTP42242732
            MWxMWxMWx
            NEWxNEWxNEWx
            CA WxCA WxCA Wx
            CA WxCA WxCA Wx
            SF-NCSF-NCSF-NC
            MF-NCMF-NCMF-NC
            C- LiteC- LiteC- Lite
            C-RebC-RebC-Reb
            C-RebC-RebC-Reb

            SERA

            KEY APPLICATIONS OF NEBS

            MARKETING amp ROI ndashSell whatrsquos valuable to customers link to

            peers

            PROGRAM REFINEMENT ndashPositive amp Negative NEBs for

            measures barriers incentives and targeting

            TRAIN THE CHAIN ndashAlign Educate Actors on NEB

            priorities

            POLICY GOALS Quantifies Non-energy goals (eg

            Low income jobs etc)

            BC TESTS ndashRefined CE for program amp portfolio reduce bias in

            investment

            6

            Source SERA all rights reserved

            SERA

            USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS

            NegativeNEBs

            Solar WH

            Appearance -$14 NZ

            Maintenance -$9 NZ

            Source Skumatz Economic Research Associates research ndash 1998-2005

            -005

            0

            005

            01

            015

            02

            Maint

            Productiv Perf Life

            Op CostTenant S

            atComfor

            t

            Lite Safe SellEnvir

            oOther

            AampE Owner

            Commercial Example

            Residential Example

            SERA

            NEB MEASUREMENT

            8

            SERA

            NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES

            Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip

            9

            Monetized NEBs

            Source Skumatz SERA research

            NEB

            s

            Direct

            Secondary

            Model

            Survey Story of a ferryhellip then it is academic

            Corp Records Utility data

            Change x valueFinancial Calcs

            Third party jobsAnd emissions

            Specialized academic Best for some NEBs

            SERA

            MEASURING PARTICIPANTNEBs Best practices

            WTP not fruitful 7+ better options

            Net Three Net positive amp negative Net beyond standard

            efficiency Net to Gross

            Non-overlapping Consistent units Recommend in ALL

            process (impact) surveys - barriers

            HTM=Hard to measure HTA=Hard to answer

            Direct calc Accurate Small ldquonrdquo

            WTPWTA Accepted Volatile HTA

            Relative Fast strong HTA

            LMSFast

            strong clear ETA

            Unfamiliar

            Logit Conjoint

            rankStrong Slow

            complex

            Regression Defensible Limited data cost

            Market value Strong Data

            Other Exploring Cost tradeoffs

            Source SERA Research

            SERA

            HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)

            11

            Monetized NEBs

            Direct Secondary Model Survey

            Attributable Change (study)

            Value or Financial

            Calc

            X

            Total Attrib Stated Relative

            Effect

            Savings (or translation)(ldquoNormrdquo)

            Individual NEB Shares

            X

            X

            SERA

            PARTICIPANT MEASUREMENT METHODS COMPARISON

            005

            115

            225

            335

            445

            MW

            x

            NE

            Wx

            CA

            Wx

            CA

            Wx

            SF-N

            C

            MF-

            NC

            C-

            Lite

            C-R

            eb

            C-R

            eb

            multLMS VerbalWTP

            Source SERA Research

            Chart1

            mult
            LMS Verbal
            WTP
            122
            156
            42
            08
            1
            2
            098
            4
            06
            13
            12
            075
            067
            104
            115
            27
            075
            055
            32
            084
            091

            Sheet1

            Sheet1

            mult
            LMS Verbal
            WTP

            Sheet2

            Sheet3

            SERA

            PROS AND CONS OF NEBS MEASUREMENT METHODS

            13

            Source Skumatz SERA research

            Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

            Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

            Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

            Long history easy secondary sources Credible to reviewers vetted inputs

            Not available for all NEBs

            Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

            Third party peer-vetted models available for economics jobs and emissions

            Not available for all NEBs

            Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

            Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

            Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

            SERA

            PROGRESS TRANSFERABILITY OF NEBS

            14

            Source Skumatz SERA research

            Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

            bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

            Program measure dependent

            bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

            education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

            Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

            Residential Target dependent (low income or MF vs SF)

            bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

            vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

            Cost savings consistency if transferable vs risk

            SERA

            RANGES TYPICAL NEB VALUES FOR WxRETROFI

            15

            SERA

            PROGRESS AND GAPS Consistency Transferability

            Some programs types others less studied Transferability research

            Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

            measures until then Across the board Savings share Regression

            SERA

            PROGRESS AND GAPS Gas vs Electric

            Some researchndashusually program-wide not broken down International research found participant NEBs have

            similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

            MF Less-commonly-studied complicated by poor response

            and complexity of sector (decision-maker some measures in home some central) separate from low income not common

            Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

            Commercial gaps in some measures

            SERA

            USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

            Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

            18

            0

            005

            01

            015

            02

            025

            Environment

            Quality

            Quantity

            PgmAvailFlicker

            Mainten

            ance

            Lifetime

            SalesP

            roductivity

            Safety

            Noise

            Control

            Illness

            Total NEBs Multiplier=X

            $

            Savings value=$X

            Chart3

            023
            016
            014
            01
            008
            006
            006
            006
            005
            005
            003
            0

            Chart1

            Vendor
            NP
            Partic
            028
            02
            02
            028
            013
            017
            031
            022
            016
            006
            008
            014
            005
            006
            014
            004
            009
            007
            004
            003
            007
            -006
            018
            006

            Sheet1

            Sheet1

            Sheet2

            Sheet3

            SERA

            NEB VALUES

            19

            SERA

            ARE NEBS HIGH VALUE Energy savings are often the minority of

            benefits from program measuresndash Donrsquot ignore that

            20

            0 05 1 15 2 25 3

            Energy Sav

            NEBs

            NEB vs Energy Savings ValueIncluding all NEBs

            Source (SkumatzSERA 2010 amp others)

            Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

            SERA

            RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

            MF Many others

            Thousands of surveys results By measures By program types By sectors By stakeholders By geography

            Variety of end uses

            SERA

            TOP NEBS FOR WEATHERIZATION PROGRAM

            Weatherization program multipliergt1

            Source SERA Research

            SERA

            RESULTS FOR ENERGY STARreg APPLIANCES

            0002004006008

            01012014016018

            Maint

            Appear

            Perf LifeNoise Satis

            Comfort

            LiteSafe

            ty SellMove

            Water

            Enviro

            Refrig DW CW

            Source SERA Research

            SERA

            INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

            5 increments to 50

            Source SERA research

            Chart5

            EnergySmart (insul Retrofit)
            Dunedin (insul)
            001
            024
            002
            004
            002
            005
            006
            005
            007
            008
            008
            006
            013
            0
            013
            016
            049
            033

            cx

            cx

            nz

            Sheet3

            Sheet3

            Super Insulation
            Double Glazing
            Solar Design
            Solar Water Heating
            Super Insulation
            Double Glazing
            Solar Design
            Solar Water Heating
            EnergySmart (insul Retrofit)
            Dunedin (insul)

            SERA

            ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

            0

            002

            004

            006

            008

            01

            012

            Maint

            Perf LifeSatis

            f

            ComfortAesth

            NoiseSafe

            ty

            LightingSelli

            ng

            Stay in home

            Enviro

            Sick Days

            Calls

            ES Homes HP wES Energy savings multiplier=1

            Source SERA Research

            SERA

            CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

            DG CHP HVAC Equipment rebate Other

            Building codes incentives by cities

            Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

            Variety of end uses

            SERA

            AUDIT FINANCIAL PROGRAM

            0

            01

            02

            03

            04

            05

            06

            07

            Variety of measureshellip disaggregatedResults by measure type businessType etchellip

            Source SERA Research

            SERA

            CampI NEW CONSTRUCTION

            -005

            0

            005

            01

            015

            02

            AampE Owner(Pct of Total Partic NEBs)

            Source SERA Research

            SERA

            CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

            0 2 4 6 8 10 12

            m Product losses

            l Ease of selling leasing building

            n Labor requirements

            h Aesthetics appearance

            j Noise

            b Equipment maintenance

            f Tenant satisfaction

            d Equipment lifetimes

            e Productivity

            k Building safety[2]

            g Comfort

            i Lighting Quality of Light

            c Equipment performance[1]

            a Operating costs

            o Doing good for the Environment[3]

            Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

            vcv

            Source SERA Research

            Chart1

            CampI Technical Assistance NEBs
            002
            003
            004
            005
            005
            006
            006
            007
            007
            007
            009
            009
            01
            011
            011

            Sheet1

            Sheet1

            CampI Technical Assistance Program NEBs (Participant)

            Sheet2

            Sheet3

            SERA

            EXPRESSING NEBS VALUEndashCxe

            Strong value from RetroCxSource SERA Research

            SERA

            SOCIETAL NEB EXAMPLES

            31

            SERA

            JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

            32

            0

            02

            04

            06

            08

            1

            12

            HPWxRetrof Appliance

            CA

            WI

            Natl

            (Source Skumatz SERAECEEE 2007 ACEEE 2006)

            Jobs Economic

            SERA

            EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

            33

            SERA

            UTILITY NEBSUtility NEBs for Template Program

            Debt WriteOff (util)13

            Rate Subsidy(util)61

            HealthSafety(util)0

            Colln Costs (util)0

            Gas Calls (util)0

            Calls to CSRs(util)2

            TampD (util)16

            Arrears (util)0

            Reconnects (util)0

            Notices (util)7

            Shutoffs (util)1

            Rate subsidy TampD

            Payment-related

            Source Skumatz Economic Research Associates research

            Example Low Income Weatherization

            SERA

            ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

            be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

            Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

            Value of 100 HALVES Payback doubles ROI

            Source Skumatz Economics (SERA)

            35

            ( for NEBs Multiplier of 125)

            SERA

            NEBS IN COST-EFFECTIVENESS APPLICATIONS

            36

            hellip2001 ON

            SERA

            37

            NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

            For true representation of B amp C elements of NEBs address missing factor bias

            Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

            BC applications were considered early on then pulled back awaiting more quantitative evidence

            SERA

            CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

            38

            Test Benefits Costs States Using Traditionally

            Improved treat-ment with NEBs

            Utility Cost (or Program Administrator Test) (UCT or PAC)

            bull Avoided supply costs for transmission distribution and generation (TDampG)

            bull Avoided gas and water supply costs

            bull Program administration

            bull Participant incentives

            bull Increased supply cost

            CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

            Use cost only paid by the utility

            Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

            Same as above plus bull increased revenue

            Same as above plus bull Decreased revenue

            AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

            Participant cost bull Utility bill reductions bull Participant incentives

            bull Participant direct costs

            AR CA FL HI IA IN MI MN NY VA

            Participant NEBs

            Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

            Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

            SERA

            APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

            Expansion Concern with risk calculation accuracy

            39

            Source Skumatz SERA research

            SERA

            40

            METHODS TO INCLUDE NEBsIN REGULATORY TESTS

            Maximize DSM opportunities amp feedback Accuracy tailoring

            Minimize Regulatory amp Implementer Risk

            Minimize Evaluation Cost

            Adder

            Readily MeasurableHybrid

            All NEBsSource SERA Research

            SERA

            41

            STATE TREATMENT OF NEBS NEBsRegulatory

            Screening Application Utilities regions

            Program Marketing Fairly widespread use in utilities states across the country

            MO

            RE AGGRESSIVE ==gt

            Test Pgm Screen - adder

            IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

            Test Pgm Screen - readily measured

            MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

            Test Hybrid (potential adder amp measured)

            CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

            Test Pgm screen - Broad

            With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

            Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

            SERA

            42

            Key

            States with NEBs in CE tests for At least one type of program

            copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

            STATES WITH NEBS IN CE TESTS

            NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

            groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

            SERA

            SELECTED STATE PROGRESS CA

            2001 ndash LIPPT model NY

            Mid-2000s measurement amp scenarios not included CO

            Adders (10 electric 25 LI 5 gas) VT

            15 adder helped by previous research DC

            10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

            43

            Source Skumatz SERA research

            SERA

            SELECTED STATE PROGRESS Dominos ongoing

            Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

            Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

            Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

            Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

            44

            Source Skumatz SERA research

            SERA

            APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

            Expansion Concern with risk calculation accuracy

            45

            Source Skumatz SERA research

            SERA

            46

            NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

            [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

            SourceSkumatz SERA Independent research

            NTG ndash accuracy measurement incomplete

            Savings Impact repeatedly amp expensivelymeasured little variation $100K+

            NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

            EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

            Cost Complicated expensive local changes

            RELATIVE Risk from NEBs

            Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

            Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

            NEBs as accurate as many inputs (all could improve)

            SERA

            DISCOUNT RATE MATTERS

            47

            Source Skumatz SERA Research

            0

            20

            40

            60

            80

            100

            120

            1 2 3 4 5 6 7 8 9 10

            PV Ratios vs 1 Discount Rate

            10 yr PV 20 yr PV

            Discount rate - percent

            WACCSocialLTT

            Source Skumatz Electricity Journal IEPEC paper independent research

            SERA

            EUL VARIATIONS-ADOPTED VALUES

            Source Skumatz Economic Research Associates SERA independent research

            0

            50

            100

            150

            200

            250

            300

            350

            400

            450

            0 10 20 30 40 50 60 70

            EUL Variation by Measure

            67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

            Poorly supportedOldTechnology change

            SERA

            KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

            Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

            measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

            change the BC conclusion

            On the cusphellip

            If NO Yoursquore doneAnd bias addressedsufficiently

            IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

            Source SERA all rights reserved

            SERArsquos lsquoNEB-ItrdquoModel ndash

            SERA

            IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

            Utility Soc Part Conserv Recrsquom

            Rationale

            Base Percent X X X Program-invariant

            Low Income X X X X Multiple sources

            Weatherization X X X Substantial Participantimpacts

            Measure Program-specific

            X Varies by measure sector

            Other Recomrsquos Local Research

            50

            Source Skumatz SERA

            Developing values for multiple states amp utilities

            SERA

            TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

            Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

            Other sources of bias to research (EUL NTG etc) bring high variation

            5) States are incorporating NEBs dominos input in deliberations in multiple states

            6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

            THANK YOU

            Questions

            Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

            SERA

            POWER OF NEBS ndashBARRIERS DISCONNECTS

            53

            SERA

            NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

            Negative NEB values cost of barrier

            Solar Water Heat NZ$ Euros

            Solar Design NZ$ Euros

            Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

            -23 -12 (079)

            -11 -6 (06)

            Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

            Source Skumatz Economic Research Associates research

            SERA

            CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

            NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

            -5-4-3-2-1012345

            Op Co

            st

            Init C

            ost

            Maint

            Perfo

            rm

            Prod

            uctiv

            Tena

            nt Sa

            t

            Comf

            ort

            Appe

            ar

            Light

            Qual

            SellL

            ease

            Devp Ownr AampE

            Note AgtE (~4 vs ~2) for these NEBs

            Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

            Source Skumatz Economic Research Associates Inc 2002

            Owners more positive ndashUnderinvestment train

            • Soup to nuts on non-energy benefits NEIs
            • TOPICS
            • BACKGROUND NEBS
            • 20+ YEARS OF NEBS PROGRESShellip
            • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
            • KEY APPLICATIONS OF NEBS
            • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
            • Neb measurement
            • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
            • MEASURING PARTICIPANTNEBs
            • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
            • PARTICIPANT MEASUREMENT METHODS COMPARISON
            • PROS AND CONS OF NEBS MEASUREMENT METHODS
            • PROGRESS TRANSFERABILITY OF NEBS
            • RANGES TYPICAL NEB VALUES FOR WxRETROFI
            • PROGRESS AND GAPS
            • PROGRESS AND GAPS
            • USING NEB NEI VALUES
            • Slide Number 19
            • ARE NEBS HIGH VALUE
            • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
            • TOP NEBS FOR WEATHERIZATION PROGRAM
            • RESULTS FOR ENERGY STARreg APPLIANCES
            • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
            • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
            • CampI PROGRAMS ldquoNEBBEDrdquo
            • AUDIT FINANCIAL PROGRAM
            • CampI NEW CONSTRUCTION
            • CampI TECHNICAL ASSISTANCE
            • EXPRESSING NEBS VALUEndashCxe
            • Slide Number 31
            • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
            • EMISSIONS (SOCIETAL NEBS)
            • UTILITY NEBS
            • ADJUSTED PAYBACKS
            • NEBS IN COST-EFFECTIVENESS APPLICATIONS
            • NEBS IN BC
            • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
            • APPLICATION TO BC
            • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
            • STATE TREATMENT OF NEBS NEBs
            • Slide Number 42
            • SELECTED STATE PROGRESS
            • SELECTED STATE PROGRESS
            • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
            • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
            • DISCOUNT RATE MATTERS
            • EUL VARIATIONS- ADOPTED VALUES
            • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
            • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
            • TAKEAWAYSON NEBSNEIS
            • THANK YOUQuestions
            • Slide Number 53
            • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
            • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
              Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
              m Product losses20
              l Ease of selling leasing building33
              n Labor requirements45
              h Aesthetics appearance59
              j Noise50
              b Equipment maintenance660
              f Tenant satisfaction60
              d Equipment lifetimes718
              e Productivity724
              k Building safety[2]73
              g Comfort933
              i Lighting Quality of Light916
              c Equipment performance[1]1019
              a Operating costs1165
              o Doing good for the Environment[3]1132
              Other03
              [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
              [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
              [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
              m Product losses
              l Ease of selling leasing building
              n Labor requirements
              h Aesthetics appearance
              j Noise
              b Equipment maintenance
              f Tenant satisfaction
              d Equipment lifetimes
              e Productivity
              k Building safety[2]
              g Comfort
              i Lighting Quality of Light
              c Equipment performance[1]
              a Operating costs
              o Doing good for the Environment[3]
              Share of ZALEH NEBS from various measures
              Each presented as multiple of the (overall) energy savings
              Super Insulation037
              Double Glazing026
              Solar Design025
              Solar Water Heating005
              EnergySmart (insul Retrofit)Dunedin (insul)
              Other124
              Noise24
              Budgetfinances25
              Environmental65
              Health78
              Reduced moves86
              Ease of sellingrenting130
              DampnessMold1316
              Comfort-related4933
              101101
              Operational NEBs44
              DesignConstruction NEBs31
              Occupant NEBs25
              1
              OtherOther
              NoiseNoise
              BudgetfinancesBudgetfinances
              EnvironmentalEnvironmental
              HealthHealth
              Reduced movesReduced moves
              Ease of sellingrentingEase of sellingrenting
              DampnessMoldDampnessMold
              Comfort-relatedComfort-related
              Boiler
              VendorNPPartic
              Perform0280202
              Footprint028013017
              ControlFeatures031022016
              Tenant compl006008014
              Noise005006014
              Maint004009007
              Other004003007
              Lifetime-006018006
              MF New construction energy star
              Operating cost014
              Environment012
              Maintenance01
              Lifetime01
              Performance009
              Occupant Satisf009
              Comfort008
              Noise006
              Sellingleasing006
              Aesthetics005
              Light Quality005
              Power QualReliab004
              Safety002
              RIGHTLIGHTS
              Environment023
              Quality016
              Quantity014
              PgmAvail01
              Flicker008
              Maintenance006
              Lifetime006
              SalesProductivity006
              Safety005
              Noise005
              Control003
              Illness0
              PerformPerformPerform
              FootprintFootprintFootprint
              ControlFeaturesControlFeaturesControlFeatures
              Tenant complTenant complTenant compl
              NoiseNoiseNoise
              MaintMaintMaint
              OtherOtherOther
              LifetimeLifetimeLifetime
              Environment
              Quality
              Quantity
              PgmAvail
              Flicker
              Maintenance
              Lifetime
              SalesProductivity
              Safety
              Noise
              Control
              Illness

              Variability

              Relevant NEB Categories

              Program measure invariant (suitable for ldquoadderrdquo)

              middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

              Program measure dependent

              middot Economic ndash societal (depends on measures and local manufacture installation)

              middot Health and safety health care illnesses ndash societal and participant (measure)

              middot Water wastewater infrastructure and water bill savings ndash societal and participant

              middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

              Climate dependent

              middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

              Residential Target dependent (low income or MF vs SF)

              middot Payment related ndash utility (arrearages etc stronger for low income targets)

              middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

              middot Participant benefits related to hardship and payments

              middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

              Used for

              Major Advantages

              Major Disadvantages

              Direct measurement (specific studies of changes on-site)

              Comrsquol labor productivity etc

              Direct precise attributable

              Small estimation sample sizes specialized cases poor transferability expensive

              Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

              Insurance water health others

              Long history easy secondary sources

              Credible to reviewers vetted inputs

              Not available for all NEBs

              Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

              Emissions economics

              Third party peer-vetted models available for economics jobs and emissions

              Not available for all NEBs

              Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

              Wide variety of Participant NEBs

              Large sample sizes amp statistical properties

              Affordable

              Multiple estimates leading to similar ranges

              Direct method of measuring some key NEBs

              Concerns about surveys as a source of quantitative values amp reliability

              Recall from survey respondents

              Proper attribution to programs measures

              MWxNEWxCA WxCA WxSF-NCMF-NCC- LiteC-RebC-Reb
              mult1220813075104075084
              LMS Verbal15610980612067115055091
              WTP42242732
              MWxMWxMWx
              NEWxNEWxNEWx
              CA WxCA WxCA Wx
              CA WxCA WxCA Wx
              SF-NCSF-NCSF-NC
              MF-NCMF-NCMF-NC
              C- LiteC- LiteC- Lite
              C-RebC-RebC-Reb
              C-RebC-RebC-Reb

              SERA

              USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS

              NegativeNEBs

              Solar WH

              Appearance -$14 NZ

              Maintenance -$9 NZ

              Source Skumatz Economic Research Associates research ndash 1998-2005

              -005

              0

              005

              01

              015

              02

              Maint

              Productiv Perf Life

              Op CostTenant S

              atComfor

              t

              Lite Safe SellEnvir

              oOther

              AampE Owner

              Commercial Example

              Residential Example

              SERA

              NEB MEASUREMENT

              8

              SERA

              NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES

              Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip

              9

              Monetized NEBs

              Source Skumatz SERA research

              NEB

              s

              Direct

              Secondary

              Model

              Survey Story of a ferryhellip then it is academic

              Corp Records Utility data

              Change x valueFinancial Calcs

              Third party jobsAnd emissions

              Specialized academic Best for some NEBs

              SERA

              MEASURING PARTICIPANTNEBs Best practices

              WTP not fruitful 7+ better options

              Net Three Net positive amp negative Net beyond standard

              efficiency Net to Gross

              Non-overlapping Consistent units Recommend in ALL

              process (impact) surveys - barriers

              HTM=Hard to measure HTA=Hard to answer

              Direct calc Accurate Small ldquonrdquo

              WTPWTA Accepted Volatile HTA

              Relative Fast strong HTA

              LMSFast

              strong clear ETA

              Unfamiliar

              Logit Conjoint

              rankStrong Slow

              complex

              Regression Defensible Limited data cost

              Market value Strong Data

              Other Exploring Cost tradeoffs

              Source SERA Research

              SERA

              HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)

              11

              Monetized NEBs

              Direct Secondary Model Survey

              Attributable Change (study)

              Value or Financial

              Calc

              X

              Total Attrib Stated Relative

              Effect

              Savings (or translation)(ldquoNormrdquo)

              Individual NEB Shares

              X

              X

              SERA

              PARTICIPANT MEASUREMENT METHODS COMPARISON

              005

              115

              225

              335

              445

              MW

              x

              NE

              Wx

              CA

              Wx

              CA

              Wx

              SF-N

              C

              MF-

              NC

              C-

              Lite

              C-R

              eb

              C-R

              eb

              multLMS VerbalWTP

              Source SERA Research

              Chart1

              mult
              LMS Verbal
              WTP
              122
              156
              42
              08
              1
              2
              098
              4
              06
              13
              12
              075
              067
              104
              115
              27
              075
              055
              32
              084
              091

              Sheet1

              Sheet1

              mult
              LMS Verbal
              WTP

              Sheet2

              Sheet3

              SERA

              PROS AND CONS OF NEBS MEASUREMENT METHODS

              13

              Source Skumatz SERA research

              Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

              Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

              Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

              Long history easy secondary sources Credible to reviewers vetted inputs

              Not available for all NEBs

              Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

              Third party peer-vetted models available for economics jobs and emissions

              Not available for all NEBs

              Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

              Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

              Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

              SERA

              PROGRESS TRANSFERABILITY OF NEBS

              14

              Source Skumatz SERA research

              Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

              bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

              Program measure dependent

              bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

              education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

              Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

              Residential Target dependent (low income or MF vs SF)

              bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

              vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

              Cost savings consistency if transferable vs risk

              SERA

              RANGES TYPICAL NEB VALUES FOR WxRETROFI

              15

              SERA

              PROGRESS AND GAPS Consistency Transferability

              Some programs types others less studied Transferability research

              Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

              measures until then Across the board Savings share Regression

              SERA

              PROGRESS AND GAPS Gas vs Electric

              Some researchndashusually program-wide not broken down International research found participant NEBs have

              similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

              MF Less-commonly-studied complicated by poor response

              and complexity of sector (decision-maker some measures in home some central) separate from low income not common

              Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

              Commercial gaps in some measures

              SERA

              USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

              Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

              18

              0

              005

              01

              015

              02

              025

              Environment

              Quality

              Quantity

              PgmAvailFlicker

              Mainten

              ance

              Lifetime

              SalesP

              roductivity

              Safety

              Noise

              Control

              Illness

              Total NEBs Multiplier=X

              $

              Savings value=$X

              Chart3

              023
              016
              014
              01
              008
              006
              006
              006
              005
              005
              003
              0

              Chart1

              Vendor
              NP
              Partic
              028
              02
              02
              028
              013
              017
              031
              022
              016
              006
              008
              014
              005
              006
              014
              004
              009
              007
              004
              003
              007
              -006
              018
              006

              Sheet1

              Sheet1

              Sheet2

              Sheet3

              SERA

              NEB VALUES

              19

              SERA

              ARE NEBS HIGH VALUE Energy savings are often the minority of

              benefits from program measuresndash Donrsquot ignore that

              20

              0 05 1 15 2 25 3

              Energy Sav

              NEBs

              NEB vs Energy Savings ValueIncluding all NEBs

              Source (SkumatzSERA 2010 amp others)

              Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

              SERA

              RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

              MF Many others

              Thousands of surveys results By measures By program types By sectors By stakeholders By geography

              Variety of end uses

              SERA

              TOP NEBS FOR WEATHERIZATION PROGRAM

              Weatherization program multipliergt1

              Source SERA Research

              SERA

              RESULTS FOR ENERGY STARreg APPLIANCES

              0002004006008

              01012014016018

              Maint

              Appear

              Perf LifeNoise Satis

              Comfort

              LiteSafe

              ty SellMove

              Water

              Enviro

              Refrig DW CW

              Source SERA Research

              SERA

              INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

              5 increments to 50

              Source SERA research

              Chart5

              EnergySmart (insul Retrofit)
              Dunedin (insul)
              001
              024
              002
              004
              002
              005
              006
              005
              007
              008
              008
              006
              013
              0
              013
              016
              049
              033

              cx

              cx

              nz

              Sheet3

              Sheet3

              Super Insulation
              Double Glazing
              Solar Design
              Solar Water Heating
              Super Insulation
              Double Glazing
              Solar Design
              Solar Water Heating
              EnergySmart (insul Retrofit)
              Dunedin (insul)

              SERA

              ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

              0

              002

              004

              006

              008

              01

              012

              Maint

              Perf LifeSatis

              f

              ComfortAesth

              NoiseSafe

              ty

              LightingSelli

              ng

              Stay in home

              Enviro

              Sick Days

              Calls

              ES Homes HP wES Energy savings multiplier=1

              Source SERA Research

              SERA

              CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

              DG CHP HVAC Equipment rebate Other

              Building codes incentives by cities

              Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

              Variety of end uses

              SERA

              AUDIT FINANCIAL PROGRAM

              0

              01

              02

              03

              04

              05

              06

              07

              Variety of measureshellip disaggregatedResults by measure type businessType etchellip

              Source SERA Research

              SERA

              CampI NEW CONSTRUCTION

              -005

              0

              005

              01

              015

              02

              AampE Owner(Pct of Total Partic NEBs)

              Source SERA Research

              SERA

              CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

              0 2 4 6 8 10 12

              m Product losses

              l Ease of selling leasing building

              n Labor requirements

              h Aesthetics appearance

              j Noise

              b Equipment maintenance

              f Tenant satisfaction

              d Equipment lifetimes

              e Productivity

              k Building safety[2]

              g Comfort

              i Lighting Quality of Light

              c Equipment performance[1]

              a Operating costs

              o Doing good for the Environment[3]

              Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

              vcv

              Source SERA Research

              Chart1

              CampI Technical Assistance NEBs
              002
              003
              004
              005
              005
              006
              006
              007
              007
              007
              009
              009
              01
              011
              011

              Sheet1

              Sheet1

              CampI Technical Assistance Program NEBs (Participant)

              Sheet2

              Sheet3

              SERA

              EXPRESSING NEBS VALUEndashCxe

              Strong value from RetroCxSource SERA Research

              SERA

              SOCIETAL NEB EXAMPLES

              31

              SERA

              JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

              32

              0

              02

              04

              06

              08

              1

              12

              HPWxRetrof Appliance

              CA

              WI

              Natl

              (Source Skumatz SERAECEEE 2007 ACEEE 2006)

              Jobs Economic

              SERA

              EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

              33

              SERA

              UTILITY NEBSUtility NEBs for Template Program

              Debt WriteOff (util)13

              Rate Subsidy(util)61

              HealthSafety(util)0

              Colln Costs (util)0

              Gas Calls (util)0

              Calls to CSRs(util)2

              TampD (util)16

              Arrears (util)0

              Reconnects (util)0

              Notices (util)7

              Shutoffs (util)1

              Rate subsidy TampD

              Payment-related

              Source Skumatz Economic Research Associates research

              Example Low Income Weatherization

              SERA

              ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

              be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

              Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

              Value of 100 HALVES Payback doubles ROI

              Source Skumatz Economics (SERA)

              35

              ( for NEBs Multiplier of 125)

              SERA

              NEBS IN COST-EFFECTIVENESS APPLICATIONS

              36

              hellip2001 ON

              SERA

              37

              NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

              For true representation of B amp C elements of NEBs address missing factor bias

              Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

              BC applications were considered early on then pulled back awaiting more quantitative evidence

              SERA

              CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

              38

              Test Benefits Costs States Using Traditionally

              Improved treat-ment with NEBs

              Utility Cost (or Program Administrator Test) (UCT or PAC)

              bull Avoided supply costs for transmission distribution and generation (TDampG)

              bull Avoided gas and water supply costs

              bull Program administration

              bull Participant incentives

              bull Increased supply cost

              CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

              Use cost only paid by the utility

              Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

              Same as above plus bull increased revenue

              Same as above plus bull Decreased revenue

              AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

              Participant cost bull Utility bill reductions bull Participant incentives

              bull Participant direct costs

              AR CA FL HI IA IN MI MN NY VA

              Participant NEBs

              Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

              Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

              SERA

              APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

              Expansion Concern with risk calculation accuracy

              39

              Source Skumatz SERA research

              SERA

              40

              METHODS TO INCLUDE NEBsIN REGULATORY TESTS

              Maximize DSM opportunities amp feedback Accuracy tailoring

              Minimize Regulatory amp Implementer Risk

              Minimize Evaluation Cost

              Adder

              Readily MeasurableHybrid

              All NEBsSource SERA Research

              SERA

              41

              STATE TREATMENT OF NEBS NEBsRegulatory

              Screening Application Utilities regions

              Program Marketing Fairly widespread use in utilities states across the country

              MO

              RE AGGRESSIVE ==gt

              Test Pgm Screen - adder

              IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

              Test Pgm Screen - readily measured

              MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

              Test Hybrid (potential adder amp measured)

              CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

              Test Pgm screen - Broad

              With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

              Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

              SERA

              42

              Key

              States with NEBs in CE tests for At least one type of program

              copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

              STATES WITH NEBS IN CE TESTS

              NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

              groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

              SERA

              SELECTED STATE PROGRESS CA

              2001 ndash LIPPT model NY

              Mid-2000s measurement amp scenarios not included CO

              Adders (10 electric 25 LI 5 gas) VT

              15 adder helped by previous research DC

              10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

              43

              Source Skumatz SERA research

              SERA

              SELECTED STATE PROGRESS Dominos ongoing

              Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

              Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

              Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

              Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

              44

              Source Skumatz SERA research

              SERA

              APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

              Expansion Concern with risk calculation accuracy

              45

              Source Skumatz SERA research

              SERA

              46

              NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

              [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

              SourceSkumatz SERA Independent research

              NTG ndash accuracy measurement incomplete

              Savings Impact repeatedly amp expensivelymeasured little variation $100K+

              NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

              EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

              Cost Complicated expensive local changes

              RELATIVE Risk from NEBs

              Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

              Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

              NEBs as accurate as many inputs (all could improve)

              SERA

              DISCOUNT RATE MATTERS

              47

              Source Skumatz SERA Research

              0

              20

              40

              60

              80

              100

              120

              1 2 3 4 5 6 7 8 9 10

              PV Ratios vs 1 Discount Rate

              10 yr PV 20 yr PV

              Discount rate - percent

              WACCSocialLTT

              Source Skumatz Electricity Journal IEPEC paper independent research

              SERA

              EUL VARIATIONS-ADOPTED VALUES

              Source Skumatz Economic Research Associates SERA independent research

              0

              50

              100

              150

              200

              250

              300

              350

              400

              450

              0 10 20 30 40 50 60 70

              EUL Variation by Measure

              67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

              Poorly supportedOldTechnology change

              SERA

              KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

              Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

              measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

              change the BC conclusion

              On the cusphellip

              If NO Yoursquore doneAnd bias addressedsufficiently

              IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

              Source SERA all rights reserved

              SERArsquos lsquoNEB-ItrdquoModel ndash

              SERA

              IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

              Utility Soc Part Conserv Recrsquom

              Rationale

              Base Percent X X X Program-invariant

              Low Income X X X X Multiple sources

              Weatherization X X X Substantial Participantimpacts

              Measure Program-specific

              X Varies by measure sector

              Other Recomrsquos Local Research

              50

              Source Skumatz SERA

              Developing values for multiple states amp utilities

              SERA

              TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

              Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

              Other sources of bias to research (EUL NTG etc) bring high variation

              5) States are incorporating NEBs dominos input in deliberations in multiple states

              6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

              THANK YOU

              Questions

              Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

              SERA

              POWER OF NEBS ndashBARRIERS DISCONNECTS

              53

              SERA

              NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

              Negative NEB values cost of barrier

              Solar Water Heat NZ$ Euros

              Solar Design NZ$ Euros

              Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

              -23 -12 (079)

              -11 -6 (06)

              Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

              Source Skumatz Economic Research Associates research

              SERA

              CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

              NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

              -5-4-3-2-1012345

              Op Co

              st

              Init C

              ost

              Maint

              Perfo

              rm

              Prod

              uctiv

              Tena

              nt Sa

              t

              Comf

              ort

              Appe

              ar

              Light

              Qual

              SellL

              ease

              Devp Ownr AampE

              Note AgtE (~4 vs ~2) for these NEBs

              Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

              Source Skumatz Economic Research Associates Inc 2002

              Owners more positive ndashUnderinvestment train

              • Soup to nuts on non-energy benefits NEIs
              • TOPICS
              • BACKGROUND NEBS
              • 20+ YEARS OF NEBS PROGRESShellip
              • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
              • KEY APPLICATIONS OF NEBS
              • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
              • Neb measurement
              • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
              • MEASURING PARTICIPANTNEBs
              • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
              • PARTICIPANT MEASUREMENT METHODS COMPARISON
              • PROS AND CONS OF NEBS MEASUREMENT METHODS
              • PROGRESS TRANSFERABILITY OF NEBS
              • RANGES TYPICAL NEB VALUES FOR WxRETROFI
              • PROGRESS AND GAPS
              • PROGRESS AND GAPS
              • USING NEB NEI VALUES
              • Slide Number 19
              • ARE NEBS HIGH VALUE
              • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
              • TOP NEBS FOR WEATHERIZATION PROGRAM
              • RESULTS FOR ENERGY STARreg APPLIANCES
              • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
              • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
              • CampI PROGRAMS ldquoNEBBEDrdquo
              • AUDIT FINANCIAL PROGRAM
              • CampI NEW CONSTRUCTION
              • CampI TECHNICAL ASSISTANCE
              • EXPRESSING NEBS VALUEndashCxe
              • Slide Number 31
              • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
              • EMISSIONS (SOCIETAL NEBS)
              • UTILITY NEBS
              • ADJUSTED PAYBACKS
              • NEBS IN COST-EFFECTIVENESS APPLICATIONS
              • NEBS IN BC
              • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
              • APPLICATION TO BC
              • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
              • STATE TREATMENT OF NEBS NEBs
              • Slide Number 42
              • SELECTED STATE PROGRESS
              • SELECTED STATE PROGRESS
              • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
              • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
              • DISCOUNT RATE MATTERS
              • EUL VARIATIONS- ADOPTED VALUES
              • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
              • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
              • TAKEAWAYSON NEBSNEIS
              • THANK YOUQuestions
              • Slide Number 53
              • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
              • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                m Product losses20
                l Ease of selling leasing building33
                n Labor requirements45
                h Aesthetics appearance59
                j Noise50
                b Equipment maintenance660
                f Tenant satisfaction60
                d Equipment lifetimes718
                e Productivity724
                k Building safety[2]73
                g Comfort933
                i Lighting Quality of Light916
                c Equipment performance[1]1019
                a Operating costs1165
                o Doing good for the Environment[3]1132
                Other03
                [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                m Product losses
                l Ease of selling leasing building
                n Labor requirements
                h Aesthetics appearance
                j Noise
                b Equipment maintenance
                f Tenant satisfaction
                d Equipment lifetimes
                e Productivity
                k Building safety[2]
                g Comfort
                i Lighting Quality of Light
                c Equipment performance[1]
                a Operating costs
                o Doing good for the Environment[3]
                Share of ZALEH NEBS from various measures
                Each presented as multiple of the (overall) energy savings
                Super Insulation037
                Double Glazing026
                Solar Design025
                Solar Water Heating005
                EnergySmart (insul Retrofit)Dunedin (insul)
                Other124
                Noise24
                Budgetfinances25
                Environmental65
                Health78
                Reduced moves86
                Ease of sellingrenting130
                DampnessMold1316
                Comfort-related4933
                101101
                Operational NEBs44
                DesignConstruction NEBs31
                Occupant NEBs25
                1
                OtherOther
                NoiseNoise
                BudgetfinancesBudgetfinances
                EnvironmentalEnvironmental
                HealthHealth
                Reduced movesReduced moves
                Ease of sellingrentingEase of sellingrenting
                DampnessMoldDampnessMold
                Comfort-relatedComfort-related
                Boiler
                VendorNPPartic
                Perform0280202
                Footprint028013017
                ControlFeatures031022016
                Tenant compl006008014
                Noise005006014
                Maint004009007
                Other004003007
                Lifetime-006018006
                MF New construction energy star
                Operating cost014
                Environment012
                Maintenance01
                Lifetime01
                Performance009
                Occupant Satisf009
                Comfort008
                Noise006
                Sellingleasing006
                Aesthetics005
                Light Quality005
                Power QualReliab004
                Safety002
                RIGHTLIGHTS
                Environment023
                Quality016
                Quantity014
                PgmAvail01
                Flicker008
                Maintenance006
                Lifetime006
                SalesProductivity006
                Safety005
                Noise005
                Control003
                Illness0
                PerformPerformPerform
                FootprintFootprintFootprint
                ControlFeaturesControlFeaturesControlFeatures
                Tenant complTenant complTenant compl
                NoiseNoiseNoise
                MaintMaintMaint
                OtherOtherOther
                LifetimeLifetimeLifetime
                Environment
                Quality
                Quantity
                PgmAvail
                Flicker
                Maintenance
                Lifetime
                SalesProductivity
                Safety
                Noise
                Control
                Illness

                Variability

                Relevant NEB Categories

                Program measure invariant (suitable for ldquoadderrdquo)

                middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                Program measure dependent

                middot Economic ndash societal (depends on measures and local manufacture installation)

                middot Health and safety health care illnesses ndash societal and participant (measure)

                middot Water wastewater infrastructure and water bill savings ndash societal and participant

                middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                Climate dependent

                middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                Residential Target dependent (low income or MF vs SF)

                middot Payment related ndash utility (arrearages etc stronger for low income targets)

                middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

                middot Participant benefits related to hardship and payments

                middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                Used for

                Major Advantages

                Major Disadvantages

                Direct measurement (specific studies of changes on-site)

                Comrsquol labor productivity etc

                Direct precise attributable

                Small estimation sample sizes specialized cases poor transferability expensive

                Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

                Insurance water health others

                Long history easy secondary sources

                Credible to reviewers vetted inputs

                Not available for all NEBs

                Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

                Emissions economics

                Third party peer-vetted models available for economics jobs and emissions

                Not available for all NEBs

                Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

                Wide variety of Participant NEBs

                Large sample sizes amp statistical properties

                Affordable

                Multiple estimates leading to similar ranges

                Direct method of measuring some key NEBs

                Concerns about surveys as a source of quantitative values amp reliability

                Recall from survey respondents

                Proper attribution to programs measures

                MWxNEWxCA WxCA WxSF-NCMF-NCC- LiteC-RebC-Reb
                mult1220813075104075084
                LMS Verbal15610980612067115055091
                WTP42242732
                MWxMWxMWx
                NEWxNEWxNEWx
                CA WxCA WxCA Wx
                CA WxCA WxCA Wx
                SF-NCSF-NCSF-NC
                MF-NCMF-NCMF-NC
                C- LiteC- LiteC- Lite
                C-RebC-RebC-Reb
                C-RebC-RebC-Reb

                SERA

                NEB MEASUREMENT

                8

                SERA

                NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES

                Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip

                9

                Monetized NEBs

                Source Skumatz SERA research

                NEB

                s

                Direct

                Secondary

                Model

                Survey Story of a ferryhellip then it is academic

                Corp Records Utility data

                Change x valueFinancial Calcs

                Third party jobsAnd emissions

                Specialized academic Best for some NEBs

                SERA

                MEASURING PARTICIPANTNEBs Best practices

                WTP not fruitful 7+ better options

                Net Three Net positive amp negative Net beyond standard

                efficiency Net to Gross

                Non-overlapping Consistent units Recommend in ALL

                process (impact) surveys - barriers

                HTM=Hard to measure HTA=Hard to answer

                Direct calc Accurate Small ldquonrdquo

                WTPWTA Accepted Volatile HTA

                Relative Fast strong HTA

                LMSFast

                strong clear ETA

                Unfamiliar

                Logit Conjoint

                rankStrong Slow

                complex

                Regression Defensible Limited data cost

                Market value Strong Data

                Other Exploring Cost tradeoffs

                Source SERA Research

                SERA

                HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)

                11

                Monetized NEBs

                Direct Secondary Model Survey

                Attributable Change (study)

                Value or Financial

                Calc

                X

                Total Attrib Stated Relative

                Effect

                Savings (or translation)(ldquoNormrdquo)

                Individual NEB Shares

                X

                X

                SERA

                PARTICIPANT MEASUREMENT METHODS COMPARISON

                005

                115

                225

                335

                445

                MW

                x

                NE

                Wx

                CA

                Wx

                CA

                Wx

                SF-N

                C

                MF-

                NC

                C-

                Lite

                C-R

                eb

                C-R

                eb

                multLMS VerbalWTP

                Source SERA Research

                Chart1

                mult
                LMS Verbal
                WTP
                122
                156
                42
                08
                1
                2
                098
                4
                06
                13
                12
                075
                067
                104
                115
                27
                075
                055
                32
                084
                091

                Sheet1

                Sheet1

                mult
                LMS Verbal
                WTP

                Sheet2

                Sheet3

                SERA

                PROS AND CONS OF NEBS MEASUREMENT METHODS

                13

                Source Skumatz SERA research

                Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

                Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

                Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

                Long history easy secondary sources Credible to reviewers vetted inputs

                Not available for all NEBs

                Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

                Third party peer-vetted models available for economics jobs and emissions

                Not available for all NEBs

                Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

                Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

                Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

                SERA

                PROGRESS TRANSFERABILITY OF NEBS

                14

                Source Skumatz SERA research

                Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

                bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                Program measure dependent

                bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

                education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                Residential Target dependent (low income or MF vs SF)

                bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

                vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                Cost savings consistency if transferable vs risk

                SERA

                RANGES TYPICAL NEB VALUES FOR WxRETROFI

                15

                SERA

                PROGRESS AND GAPS Consistency Transferability

                Some programs types others less studied Transferability research

                Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

                measures until then Across the board Savings share Regression

                SERA

                PROGRESS AND GAPS Gas vs Electric

                Some researchndashusually program-wide not broken down International research found participant NEBs have

                similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

                MF Less-commonly-studied complicated by poor response

                and complexity of sector (decision-maker some measures in home some central) separate from low income not common

                Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

                Commercial gaps in some measures

                SERA

                USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                18

                0

                005

                01

                015

                02

                025

                Environment

                Quality

                Quantity

                PgmAvailFlicker

                Mainten

                ance

                Lifetime

                SalesP

                roductivity

                Safety

                Noise

                Control

                Illness

                Total NEBs Multiplier=X

                $

                Savings value=$X

                Chart3

                023
                016
                014
                01
                008
                006
                006
                006
                005
                005
                003
                0

                Chart1

                Vendor
                NP
                Partic
                028
                02
                02
                028
                013
                017
                031
                022
                016
                006
                008
                014
                005
                006
                014
                004
                009
                007
                004
                003
                007
                -006
                018
                006

                Sheet1

                Sheet1

                Sheet2

                Sheet3

                SERA

                NEB VALUES

                19

                SERA

                ARE NEBS HIGH VALUE Energy savings are often the minority of

                benefits from program measuresndash Donrsquot ignore that

                20

                0 05 1 15 2 25 3

                Energy Sav

                NEBs

                NEB vs Energy Savings ValueIncluding all NEBs

                Source (SkumatzSERA 2010 amp others)

                Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                SERA

                RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                MF Many others

                Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                Variety of end uses

                SERA

                TOP NEBS FOR WEATHERIZATION PROGRAM

                Weatherization program multipliergt1

                Source SERA Research

                SERA

                RESULTS FOR ENERGY STARreg APPLIANCES

                0002004006008

                01012014016018

                Maint

                Appear

                Perf LifeNoise Satis

                Comfort

                LiteSafe

                ty SellMove

                Water

                Enviro

                Refrig DW CW

                Source SERA Research

                SERA

                INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                5 increments to 50

                Source SERA research

                Chart5

                EnergySmart (insul Retrofit)
                Dunedin (insul)
                001
                024
                002
                004
                002
                005
                006
                005
                007
                008
                008
                006
                013
                0
                013
                016
                049
                033

                cx

                cx

                nz

                Sheet3

                Sheet3

                Super Insulation
                Double Glazing
                Solar Design
                Solar Water Heating
                Super Insulation
                Double Glazing
                Solar Design
                Solar Water Heating
                EnergySmart (insul Retrofit)
                Dunedin (insul)

                SERA

                ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                0

                002

                004

                006

                008

                01

                012

                Maint

                Perf LifeSatis

                f

                ComfortAesth

                NoiseSafe

                ty

                LightingSelli

                ng

                Stay in home

                Enviro

                Sick Days

                Calls

                ES Homes HP wES Energy savings multiplier=1

                Source SERA Research

                SERA

                CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                DG CHP HVAC Equipment rebate Other

                Building codes incentives by cities

                Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                Variety of end uses

                SERA

                AUDIT FINANCIAL PROGRAM

                0

                01

                02

                03

                04

                05

                06

                07

                Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                Source SERA Research

                SERA

                CampI NEW CONSTRUCTION

                -005

                0

                005

                01

                015

                02

                AampE Owner(Pct of Total Partic NEBs)

                Source SERA Research

                SERA

                CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                0 2 4 6 8 10 12

                m Product losses

                l Ease of selling leasing building

                n Labor requirements

                h Aesthetics appearance

                j Noise

                b Equipment maintenance

                f Tenant satisfaction

                d Equipment lifetimes

                e Productivity

                k Building safety[2]

                g Comfort

                i Lighting Quality of Light

                c Equipment performance[1]

                a Operating costs

                o Doing good for the Environment[3]

                Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                vcv

                Source SERA Research

                Chart1

                CampI Technical Assistance NEBs
                002
                003
                004
                005
                005
                006
                006
                007
                007
                007
                009
                009
                01
                011
                011

                Sheet1

                Sheet1

                CampI Technical Assistance Program NEBs (Participant)

                Sheet2

                Sheet3

                SERA

                EXPRESSING NEBS VALUEndashCxe

                Strong value from RetroCxSource SERA Research

                SERA

                SOCIETAL NEB EXAMPLES

                31

                SERA

                JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                32

                0

                02

                04

                06

                08

                1

                12

                HPWxRetrof Appliance

                CA

                WI

                Natl

                (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                Jobs Economic

                SERA

                EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                33

                SERA

                UTILITY NEBSUtility NEBs for Template Program

                Debt WriteOff (util)13

                Rate Subsidy(util)61

                HealthSafety(util)0

                Colln Costs (util)0

                Gas Calls (util)0

                Calls to CSRs(util)2

                TampD (util)16

                Arrears (util)0

                Reconnects (util)0

                Notices (util)7

                Shutoffs (util)1

                Rate subsidy TampD

                Payment-related

                Source Skumatz Economic Research Associates research

                Example Low Income Weatherization

                SERA

                ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                Value of 100 HALVES Payback doubles ROI

                Source Skumatz Economics (SERA)

                35

                ( for NEBs Multiplier of 125)

                SERA

                NEBS IN COST-EFFECTIVENESS APPLICATIONS

                36

                hellip2001 ON

                SERA

                37

                NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                For true representation of B amp C elements of NEBs address missing factor bias

                Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                BC applications were considered early on then pulled back awaiting more quantitative evidence

                SERA

                CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                38

                Test Benefits Costs States Using Traditionally

                Improved treat-ment with NEBs

                Utility Cost (or Program Administrator Test) (UCT or PAC)

                bull Avoided supply costs for transmission distribution and generation (TDampG)

                bull Avoided gas and water supply costs

                bull Program administration

                bull Participant incentives

                bull Increased supply cost

                CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                Use cost only paid by the utility

                Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                Same as above plus bull increased revenue

                Same as above plus bull Decreased revenue

                AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                Participant cost bull Utility bill reductions bull Participant incentives

                bull Participant direct costs

                AR CA FL HI IA IN MI MN NY VA

                Participant NEBs

                Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                SERA

                APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                Expansion Concern with risk calculation accuracy

                39

                Source Skumatz SERA research

                SERA

                40

                METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                Maximize DSM opportunities amp feedback Accuracy tailoring

                Minimize Regulatory amp Implementer Risk

                Minimize Evaluation Cost

                Adder

                Readily MeasurableHybrid

                All NEBsSource SERA Research

                SERA

                41

                STATE TREATMENT OF NEBS NEBsRegulatory

                Screening Application Utilities regions

                Program Marketing Fairly widespread use in utilities states across the country

                MO

                RE AGGRESSIVE ==gt

                Test Pgm Screen - adder

                IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                Test Pgm Screen - readily measured

                MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                Test Hybrid (potential adder amp measured)

                CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                Test Pgm screen - Broad

                With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                SERA

                42

                Key

                States with NEBs in CE tests for At least one type of program

                copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                STATES WITH NEBS IN CE TESTS

                NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                SERA

                SELECTED STATE PROGRESS CA

                2001 ndash LIPPT model NY

                Mid-2000s measurement amp scenarios not included CO

                Adders (10 electric 25 LI 5 gas) VT

                15 adder helped by previous research DC

                10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                43

                Source Skumatz SERA research

                SERA

                SELECTED STATE PROGRESS Dominos ongoing

                Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                44

                Source Skumatz SERA research

                SERA

                APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                Expansion Concern with risk calculation accuracy

                45

                Source Skumatz SERA research

                SERA

                46

                NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                SourceSkumatz SERA Independent research

                NTG ndash accuracy measurement incomplete

                Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                Cost Complicated expensive local changes

                RELATIVE Risk from NEBs

                Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                NEBs as accurate as many inputs (all could improve)

                SERA

                DISCOUNT RATE MATTERS

                47

                Source Skumatz SERA Research

                0

                20

                40

                60

                80

                100

                120

                1 2 3 4 5 6 7 8 9 10

                PV Ratios vs 1 Discount Rate

                10 yr PV 20 yr PV

                Discount rate - percent

                WACCSocialLTT

                Source Skumatz Electricity Journal IEPEC paper independent research

                SERA

                EUL VARIATIONS-ADOPTED VALUES

                Source Skumatz Economic Research Associates SERA independent research

                0

                50

                100

                150

                200

                250

                300

                350

                400

                450

                0 10 20 30 40 50 60 70

                EUL Variation by Measure

                67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                Poorly supportedOldTechnology change

                SERA

                KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                change the BC conclusion

                On the cusphellip

                If NO Yoursquore doneAnd bias addressedsufficiently

                IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                Source SERA all rights reserved

                SERArsquos lsquoNEB-ItrdquoModel ndash

                SERA

                IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                Utility Soc Part Conserv Recrsquom

                Rationale

                Base Percent X X X Program-invariant

                Low Income X X X X Multiple sources

                Weatherization X X X Substantial Participantimpacts

                Measure Program-specific

                X Varies by measure sector

                Other Recomrsquos Local Research

                50

                Source Skumatz SERA

                Developing values for multiple states amp utilities

                SERA

                TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                Other sources of bias to research (EUL NTG etc) bring high variation

                5) States are incorporating NEBs dominos input in deliberations in multiple states

                6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                THANK YOU

                Questions

                Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                SERA

                POWER OF NEBS ndashBARRIERS DISCONNECTS

                53

                SERA

                NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                Negative NEB values cost of barrier

                Solar Water Heat NZ$ Euros

                Solar Design NZ$ Euros

                Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                -23 -12 (079)

                -11 -6 (06)

                Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                Source Skumatz Economic Research Associates research

                SERA

                CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                -5-4-3-2-1012345

                Op Co

                st

                Init C

                ost

                Maint

                Perfo

                rm

                Prod

                uctiv

                Tena

                nt Sa

                t

                Comf

                ort

                Appe

                ar

                Light

                Qual

                SellL

                ease

                Devp Ownr AampE

                Note AgtE (~4 vs ~2) for these NEBs

                Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                Source Skumatz Economic Research Associates Inc 2002

                Owners more positive ndashUnderinvestment train

                • Soup to nuts on non-energy benefits NEIs
                • TOPICS
                • BACKGROUND NEBS
                • 20+ YEARS OF NEBS PROGRESShellip
                • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                • KEY APPLICATIONS OF NEBS
                • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                • Neb measurement
                • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                • MEASURING PARTICIPANTNEBs
                • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                • PARTICIPANT MEASUREMENT METHODS COMPARISON
                • PROS AND CONS OF NEBS MEASUREMENT METHODS
                • PROGRESS TRANSFERABILITY OF NEBS
                • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                • PROGRESS AND GAPS
                • PROGRESS AND GAPS
                • USING NEB NEI VALUES
                • Slide Number 19
                • ARE NEBS HIGH VALUE
                • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                • TOP NEBS FOR WEATHERIZATION PROGRAM
                • RESULTS FOR ENERGY STARreg APPLIANCES
                • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                • CampI PROGRAMS ldquoNEBBEDrdquo
                • AUDIT FINANCIAL PROGRAM
                • CampI NEW CONSTRUCTION
                • CampI TECHNICAL ASSISTANCE
                • EXPRESSING NEBS VALUEndashCxe
                • Slide Number 31
                • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                • EMISSIONS (SOCIETAL NEBS)
                • UTILITY NEBS
                • ADJUSTED PAYBACKS
                • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                • NEBS IN BC
                • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                • APPLICATION TO BC
                • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                • STATE TREATMENT OF NEBS NEBs
                • Slide Number 42
                • SELECTED STATE PROGRESS
                • SELECTED STATE PROGRESS
                • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                • DISCOUNT RATE MATTERS
                • EUL VARIATIONS- ADOPTED VALUES
                • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                • TAKEAWAYSON NEBSNEIS
                • THANK YOUQuestions
                • Slide Number 53
                • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                  Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                  m Product losses20
                  l Ease of selling leasing building33
                  n Labor requirements45
                  h Aesthetics appearance59
                  j Noise50
                  b Equipment maintenance660
                  f Tenant satisfaction60
                  d Equipment lifetimes718
                  e Productivity724
                  k Building safety[2]73
                  g Comfort933
                  i Lighting Quality of Light916
                  c Equipment performance[1]1019
                  a Operating costs1165
                  o Doing good for the Environment[3]1132
                  Other03
                  [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                  [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                  [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                  m Product losses
                  l Ease of selling leasing building
                  n Labor requirements
                  h Aesthetics appearance
                  j Noise
                  b Equipment maintenance
                  f Tenant satisfaction
                  d Equipment lifetimes
                  e Productivity
                  k Building safety[2]
                  g Comfort
                  i Lighting Quality of Light
                  c Equipment performance[1]
                  a Operating costs
                  o Doing good for the Environment[3]
                  Share of ZALEH NEBS from various measures
                  Each presented as multiple of the (overall) energy savings
                  Super Insulation037
                  Double Glazing026
                  Solar Design025
                  Solar Water Heating005
                  EnergySmart (insul Retrofit)Dunedin (insul)
                  Other124
                  Noise24
                  Budgetfinances25
                  Environmental65
                  Health78
                  Reduced moves86
                  Ease of sellingrenting130
                  DampnessMold1316
                  Comfort-related4933
                  101101
                  Operational NEBs44
                  DesignConstruction NEBs31
                  Occupant NEBs25
                  1
                  OtherOther
                  NoiseNoise
                  BudgetfinancesBudgetfinances
                  EnvironmentalEnvironmental
                  HealthHealth
                  Reduced movesReduced moves
                  Ease of sellingrentingEase of sellingrenting
                  DampnessMoldDampnessMold
                  Comfort-relatedComfort-related
                  Boiler
                  VendorNPPartic
                  Perform0280202
                  Footprint028013017
                  ControlFeatures031022016
                  Tenant compl006008014
                  Noise005006014
                  Maint004009007
                  Other004003007
                  Lifetime-006018006
                  MF New construction energy star
                  Operating cost014
                  Environment012
                  Maintenance01
                  Lifetime01
                  Performance009
                  Occupant Satisf009
                  Comfort008
                  Noise006
                  Sellingleasing006
                  Aesthetics005
                  Light Quality005
                  Power QualReliab004
                  Safety002
                  RIGHTLIGHTS
                  Environment023
                  Quality016
                  Quantity014
                  PgmAvail01
                  Flicker008
                  Maintenance006
                  Lifetime006
                  SalesProductivity006
                  Safety005
                  Noise005
                  Control003
                  Illness0
                  PerformPerformPerform
                  FootprintFootprintFootprint
                  ControlFeaturesControlFeaturesControlFeatures
                  Tenant complTenant complTenant compl
                  NoiseNoiseNoise
                  MaintMaintMaint
                  OtherOtherOther
                  LifetimeLifetimeLifetime
                  Environment
                  Quality
                  Quantity
                  PgmAvail
                  Flicker
                  Maintenance
                  Lifetime
                  SalesProductivity
                  Safety
                  Noise
                  Control
                  Illness

                  Variability

                  Relevant NEB Categories

                  Program measure invariant (suitable for ldquoadderrdquo)

                  middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                  Program measure dependent

                  middot Economic ndash societal (depends on measures and local manufacture installation)

                  middot Health and safety health care illnesses ndash societal and participant (measure)

                  middot Water wastewater infrastructure and water bill savings ndash societal and participant

                  middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                  Climate dependent

                  middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                  Residential Target dependent (low income or MF vs SF)

                  middot Payment related ndash utility (arrearages etc stronger for low income targets)

                  middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

                  middot Participant benefits related to hardship and payments

                  middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                  Used for

                  Major Advantages

                  Major Disadvantages

                  Direct measurement (specific studies of changes on-site)

                  Comrsquol labor productivity etc

                  Direct precise attributable

                  Small estimation sample sizes specialized cases poor transferability expensive

                  Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

                  Insurance water health others

                  Long history easy secondary sources

                  Credible to reviewers vetted inputs

                  Not available for all NEBs

                  Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

                  Emissions economics

                  Third party peer-vetted models available for economics jobs and emissions

                  Not available for all NEBs

                  Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

                  Wide variety of Participant NEBs

                  Large sample sizes amp statistical properties

                  Affordable

                  Multiple estimates leading to similar ranges

                  Direct method of measuring some key NEBs

                  Concerns about surveys as a source of quantitative values amp reliability

                  Recall from survey respondents

                  Proper attribution to programs measures

                  MWxNEWxCA WxCA WxSF-NCMF-NCC- LiteC-RebC-Reb
                  mult1220813075104075084
                  LMS Verbal15610980612067115055091
                  WTP42242732
                  MWxMWxMWx
                  NEWxNEWxNEWx
                  CA WxCA WxCA Wx
                  CA WxCA WxCA Wx
                  SF-NCSF-NCSF-NC
                  MF-NCMF-NCMF-NC
                  C- LiteC- LiteC- Lite
                  C-RebC-RebC-Reb
                  C-RebC-RebC-Reb

                  SERA

                  NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES

                  Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip

                  9

                  Monetized NEBs

                  Source Skumatz SERA research

                  NEB

                  s

                  Direct

                  Secondary

                  Model

                  Survey Story of a ferryhellip then it is academic

                  Corp Records Utility data

                  Change x valueFinancial Calcs

                  Third party jobsAnd emissions

                  Specialized academic Best for some NEBs

                  SERA

                  MEASURING PARTICIPANTNEBs Best practices

                  WTP not fruitful 7+ better options

                  Net Three Net positive amp negative Net beyond standard

                  efficiency Net to Gross

                  Non-overlapping Consistent units Recommend in ALL

                  process (impact) surveys - barriers

                  HTM=Hard to measure HTA=Hard to answer

                  Direct calc Accurate Small ldquonrdquo

                  WTPWTA Accepted Volatile HTA

                  Relative Fast strong HTA

                  LMSFast

                  strong clear ETA

                  Unfamiliar

                  Logit Conjoint

                  rankStrong Slow

                  complex

                  Regression Defensible Limited data cost

                  Market value Strong Data

                  Other Exploring Cost tradeoffs

                  Source SERA Research

                  SERA

                  HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)

                  11

                  Monetized NEBs

                  Direct Secondary Model Survey

                  Attributable Change (study)

                  Value or Financial

                  Calc

                  X

                  Total Attrib Stated Relative

                  Effect

                  Savings (or translation)(ldquoNormrdquo)

                  Individual NEB Shares

                  X

                  X

                  SERA

                  PARTICIPANT MEASUREMENT METHODS COMPARISON

                  005

                  115

                  225

                  335

                  445

                  MW

                  x

                  NE

                  Wx

                  CA

                  Wx

                  CA

                  Wx

                  SF-N

                  C

                  MF-

                  NC

                  C-

                  Lite

                  C-R

                  eb

                  C-R

                  eb

                  multLMS VerbalWTP

                  Source SERA Research

                  Chart1

                  mult
                  LMS Verbal
                  WTP
                  122
                  156
                  42
                  08
                  1
                  2
                  098
                  4
                  06
                  13
                  12
                  075
                  067
                  104
                  115
                  27
                  075
                  055
                  32
                  084
                  091

                  Sheet1

                  Sheet1

                  mult
                  LMS Verbal
                  WTP

                  Sheet2

                  Sheet3

                  SERA

                  PROS AND CONS OF NEBS MEASUREMENT METHODS

                  13

                  Source Skumatz SERA research

                  Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

                  Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

                  Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

                  Long history easy secondary sources Credible to reviewers vetted inputs

                  Not available for all NEBs

                  Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

                  Third party peer-vetted models available for economics jobs and emissions

                  Not available for all NEBs

                  Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

                  Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

                  Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

                  SERA

                  PROGRESS TRANSFERABILITY OF NEBS

                  14

                  Source Skumatz SERA research

                  Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

                  bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                  Program measure dependent

                  bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

                  education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                  Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                  Residential Target dependent (low income or MF vs SF)

                  bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

                  vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                  Cost savings consistency if transferable vs risk

                  SERA

                  RANGES TYPICAL NEB VALUES FOR WxRETROFI

                  15

                  SERA

                  PROGRESS AND GAPS Consistency Transferability

                  Some programs types others less studied Transferability research

                  Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

                  measures until then Across the board Savings share Regression

                  SERA

                  PROGRESS AND GAPS Gas vs Electric

                  Some researchndashusually program-wide not broken down International research found participant NEBs have

                  similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

                  MF Less-commonly-studied complicated by poor response

                  and complexity of sector (decision-maker some measures in home some central) separate from low income not common

                  Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

                  Commercial gaps in some measures

                  SERA

                  USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                  Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                  18

                  0

                  005

                  01

                  015

                  02

                  025

                  Environment

                  Quality

                  Quantity

                  PgmAvailFlicker

                  Mainten

                  ance

                  Lifetime

                  SalesP

                  roductivity

                  Safety

                  Noise

                  Control

                  Illness

                  Total NEBs Multiplier=X

                  $

                  Savings value=$X

                  Chart3

                  023
                  016
                  014
                  01
                  008
                  006
                  006
                  006
                  005
                  005
                  003
                  0

                  Chart1

                  Vendor
                  NP
                  Partic
                  028
                  02
                  02
                  028
                  013
                  017
                  031
                  022
                  016
                  006
                  008
                  014
                  005
                  006
                  014
                  004
                  009
                  007
                  004
                  003
                  007
                  -006
                  018
                  006

                  Sheet1

                  Sheet1

                  Sheet2

                  Sheet3

                  SERA

                  NEB VALUES

                  19

                  SERA

                  ARE NEBS HIGH VALUE Energy savings are often the minority of

                  benefits from program measuresndash Donrsquot ignore that

                  20

                  0 05 1 15 2 25 3

                  Energy Sav

                  NEBs

                  NEB vs Energy Savings ValueIncluding all NEBs

                  Source (SkumatzSERA 2010 amp others)

                  Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                  SERA

                  RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                  MF Many others

                  Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                  Variety of end uses

                  SERA

                  TOP NEBS FOR WEATHERIZATION PROGRAM

                  Weatherization program multipliergt1

                  Source SERA Research

                  SERA

                  RESULTS FOR ENERGY STARreg APPLIANCES

                  0002004006008

                  01012014016018

                  Maint

                  Appear

                  Perf LifeNoise Satis

                  Comfort

                  LiteSafe

                  ty SellMove

                  Water

                  Enviro

                  Refrig DW CW

                  Source SERA Research

                  SERA

                  INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                  5 increments to 50

                  Source SERA research

                  Chart5

                  EnergySmart (insul Retrofit)
                  Dunedin (insul)
                  001
                  024
                  002
                  004
                  002
                  005
                  006
                  005
                  007
                  008
                  008
                  006
                  013
                  0
                  013
                  016
                  049
                  033

                  cx

                  cx

                  nz

                  Sheet3

                  Sheet3

                  Super Insulation
                  Double Glazing
                  Solar Design
                  Solar Water Heating
                  Super Insulation
                  Double Glazing
                  Solar Design
                  Solar Water Heating
                  EnergySmart (insul Retrofit)
                  Dunedin (insul)

                  SERA

                  ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                  0

                  002

                  004

                  006

                  008

                  01

                  012

                  Maint

                  Perf LifeSatis

                  f

                  ComfortAesth

                  NoiseSafe

                  ty

                  LightingSelli

                  ng

                  Stay in home

                  Enviro

                  Sick Days

                  Calls

                  ES Homes HP wES Energy savings multiplier=1

                  Source SERA Research

                  SERA

                  CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                  DG CHP HVAC Equipment rebate Other

                  Building codes incentives by cities

                  Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                  Variety of end uses

                  SERA

                  AUDIT FINANCIAL PROGRAM

                  0

                  01

                  02

                  03

                  04

                  05

                  06

                  07

                  Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                  Source SERA Research

                  SERA

                  CampI NEW CONSTRUCTION

                  -005

                  0

                  005

                  01

                  015

                  02

                  AampE Owner(Pct of Total Partic NEBs)

                  Source SERA Research

                  SERA

                  CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                  0 2 4 6 8 10 12

                  m Product losses

                  l Ease of selling leasing building

                  n Labor requirements

                  h Aesthetics appearance

                  j Noise

                  b Equipment maintenance

                  f Tenant satisfaction

                  d Equipment lifetimes

                  e Productivity

                  k Building safety[2]

                  g Comfort

                  i Lighting Quality of Light

                  c Equipment performance[1]

                  a Operating costs

                  o Doing good for the Environment[3]

                  Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                  vcv

                  Source SERA Research

                  Chart1

                  CampI Technical Assistance NEBs
                  002
                  003
                  004
                  005
                  005
                  006
                  006
                  007
                  007
                  007
                  009
                  009
                  01
                  011
                  011

                  Sheet1

                  Sheet1

                  CampI Technical Assistance Program NEBs (Participant)

                  Sheet2

                  Sheet3

                  SERA

                  EXPRESSING NEBS VALUEndashCxe

                  Strong value from RetroCxSource SERA Research

                  SERA

                  SOCIETAL NEB EXAMPLES

                  31

                  SERA

                  JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                  32

                  0

                  02

                  04

                  06

                  08

                  1

                  12

                  HPWxRetrof Appliance

                  CA

                  WI

                  Natl

                  (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                  Jobs Economic

                  SERA

                  EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                  33

                  SERA

                  UTILITY NEBSUtility NEBs for Template Program

                  Debt WriteOff (util)13

                  Rate Subsidy(util)61

                  HealthSafety(util)0

                  Colln Costs (util)0

                  Gas Calls (util)0

                  Calls to CSRs(util)2

                  TampD (util)16

                  Arrears (util)0

                  Reconnects (util)0

                  Notices (util)7

                  Shutoffs (util)1

                  Rate subsidy TampD

                  Payment-related

                  Source Skumatz Economic Research Associates research

                  Example Low Income Weatherization

                  SERA

                  ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                  be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                  Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                  Value of 100 HALVES Payback doubles ROI

                  Source Skumatz Economics (SERA)

                  35

                  ( for NEBs Multiplier of 125)

                  SERA

                  NEBS IN COST-EFFECTIVENESS APPLICATIONS

                  36

                  hellip2001 ON

                  SERA

                  37

                  NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                  For true representation of B amp C elements of NEBs address missing factor bias

                  Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                  BC applications were considered early on then pulled back awaiting more quantitative evidence

                  SERA

                  CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                  38

                  Test Benefits Costs States Using Traditionally

                  Improved treat-ment with NEBs

                  Utility Cost (or Program Administrator Test) (UCT or PAC)

                  bull Avoided supply costs for transmission distribution and generation (TDampG)

                  bull Avoided gas and water supply costs

                  bull Program administration

                  bull Participant incentives

                  bull Increased supply cost

                  CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                  Use cost only paid by the utility

                  Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                  Same as above plus bull increased revenue

                  Same as above plus bull Decreased revenue

                  AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                  Participant cost bull Utility bill reductions bull Participant incentives

                  bull Participant direct costs

                  AR CA FL HI IA IN MI MN NY VA

                  Participant NEBs

                  Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                  Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                  SERA

                  APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                  Expansion Concern with risk calculation accuracy

                  39

                  Source Skumatz SERA research

                  SERA

                  40

                  METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                  Maximize DSM opportunities amp feedback Accuracy tailoring

                  Minimize Regulatory amp Implementer Risk

                  Minimize Evaluation Cost

                  Adder

                  Readily MeasurableHybrid

                  All NEBsSource SERA Research

                  SERA

                  41

                  STATE TREATMENT OF NEBS NEBsRegulatory

                  Screening Application Utilities regions

                  Program Marketing Fairly widespread use in utilities states across the country

                  MO

                  RE AGGRESSIVE ==gt

                  Test Pgm Screen - adder

                  IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                  Test Pgm Screen - readily measured

                  MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                  Test Hybrid (potential adder amp measured)

                  CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                  Test Pgm screen - Broad

                  With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                  Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                  SERA

                  42

                  Key

                  States with NEBs in CE tests for At least one type of program

                  copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                  STATES WITH NEBS IN CE TESTS

                  NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                  groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                  SERA

                  SELECTED STATE PROGRESS CA

                  2001 ndash LIPPT model NY

                  Mid-2000s measurement amp scenarios not included CO

                  Adders (10 electric 25 LI 5 gas) VT

                  15 adder helped by previous research DC

                  10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                  43

                  Source Skumatz SERA research

                  SERA

                  SELECTED STATE PROGRESS Dominos ongoing

                  Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                  Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                  Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                  Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                  44

                  Source Skumatz SERA research

                  SERA

                  APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                  Expansion Concern with risk calculation accuracy

                  45

                  Source Skumatz SERA research

                  SERA

                  46

                  NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                  [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                  SourceSkumatz SERA Independent research

                  NTG ndash accuracy measurement incomplete

                  Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                  NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                  EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                  Cost Complicated expensive local changes

                  RELATIVE Risk from NEBs

                  Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                  Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                  NEBs as accurate as many inputs (all could improve)

                  SERA

                  DISCOUNT RATE MATTERS

                  47

                  Source Skumatz SERA Research

                  0

                  20

                  40

                  60

                  80

                  100

                  120

                  1 2 3 4 5 6 7 8 9 10

                  PV Ratios vs 1 Discount Rate

                  10 yr PV 20 yr PV

                  Discount rate - percent

                  WACCSocialLTT

                  Source Skumatz Electricity Journal IEPEC paper independent research

                  SERA

                  EUL VARIATIONS-ADOPTED VALUES

                  Source Skumatz Economic Research Associates SERA independent research

                  0

                  50

                  100

                  150

                  200

                  250

                  300

                  350

                  400

                  450

                  0 10 20 30 40 50 60 70

                  EUL Variation by Measure

                  67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                  Poorly supportedOldTechnology change

                  SERA

                  KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                  Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                  measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                  change the BC conclusion

                  On the cusphellip

                  If NO Yoursquore doneAnd bias addressedsufficiently

                  IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                  Source SERA all rights reserved

                  SERArsquos lsquoNEB-ItrdquoModel ndash

                  SERA

                  IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                  Utility Soc Part Conserv Recrsquom

                  Rationale

                  Base Percent X X X Program-invariant

                  Low Income X X X X Multiple sources

                  Weatherization X X X Substantial Participantimpacts

                  Measure Program-specific

                  X Varies by measure sector

                  Other Recomrsquos Local Research

                  50

                  Source Skumatz SERA

                  Developing values for multiple states amp utilities

                  SERA

                  TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                  Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                  Other sources of bias to research (EUL NTG etc) bring high variation

                  5) States are incorporating NEBs dominos input in deliberations in multiple states

                  6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                  THANK YOU

                  Questions

                  Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                  SERA

                  POWER OF NEBS ndashBARRIERS DISCONNECTS

                  53

                  SERA

                  NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                  Negative NEB values cost of barrier

                  Solar Water Heat NZ$ Euros

                  Solar Design NZ$ Euros

                  Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                  -23 -12 (079)

                  -11 -6 (06)

                  Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                  Source Skumatz Economic Research Associates research

                  SERA

                  CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                  NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                  -5-4-3-2-1012345

                  Op Co

                  st

                  Init C

                  ost

                  Maint

                  Perfo

                  rm

                  Prod

                  uctiv

                  Tena

                  nt Sa

                  t

                  Comf

                  ort

                  Appe

                  ar

                  Light

                  Qual

                  SellL

                  ease

                  Devp Ownr AampE

                  Note AgtE (~4 vs ~2) for these NEBs

                  Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                  Source Skumatz Economic Research Associates Inc 2002

                  Owners more positive ndashUnderinvestment train

                  • Soup to nuts on non-energy benefits NEIs
                  • TOPICS
                  • BACKGROUND NEBS
                  • 20+ YEARS OF NEBS PROGRESShellip
                  • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                  • KEY APPLICATIONS OF NEBS
                  • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                  • Neb measurement
                  • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                  • MEASURING PARTICIPANTNEBs
                  • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                  • PARTICIPANT MEASUREMENT METHODS COMPARISON
                  • PROS AND CONS OF NEBS MEASUREMENT METHODS
                  • PROGRESS TRANSFERABILITY OF NEBS
                  • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                  • PROGRESS AND GAPS
                  • PROGRESS AND GAPS
                  • USING NEB NEI VALUES
                  • Slide Number 19
                  • ARE NEBS HIGH VALUE
                  • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                  • TOP NEBS FOR WEATHERIZATION PROGRAM
                  • RESULTS FOR ENERGY STARreg APPLIANCES
                  • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                  • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                  • CampI PROGRAMS ldquoNEBBEDrdquo
                  • AUDIT FINANCIAL PROGRAM
                  • CampI NEW CONSTRUCTION
                  • CampI TECHNICAL ASSISTANCE
                  • EXPRESSING NEBS VALUEndashCxe
                  • Slide Number 31
                  • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                  • EMISSIONS (SOCIETAL NEBS)
                  • UTILITY NEBS
                  • ADJUSTED PAYBACKS
                  • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                  • NEBS IN BC
                  • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                  • APPLICATION TO BC
                  • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                  • STATE TREATMENT OF NEBS NEBs
                  • Slide Number 42
                  • SELECTED STATE PROGRESS
                  • SELECTED STATE PROGRESS
                  • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                  • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                  • DISCOUNT RATE MATTERS
                  • EUL VARIATIONS- ADOPTED VALUES
                  • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                  • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                  • TAKEAWAYSON NEBSNEIS
                  • THANK YOUQuestions
                  • Slide Number 53
                  • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                  • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                    Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                    m Product losses20
                    l Ease of selling leasing building33
                    n Labor requirements45
                    h Aesthetics appearance59
                    j Noise50
                    b Equipment maintenance660
                    f Tenant satisfaction60
                    d Equipment lifetimes718
                    e Productivity724
                    k Building safety[2]73
                    g Comfort933
                    i Lighting Quality of Light916
                    c Equipment performance[1]1019
                    a Operating costs1165
                    o Doing good for the Environment[3]1132
                    Other03
                    [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                    [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                    [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                    m Product losses
                    l Ease of selling leasing building
                    n Labor requirements
                    h Aesthetics appearance
                    j Noise
                    b Equipment maintenance
                    f Tenant satisfaction
                    d Equipment lifetimes
                    e Productivity
                    k Building safety[2]
                    g Comfort
                    i Lighting Quality of Light
                    c Equipment performance[1]
                    a Operating costs
                    o Doing good for the Environment[3]
                    Share of ZALEH NEBS from various measures
                    Each presented as multiple of the (overall) energy savings
                    Super Insulation037
                    Double Glazing026
                    Solar Design025
                    Solar Water Heating005
                    EnergySmart (insul Retrofit)Dunedin (insul)
                    Other124
                    Noise24
                    Budgetfinances25
                    Environmental65
                    Health78
                    Reduced moves86
                    Ease of sellingrenting130
                    DampnessMold1316
                    Comfort-related4933
                    101101
                    Operational NEBs44
                    DesignConstruction NEBs31
                    Occupant NEBs25
                    1
                    OtherOther
                    NoiseNoise
                    BudgetfinancesBudgetfinances
                    EnvironmentalEnvironmental
                    HealthHealth
                    Reduced movesReduced moves
                    Ease of sellingrentingEase of sellingrenting
                    DampnessMoldDampnessMold
                    Comfort-relatedComfort-related
                    Boiler
                    VendorNPPartic
                    Perform0280202
                    Footprint028013017
                    ControlFeatures031022016
                    Tenant compl006008014
                    Noise005006014
                    Maint004009007
                    Other004003007
                    Lifetime-006018006
                    MF New construction energy star
                    Operating cost014
                    Environment012
                    Maintenance01
                    Lifetime01
                    Performance009
                    Occupant Satisf009
                    Comfort008
                    Noise006
                    Sellingleasing006
                    Aesthetics005
                    Light Quality005
                    Power QualReliab004
                    Safety002
                    RIGHTLIGHTS
                    Environment023
                    Quality016
                    Quantity014
                    PgmAvail01
                    Flicker008
                    Maintenance006
                    Lifetime006
                    SalesProductivity006
                    Safety005
                    Noise005
                    Control003
                    Illness0
                    PerformPerformPerform
                    FootprintFootprintFootprint
                    ControlFeaturesControlFeaturesControlFeatures
                    Tenant complTenant complTenant compl
                    NoiseNoiseNoise
                    MaintMaintMaint
                    OtherOtherOther
                    LifetimeLifetimeLifetime
                    Environment
                    Quality
                    Quantity
                    PgmAvail
                    Flicker
                    Maintenance
                    Lifetime
                    SalesProductivity
                    Safety
                    Noise
                    Control
                    Illness

                    Variability

                    Relevant NEB Categories

                    Program measure invariant (suitable for ldquoadderrdquo)

                    middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                    Program measure dependent

                    middot Economic ndash societal (depends on measures and local manufacture installation)

                    middot Health and safety health care illnesses ndash societal and participant (measure)

                    middot Water wastewater infrastructure and water bill savings ndash societal and participant

                    middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                    Climate dependent

                    middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                    Residential Target dependent (low income or MF vs SF)

                    middot Payment related ndash utility (arrearages etc stronger for low income targets)

                    middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

                    middot Participant benefits related to hardship and payments

                    middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                    Used for

                    Major Advantages

                    Major Disadvantages

                    Direct measurement (specific studies of changes on-site)

                    Comrsquol labor productivity etc

                    Direct precise attributable

                    Small estimation sample sizes specialized cases poor transferability expensive

                    Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

                    Insurance water health others

                    Long history easy secondary sources

                    Credible to reviewers vetted inputs

                    Not available for all NEBs

                    Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

                    Emissions economics

                    Third party peer-vetted models available for economics jobs and emissions

                    Not available for all NEBs

                    Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

                    Wide variety of Participant NEBs

                    Large sample sizes amp statistical properties

                    Affordable

                    Multiple estimates leading to similar ranges

                    Direct method of measuring some key NEBs

                    Concerns about surveys as a source of quantitative values amp reliability

                    Recall from survey respondents

                    Proper attribution to programs measures

                    MWxNEWxCA WxCA WxSF-NCMF-NCC- LiteC-RebC-Reb
                    mult1220813075104075084
                    LMS Verbal15610980612067115055091
                    WTP42242732
                    MWxMWxMWx
                    NEWxNEWxNEWx
                    CA WxCA WxCA Wx
                    CA WxCA WxCA Wx
                    SF-NCSF-NCSF-NC
                    MF-NCMF-NCMF-NC
                    C- LiteC- LiteC- Lite
                    C-RebC-RebC-Reb
                    C-RebC-RebC-Reb

                    SERA

                    MEASURING PARTICIPANTNEBs Best practices

                    WTP not fruitful 7+ better options

                    Net Three Net positive amp negative Net beyond standard

                    efficiency Net to Gross

                    Non-overlapping Consistent units Recommend in ALL

                    process (impact) surveys - barriers

                    HTM=Hard to measure HTA=Hard to answer

                    Direct calc Accurate Small ldquonrdquo

                    WTPWTA Accepted Volatile HTA

                    Relative Fast strong HTA

                    LMSFast

                    strong clear ETA

                    Unfamiliar

                    Logit Conjoint

                    rankStrong Slow

                    complex

                    Regression Defensible Limited data cost

                    Market value Strong Data

                    Other Exploring Cost tradeoffs

                    Source SERA Research

                    SERA

                    HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)

                    11

                    Monetized NEBs

                    Direct Secondary Model Survey

                    Attributable Change (study)

                    Value or Financial

                    Calc

                    X

                    Total Attrib Stated Relative

                    Effect

                    Savings (or translation)(ldquoNormrdquo)

                    Individual NEB Shares

                    X

                    X

                    SERA

                    PARTICIPANT MEASUREMENT METHODS COMPARISON

                    005

                    115

                    225

                    335

                    445

                    MW

                    x

                    NE

                    Wx

                    CA

                    Wx

                    CA

                    Wx

                    SF-N

                    C

                    MF-

                    NC

                    C-

                    Lite

                    C-R

                    eb

                    C-R

                    eb

                    multLMS VerbalWTP

                    Source SERA Research

                    Chart1

                    mult
                    LMS Verbal
                    WTP
                    122
                    156
                    42
                    08
                    1
                    2
                    098
                    4
                    06
                    13
                    12
                    075
                    067
                    104
                    115
                    27
                    075
                    055
                    32
                    084
                    091

                    Sheet1

                    Sheet1

                    mult
                    LMS Verbal
                    WTP

                    Sheet2

                    Sheet3

                    SERA

                    PROS AND CONS OF NEBS MEASUREMENT METHODS

                    13

                    Source Skumatz SERA research

                    Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

                    Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

                    Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

                    Long history easy secondary sources Credible to reviewers vetted inputs

                    Not available for all NEBs

                    Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

                    Third party peer-vetted models available for economics jobs and emissions

                    Not available for all NEBs

                    Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

                    Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

                    Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

                    SERA

                    PROGRESS TRANSFERABILITY OF NEBS

                    14

                    Source Skumatz SERA research

                    Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

                    bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                    Program measure dependent

                    bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

                    education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                    Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                    Residential Target dependent (low income or MF vs SF)

                    bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

                    vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                    Cost savings consistency if transferable vs risk

                    SERA

                    RANGES TYPICAL NEB VALUES FOR WxRETROFI

                    15

                    SERA

                    PROGRESS AND GAPS Consistency Transferability

                    Some programs types others less studied Transferability research

                    Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

                    measures until then Across the board Savings share Regression

                    SERA

                    PROGRESS AND GAPS Gas vs Electric

                    Some researchndashusually program-wide not broken down International research found participant NEBs have

                    similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

                    MF Less-commonly-studied complicated by poor response

                    and complexity of sector (decision-maker some measures in home some central) separate from low income not common

                    Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

                    Commercial gaps in some measures

                    SERA

                    USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                    Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                    18

                    0

                    005

                    01

                    015

                    02

                    025

                    Environment

                    Quality

                    Quantity

                    PgmAvailFlicker

                    Mainten

                    ance

                    Lifetime

                    SalesP

                    roductivity

                    Safety

                    Noise

                    Control

                    Illness

                    Total NEBs Multiplier=X

                    $

                    Savings value=$X

                    Chart3

                    023
                    016
                    014
                    01
                    008
                    006
                    006
                    006
                    005
                    005
                    003
                    0

                    Chart1

                    Vendor
                    NP
                    Partic
                    028
                    02
                    02
                    028
                    013
                    017
                    031
                    022
                    016
                    006
                    008
                    014
                    005
                    006
                    014
                    004
                    009
                    007
                    004
                    003
                    007
                    -006
                    018
                    006

                    Sheet1

                    Sheet1

                    Sheet2

                    Sheet3

                    SERA

                    NEB VALUES

                    19

                    SERA

                    ARE NEBS HIGH VALUE Energy savings are often the minority of

                    benefits from program measuresndash Donrsquot ignore that

                    20

                    0 05 1 15 2 25 3

                    Energy Sav

                    NEBs

                    NEB vs Energy Savings ValueIncluding all NEBs

                    Source (SkumatzSERA 2010 amp others)

                    Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                    SERA

                    RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                    MF Many others

                    Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                    Variety of end uses

                    SERA

                    TOP NEBS FOR WEATHERIZATION PROGRAM

                    Weatherization program multipliergt1

                    Source SERA Research

                    SERA

                    RESULTS FOR ENERGY STARreg APPLIANCES

                    0002004006008

                    01012014016018

                    Maint

                    Appear

                    Perf LifeNoise Satis

                    Comfort

                    LiteSafe

                    ty SellMove

                    Water

                    Enviro

                    Refrig DW CW

                    Source SERA Research

                    SERA

                    INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                    5 increments to 50

                    Source SERA research

                    Chart5

                    EnergySmart (insul Retrofit)
                    Dunedin (insul)
                    001
                    024
                    002
                    004
                    002
                    005
                    006
                    005
                    007
                    008
                    008
                    006
                    013
                    0
                    013
                    016
                    049
                    033

                    cx

                    cx

                    nz

                    Sheet3

                    Sheet3

                    Super Insulation
                    Double Glazing
                    Solar Design
                    Solar Water Heating
                    Super Insulation
                    Double Glazing
                    Solar Design
                    Solar Water Heating
                    EnergySmart (insul Retrofit)
                    Dunedin (insul)

                    SERA

                    ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                    0

                    002

                    004

                    006

                    008

                    01

                    012

                    Maint

                    Perf LifeSatis

                    f

                    ComfortAesth

                    NoiseSafe

                    ty

                    LightingSelli

                    ng

                    Stay in home

                    Enviro

                    Sick Days

                    Calls

                    ES Homes HP wES Energy savings multiplier=1

                    Source SERA Research

                    SERA

                    CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                    DG CHP HVAC Equipment rebate Other

                    Building codes incentives by cities

                    Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                    Variety of end uses

                    SERA

                    AUDIT FINANCIAL PROGRAM

                    0

                    01

                    02

                    03

                    04

                    05

                    06

                    07

                    Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                    Source SERA Research

                    SERA

                    CampI NEW CONSTRUCTION

                    -005

                    0

                    005

                    01

                    015

                    02

                    AampE Owner(Pct of Total Partic NEBs)

                    Source SERA Research

                    SERA

                    CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                    0 2 4 6 8 10 12

                    m Product losses

                    l Ease of selling leasing building

                    n Labor requirements

                    h Aesthetics appearance

                    j Noise

                    b Equipment maintenance

                    f Tenant satisfaction

                    d Equipment lifetimes

                    e Productivity

                    k Building safety[2]

                    g Comfort

                    i Lighting Quality of Light

                    c Equipment performance[1]

                    a Operating costs

                    o Doing good for the Environment[3]

                    Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                    vcv

                    Source SERA Research

                    Chart1

                    CampI Technical Assistance NEBs
                    002
                    003
                    004
                    005
                    005
                    006
                    006
                    007
                    007
                    007
                    009
                    009
                    01
                    011
                    011

                    Sheet1

                    Sheet1

                    CampI Technical Assistance Program NEBs (Participant)

                    Sheet2

                    Sheet3

                    SERA

                    EXPRESSING NEBS VALUEndashCxe

                    Strong value from RetroCxSource SERA Research

                    SERA

                    SOCIETAL NEB EXAMPLES

                    31

                    SERA

                    JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                    32

                    0

                    02

                    04

                    06

                    08

                    1

                    12

                    HPWxRetrof Appliance

                    CA

                    WI

                    Natl

                    (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                    Jobs Economic

                    SERA

                    EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                    33

                    SERA

                    UTILITY NEBSUtility NEBs for Template Program

                    Debt WriteOff (util)13

                    Rate Subsidy(util)61

                    HealthSafety(util)0

                    Colln Costs (util)0

                    Gas Calls (util)0

                    Calls to CSRs(util)2

                    TampD (util)16

                    Arrears (util)0

                    Reconnects (util)0

                    Notices (util)7

                    Shutoffs (util)1

                    Rate subsidy TampD

                    Payment-related

                    Source Skumatz Economic Research Associates research

                    Example Low Income Weatherization

                    SERA

                    ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                    be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                    Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                    Value of 100 HALVES Payback doubles ROI

                    Source Skumatz Economics (SERA)

                    35

                    ( for NEBs Multiplier of 125)

                    SERA

                    NEBS IN COST-EFFECTIVENESS APPLICATIONS

                    36

                    hellip2001 ON

                    SERA

                    37

                    NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                    For true representation of B amp C elements of NEBs address missing factor bias

                    Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                    BC applications were considered early on then pulled back awaiting more quantitative evidence

                    SERA

                    CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                    38

                    Test Benefits Costs States Using Traditionally

                    Improved treat-ment with NEBs

                    Utility Cost (or Program Administrator Test) (UCT or PAC)

                    bull Avoided supply costs for transmission distribution and generation (TDampG)

                    bull Avoided gas and water supply costs

                    bull Program administration

                    bull Participant incentives

                    bull Increased supply cost

                    CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                    Use cost only paid by the utility

                    Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                    Same as above plus bull increased revenue

                    Same as above plus bull Decreased revenue

                    AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                    Participant cost bull Utility bill reductions bull Participant incentives

                    bull Participant direct costs

                    AR CA FL HI IA IN MI MN NY VA

                    Participant NEBs

                    Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                    Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                    SERA

                    APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                    Expansion Concern with risk calculation accuracy

                    39

                    Source Skumatz SERA research

                    SERA

                    40

                    METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                    Maximize DSM opportunities amp feedback Accuracy tailoring

                    Minimize Regulatory amp Implementer Risk

                    Minimize Evaluation Cost

                    Adder

                    Readily MeasurableHybrid

                    All NEBsSource SERA Research

                    SERA

                    41

                    STATE TREATMENT OF NEBS NEBsRegulatory

                    Screening Application Utilities regions

                    Program Marketing Fairly widespread use in utilities states across the country

                    MO

                    RE AGGRESSIVE ==gt

                    Test Pgm Screen - adder

                    IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                    Test Pgm Screen - readily measured

                    MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                    Test Hybrid (potential adder amp measured)

                    CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                    Test Pgm screen - Broad

                    With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                    Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                    SERA

                    42

                    Key

                    States with NEBs in CE tests for At least one type of program

                    copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                    STATES WITH NEBS IN CE TESTS

                    NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                    groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                    SERA

                    SELECTED STATE PROGRESS CA

                    2001 ndash LIPPT model NY

                    Mid-2000s measurement amp scenarios not included CO

                    Adders (10 electric 25 LI 5 gas) VT

                    15 adder helped by previous research DC

                    10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                    43

                    Source Skumatz SERA research

                    SERA

                    SELECTED STATE PROGRESS Dominos ongoing

                    Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                    Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                    Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                    Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                    44

                    Source Skumatz SERA research

                    SERA

                    APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                    Expansion Concern with risk calculation accuracy

                    45

                    Source Skumatz SERA research

                    SERA

                    46

                    NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                    [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                    SourceSkumatz SERA Independent research

                    NTG ndash accuracy measurement incomplete

                    Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                    NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                    EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                    Cost Complicated expensive local changes

                    RELATIVE Risk from NEBs

                    Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                    Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                    NEBs as accurate as many inputs (all could improve)

                    SERA

                    DISCOUNT RATE MATTERS

                    47

                    Source Skumatz SERA Research

                    0

                    20

                    40

                    60

                    80

                    100

                    120

                    1 2 3 4 5 6 7 8 9 10

                    PV Ratios vs 1 Discount Rate

                    10 yr PV 20 yr PV

                    Discount rate - percent

                    WACCSocialLTT

                    Source Skumatz Electricity Journal IEPEC paper independent research

                    SERA

                    EUL VARIATIONS-ADOPTED VALUES

                    Source Skumatz Economic Research Associates SERA independent research

                    0

                    50

                    100

                    150

                    200

                    250

                    300

                    350

                    400

                    450

                    0 10 20 30 40 50 60 70

                    EUL Variation by Measure

                    67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                    Poorly supportedOldTechnology change

                    SERA

                    KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                    Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                    measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                    change the BC conclusion

                    On the cusphellip

                    If NO Yoursquore doneAnd bias addressedsufficiently

                    IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                    Source SERA all rights reserved

                    SERArsquos lsquoNEB-ItrdquoModel ndash

                    SERA

                    IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                    Utility Soc Part Conserv Recrsquom

                    Rationale

                    Base Percent X X X Program-invariant

                    Low Income X X X X Multiple sources

                    Weatherization X X X Substantial Participantimpacts

                    Measure Program-specific

                    X Varies by measure sector

                    Other Recomrsquos Local Research

                    50

                    Source Skumatz SERA

                    Developing values for multiple states amp utilities

                    SERA

                    TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                    Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                    Other sources of bias to research (EUL NTG etc) bring high variation

                    5) States are incorporating NEBs dominos input in deliberations in multiple states

                    6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                    THANK YOU

                    Questions

                    Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                    SERA

                    POWER OF NEBS ndashBARRIERS DISCONNECTS

                    53

                    SERA

                    NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                    Negative NEB values cost of barrier

                    Solar Water Heat NZ$ Euros

                    Solar Design NZ$ Euros

                    Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                    -23 -12 (079)

                    -11 -6 (06)

                    Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                    Source Skumatz Economic Research Associates research

                    SERA

                    CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                    NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                    -5-4-3-2-1012345

                    Op Co

                    st

                    Init C

                    ost

                    Maint

                    Perfo

                    rm

                    Prod

                    uctiv

                    Tena

                    nt Sa

                    t

                    Comf

                    ort

                    Appe

                    ar

                    Light

                    Qual

                    SellL

                    ease

                    Devp Ownr AampE

                    Note AgtE (~4 vs ~2) for these NEBs

                    Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                    Source Skumatz Economic Research Associates Inc 2002

                    Owners more positive ndashUnderinvestment train

                    • Soup to nuts on non-energy benefits NEIs
                    • TOPICS
                    • BACKGROUND NEBS
                    • 20+ YEARS OF NEBS PROGRESShellip
                    • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                    • KEY APPLICATIONS OF NEBS
                    • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                    • Neb measurement
                    • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                    • MEASURING PARTICIPANTNEBs
                    • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                    • PARTICIPANT MEASUREMENT METHODS COMPARISON
                    • PROS AND CONS OF NEBS MEASUREMENT METHODS
                    • PROGRESS TRANSFERABILITY OF NEBS
                    • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                    • PROGRESS AND GAPS
                    • PROGRESS AND GAPS
                    • USING NEB NEI VALUES
                    • Slide Number 19
                    • ARE NEBS HIGH VALUE
                    • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                    • TOP NEBS FOR WEATHERIZATION PROGRAM
                    • RESULTS FOR ENERGY STARreg APPLIANCES
                    • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                    • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                    • CampI PROGRAMS ldquoNEBBEDrdquo
                    • AUDIT FINANCIAL PROGRAM
                    • CampI NEW CONSTRUCTION
                    • CampI TECHNICAL ASSISTANCE
                    • EXPRESSING NEBS VALUEndashCxe
                    • Slide Number 31
                    • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                    • EMISSIONS (SOCIETAL NEBS)
                    • UTILITY NEBS
                    • ADJUSTED PAYBACKS
                    • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                    • NEBS IN BC
                    • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                    • APPLICATION TO BC
                    • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                    • STATE TREATMENT OF NEBS NEBs
                    • Slide Number 42
                    • SELECTED STATE PROGRESS
                    • SELECTED STATE PROGRESS
                    • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                    • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                    • DISCOUNT RATE MATTERS
                    • EUL VARIATIONS- ADOPTED VALUES
                    • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                    • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                    • TAKEAWAYSON NEBSNEIS
                    • THANK YOUQuestions
                    • Slide Number 53
                    • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                    • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                      Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                      m Product losses20
                      l Ease of selling leasing building33
                      n Labor requirements45
                      h Aesthetics appearance59
                      j Noise50
                      b Equipment maintenance660
                      f Tenant satisfaction60
                      d Equipment lifetimes718
                      e Productivity724
                      k Building safety[2]73
                      g Comfort933
                      i Lighting Quality of Light916
                      c Equipment performance[1]1019
                      a Operating costs1165
                      o Doing good for the Environment[3]1132
                      Other03
                      [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                      [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                      [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                      m Product losses
                      l Ease of selling leasing building
                      n Labor requirements
                      h Aesthetics appearance
                      j Noise
                      b Equipment maintenance
                      f Tenant satisfaction
                      d Equipment lifetimes
                      e Productivity
                      k Building safety[2]
                      g Comfort
                      i Lighting Quality of Light
                      c Equipment performance[1]
                      a Operating costs
                      o Doing good for the Environment[3]
                      Share of ZALEH NEBS from various measures
                      Each presented as multiple of the (overall) energy savings
                      Super Insulation037
                      Double Glazing026
                      Solar Design025
                      Solar Water Heating005
                      EnergySmart (insul Retrofit)Dunedin (insul)
                      Other124
                      Noise24
                      Budgetfinances25
                      Environmental65
                      Health78
                      Reduced moves86
                      Ease of sellingrenting130
                      DampnessMold1316
                      Comfort-related4933
                      101101
                      Operational NEBs44
                      DesignConstruction NEBs31
                      Occupant NEBs25
                      1
                      OtherOther
                      NoiseNoise
                      BudgetfinancesBudgetfinances
                      EnvironmentalEnvironmental
                      HealthHealth
                      Reduced movesReduced moves
                      Ease of sellingrentingEase of sellingrenting
                      DampnessMoldDampnessMold
                      Comfort-relatedComfort-related
                      Boiler
                      VendorNPPartic
                      Perform0280202
                      Footprint028013017
                      ControlFeatures031022016
                      Tenant compl006008014
                      Noise005006014
                      Maint004009007
                      Other004003007
                      Lifetime-006018006
                      MF New construction energy star
                      Operating cost014
                      Environment012
                      Maintenance01
                      Lifetime01
                      Performance009
                      Occupant Satisf009
                      Comfort008
                      Noise006
                      Sellingleasing006
                      Aesthetics005
                      Light Quality005
                      Power QualReliab004
                      Safety002
                      RIGHTLIGHTS
                      Environment023
                      Quality016
                      Quantity014
                      PgmAvail01
                      Flicker008
                      Maintenance006
                      Lifetime006
                      SalesProductivity006
                      Safety005
                      Noise005
                      Control003
                      Illness0
                      PerformPerformPerform
                      FootprintFootprintFootprint
                      ControlFeaturesControlFeaturesControlFeatures
                      Tenant complTenant complTenant compl
                      NoiseNoiseNoise
                      MaintMaintMaint
                      OtherOtherOther
                      LifetimeLifetimeLifetime
                      Environment
                      Quality
                      Quantity
                      PgmAvail
                      Flicker
                      Maintenance
                      Lifetime
                      SalesProductivity
                      Safety
                      Noise
                      Control
                      Illness

                      Variability

                      Relevant NEB Categories

                      Program measure invariant (suitable for ldquoadderrdquo)

                      middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                      Program measure dependent

                      middot Economic ndash societal (depends on measures and local manufacture installation)

                      middot Health and safety health care illnesses ndash societal and participant (measure)

                      middot Water wastewater infrastructure and water bill savings ndash societal and participant

                      middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                      Climate dependent

                      middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                      Residential Target dependent (low income or MF vs SF)

                      middot Payment related ndash utility (arrearages etc stronger for low income targets)

                      middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

                      middot Participant benefits related to hardship and payments

                      middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                      Used for

                      Major Advantages

                      Major Disadvantages

                      Direct measurement (specific studies of changes on-site)

                      Comrsquol labor productivity etc

                      Direct precise attributable

                      Small estimation sample sizes specialized cases poor transferability expensive

                      Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

                      Insurance water health others

                      Long history easy secondary sources

                      Credible to reviewers vetted inputs

                      Not available for all NEBs

                      Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

                      Emissions economics

                      Third party peer-vetted models available for economics jobs and emissions

                      Not available for all NEBs

                      Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

                      Wide variety of Participant NEBs

                      Large sample sizes amp statistical properties

                      Affordable

                      Multiple estimates leading to similar ranges

                      Direct method of measuring some key NEBs

                      Concerns about surveys as a source of quantitative values amp reliability

                      Recall from survey respondents

                      Proper attribution to programs measures

                      MWxNEWxCA WxCA WxSF-NCMF-NCC- LiteC-RebC-Reb
                      mult1220813075104075084
                      LMS Verbal15610980612067115055091
                      WTP42242732
                      MWxMWxMWx
                      NEWxNEWxNEWx
                      CA WxCA WxCA Wx
                      CA WxCA WxCA Wx
                      SF-NCSF-NCSF-NC
                      MF-NCMF-NCMF-NC
                      C- LiteC- LiteC- Lite
                      C-RebC-RebC-Reb
                      C-RebC-RebC-Reb

                      SERA

                      HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)

                      11

                      Monetized NEBs

                      Direct Secondary Model Survey

                      Attributable Change (study)

                      Value or Financial

                      Calc

                      X

                      Total Attrib Stated Relative

                      Effect

                      Savings (or translation)(ldquoNormrdquo)

                      Individual NEB Shares

                      X

                      X

                      SERA

                      PARTICIPANT MEASUREMENT METHODS COMPARISON

                      005

                      115

                      225

                      335

                      445

                      MW

                      x

                      NE

                      Wx

                      CA

                      Wx

                      CA

                      Wx

                      SF-N

                      C

                      MF-

                      NC

                      C-

                      Lite

                      C-R

                      eb

                      C-R

                      eb

                      multLMS VerbalWTP

                      Source SERA Research

                      Chart1

                      mult
                      LMS Verbal
                      WTP
                      122
                      156
                      42
                      08
                      1
                      2
                      098
                      4
                      06
                      13
                      12
                      075
                      067
                      104
                      115
                      27
                      075
                      055
                      32
                      084
                      091

                      Sheet1

                      Sheet1

                      mult
                      LMS Verbal
                      WTP

                      Sheet2

                      Sheet3

                      SERA

                      PROS AND CONS OF NEBS MEASUREMENT METHODS

                      13

                      Source Skumatz SERA research

                      Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

                      Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

                      Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

                      Long history easy secondary sources Credible to reviewers vetted inputs

                      Not available for all NEBs

                      Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

                      Third party peer-vetted models available for economics jobs and emissions

                      Not available for all NEBs

                      Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

                      Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

                      Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

                      SERA

                      PROGRESS TRANSFERABILITY OF NEBS

                      14

                      Source Skumatz SERA research

                      Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

                      bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                      Program measure dependent

                      bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

                      education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                      Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                      Residential Target dependent (low income or MF vs SF)

                      bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

                      vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                      Cost savings consistency if transferable vs risk

                      SERA

                      RANGES TYPICAL NEB VALUES FOR WxRETROFI

                      15

                      SERA

                      PROGRESS AND GAPS Consistency Transferability

                      Some programs types others less studied Transferability research

                      Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

                      measures until then Across the board Savings share Regression

                      SERA

                      PROGRESS AND GAPS Gas vs Electric

                      Some researchndashusually program-wide not broken down International research found participant NEBs have

                      similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

                      MF Less-commonly-studied complicated by poor response

                      and complexity of sector (decision-maker some measures in home some central) separate from low income not common

                      Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

                      Commercial gaps in some measures

                      SERA

                      USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                      Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                      18

                      0

                      005

                      01

                      015

                      02

                      025

                      Environment

                      Quality

                      Quantity

                      PgmAvailFlicker

                      Mainten

                      ance

                      Lifetime

                      SalesP

                      roductivity

                      Safety

                      Noise

                      Control

                      Illness

                      Total NEBs Multiplier=X

                      $

                      Savings value=$X

                      Chart3

                      023
                      016
                      014
                      01
                      008
                      006
                      006
                      006
                      005
                      005
                      003
                      0

                      Chart1

                      Vendor
                      NP
                      Partic
                      028
                      02
                      02
                      028
                      013
                      017
                      031
                      022
                      016
                      006
                      008
                      014
                      005
                      006
                      014
                      004
                      009
                      007
                      004
                      003
                      007
                      -006
                      018
                      006

                      Sheet1

                      Sheet1

                      Sheet2

                      Sheet3

                      SERA

                      NEB VALUES

                      19

                      SERA

                      ARE NEBS HIGH VALUE Energy savings are often the minority of

                      benefits from program measuresndash Donrsquot ignore that

                      20

                      0 05 1 15 2 25 3

                      Energy Sav

                      NEBs

                      NEB vs Energy Savings ValueIncluding all NEBs

                      Source (SkumatzSERA 2010 amp others)

                      Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                      SERA

                      RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                      MF Many others

                      Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                      Variety of end uses

                      SERA

                      TOP NEBS FOR WEATHERIZATION PROGRAM

                      Weatherization program multipliergt1

                      Source SERA Research

                      SERA

                      RESULTS FOR ENERGY STARreg APPLIANCES

                      0002004006008

                      01012014016018

                      Maint

                      Appear

                      Perf LifeNoise Satis

                      Comfort

                      LiteSafe

                      ty SellMove

                      Water

                      Enviro

                      Refrig DW CW

                      Source SERA Research

                      SERA

                      INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                      5 increments to 50

                      Source SERA research

                      Chart5

                      EnergySmart (insul Retrofit)
                      Dunedin (insul)
                      001
                      024
                      002
                      004
                      002
                      005
                      006
                      005
                      007
                      008
                      008
                      006
                      013
                      0
                      013
                      016
                      049
                      033

                      cx

                      cx

                      nz

                      Sheet3

                      Sheet3

                      Super Insulation
                      Double Glazing
                      Solar Design
                      Solar Water Heating
                      Super Insulation
                      Double Glazing
                      Solar Design
                      Solar Water Heating
                      EnergySmart (insul Retrofit)
                      Dunedin (insul)

                      SERA

                      ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                      0

                      002

                      004

                      006

                      008

                      01

                      012

                      Maint

                      Perf LifeSatis

                      f

                      ComfortAesth

                      NoiseSafe

                      ty

                      LightingSelli

                      ng

                      Stay in home

                      Enviro

                      Sick Days

                      Calls

                      ES Homes HP wES Energy savings multiplier=1

                      Source SERA Research

                      SERA

                      CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                      DG CHP HVAC Equipment rebate Other

                      Building codes incentives by cities

                      Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                      Variety of end uses

                      SERA

                      AUDIT FINANCIAL PROGRAM

                      0

                      01

                      02

                      03

                      04

                      05

                      06

                      07

                      Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                      Source SERA Research

                      SERA

                      CampI NEW CONSTRUCTION

                      -005

                      0

                      005

                      01

                      015

                      02

                      AampE Owner(Pct of Total Partic NEBs)

                      Source SERA Research

                      SERA

                      CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                      0 2 4 6 8 10 12

                      m Product losses

                      l Ease of selling leasing building

                      n Labor requirements

                      h Aesthetics appearance

                      j Noise

                      b Equipment maintenance

                      f Tenant satisfaction

                      d Equipment lifetimes

                      e Productivity

                      k Building safety[2]

                      g Comfort

                      i Lighting Quality of Light

                      c Equipment performance[1]

                      a Operating costs

                      o Doing good for the Environment[3]

                      Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                      vcv

                      Source SERA Research

                      Chart1

                      CampI Technical Assistance NEBs
                      002
                      003
                      004
                      005
                      005
                      006
                      006
                      007
                      007
                      007
                      009
                      009
                      01
                      011
                      011

                      Sheet1

                      Sheet1

                      CampI Technical Assistance Program NEBs (Participant)

                      Sheet2

                      Sheet3

                      SERA

                      EXPRESSING NEBS VALUEndashCxe

                      Strong value from RetroCxSource SERA Research

                      SERA

                      SOCIETAL NEB EXAMPLES

                      31

                      SERA

                      JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                      32

                      0

                      02

                      04

                      06

                      08

                      1

                      12

                      HPWxRetrof Appliance

                      CA

                      WI

                      Natl

                      (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                      Jobs Economic

                      SERA

                      EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                      33

                      SERA

                      UTILITY NEBSUtility NEBs for Template Program

                      Debt WriteOff (util)13

                      Rate Subsidy(util)61

                      HealthSafety(util)0

                      Colln Costs (util)0

                      Gas Calls (util)0

                      Calls to CSRs(util)2

                      TampD (util)16

                      Arrears (util)0

                      Reconnects (util)0

                      Notices (util)7

                      Shutoffs (util)1

                      Rate subsidy TampD

                      Payment-related

                      Source Skumatz Economic Research Associates research

                      Example Low Income Weatherization

                      SERA

                      ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                      be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                      Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                      Value of 100 HALVES Payback doubles ROI

                      Source Skumatz Economics (SERA)

                      35

                      ( for NEBs Multiplier of 125)

                      SERA

                      NEBS IN COST-EFFECTIVENESS APPLICATIONS

                      36

                      hellip2001 ON

                      SERA

                      37

                      NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                      For true representation of B amp C elements of NEBs address missing factor bias

                      Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                      BC applications were considered early on then pulled back awaiting more quantitative evidence

                      SERA

                      CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                      38

                      Test Benefits Costs States Using Traditionally

                      Improved treat-ment with NEBs

                      Utility Cost (or Program Administrator Test) (UCT or PAC)

                      bull Avoided supply costs for transmission distribution and generation (TDampG)

                      bull Avoided gas and water supply costs

                      bull Program administration

                      bull Participant incentives

                      bull Increased supply cost

                      CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                      Use cost only paid by the utility

                      Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                      Same as above plus bull increased revenue

                      Same as above plus bull Decreased revenue

                      AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                      Participant cost bull Utility bill reductions bull Participant incentives

                      bull Participant direct costs

                      AR CA FL HI IA IN MI MN NY VA

                      Participant NEBs

                      Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                      Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                      SERA

                      APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                      Expansion Concern with risk calculation accuracy

                      39

                      Source Skumatz SERA research

                      SERA

                      40

                      METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                      Maximize DSM opportunities amp feedback Accuracy tailoring

                      Minimize Regulatory amp Implementer Risk

                      Minimize Evaluation Cost

                      Adder

                      Readily MeasurableHybrid

                      All NEBsSource SERA Research

                      SERA

                      41

                      STATE TREATMENT OF NEBS NEBsRegulatory

                      Screening Application Utilities regions

                      Program Marketing Fairly widespread use in utilities states across the country

                      MO

                      RE AGGRESSIVE ==gt

                      Test Pgm Screen - adder

                      IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                      Test Pgm Screen - readily measured

                      MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                      Test Hybrid (potential adder amp measured)

                      CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                      Test Pgm screen - Broad

                      With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                      Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                      SERA

                      42

                      Key

                      States with NEBs in CE tests for At least one type of program

                      copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                      STATES WITH NEBS IN CE TESTS

                      NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                      groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                      SERA

                      SELECTED STATE PROGRESS CA

                      2001 ndash LIPPT model NY

                      Mid-2000s measurement amp scenarios not included CO

                      Adders (10 electric 25 LI 5 gas) VT

                      15 adder helped by previous research DC

                      10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                      43

                      Source Skumatz SERA research

                      SERA

                      SELECTED STATE PROGRESS Dominos ongoing

                      Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                      Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                      Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                      Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                      44

                      Source Skumatz SERA research

                      SERA

                      APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                      Expansion Concern with risk calculation accuracy

                      45

                      Source Skumatz SERA research

                      SERA

                      46

                      NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                      [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                      SourceSkumatz SERA Independent research

                      NTG ndash accuracy measurement incomplete

                      Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                      NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                      EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                      Cost Complicated expensive local changes

                      RELATIVE Risk from NEBs

                      Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                      Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                      NEBs as accurate as many inputs (all could improve)

                      SERA

                      DISCOUNT RATE MATTERS

                      47

                      Source Skumatz SERA Research

                      0

                      20

                      40

                      60

                      80

                      100

                      120

                      1 2 3 4 5 6 7 8 9 10

                      PV Ratios vs 1 Discount Rate

                      10 yr PV 20 yr PV

                      Discount rate - percent

                      WACCSocialLTT

                      Source Skumatz Electricity Journal IEPEC paper independent research

                      SERA

                      EUL VARIATIONS-ADOPTED VALUES

                      Source Skumatz Economic Research Associates SERA independent research

                      0

                      50

                      100

                      150

                      200

                      250

                      300

                      350

                      400

                      450

                      0 10 20 30 40 50 60 70

                      EUL Variation by Measure

                      67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                      Poorly supportedOldTechnology change

                      SERA

                      KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                      Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                      measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                      change the BC conclusion

                      On the cusphellip

                      If NO Yoursquore doneAnd bias addressedsufficiently

                      IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                      Source SERA all rights reserved

                      SERArsquos lsquoNEB-ItrdquoModel ndash

                      SERA

                      IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                      Utility Soc Part Conserv Recrsquom

                      Rationale

                      Base Percent X X X Program-invariant

                      Low Income X X X X Multiple sources

                      Weatherization X X X Substantial Participantimpacts

                      Measure Program-specific

                      X Varies by measure sector

                      Other Recomrsquos Local Research

                      50

                      Source Skumatz SERA

                      Developing values for multiple states amp utilities

                      SERA

                      TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                      Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                      Other sources of bias to research (EUL NTG etc) bring high variation

                      5) States are incorporating NEBs dominos input in deliberations in multiple states

                      6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                      THANK YOU

                      Questions

                      Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                      SERA

                      POWER OF NEBS ndashBARRIERS DISCONNECTS

                      53

                      SERA

                      NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                      Negative NEB values cost of barrier

                      Solar Water Heat NZ$ Euros

                      Solar Design NZ$ Euros

                      Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                      -23 -12 (079)

                      -11 -6 (06)

                      Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                      Source Skumatz Economic Research Associates research

                      SERA

                      CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                      NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                      -5-4-3-2-1012345

                      Op Co

                      st

                      Init C

                      ost

                      Maint

                      Perfo

                      rm

                      Prod

                      uctiv

                      Tena

                      nt Sa

                      t

                      Comf

                      ort

                      Appe

                      ar

                      Light

                      Qual

                      SellL

                      ease

                      Devp Ownr AampE

                      Note AgtE (~4 vs ~2) for these NEBs

                      Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                      Source Skumatz Economic Research Associates Inc 2002

                      Owners more positive ndashUnderinvestment train

                      • Soup to nuts on non-energy benefits NEIs
                      • TOPICS
                      • BACKGROUND NEBS
                      • 20+ YEARS OF NEBS PROGRESShellip
                      • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                      • KEY APPLICATIONS OF NEBS
                      • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                      • Neb measurement
                      • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                      • MEASURING PARTICIPANTNEBs
                      • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                      • PARTICIPANT MEASUREMENT METHODS COMPARISON
                      • PROS AND CONS OF NEBS MEASUREMENT METHODS
                      • PROGRESS TRANSFERABILITY OF NEBS
                      • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                      • PROGRESS AND GAPS
                      • PROGRESS AND GAPS
                      • USING NEB NEI VALUES
                      • Slide Number 19
                      • ARE NEBS HIGH VALUE
                      • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                      • TOP NEBS FOR WEATHERIZATION PROGRAM
                      • RESULTS FOR ENERGY STARreg APPLIANCES
                      • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                      • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                      • CampI PROGRAMS ldquoNEBBEDrdquo
                      • AUDIT FINANCIAL PROGRAM
                      • CampI NEW CONSTRUCTION
                      • CampI TECHNICAL ASSISTANCE
                      • EXPRESSING NEBS VALUEndashCxe
                      • Slide Number 31
                      • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                      • EMISSIONS (SOCIETAL NEBS)
                      • UTILITY NEBS
                      • ADJUSTED PAYBACKS
                      • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                      • NEBS IN BC
                      • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                      • APPLICATION TO BC
                      • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                      • STATE TREATMENT OF NEBS NEBs
                      • Slide Number 42
                      • SELECTED STATE PROGRESS
                      • SELECTED STATE PROGRESS
                      • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                      • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                      • DISCOUNT RATE MATTERS
                      • EUL VARIATIONS- ADOPTED VALUES
                      • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                      • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                      • TAKEAWAYSON NEBSNEIS
                      • THANK YOUQuestions
                      • Slide Number 53
                      • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                      • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                        Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                        m Product losses20
                        l Ease of selling leasing building33
                        n Labor requirements45
                        h Aesthetics appearance59
                        j Noise50
                        b Equipment maintenance660
                        f Tenant satisfaction60
                        d Equipment lifetimes718
                        e Productivity724
                        k Building safety[2]73
                        g Comfort933
                        i Lighting Quality of Light916
                        c Equipment performance[1]1019
                        a Operating costs1165
                        o Doing good for the Environment[3]1132
                        Other03
                        [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                        [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                        [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                        m Product losses
                        l Ease of selling leasing building
                        n Labor requirements
                        h Aesthetics appearance
                        j Noise
                        b Equipment maintenance
                        f Tenant satisfaction
                        d Equipment lifetimes
                        e Productivity
                        k Building safety[2]
                        g Comfort
                        i Lighting Quality of Light
                        c Equipment performance[1]
                        a Operating costs
                        o Doing good for the Environment[3]
                        Share of ZALEH NEBS from various measures
                        Each presented as multiple of the (overall) energy savings
                        Super Insulation037
                        Double Glazing026
                        Solar Design025
                        Solar Water Heating005
                        EnergySmart (insul Retrofit)Dunedin (insul)
                        Other124
                        Noise24
                        Budgetfinances25
                        Environmental65
                        Health78
                        Reduced moves86
                        Ease of sellingrenting130
                        DampnessMold1316
                        Comfort-related4933
                        101101
                        Operational NEBs44
                        DesignConstruction NEBs31
                        Occupant NEBs25
                        1
                        OtherOther
                        NoiseNoise
                        BudgetfinancesBudgetfinances
                        EnvironmentalEnvironmental
                        HealthHealth
                        Reduced movesReduced moves
                        Ease of sellingrentingEase of sellingrenting
                        DampnessMoldDampnessMold
                        Comfort-relatedComfort-related
                        Boiler
                        VendorNPPartic
                        Perform0280202
                        Footprint028013017
                        ControlFeatures031022016
                        Tenant compl006008014
                        Noise005006014
                        Maint004009007
                        Other004003007
                        Lifetime-006018006
                        MF New construction energy star
                        Operating cost014
                        Environment012
                        Maintenance01
                        Lifetime01
                        Performance009
                        Occupant Satisf009
                        Comfort008
                        Noise006
                        Sellingleasing006
                        Aesthetics005
                        Light Quality005
                        Power QualReliab004
                        Safety002
                        RIGHTLIGHTS
                        Environment023
                        Quality016
                        Quantity014
                        PgmAvail01
                        Flicker008
                        Maintenance006
                        Lifetime006
                        SalesProductivity006
                        Safety005
                        Noise005
                        Control003
                        Illness0
                        PerformPerformPerform
                        FootprintFootprintFootprint
                        ControlFeaturesControlFeaturesControlFeatures
                        Tenant complTenant complTenant compl
                        NoiseNoiseNoise
                        MaintMaintMaint
                        OtherOtherOther
                        LifetimeLifetimeLifetime
                        Environment
                        Quality
                        Quantity
                        PgmAvail
                        Flicker
                        Maintenance
                        Lifetime
                        SalesProductivity
                        Safety
                        Noise
                        Control
                        Illness

                        Variability

                        Relevant NEB Categories

                        Program measure invariant (suitable for ldquoadderrdquo)

                        middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                        Program measure dependent

                        middot Economic ndash societal (depends on measures and local manufacture installation)

                        middot Health and safety health care illnesses ndash societal and participant (measure)

                        middot Water wastewater infrastructure and water bill savings ndash societal and participant

                        middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                        Climate dependent

                        middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                        Residential Target dependent (low income or MF vs SF)

                        middot Payment related ndash utility (arrearages etc stronger for low income targets)

                        middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

                        middot Participant benefits related to hardship and payments

                        middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                        Used for

                        Major Advantages

                        Major Disadvantages

                        Direct measurement (specific studies of changes on-site)

                        Comrsquol labor productivity etc

                        Direct precise attributable

                        Small estimation sample sizes specialized cases poor transferability expensive

                        Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

                        Insurance water health others

                        Long history easy secondary sources

                        Credible to reviewers vetted inputs

                        Not available for all NEBs

                        Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

                        Emissions economics

                        Third party peer-vetted models available for economics jobs and emissions

                        Not available for all NEBs

                        Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

                        Wide variety of Participant NEBs

                        Large sample sizes amp statistical properties

                        Affordable

                        Multiple estimates leading to similar ranges

                        Direct method of measuring some key NEBs

                        Concerns about surveys as a source of quantitative values amp reliability

                        Recall from survey respondents

                        Proper attribution to programs measures

                        MWxNEWxCA WxCA WxSF-NCMF-NCC- LiteC-RebC-Reb
                        mult1220813075104075084
                        LMS Verbal15610980612067115055091
                        WTP42242732
                        MWxMWxMWx
                        NEWxNEWxNEWx
                        CA WxCA WxCA Wx
                        CA WxCA WxCA Wx
                        SF-NCSF-NCSF-NC
                        MF-NCMF-NCMF-NC
                        C- LiteC- LiteC- Lite
                        C-RebC-RebC-Reb
                        C-RebC-RebC-Reb

                        SERA

                        PARTICIPANT MEASUREMENT METHODS COMPARISON

                        005

                        115

                        225

                        335

                        445

                        MW

                        x

                        NE

                        Wx

                        CA

                        Wx

                        CA

                        Wx

                        SF-N

                        C

                        MF-

                        NC

                        C-

                        Lite

                        C-R

                        eb

                        C-R

                        eb

                        multLMS VerbalWTP

                        Source SERA Research

                        Chart1

                        mult
                        LMS Verbal
                        WTP
                        122
                        156
                        42
                        08
                        1
                        2
                        098
                        4
                        06
                        13
                        12
                        075
                        067
                        104
                        115
                        27
                        075
                        055
                        32
                        084
                        091

                        Sheet1

                        Sheet1

                        mult
                        LMS Verbal
                        WTP

                        Sheet2

                        Sheet3

                        SERA

                        PROS AND CONS OF NEBS MEASUREMENT METHODS

                        13

                        Source Skumatz SERA research

                        Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

                        Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

                        Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

                        Long history easy secondary sources Credible to reviewers vetted inputs

                        Not available for all NEBs

                        Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

                        Third party peer-vetted models available for economics jobs and emissions

                        Not available for all NEBs

                        Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

                        Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

                        Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

                        SERA

                        PROGRESS TRANSFERABILITY OF NEBS

                        14

                        Source Skumatz SERA research

                        Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

                        bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                        Program measure dependent

                        bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

                        education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                        Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                        Residential Target dependent (low income or MF vs SF)

                        bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

                        vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                        Cost savings consistency if transferable vs risk

                        SERA

                        RANGES TYPICAL NEB VALUES FOR WxRETROFI

                        15

                        SERA

                        PROGRESS AND GAPS Consistency Transferability

                        Some programs types others less studied Transferability research

                        Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

                        measures until then Across the board Savings share Regression

                        SERA

                        PROGRESS AND GAPS Gas vs Electric

                        Some researchndashusually program-wide not broken down International research found participant NEBs have

                        similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

                        MF Less-commonly-studied complicated by poor response

                        and complexity of sector (decision-maker some measures in home some central) separate from low income not common

                        Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

                        Commercial gaps in some measures

                        SERA

                        USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                        Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                        18

                        0

                        005

                        01

                        015

                        02

                        025

                        Environment

                        Quality

                        Quantity

                        PgmAvailFlicker

                        Mainten

                        ance

                        Lifetime

                        SalesP

                        roductivity

                        Safety

                        Noise

                        Control

                        Illness

                        Total NEBs Multiplier=X

                        $

                        Savings value=$X

                        Chart3

                        023
                        016
                        014
                        01
                        008
                        006
                        006
                        006
                        005
                        005
                        003
                        0

                        Chart1

                        Vendor
                        NP
                        Partic
                        028
                        02
                        02
                        028
                        013
                        017
                        031
                        022
                        016
                        006
                        008
                        014
                        005
                        006
                        014
                        004
                        009
                        007
                        004
                        003
                        007
                        -006
                        018
                        006

                        Sheet1

                        Sheet1

                        Sheet2

                        Sheet3

                        SERA

                        NEB VALUES

                        19

                        SERA

                        ARE NEBS HIGH VALUE Energy savings are often the minority of

                        benefits from program measuresndash Donrsquot ignore that

                        20

                        0 05 1 15 2 25 3

                        Energy Sav

                        NEBs

                        NEB vs Energy Savings ValueIncluding all NEBs

                        Source (SkumatzSERA 2010 amp others)

                        Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                        SERA

                        RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                        MF Many others

                        Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                        Variety of end uses

                        SERA

                        TOP NEBS FOR WEATHERIZATION PROGRAM

                        Weatherization program multipliergt1

                        Source SERA Research

                        SERA

                        RESULTS FOR ENERGY STARreg APPLIANCES

                        0002004006008

                        01012014016018

                        Maint

                        Appear

                        Perf LifeNoise Satis

                        Comfort

                        LiteSafe

                        ty SellMove

                        Water

                        Enviro

                        Refrig DW CW

                        Source SERA Research

                        SERA

                        INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                        5 increments to 50

                        Source SERA research

                        Chart5

                        EnergySmart (insul Retrofit)
                        Dunedin (insul)
                        001
                        024
                        002
                        004
                        002
                        005
                        006
                        005
                        007
                        008
                        008
                        006
                        013
                        0
                        013
                        016
                        049
                        033

                        cx

                        cx

                        nz

                        Sheet3

                        Sheet3

                        Super Insulation
                        Double Glazing
                        Solar Design
                        Solar Water Heating
                        Super Insulation
                        Double Glazing
                        Solar Design
                        Solar Water Heating
                        EnergySmart (insul Retrofit)
                        Dunedin (insul)

                        SERA

                        ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                        0

                        002

                        004

                        006

                        008

                        01

                        012

                        Maint

                        Perf LifeSatis

                        f

                        ComfortAesth

                        NoiseSafe

                        ty

                        LightingSelli

                        ng

                        Stay in home

                        Enviro

                        Sick Days

                        Calls

                        ES Homes HP wES Energy savings multiplier=1

                        Source SERA Research

                        SERA

                        CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                        DG CHP HVAC Equipment rebate Other

                        Building codes incentives by cities

                        Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                        Variety of end uses

                        SERA

                        AUDIT FINANCIAL PROGRAM

                        0

                        01

                        02

                        03

                        04

                        05

                        06

                        07

                        Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                        Source SERA Research

                        SERA

                        CampI NEW CONSTRUCTION

                        -005

                        0

                        005

                        01

                        015

                        02

                        AampE Owner(Pct of Total Partic NEBs)

                        Source SERA Research

                        SERA

                        CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                        0 2 4 6 8 10 12

                        m Product losses

                        l Ease of selling leasing building

                        n Labor requirements

                        h Aesthetics appearance

                        j Noise

                        b Equipment maintenance

                        f Tenant satisfaction

                        d Equipment lifetimes

                        e Productivity

                        k Building safety[2]

                        g Comfort

                        i Lighting Quality of Light

                        c Equipment performance[1]

                        a Operating costs

                        o Doing good for the Environment[3]

                        Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                        vcv

                        Source SERA Research

                        Chart1

                        CampI Technical Assistance NEBs
                        002
                        003
                        004
                        005
                        005
                        006
                        006
                        007
                        007
                        007
                        009
                        009
                        01
                        011
                        011

                        Sheet1

                        Sheet1

                        CampI Technical Assistance Program NEBs (Participant)

                        Sheet2

                        Sheet3

                        SERA

                        EXPRESSING NEBS VALUEndashCxe

                        Strong value from RetroCxSource SERA Research

                        SERA

                        SOCIETAL NEB EXAMPLES

                        31

                        SERA

                        JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                        32

                        0

                        02

                        04

                        06

                        08

                        1

                        12

                        HPWxRetrof Appliance

                        CA

                        WI

                        Natl

                        (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                        Jobs Economic

                        SERA

                        EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                        33

                        SERA

                        UTILITY NEBSUtility NEBs for Template Program

                        Debt WriteOff (util)13

                        Rate Subsidy(util)61

                        HealthSafety(util)0

                        Colln Costs (util)0

                        Gas Calls (util)0

                        Calls to CSRs(util)2

                        TampD (util)16

                        Arrears (util)0

                        Reconnects (util)0

                        Notices (util)7

                        Shutoffs (util)1

                        Rate subsidy TampD

                        Payment-related

                        Source Skumatz Economic Research Associates research

                        Example Low Income Weatherization

                        SERA

                        ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                        be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                        Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                        Value of 100 HALVES Payback doubles ROI

                        Source Skumatz Economics (SERA)

                        35

                        ( for NEBs Multiplier of 125)

                        SERA

                        NEBS IN COST-EFFECTIVENESS APPLICATIONS

                        36

                        hellip2001 ON

                        SERA

                        37

                        NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                        For true representation of B amp C elements of NEBs address missing factor bias

                        Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                        BC applications were considered early on then pulled back awaiting more quantitative evidence

                        SERA

                        CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                        38

                        Test Benefits Costs States Using Traditionally

                        Improved treat-ment with NEBs

                        Utility Cost (or Program Administrator Test) (UCT or PAC)

                        bull Avoided supply costs for transmission distribution and generation (TDampG)

                        bull Avoided gas and water supply costs

                        bull Program administration

                        bull Participant incentives

                        bull Increased supply cost

                        CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                        Use cost only paid by the utility

                        Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                        Same as above plus bull increased revenue

                        Same as above plus bull Decreased revenue

                        AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                        Participant cost bull Utility bill reductions bull Participant incentives

                        bull Participant direct costs

                        AR CA FL HI IA IN MI MN NY VA

                        Participant NEBs

                        Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                        Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                        SERA

                        APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                        Expansion Concern with risk calculation accuracy

                        39

                        Source Skumatz SERA research

                        SERA

                        40

                        METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                        Maximize DSM opportunities amp feedback Accuracy tailoring

                        Minimize Regulatory amp Implementer Risk

                        Minimize Evaluation Cost

                        Adder

                        Readily MeasurableHybrid

                        All NEBsSource SERA Research

                        SERA

                        41

                        STATE TREATMENT OF NEBS NEBsRegulatory

                        Screening Application Utilities regions

                        Program Marketing Fairly widespread use in utilities states across the country

                        MO

                        RE AGGRESSIVE ==gt

                        Test Pgm Screen - adder

                        IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                        Test Pgm Screen - readily measured

                        MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                        Test Hybrid (potential adder amp measured)

                        CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                        Test Pgm screen - Broad

                        With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                        Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                        SERA

                        42

                        Key

                        States with NEBs in CE tests for At least one type of program

                        copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                        STATES WITH NEBS IN CE TESTS

                        NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                        groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                        SERA

                        SELECTED STATE PROGRESS CA

                        2001 ndash LIPPT model NY

                        Mid-2000s measurement amp scenarios not included CO

                        Adders (10 electric 25 LI 5 gas) VT

                        15 adder helped by previous research DC

                        10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                        43

                        Source Skumatz SERA research

                        SERA

                        SELECTED STATE PROGRESS Dominos ongoing

                        Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                        Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                        Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                        Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                        44

                        Source Skumatz SERA research

                        SERA

                        APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                        Expansion Concern with risk calculation accuracy

                        45

                        Source Skumatz SERA research

                        SERA

                        46

                        NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                        [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                        SourceSkumatz SERA Independent research

                        NTG ndash accuracy measurement incomplete

                        Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                        NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                        EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                        Cost Complicated expensive local changes

                        RELATIVE Risk from NEBs

                        Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                        Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                        NEBs as accurate as many inputs (all could improve)

                        SERA

                        DISCOUNT RATE MATTERS

                        47

                        Source Skumatz SERA Research

                        0

                        20

                        40

                        60

                        80

                        100

                        120

                        1 2 3 4 5 6 7 8 9 10

                        PV Ratios vs 1 Discount Rate

                        10 yr PV 20 yr PV

                        Discount rate - percent

                        WACCSocialLTT

                        Source Skumatz Electricity Journal IEPEC paper independent research

                        SERA

                        EUL VARIATIONS-ADOPTED VALUES

                        Source Skumatz Economic Research Associates SERA independent research

                        0

                        50

                        100

                        150

                        200

                        250

                        300

                        350

                        400

                        450

                        0 10 20 30 40 50 60 70

                        EUL Variation by Measure

                        67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                        Poorly supportedOldTechnology change

                        SERA

                        KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                        Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                        measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                        change the BC conclusion

                        On the cusphellip

                        If NO Yoursquore doneAnd bias addressedsufficiently

                        IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                        Source SERA all rights reserved

                        SERArsquos lsquoNEB-ItrdquoModel ndash

                        SERA

                        IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                        Utility Soc Part Conserv Recrsquom

                        Rationale

                        Base Percent X X X Program-invariant

                        Low Income X X X X Multiple sources

                        Weatherization X X X Substantial Participantimpacts

                        Measure Program-specific

                        X Varies by measure sector

                        Other Recomrsquos Local Research

                        50

                        Source Skumatz SERA

                        Developing values for multiple states amp utilities

                        SERA

                        TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                        Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                        Other sources of bias to research (EUL NTG etc) bring high variation

                        5) States are incorporating NEBs dominos input in deliberations in multiple states

                        6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                        THANK YOU

                        Questions

                        Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                        SERA

                        POWER OF NEBS ndashBARRIERS DISCONNECTS

                        53

                        SERA

                        NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                        Negative NEB values cost of barrier

                        Solar Water Heat NZ$ Euros

                        Solar Design NZ$ Euros

                        Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                        -23 -12 (079)

                        -11 -6 (06)

                        Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                        Source Skumatz Economic Research Associates research

                        SERA

                        CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                        NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                        -5-4-3-2-1012345

                        Op Co

                        st

                        Init C

                        ost

                        Maint

                        Perfo

                        rm

                        Prod

                        uctiv

                        Tena

                        nt Sa

                        t

                        Comf

                        ort

                        Appe

                        ar

                        Light

                        Qual

                        SellL

                        ease

                        Devp Ownr AampE

                        Note AgtE (~4 vs ~2) for these NEBs

                        Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                        Source Skumatz Economic Research Associates Inc 2002

                        Owners more positive ndashUnderinvestment train

                        • Soup to nuts on non-energy benefits NEIs
                        • TOPICS
                        • BACKGROUND NEBS
                        • 20+ YEARS OF NEBS PROGRESShellip
                        • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                        • KEY APPLICATIONS OF NEBS
                        • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                        • Neb measurement
                        • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                        • MEASURING PARTICIPANTNEBs
                        • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                        • PARTICIPANT MEASUREMENT METHODS COMPARISON
                        • PROS AND CONS OF NEBS MEASUREMENT METHODS
                        • PROGRESS TRANSFERABILITY OF NEBS
                        • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                        • PROGRESS AND GAPS
                        • PROGRESS AND GAPS
                        • USING NEB NEI VALUES
                        • Slide Number 19
                        • ARE NEBS HIGH VALUE
                        • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                        • TOP NEBS FOR WEATHERIZATION PROGRAM
                        • RESULTS FOR ENERGY STARreg APPLIANCES
                        • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                        • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                        • CampI PROGRAMS ldquoNEBBEDrdquo
                        • AUDIT FINANCIAL PROGRAM
                        • CampI NEW CONSTRUCTION
                        • CampI TECHNICAL ASSISTANCE
                        • EXPRESSING NEBS VALUEndashCxe
                        • Slide Number 31
                        • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                        • EMISSIONS (SOCIETAL NEBS)
                        • UTILITY NEBS
                        • ADJUSTED PAYBACKS
                        • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                        • NEBS IN BC
                        • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                        • APPLICATION TO BC
                        • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                        • STATE TREATMENT OF NEBS NEBs
                        • Slide Number 42
                        • SELECTED STATE PROGRESS
                        • SELECTED STATE PROGRESS
                        • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                        • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                        • DISCOUNT RATE MATTERS
                        • EUL VARIATIONS- ADOPTED VALUES
                        • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                        • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                        • TAKEAWAYSON NEBSNEIS
                        • THANK YOUQuestions
                        • Slide Number 53
                        • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                        • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                          Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                          m Product losses20
                          l Ease of selling leasing building33
                          n Labor requirements45
                          h Aesthetics appearance59
                          j Noise50
                          b Equipment maintenance660
                          f Tenant satisfaction60
                          d Equipment lifetimes718
                          e Productivity724
                          k Building safety[2]73
                          g Comfort933
                          i Lighting Quality of Light916
                          c Equipment performance[1]1019
                          a Operating costs1165
                          o Doing good for the Environment[3]1132
                          Other03
                          [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                          [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                          [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                          m Product losses
                          l Ease of selling leasing building
                          n Labor requirements
                          h Aesthetics appearance
                          j Noise
                          b Equipment maintenance
                          f Tenant satisfaction
                          d Equipment lifetimes
                          e Productivity
                          k Building safety[2]
                          g Comfort
                          i Lighting Quality of Light
                          c Equipment performance[1]
                          a Operating costs
                          o Doing good for the Environment[3]
                          Share of ZALEH NEBS from various measures
                          Each presented as multiple of the (overall) energy savings
                          Super Insulation037
                          Double Glazing026
                          Solar Design025
                          Solar Water Heating005
                          EnergySmart (insul Retrofit)Dunedin (insul)
                          Other124
                          Noise24
                          Budgetfinances25
                          Environmental65
                          Health78
                          Reduced moves86
                          Ease of sellingrenting130
                          DampnessMold1316
                          Comfort-related4933
                          101101
                          Operational NEBs44
                          DesignConstruction NEBs31
                          Occupant NEBs25
                          1
                          OtherOther
                          NoiseNoise
                          BudgetfinancesBudgetfinances
                          EnvironmentalEnvironmental
                          HealthHealth
                          Reduced movesReduced moves
                          Ease of sellingrentingEase of sellingrenting
                          DampnessMoldDampnessMold
                          Comfort-relatedComfort-related
                          Boiler
                          VendorNPPartic
                          Perform0280202
                          Footprint028013017
                          ControlFeatures031022016
                          Tenant compl006008014
                          Noise005006014
                          Maint004009007
                          Other004003007
                          Lifetime-006018006
                          MF New construction energy star
                          Operating cost014
                          Environment012
                          Maintenance01
                          Lifetime01
                          Performance009
                          Occupant Satisf009
                          Comfort008
                          Noise006
                          Sellingleasing006
                          Aesthetics005
                          Light Quality005
                          Power QualReliab004
                          Safety002
                          RIGHTLIGHTS
                          Environment023
                          Quality016
                          Quantity014
                          PgmAvail01
                          Flicker008
                          Maintenance006
                          Lifetime006
                          SalesProductivity006
                          Safety005
                          Noise005
                          Control003
                          Illness0
                          PerformPerformPerform
                          FootprintFootprintFootprint
                          ControlFeaturesControlFeaturesControlFeatures
                          Tenant complTenant complTenant compl
                          NoiseNoiseNoise
                          MaintMaintMaint
                          OtherOtherOther
                          LifetimeLifetimeLifetime
                          Environment
                          Quality
                          Quantity
                          PgmAvail
                          Flicker
                          Maintenance
                          Lifetime
                          SalesProductivity
                          Safety
                          Noise
                          Control
                          Illness

                          Variability

                          Relevant NEB Categories

                          Program measure invariant (suitable for ldquoadderrdquo)

                          middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                          Program measure dependent

                          middot Economic ndash societal (depends on measures and local manufacture installation)

                          middot Health and safety health care illnesses ndash societal and participant (measure)

                          middot Water wastewater infrastructure and water bill savings ndash societal and participant

                          middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                          Climate dependent

                          middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                          Residential Target dependent (low income or MF vs SF)

                          middot Payment related ndash utility (arrearages etc stronger for low income targets)

                          middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

                          middot Participant benefits related to hardship and payments

                          middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                          Used for

                          Major Advantages

                          Major Disadvantages

                          Direct measurement (specific studies of changes on-site)

                          Comrsquol labor productivity etc

                          Direct precise attributable

                          Small estimation sample sizes specialized cases poor transferability expensive

                          Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

                          Insurance water health others

                          Long history easy secondary sources

                          Credible to reviewers vetted inputs

                          Not available for all NEBs

                          Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

                          Emissions economics

                          Third party peer-vetted models available for economics jobs and emissions

                          Not available for all NEBs

                          Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

                          Wide variety of Participant NEBs

                          Large sample sizes amp statistical properties

                          Affordable

                          Multiple estimates leading to similar ranges

                          Direct method of measuring some key NEBs

                          Concerns about surveys as a source of quantitative values amp reliability

                          Recall from survey respondents

                          Proper attribution to programs measures

                          MWxNEWxCA WxCA WxSF-NCMF-NCC- LiteC-RebC-Reb
                          mult1220813075104075084
                          LMS Verbal15610980612067115055091
                          WTP42242732
                          MWxMWxMWx
                          NEWxNEWxNEWx
                          CA WxCA WxCA Wx
                          CA WxCA WxCA Wx
                          SF-NCSF-NCSF-NC
                          MF-NCMF-NCMF-NC
                          C- LiteC- LiteC- Lite
                          C-RebC-RebC-Reb
                          C-RebC-RebC-Reb

                          Chart1

                          mult
                          LMS Verbal
                          WTP
                          122
                          156
                          42
                          08
                          1
                          2
                          098
                          4
                          06
                          13
                          12
                          075
                          067
                          104
                          115
                          27
                          075
                          055
                          32
                          084
                          091

                          Sheet1

                          Sheet1

                          mult
                          LMS Verbal
                          WTP

                          Sheet2

                          Sheet3

                          SERA

                          PROS AND CONS OF NEBS MEASUREMENT METHODS

                          13

                          Source Skumatz SERA research

                          Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

                          Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

                          Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

                          Long history easy secondary sources Credible to reviewers vetted inputs

                          Not available for all NEBs

                          Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

                          Third party peer-vetted models available for economics jobs and emissions

                          Not available for all NEBs

                          Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

                          Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

                          Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

                          SERA

                          PROGRESS TRANSFERABILITY OF NEBS

                          14

                          Source Skumatz SERA research

                          Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

                          bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                          Program measure dependent

                          bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

                          education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                          Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                          Residential Target dependent (low income or MF vs SF)

                          bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

                          vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                          Cost savings consistency if transferable vs risk

                          SERA

                          RANGES TYPICAL NEB VALUES FOR WxRETROFI

                          15

                          SERA

                          PROGRESS AND GAPS Consistency Transferability

                          Some programs types others less studied Transferability research

                          Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

                          measures until then Across the board Savings share Regression

                          SERA

                          PROGRESS AND GAPS Gas vs Electric

                          Some researchndashusually program-wide not broken down International research found participant NEBs have

                          similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

                          MF Less-commonly-studied complicated by poor response

                          and complexity of sector (decision-maker some measures in home some central) separate from low income not common

                          Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

                          Commercial gaps in some measures

                          SERA

                          USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                          Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                          18

                          0

                          005

                          01

                          015

                          02

                          025

                          Environment

                          Quality

                          Quantity

                          PgmAvailFlicker

                          Mainten

                          ance

                          Lifetime

                          SalesP

                          roductivity

                          Safety

                          Noise

                          Control

                          Illness

                          Total NEBs Multiplier=X

                          $

                          Savings value=$X

                          Chart3

                          023
                          016
                          014
                          01
                          008
                          006
                          006
                          006
                          005
                          005
                          003
                          0

                          Chart1

                          Vendor
                          NP
                          Partic
                          028
                          02
                          02
                          028
                          013
                          017
                          031
                          022
                          016
                          006
                          008
                          014
                          005
                          006
                          014
                          004
                          009
                          007
                          004
                          003
                          007
                          -006
                          018
                          006

                          Sheet1

                          Sheet1

                          Sheet2

                          Sheet3

                          SERA

                          NEB VALUES

                          19

                          SERA

                          ARE NEBS HIGH VALUE Energy savings are often the minority of

                          benefits from program measuresndash Donrsquot ignore that

                          20

                          0 05 1 15 2 25 3

                          Energy Sav

                          NEBs

                          NEB vs Energy Savings ValueIncluding all NEBs

                          Source (SkumatzSERA 2010 amp others)

                          Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                          SERA

                          RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                          MF Many others

                          Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                          Variety of end uses

                          SERA

                          TOP NEBS FOR WEATHERIZATION PROGRAM

                          Weatherization program multipliergt1

                          Source SERA Research

                          SERA

                          RESULTS FOR ENERGY STARreg APPLIANCES

                          0002004006008

                          01012014016018

                          Maint

                          Appear

                          Perf LifeNoise Satis

                          Comfort

                          LiteSafe

                          ty SellMove

                          Water

                          Enviro

                          Refrig DW CW

                          Source SERA Research

                          SERA

                          INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                          5 increments to 50

                          Source SERA research

                          Chart5

                          EnergySmart (insul Retrofit)
                          Dunedin (insul)
                          001
                          024
                          002
                          004
                          002
                          005
                          006
                          005
                          007
                          008
                          008
                          006
                          013
                          0
                          013
                          016
                          049
                          033

                          cx

                          cx

                          nz

                          Sheet3

                          Sheet3

                          Super Insulation
                          Double Glazing
                          Solar Design
                          Solar Water Heating
                          Super Insulation
                          Double Glazing
                          Solar Design
                          Solar Water Heating
                          EnergySmart (insul Retrofit)
                          Dunedin (insul)

                          SERA

                          ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                          0

                          002

                          004

                          006

                          008

                          01

                          012

                          Maint

                          Perf LifeSatis

                          f

                          ComfortAesth

                          NoiseSafe

                          ty

                          LightingSelli

                          ng

                          Stay in home

                          Enviro

                          Sick Days

                          Calls

                          ES Homes HP wES Energy savings multiplier=1

                          Source SERA Research

                          SERA

                          CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                          DG CHP HVAC Equipment rebate Other

                          Building codes incentives by cities

                          Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                          Variety of end uses

                          SERA

                          AUDIT FINANCIAL PROGRAM

                          0

                          01

                          02

                          03

                          04

                          05

                          06

                          07

                          Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                          Source SERA Research

                          SERA

                          CampI NEW CONSTRUCTION

                          -005

                          0

                          005

                          01

                          015

                          02

                          AampE Owner(Pct of Total Partic NEBs)

                          Source SERA Research

                          SERA

                          CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                          0 2 4 6 8 10 12

                          m Product losses

                          l Ease of selling leasing building

                          n Labor requirements

                          h Aesthetics appearance

                          j Noise

                          b Equipment maintenance

                          f Tenant satisfaction

                          d Equipment lifetimes

                          e Productivity

                          k Building safety[2]

                          g Comfort

                          i Lighting Quality of Light

                          c Equipment performance[1]

                          a Operating costs

                          o Doing good for the Environment[3]

                          Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                          vcv

                          Source SERA Research

                          Chart1

                          CampI Technical Assistance NEBs
                          002
                          003
                          004
                          005
                          005
                          006
                          006
                          007
                          007
                          007
                          009
                          009
                          01
                          011
                          011

                          Sheet1

                          Sheet1

                          CampI Technical Assistance Program NEBs (Participant)

                          Sheet2

                          Sheet3

                          SERA

                          EXPRESSING NEBS VALUEndashCxe

                          Strong value from RetroCxSource SERA Research

                          SERA

                          SOCIETAL NEB EXAMPLES

                          31

                          SERA

                          JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                          32

                          0

                          02

                          04

                          06

                          08

                          1

                          12

                          HPWxRetrof Appliance

                          CA

                          WI

                          Natl

                          (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                          Jobs Economic

                          SERA

                          EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                          33

                          SERA

                          UTILITY NEBSUtility NEBs for Template Program

                          Debt WriteOff (util)13

                          Rate Subsidy(util)61

                          HealthSafety(util)0

                          Colln Costs (util)0

                          Gas Calls (util)0

                          Calls to CSRs(util)2

                          TampD (util)16

                          Arrears (util)0

                          Reconnects (util)0

                          Notices (util)7

                          Shutoffs (util)1

                          Rate subsidy TampD

                          Payment-related

                          Source Skumatz Economic Research Associates research

                          Example Low Income Weatherization

                          SERA

                          ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                          be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                          Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                          Value of 100 HALVES Payback doubles ROI

                          Source Skumatz Economics (SERA)

                          35

                          ( for NEBs Multiplier of 125)

                          SERA

                          NEBS IN COST-EFFECTIVENESS APPLICATIONS

                          36

                          hellip2001 ON

                          SERA

                          37

                          NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                          For true representation of B amp C elements of NEBs address missing factor bias

                          Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                          BC applications were considered early on then pulled back awaiting more quantitative evidence

                          SERA

                          CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                          38

                          Test Benefits Costs States Using Traditionally

                          Improved treat-ment with NEBs

                          Utility Cost (or Program Administrator Test) (UCT or PAC)

                          bull Avoided supply costs for transmission distribution and generation (TDampG)

                          bull Avoided gas and water supply costs

                          bull Program administration

                          bull Participant incentives

                          bull Increased supply cost

                          CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                          Use cost only paid by the utility

                          Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                          Same as above plus bull increased revenue

                          Same as above plus bull Decreased revenue

                          AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                          Participant cost bull Utility bill reductions bull Participant incentives

                          bull Participant direct costs

                          AR CA FL HI IA IN MI MN NY VA

                          Participant NEBs

                          Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                          Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                          SERA

                          APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                          Expansion Concern with risk calculation accuracy

                          39

                          Source Skumatz SERA research

                          SERA

                          40

                          METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                          Maximize DSM opportunities amp feedback Accuracy tailoring

                          Minimize Regulatory amp Implementer Risk

                          Minimize Evaluation Cost

                          Adder

                          Readily MeasurableHybrid

                          All NEBsSource SERA Research

                          SERA

                          41

                          STATE TREATMENT OF NEBS NEBsRegulatory

                          Screening Application Utilities regions

                          Program Marketing Fairly widespread use in utilities states across the country

                          MO

                          RE AGGRESSIVE ==gt

                          Test Pgm Screen - adder

                          IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                          Test Pgm Screen - readily measured

                          MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                          Test Hybrid (potential adder amp measured)

                          CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                          Test Pgm screen - Broad

                          With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                          Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                          SERA

                          42

                          Key

                          States with NEBs in CE tests for At least one type of program

                          copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                          STATES WITH NEBS IN CE TESTS

                          NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                          groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                          SERA

                          SELECTED STATE PROGRESS CA

                          2001 ndash LIPPT model NY

                          Mid-2000s measurement amp scenarios not included CO

                          Adders (10 electric 25 LI 5 gas) VT

                          15 adder helped by previous research DC

                          10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                          43

                          Source Skumatz SERA research

                          SERA

                          SELECTED STATE PROGRESS Dominos ongoing

                          Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                          Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                          Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                          Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                          44

                          Source Skumatz SERA research

                          SERA

                          APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                          Expansion Concern with risk calculation accuracy

                          45

                          Source Skumatz SERA research

                          SERA

                          46

                          NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                          [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                          SourceSkumatz SERA Independent research

                          NTG ndash accuracy measurement incomplete

                          Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                          NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                          EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                          Cost Complicated expensive local changes

                          RELATIVE Risk from NEBs

                          Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                          Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                          NEBs as accurate as many inputs (all could improve)

                          SERA

                          DISCOUNT RATE MATTERS

                          47

                          Source Skumatz SERA Research

                          0

                          20

                          40

                          60

                          80

                          100

                          120

                          1 2 3 4 5 6 7 8 9 10

                          PV Ratios vs 1 Discount Rate

                          10 yr PV 20 yr PV

                          Discount rate - percent

                          WACCSocialLTT

                          Source Skumatz Electricity Journal IEPEC paper independent research

                          SERA

                          EUL VARIATIONS-ADOPTED VALUES

                          Source Skumatz Economic Research Associates SERA independent research

                          0

                          50

                          100

                          150

                          200

                          250

                          300

                          350

                          400

                          450

                          0 10 20 30 40 50 60 70

                          EUL Variation by Measure

                          67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                          Poorly supportedOldTechnology change

                          SERA

                          KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                          Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                          measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                          change the BC conclusion

                          On the cusphellip

                          If NO Yoursquore doneAnd bias addressedsufficiently

                          IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                          Source SERA all rights reserved

                          SERArsquos lsquoNEB-ItrdquoModel ndash

                          SERA

                          IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                          Utility Soc Part Conserv Recrsquom

                          Rationale

                          Base Percent X X X Program-invariant

                          Low Income X X X X Multiple sources

                          Weatherization X X X Substantial Participantimpacts

                          Measure Program-specific

                          X Varies by measure sector

                          Other Recomrsquos Local Research

                          50

                          Source Skumatz SERA

                          Developing values for multiple states amp utilities

                          SERA

                          TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                          Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                          Other sources of bias to research (EUL NTG etc) bring high variation

                          5) States are incorporating NEBs dominos input in deliberations in multiple states

                          6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                          THANK YOU

                          Questions

                          Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                          SERA

                          POWER OF NEBS ndashBARRIERS DISCONNECTS

                          53

                          SERA

                          NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                          Negative NEB values cost of barrier

                          Solar Water Heat NZ$ Euros

                          Solar Design NZ$ Euros

                          Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                          -23 -12 (079)

                          -11 -6 (06)

                          Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                          Source Skumatz Economic Research Associates research

                          SERA

                          CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                          NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                          -5-4-3-2-1012345

                          Op Co

                          st

                          Init C

                          ost

                          Maint

                          Perfo

                          rm

                          Prod

                          uctiv

                          Tena

                          nt Sa

                          t

                          Comf

                          ort

                          Appe

                          ar

                          Light

                          Qual

                          SellL

                          ease

                          Devp Ownr AampE

                          Note AgtE (~4 vs ~2) for these NEBs

                          Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                          Source Skumatz Economic Research Associates Inc 2002

                          Owners more positive ndashUnderinvestment train

                          • Soup to nuts on non-energy benefits NEIs
                          • TOPICS
                          • BACKGROUND NEBS
                          • 20+ YEARS OF NEBS PROGRESShellip
                          • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                          • KEY APPLICATIONS OF NEBS
                          • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                          • Neb measurement
                          • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                          • MEASURING PARTICIPANTNEBs
                          • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                          • PARTICIPANT MEASUREMENT METHODS COMPARISON
                          • PROS AND CONS OF NEBS MEASUREMENT METHODS
                          • PROGRESS TRANSFERABILITY OF NEBS
                          • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                          • PROGRESS AND GAPS
                          • PROGRESS AND GAPS
                          • USING NEB NEI VALUES
                          • Slide Number 19
                          • ARE NEBS HIGH VALUE
                          • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                          • TOP NEBS FOR WEATHERIZATION PROGRAM
                          • RESULTS FOR ENERGY STARreg APPLIANCES
                          • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                          • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                          • CampI PROGRAMS ldquoNEBBEDrdquo
                          • AUDIT FINANCIAL PROGRAM
                          • CampI NEW CONSTRUCTION
                          • CampI TECHNICAL ASSISTANCE
                          • EXPRESSING NEBS VALUEndashCxe
                          • Slide Number 31
                          • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                          • EMISSIONS (SOCIETAL NEBS)
                          • UTILITY NEBS
                          • ADJUSTED PAYBACKS
                          • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                          • NEBS IN BC
                          • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                          • APPLICATION TO BC
                          • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                          • STATE TREATMENT OF NEBS NEBs
                          • Slide Number 42
                          • SELECTED STATE PROGRESS
                          • SELECTED STATE PROGRESS
                          • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                          • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                          • DISCOUNT RATE MATTERS
                          • EUL VARIATIONS- ADOPTED VALUES
                          • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                          • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                          • TAKEAWAYSON NEBSNEIS
                          • THANK YOUQuestions
                          • Slide Number 53
                          • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                          • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                            Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                            m Product losses20
                            l Ease of selling leasing building33
                            n Labor requirements45
                            h Aesthetics appearance59
                            j Noise50
                            b Equipment maintenance660
                            f Tenant satisfaction60
                            d Equipment lifetimes718
                            e Productivity724
                            k Building safety[2]73
                            g Comfort933
                            i Lighting Quality of Light916
                            c Equipment performance[1]1019
                            a Operating costs1165
                            o Doing good for the Environment[3]1132
                            Other03
                            [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                            [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                            [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                            m Product losses
                            l Ease of selling leasing building
                            n Labor requirements
                            h Aesthetics appearance
                            j Noise
                            b Equipment maintenance
                            f Tenant satisfaction
                            d Equipment lifetimes
                            e Productivity
                            k Building safety[2]
                            g Comfort
                            i Lighting Quality of Light
                            c Equipment performance[1]
                            a Operating costs
                            o Doing good for the Environment[3]
                            Share of ZALEH NEBS from various measures
                            Each presented as multiple of the (overall) energy savings
                            Super Insulation037
                            Double Glazing026
                            Solar Design025
                            Solar Water Heating005
                            EnergySmart (insul Retrofit)Dunedin (insul)
                            Other124
                            Noise24
                            Budgetfinances25
                            Environmental65
                            Health78
                            Reduced moves86
                            Ease of sellingrenting130
                            DampnessMold1316
                            Comfort-related4933
                            101101
                            Operational NEBs44
                            DesignConstruction NEBs31
                            Occupant NEBs25
                            1
                            OtherOther
                            NoiseNoise
                            BudgetfinancesBudgetfinances
                            EnvironmentalEnvironmental
                            HealthHealth
                            Reduced movesReduced moves
                            Ease of sellingrentingEase of sellingrenting
                            DampnessMoldDampnessMold
                            Comfort-relatedComfort-related
                            Boiler
                            VendorNPPartic
                            Perform0280202
                            Footprint028013017
                            ControlFeatures031022016
                            Tenant compl006008014
                            Noise005006014
                            Maint004009007
                            Other004003007
                            Lifetime-006018006
                            MF New construction energy star
                            Operating cost014
                            Environment012
                            Maintenance01
                            Lifetime01
                            Performance009
                            Occupant Satisf009
                            Comfort008
                            Noise006
                            Sellingleasing006
                            Aesthetics005
                            Light Quality005
                            Power QualReliab004
                            Safety002
                            RIGHTLIGHTS
                            Environment023
                            Quality016
                            Quantity014
                            PgmAvail01
                            Flicker008
                            Maintenance006
                            Lifetime006
                            SalesProductivity006
                            Safety005
                            Noise005
                            Control003
                            Illness0
                            PerformPerformPerform
                            FootprintFootprintFootprint
                            ControlFeaturesControlFeaturesControlFeatures
                            Tenant complTenant complTenant compl
                            NoiseNoiseNoise
                            MaintMaintMaint
                            OtherOtherOther
                            LifetimeLifetimeLifetime
                            Environment
                            Quality
                            Quantity
                            PgmAvail
                            Flicker
                            Maintenance
                            Lifetime
                            SalesProductivity
                            Safety
                            Noise
                            Control
                            Illness

                            Variability

                            Relevant NEB Categories

                            Program measure invariant (suitable for ldquoadderrdquo)

                            middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                            Program measure dependent

                            middot Economic ndash societal (depends on measures and local manufacture installation)

                            middot Health and safety health care illnesses ndash societal and participant (measure)

                            middot Water wastewater infrastructure and water bill savings ndash societal and participant

                            middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                            Climate dependent

                            middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                            Residential Target dependent (low income or MF vs SF)

                            middot Payment related ndash utility (arrearages etc stronger for low income targets)

                            middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

                            middot Participant benefits related to hardship and payments

                            middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                            Used for

                            Major Advantages

                            Major Disadvantages

                            Direct measurement (specific studies of changes on-site)

                            Comrsquol labor productivity etc

                            Direct precise attributable

                            Small estimation sample sizes specialized cases poor transferability expensive

                            Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

                            Insurance water health others

                            Long history easy secondary sources

                            Credible to reviewers vetted inputs

                            Not available for all NEBs

                            Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

                            Emissions economics

                            Third party peer-vetted models available for economics jobs and emissions

                            Not available for all NEBs

                            Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

                            Wide variety of Participant NEBs

                            Large sample sizes amp statistical properties

                            Affordable

                            Multiple estimates leading to similar ranges

                            Direct method of measuring some key NEBs

                            Concerns about surveys as a source of quantitative values amp reliability

                            Recall from survey respondents

                            Proper attribution to programs measures

                            MWxNEWxCA WxCA WxSF-NCMF-NCC- LiteC-RebC-Reb
                            mult1220813075104075084
                            LMS Verbal15610980612067115055091
                            WTP42242732
                            MWxMWxMWx
                            NEWxNEWxNEWx
                            CA WxCA WxCA Wx
                            CA WxCA WxCA Wx
                            SF-NCSF-NCSF-NC
                            MF-NCMF-NCMF-NC
                            C- LiteC- LiteC- Lite
                            C-RebC-RebC-Reb
                            C-RebC-RebC-Reb

                            Sheet1

                            Sheet1

                            mult
                            LMS Verbal
                            WTP

                            Sheet2

                            Sheet3

                            SERA

                            PROS AND CONS OF NEBS MEASUREMENT METHODS

                            13

                            Source Skumatz SERA research

                            Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

                            Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

                            Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

                            Long history easy secondary sources Credible to reviewers vetted inputs

                            Not available for all NEBs

                            Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

                            Third party peer-vetted models available for economics jobs and emissions

                            Not available for all NEBs

                            Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

                            Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

                            Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

                            SERA

                            PROGRESS TRANSFERABILITY OF NEBS

                            14

                            Source Skumatz SERA research

                            Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

                            bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                            Program measure dependent

                            bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

                            education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                            Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                            Residential Target dependent (low income or MF vs SF)

                            bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

                            vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                            Cost savings consistency if transferable vs risk

                            SERA

                            RANGES TYPICAL NEB VALUES FOR WxRETROFI

                            15

                            SERA

                            PROGRESS AND GAPS Consistency Transferability

                            Some programs types others less studied Transferability research

                            Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

                            measures until then Across the board Savings share Regression

                            SERA

                            PROGRESS AND GAPS Gas vs Electric

                            Some researchndashusually program-wide not broken down International research found participant NEBs have

                            similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

                            MF Less-commonly-studied complicated by poor response

                            and complexity of sector (decision-maker some measures in home some central) separate from low income not common

                            Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

                            Commercial gaps in some measures

                            SERA

                            USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                            Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                            18

                            0

                            005

                            01

                            015

                            02

                            025

                            Environment

                            Quality

                            Quantity

                            PgmAvailFlicker

                            Mainten

                            ance

                            Lifetime

                            SalesP

                            roductivity

                            Safety

                            Noise

                            Control

                            Illness

                            Total NEBs Multiplier=X

                            $

                            Savings value=$X

                            Chart3

                            023
                            016
                            014
                            01
                            008
                            006
                            006
                            006
                            005
                            005
                            003
                            0

                            Chart1

                            Vendor
                            NP
                            Partic
                            028
                            02
                            02
                            028
                            013
                            017
                            031
                            022
                            016
                            006
                            008
                            014
                            005
                            006
                            014
                            004
                            009
                            007
                            004
                            003
                            007
                            -006
                            018
                            006

                            Sheet1

                            Sheet1

                            Sheet2

                            Sheet3

                            SERA

                            NEB VALUES

                            19

                            SERA

                            ARE NEBS HIGH VALUE Energy savings are often the minority of

                            benefits from program measuresndash Donrsquot ignore that

                            20

                            0 05 1 15 2 25 3

                            Energy Sav

                            NEBs

                            NEB vs Energy Savings ValueIncluding all NEBs

                            Source (SkumatzSERA 2010 amp others)

                            Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                            SERA

                            RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                            MF Many others

                            Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                            Variety of end uses

                            SERA

                            TOP NEBS FOR WEATHERIZATION PROGRAM

                            Weatherization program multipliergt1

                            Source SERA Research

                            SERA

                            RESULTS FOR ENERGY STARreg APPLIANCES

                            0002004006008

                            01012014016018

                            Maint

                            Appear

                            Perf LifeNoise Satis

                            Comfort

                            LiteSafe

                            ty SellMove

                            Water

                            Enviro

                            Refrig DW CW

                            Source SERA Research

                            SERA

                            INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                            5 increments to 50

                            Source SERA research

                            Chart5

                            EnergySmart (insul Retrofit)
                            Dunedin (insul)
                            001
                            024
                            002
                            004
                            002
                            005
                            006
                            005
                            007
                            008
                            008
                            006
                            013
                            0
                            013
                            016
                            049
                            033

                            cx

                            cx

                            nz

                            Sheet3

                            Sheet3

                            Super Insulation
                            Double Glazing
                            Solar Design
                            Solar Water Heating
                            Super Insulation
                            Double Glazing
                            Solar Design
                            Solar Water Heating
                            EnergySmart (insul Retrofit)
                            Dunedin (insul)

                            SERA

                            ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                            0

                            002

                            004

                            006

                            008

                            01

                            012

                            Maint

                            Perf LifeSatis

                            f

                            ComfortAesth

                            NoiseSafe

                            ty

                            LightingSelli

                            ng

                            Stay in home

                            Enviro

                            Sick Days

                            Calls

                            ES Homes HP wES Energy savings multiplier=1

                            Source SERA Research

                            SERA

                            CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                            DG CHP HVAC Equipment rebate Other

                            Building codes incentives by cities

                            Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                            Variety of end uses

                            SERA

                            AUDIT FINANCIAL PROGRAM

                            0

                            01

                            02

                            03

                            04

                            05

                            06

                            07

                            Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                            Source SERA Research

                            SERA

                            CampI NEW CONSTRUCTION

                            -005

                            0

                            005

                            01

                            015

                            02

                            AampE Owner(Pct of Total Partic NEBs)

                            Source SERA Research

                            SERA

                            CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                            0 2 4 6 8 10 12

                            m Product losses

                            l Ease of selling leasing building

                            n Labor requirements

                            h Aesthetics appearance

                            j Noise

                            b Equipment maintenance

                            f Tenant satisfaction

                            d Equipment lifetimes

                            e Productivity

                            k Building safety[2]

                            g Comfort

                            i Lighting Quality of Light

                            c Equipment performance[1]

                            a Operating costs

                            o Doing good for the Environment[3]

                            Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                            vcv

                            Source SERA Research

                            Chart1

                            CampI Technical Assistance NEBs
                            002
                            003
                            004
                            005
                            005
                            006
                            006
                            007
                            007
                            007
                            009
                            009
                            01
                            011
                            011

                            Sheet1

                            Sheet1

                            CampI Technical Assistance Program NEBs (Participant)

                            Sheet2

                            Sheet3

                            SERA

                            EXPRESSING NEBS VALUEndashCxe

                            Strong value from RetroCxSource SERA Research

                            SERA

                            SOCIETAL NEB EXAMPLES

                            31

                            SERA

                            JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                            32

                            0

                            02

                            04

                            06

                            08

                            1

                            12

                            HPWxRetrof Appliance

                            CA

                            WI

                            Natl

                            (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                            Jobs Economic

                            SERA

                            EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                            33

                            SERA

                            UTILITY NEBSUtility NEBs for Template Program

                            Debt WriteOff (util)13

                            Rate Subsidy(util)61

                            HealthSafety(util)0

                            Colln Costs (util)0

                            Gas Calls (util)0

                            Calls to CSRs(util)2

                            TampD (util)16

                            Arrears (util)0

                            Reconnects (util)0

                            Notices (util)7

                            Shutoffs (util)1

                            Rate subsidy TampD

                            Payment-related

                            Source Skumatz Economic Research Associates research

                            Example Low Income Weatherization

                            SERA

                            ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                            be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                            Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                            Value of 100 HALVES Payback doubles ROI

                            Source Skumatz Economics (SERA)

                            35

                            ( for NEBs Multiplier of 125)

                            SERA

                            NEBS IN COST-EFFECTIVENESS APPLICATIONS

                            36

                            hellip2001 ON

                            SERA

                            37

                            NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                            For true representation of B amp C elements of NEBs address missing factor bias

                            Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                            BC applications were considered early on then pulled back awaiting more quantitative evidence

                            SERA

                            CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                            38

                            Test Benefits Costs States Using Traditionally

                            Improved treat-ment with NEBs

                            Utility Cost (or Program Administrator Test) (UCT or PAC)

                            bull Avoided supply costs for transmission distribution and generation (TDampG)

                            bull Avoided gas and water supply costs

                            bull Program administration

                            bull Participant incentives

                            bull Increased supply cost

                            CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                            Use cost only paid by the utility

                            Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                            Same as above plus bull increased revenue

                            Same as above plus bull Decreased revenue

                            AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                            Participant cost bull Utility bill reductions bull Participant incentives

                            bull Participant direct costs

                            AR CA FL HI IA IN MI MN NY VA

                            Participant NEBs

                            Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                            Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                            SERA

                            APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                            Expansion Concern with risk calculation accuracy

                            39

                            Source Skumatz SERA research

                            SERA

                            40

                            METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                            Maximize DSM opportunities amp feedback Accuracy tailoring

                            Minimize Regulatory amp Implementer Risk

                            Minimize Evaluation Cost

                            Adder

                            Readily MeasurableHybrid

                            All NEBsSource SERA Research

                            SERA

                            41

                            STATE TREATMENT OF NEBS NEBsRegulatory

                            Screening Application Utilities regions

                            Program Marketing Fairly widespread use in utilities states across the country

                            MO

                            RE AGGRESSIVE ==gt

                            Test Pgm Screen - adder

                            IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                            Test Pgm Screen - readily measured

                            MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                            Test Hybrid (potential adder amp measured)

                            CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                            Test Pgm screen - Broad

                            With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                            Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                            SERA

                            42

                            Key

                            States with NEBs in CE tests for At least one type of program

                            copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                            STATES WITH NEBS IN CE TESTS

                            NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                            groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                            SERA

                            SELECTED STATE PROGRESS CA

                            2001 ndash LIPPT model NY

                            Mid-2000s measurement amp scenarios not included CO

                            Adders (10 electric 25 LI 5 gas) VT

                            15 adder helped by previous research DC

                            10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                            43

                            Source Skumatz SERA research

                            SERA

                            SELECTED STATE PROGRESS Dominos ongoing

                            Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                            Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                            Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                            Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                            44

                            Source Skumatz SERA research

                            SERA

                            APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                            Expansion Concern with risk calculation accuracy

                            45

                            Source Skumatz SERA research

                            SERA

                            46

                            NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                            [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                            SourceSkumatz SERA Independent research

                            NTG ndash accuracy measurement incomplete

                            Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                            NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                            EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                            Cost Complicated expensive local changes

                            RELATIVE Risk from NEBs

                            Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                            Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                            NEBs as accurate as many inputs (all could improve)

                            SERA

                            DISCOUNT RATE MATTERS

                            47

                            Source Skumatz SERA Research

                            0

                            20

                            40

                            60

                            80

                            100

                            120

                            1 2 3 4 5 6 7 8 9 10

                            PV Ratios vs 1 Discount Rate

                            10 yr PV 20 yr PV

                            Discount rate - percent

                            WACCSocialLTT

                            Source Skumatz Electricity Journal IEPEC paper independent research

                            SERA

                            EUL VARIATIONS-ADOPTED VALUES

                            Source Skumatz Economic Research Associates SERA independent research

                            0

                            50

                            100

                            150

                            200

                            250

                            300

                            350

                            400

                            450

                            0 10 20 30 40 50 60 70

                            EUL Variation by Measure

                            67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                            Poorly supportedOldTechnology change

                            SERA

                            KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                            Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                            measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                            change the BC conclusion

                            On the cusphellip

                            If NO Yoursquore doneAnd bias addressedsufficiently

                            IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                            Source SERA all rights reserved

                            SERArsquos lsquoNEB-ItrdquoModel ndash

                            SERA

                            IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                            Utility Soc Part Conserv Recrsquom

                            Rationale

                            Base Percent X X X Program-invariant

                            Low Income X X X X Multiple sources

                            Weatherization X X X Substantial Participantimpacts

                            Measure Program-specific

                            X Varies by measure sector

                            Other Recomrsquos Local Research

                            50

                            Source Skumatz SERA

                            Developing values for multiple states amp utilities

                            SERA

                            TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                            Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                            Other sources of bias to research (EUL NTG etc) bring high variation

                            5) States are incorporating NEBs dominos input in deliberations in multiple states

                            6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                            THANK YOU

                            Questions

                            Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                            SERA

                            POWER OF NEBS ndashBARRIERS DISCONNECTS

                            53

                            SERA

                            NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                            Negative NEB values cost of barrier

                            Solar Water Heat NZ$ Euros

                            Solar Design NZ$ Euros

                            Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                            -23 -12 (079)

                            -11 -6 (06)

                            Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                            Source Skumatz Economic Research Associates research

                            SERA

                            CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                            NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                            -5-4-3-2-1012345

                            Op Co

                            st

                            Init C

                            ost

                            Maint

                            Perfo

                            rm

                            Prod

                            uctiv

                            Tena

                            nt Sa

                            t

                            Comf

                            ort

                            Appe

                            ar

                            Light

                            Qual

                            SellL

                            ease

                            Devp Ownr AampE

                            Note AgtE (~4 vs ~2) for these NEBs

                            Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                            Source Skumatz Economic Research Associates Inc 2002

                            Owners more positive ndashUnderinvestment train

                            • Soup to nuts on non-energy benefits NEIs
                            • TOPICS
                            • BACKGROUND NEBS
                            • 20+ YEARS OF NEBS PROGRESShellip
                            • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                            • KEY APPLICATIONS OF NEBS
                            • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                            • Neb measurement
                            • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                            • MEASURING PARTICIPANTNEBs
                            • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                            • PARTICIPANT MEASUREMENT METHODS COMPARISON
                            • PROS AND CONS OF NEBS MEASUREMENT METHODS
                            • PROGRESS TRANSFERABILITY OF NEBS
                            • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                            • PROGRESS AND GAPS
                            • PROGRESS AND GAPS
                            • USING NEB NEI VALUES
                            • Slide Number 19
                            • ARE NEBS HIGH VALUE
                            • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                            • TOP NEBS FOR WEATHERIZATION PROGRAM
                            • RESULTS FOR ENERGY STARreg APPLIANCES
                            • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                            • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                            • CampI PROGRAMS ldquoNEBBEDrdquo
                            • AUDIT FINANCIAL PROGRAM
                            • CampI NEW CONSTRUCTION
                            • CampI TECHNICAL ASSISTANCE
                            • EXPRESSING NEBS VALUEndashCxe
                            • Slide Number 31
                            • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                            • EMISSIONS (SOCIETAL NEBS)
                            • UTILITY NEBS
                            • ADJUSTED PAYBACKS
                            • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                            • NEBS IN BC
                            • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                            • APPLICATION TO BC
                            • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                            • STATE TREATMENT OF NEBS NEBs
                            • Slide Number 42
                            • SELECTED STATE PROGRESS
                            • SELECTED STATE PROGRESS
                            • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                            • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                            • DISCOUNT RATE MATTERS
                            • EUL VARIATIONS- ADOPTED VALUES
                            • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                            • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                            • TAKEAWAYSON NEBSNEIS
                            • THANK YOUQuestions
                            • Slide Number 53
                            • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                            • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                              Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                              m Product losses20
                              l Ease of selling leasing building33
                              n Labor requirements45
                              h Aesthetics appearance59
                              j Noise50
                              b Equipment maintenance660
                              f Tenant satisfaction60
                              d Equipment lifetimes718
                              e Productivity724
                              k Building safety[2]73
                              g Comfort933
                              i Lighting Quality of Light916
                              c Equipment performance[1]1019
                              a Operating costs1165
                              o Doing good for the Environment[3]1132
                              Other03
                              [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                              [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                              [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                              m Product losses
                              l Ease of selling leasing building
                              n Labor requirements
                              h Aesthetics appearance
                              j Noise
                              b Equipment maintenance
                              f Tenant satisfaction
                              d Equipment lifetimes
                              e Productivity
                              k Building safety[2]
                              g Comfort
                              i Lighting Quality of Light
                              c Equipment performance[1]
                              a Operating costs
                              o Doing good for the Environment[3]
                              Share of ZALEH NEBS from various measures
                              Each presented as multiple of the (overall) energy savings
                              Super Insulation037
                              Double Glazing026
                              Solar Design025
                              Solar Water Heating005
                              EnergySmart (insul Retrofit)Dunedin (insul)
                              Other124
                              Noise24
                              Budgetfinances25
                              Environmental65
                              Health78
                              Reduced moves86
                              Ease of sellingrenting130
                              DampnessMold1316
                              Comfort-related4933
                              101101
                              Operational NEBs44
                              DesignConstruction NEBs31
                              Occupant NEBs25
                              1
                              OtherOther
                              NoiseNoise
                              BudgetfinancesBudgetfinances
                              EnvironmentalEnvironmental
                              HealthHealth
                              Reduced movesReduced moves
                              Ease of sellingrentingEase of sellingrenting
                              DampnessMoldDampnessMold
                              Comfort-relatedComfort-related
                              Boiler
                              VendorNPPartic
                              Perform0280202
                              Footprint028013017
                              ControlFeatures031022016
                              Tenant compl006008014
                              Noise005006014
                              Maint004009007
                              Other004003007
                              Lifetime-006018006
                              MF New construction energy star
                              Operating cost014
                              Environment012
                              Maintenance01
                              Lifetime01
                              Performance009
                              Occupant Satisf009
                              Comfort008
                              Noise006
                              Sellingleasing006
                              Aesthetics005
                              Light Quality005
                              Power QualReliab004
                              Safety002
                              RIGHTLIGHTS
                              Environment023
                              Quality016
                              Quantity014
                              PgmAvail01
                              Flicker008
                              Maintenance006
                              Lifetime006
                              SalesProductivity006
                              Safety005
                              Noise005
                              Control003
                              Illness0
                              PerformPerformPerform
                              FootprintFootprintFootprint
                              ControlFeaturesControlFeaturesControlFeatures
                              Tenant complTenant complTenant compl
                              NoiseNoiseNoise
                              MaintMaintMaint
                              OtherOtherOther
                              LifetimeLifetimeLifetime
                              Environment
                              Quality
                              Quantity
                              PgmAvail
                              Flicker
                              Maintenance
                              Lifetime
                              SalesProductivity
                              Safety
                              Noise
                              Control
                              Illness

                              Variability

                              Relevant NEB Categories

                              Program measure invariant (suitable for ldquoadderrdquo)

                              middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                              Program measure dependent

                              middot Economic ndash societal (depends on measures and local manufacture installation)

                              middot Health and safety health care illnesses ndash societal and participant (measure)

                              middot Water wastewater infrastructure and water bill savings ndash societal and participant

                              middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                              Climate dependent

                              middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                              Residential Target dependent (low income or MF vs SF)

                              middot Payment related ndash utility (arrearages etc stronger for low income targets)

                              middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

                              middot Participant benefits related to hardship and payments

                              middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                              Used for

                              Major Advantages

                              Major Disadvantages

                              Direct measurement (specific studies of changes on-site)

                              Comrsquol labor productivity etc

                              Direct precise attributable

                              Small estimation sample sizes specialized cases poor transferability expensive

                              Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

                              Insurance water health others

                              Long history easy secondary sources

                              Credible to reviewers vetted inputs

                              Not available for all NEBs

                              Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

                              Emissions economics

                              Third party peer-vetted models available for economics jobs and emissions

                              Not available for all NEBs

                              Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

                              Wide variety of Participant NEBs

                              Large sample sizes amp statistical properties

                              Affordable

                              Multiple estimates leading to similar ranges

                              Direct method of measuring some key NEBs

                              Concerns about surveys as a source of quantitative values amp reliability

                              Recall from survey respondents

                              Proper attribution to programs measures

                              MWxNEWxCA WxCA WxSF-NCMF-NCC- LiteC-RebC-Reb
                              mult1220813075104075084
                              LMS Verbal15610980612067115055091
                              WTP42242732

                              Sheet1

                              mult
                              LMS Verbal
                              WTP

                              Sheet2

                              Sheet3

                              SERA

                              PROS AND CONS OF NEBS MEASUREMENT METHODS

                              13

                              Source Skumatz SERA research

                              Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

                              Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

                              Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

                              Long history easy secondary sources Credible to reviewers vetted inputs

                              Not available for all NEBs

                              Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

                              Third party peer-vetted models available for economics jobs and emissions

                              Not available for all NEBs

                              Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

                              Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

                              Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

                              SERA

                              PROGRESS TRANSFERABILITY OF NEBS

                              14

                              Source Skumatz SERA research

                              Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

                              bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                              Program measure dependent

                              bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

                              education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                              Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                              Residential Target dependent (low income or MF vs SF)

                              bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

                              vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                              Cost savings consistency if transferable vs risk

                              SERA

                              RANGES TYPICAL NEB VALUES FOR WxRETROFI

                              15

                              SERA

                              PROGRESS AND GAPS Consistency Transferability

                              Some programs types others less studied Transferability research

                              Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

                              measures until then Across the board Savings share Regression

                              SERA

                              PROGRESS AND GAPS Gas vs Electric

                              Some researchndashusually program-wide not broken down International research found participant NEBs have

                              similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

                              MF Less-commonly-studied complicated by poor response

                              and complexity of sector (decision-maker some measures in home some central) separate from low income not common

                              Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

                              Commercial gaps in some measures

                              SERA

                              USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                              Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                              18

                              0

                              005

                              01

                              015

                              02

                              025

                              Environment

                              Quality

                              Quantity

                              PgmAvailFlicker

                              Mainten

                              ance

                              Lifetime

                              SalesP

                              roductivity

                              Safety

                              Noise

                              Control

                              Illness

                              Total NEBs Multiplier=X

                              $

                              Savings value=$X

                              Chart3

                              023
                              016
                              014
                              01
                              008
                              006
                              006
                              006
                              005
                              005
                              003
                              0

                              Chart1

                              Vendor
                              NP
                              Partic
                              028
                              02
                              02
                              028
                              013
                              017
                              031
                              022
                              016
                              006
                              008
                              014
                              005
                              006
                              014
                              004
                              009
                              007
                              004
                              003
                              007
                              -006
                              018
                              006

                              Sheet1

                              Sheet1

                              Sheet2

                              Sheet3

                              SERA

                              NEB VALUES

                              19

                              SERA

                              ARE NEBS HIGH VALUE Energy savings are often the minority of

                              benefits from program measuresndash Donrsquot ignore that

                              20

                              0 05 1 15 2 25 3

                              Energy Sav

                              NEBs

                              NEB vs Energy Savings ValueIncluding all NEBs

                              Source (SkumatzSERA 2010 amp others)

                              Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                              SERA

                              RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                              MF Many others

                              Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                              Variety of end uses

                              SERA

                              TOP NEBS FOR WEATHERIZATION PROGRAM

                              Weatherization program multipliergt1

                              Source SERA Research

                              SERA

                              RESULTS FOR ENERGY STARreg APPLIANCES

                              0002004006008

                              01012014016018

                              Maint

                              Appear

                              Perf LifeNoise Satis

                              Comfort

                              LiteSafe

                              ty SellMove

                              Water

                              Enviro

                              Refrig DW CW

                              Source SERA Research

                              SERA

                              INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                              5 increments to 50

                              Source SERA research

                              Chart5

                              EnergySmart (insul Retrofit)
                              Dunedin (insul)
                              001
                              024
                              002
                              004
                              002
                              005
                              006
                              005
                              007
                              008
                              008
                              006
                              013
                              0
                              013
                              016
                              049
                              033

                              cx

                              cx

                              nz

                              Sheet3

                              Sheet3

                              Super Insulation
                              Double Glazing
                              Solar Design
                              Solar Water Heating
                              Super Insulation
                              Double Glazing
                              Solar Design
                              Solar Water Heating
                              EnergySmart (insul Retrofit)
                              Dunedin (insul)

                              SERA

                              ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                              0

                              002

                              004

                              006

                              008

                              01

                              012

                              Maint

                              Perf LifeSatis

                              f

                              ComfortAesth

                              NoiseSafe

                              ty

                              LightingSelli

                              ng

                              Stay in home

                              Enviro

                              Sick Days

                              Calls

                              ES Homes HP wES Energy savings multiplier=1

                              Source SERA Research

                              SERA

                              CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                              DG CHP HVAC Equipment rebate Other

                              Building codes incentives by cities

                              Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                              Variety of end uses

                              SERA

                              AUDIT FINANCIAL PROGRAM

                              0

                              01

                              02

                              03

                              04

                              05

                              06

                              07

                              Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                              Source SERA Research

                              SERA

                              CampI NEW CONSTRUCTION

                              -005

                              0

                              005

                              01

                              015

                              02

                              AampE Owner(Pct of Total Partic NEBs)

                              Source SERA Research

                              SERA

                              CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                              0 2 4 6 8 10 12

                              m Product losses

                              l Ease of selling leasing building

                              n Labor requirements

                              h Aesthetics appearance

                              j Noise

                              b Equipment maintenance

                              f Tenant satisfaction

                              d Equipment lifetimes

                              e Productivity

                              k Building safety[2]

                              g Comfort

                              i Lighting Quality of Light

                              c Equipment performance[1]

                              a Operating costs

                              o Doing good for the Environment[3]

                              Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                              vcv

                              Source SERA Research

                              Chart1

                              CampI Technical Assistance NEBs
                              002
                              003
                              004
                              005
                              005
                              006
                              006
                              007
                              007
                              007
                              009
                              009
                              01
                              011
                              011

                              Sheet1

                              Sheet1

                              CampI Technical Assistance Program NEBs (Participant)

                              Sheet2

                              Sheet3

                              SERA

                              EXPRESSING NEBS VALUEndashCxe

                              Strong value from RetroCxSource SERA Research

                              SERA

                              SOCIETAL NEB EXAMPLES

                              31

                              SERA

                              JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                              32

                              0

                              02

                              04

                              06

                              08

                              1

                              12

                              HPWxRetrof Appliance

                              CA

                              WI

                              Natl

                              (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                              Jobs Economic

                              SERA

                              EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                              33

                              SERA

                              UTILITY NEBSUtility NEBs for Template Program

                              Debt WriteOff (util)13

                              Rate Subsidy(util)61

                              HealthSafety(util)0

                              Colln Costs (util)0

                              Gas Calls (util)0

                              Calls to CSRs(util)2

                              TampD (util)16

                              Arrears (util)0

                              Reconnects (util)0

                              Notices (util)7

                              Shutoffs (util)1

                              Rate subsidy TampD

                              Payment-related

                              Source Skumatz Economic Research Associates research

                              Example Low Income Weatherization

                              SERA

                              ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                              be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                              Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                              Value of 100 HALVES Payback doubles ROI

                              Source Skumatz Economics (SERA)

                              35

                              ( for NEBs Multiplier of 125)

                              SERA

                              NEBS IN COST-EFFECTIVENESS APPLICATIONS

                              36

                              hellip2001 ON

                              SERA

                              37

                              NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                              For true representation of B amp C elements of NEBs address missing factor bias

                              Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                              BC applications were considered early on then pulled back awaiting more quantitative evidence

                              SERA

                              CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                              38

                              Test Benefits Costs States Using Traditionally

                              Improved treat-ment with NEBs

                              Utility Cost (or Program Administrator Test) (UCT or PAC)

                              bull Avoided supply costs for transmission distribution and generation (TDampG)

                              bull Avoided gas and water supply costs

                              bull Program administration

                              bull Participant incentives

                              bull Increased supply cost

                              CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                              Use cost only paid by the utility

                              Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                              Same as above plus bull increased revenue

                              Same as above plus bull Decreased revenue

                              AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                              Participant cost bull Utility bill reductions bull Participant incentives

                              bull Participant direct costs

                              AR CA FL HI IA IN MI MN NY VA

                              Participant NEBs

                              Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                              Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                              SERA

                              APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                              Expansion Concern with risk calculation accuracy

                              39

                              Source Skumatz SERA research

                              SERA

                              40

                              METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                              Maximize DSM opportunities amp feedback Accuracy tailoring

                              Minimize Regulatory amp Implementer Risk

                              Minimize Evaluation Cost

                              Adder

                              Readily MeasurableHybrid

                              All NEBsSource SERA Research

                              SERA

                              41

                              STATE TREATMENT OF NEBS NEBsRegulatory

                              Screening Application Utilities regions

                              Program Marketing Fairly widespread use in utilities states across the country

                              MO

                              RE AGGRESSIVE ==gt

                              Test Pgm Screen - adder

                              IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                              Test Pgm Screen - readily measured

                              MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                              Test Hybrid (potential adder amp measured)

                              CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                              Test Pgm screen - Broad

                              With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                              Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                              SERA

                              42

                              Key

                              States with NEBs in CE tests for At least one type of program

                              copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                              STATES WITH NEBS IN CE TESTS

                              NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                              groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                              SERA

                              SELECTED STATE PROGRESS CA

                              2001 ndash LIPPT model NY

                              Mid-2000s measurement amp scenarios not included CO

                              Adders (10 electric 25 LI 5 gas) VT

                              15 adder helped by previous research DC

                              10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                              43

                              Source Skumatz SERA research

                              SERA

                              SELECTED STATE PROGRESS Dominos ongoing

                              Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                              Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                              Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                              Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                              44

                              Source Skumatz SERA research

                              SERA

                              APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                              Expansion Concern with risk calculation accuracy

                              45

                              Source Skumatz SERA research

                              SERA

                              46

                              NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                              [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                              SourceSkumatz SERA Independent research

                              NTG ndash accuracy measurement incomplete

                              Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                              NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                              EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                              Cost Complicated expensive local changes

                              RELATIVE Risk from NEBs

                              Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                              Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                              NEBs as accurate as many inputs (all could improve)

                              SERA

                              DISCOUNT RATE MATTERS

                              47

                              Source Skumatz SERA Research

                              0

                              20

                              40

                              60

                              80

                              100

                              120

                              1 2 3 4 5 6 7 8 9 10

                              PV Ratios vs 1 Discount Rate

                              10 yr PV 20 yr PV

                              Discount rate - percent

                              WACCSocialLTT

                              Source Skumatz Electricity Journal IEPEC paper independent research

                              SERA

                              EUL VARIATIONS-ADOPTED VALUES

                              Source Skumatz Economic Research Associates SERA independent research

                              0

                              50

                              100

                              150

                              200

                              250

                              300

                              350

                              400

                              450

                              0 10 20 30 40 50 60 70

                              EUL Variation by Measure

                              67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                              Poorly supportedOldTechnology change

                              SERA

                              KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                              Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                              measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                              change the BC conclusion

                              On the cusphellip

                              If NO Yoursquore doneAnd bias addressedsufficiently

                              IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                              Source SERA all rights reserved

                              SERArsquos lsquoNEB-ItrdquoModel ndash

                              SERA

                              IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                              Utility Soc Part Conserv Recrsquom

                              Rationale

                              Base Percent X X X Program-invariant

                              Low Income X X X X Multiple sources

                              Weatherization X X X Substantial Participantimpacts

                              Measure Program-specific

                              X Varies by measure sector

                              Other Recomrsquos Local Research

                              50

                              Source Skumatz SERA

                              Developing values for multiple states amp utilities

                              SERA

                              TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                              Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                              Other sources of bias to research (EUL NTG etc) bring high variation

                              5) States are incorporating NEBs dominos input in deliberations in multiple states

                              6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                              THANK YOU

                              Questions

                              Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                              SERA

                              POWER OF NEBS ndashBARRIERS DISCONNECTS

                              53

                              SERA

                              NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                              Negative NEB values cost of barrier

                              Solar Water Heat NZ$ Euros

                              Solar Design NZ$ Euros

                              Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                              -23 -12 (079)

                              -11 -6 (06)

                              Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                              Source Skumatz Economic Research Associates research

                              SERA

                              CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                              NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                              -5-4-3-2-1012345

                              Op Co

                              st

                              Init C

                              ost

                              Maint

                              Perfo

                              rm

                              Prod

                              uctiv

                              Tena

                              nt Sa

                              t

                              Comf

                              ort

                              Appe

                              ar

                              Light

                              Qual

                              SellL

                              ease

                              Devp Ownr AampE

                              Note AgtE (~4 vs ~2) for these NEBs

                              Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                              Source Skumatz Economic Research Associates Inc 2002

                              Owners more positive ndashUnderinvestment train

                              • Soup to nuts on non-energy benefits NEIs
                              • TOPICS
                              • BACKGROUND NEBS
                              • 20+ YEARS OF NEBS PROGRESShellip
                              • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                              • KEY APPLICATIONS OF NEBS
                              • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                              • Neb measurement
                              • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                              • MEASURING PARTICIPANTNEBs
                              • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                              • PARTICIPANT MEASUREMENT METHODS COMPARISON
                              • PROS AND CONS OF NEBS MEASUREMENT METHODS
                              • PROGRESS TRANSFERABILITY OF NEBS
                              • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                              • PROGRESS AND GAPS
                              • PROGRESS AND GAPS
                              • USING NEB NEI VALUES
                              • Slide Number 19
                              • ARE NEBS HIGH VALUE
                              • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                              • TOP NEBS FOR WEATHERIZATION PROGRAM
                              • RESULTS FOR ENERGY STARreg APPLIANCES
                              • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                              • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                              • CampI PROGRAMS ldquoNEBBEDrdquo
                              • AUDIT FINANCIAL PROGRAM
                              • CampI NEW CONSTRUCTION
                              • CampI TECHNICAL ASSISTANCE
                              • EXPRESSING NEBS VALUEndashCxe
                              • Slide Number 31
                              • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                              • EMISSIONS (SOCIETAL NEBS)
                              • UTILITY NEBS
                              • ADJUSTED PAYBACKS
                              • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                              • NEBS IN BC
                              • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                              • APPLICATION TO BC
                              • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                              • STATE TREATMENT OF NEBS NEBs
                              • Slide Number 42
                              • SELECTED STATE PROGRESS
                              • SELECTED STATE PROGRESS
                              • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                              • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                              • DISCOUNT RATE MATTERS
                              • EUL VARIATIONS- ADOPTED VALUES
                              • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                              • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                              • TAKEAWAYSON NEBSNEIS
                              • THANK YOUQuestions
                              • Slide Number 53
                              • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                              • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                m Product losses20
                                l Ease of selling leasing building33
                                n Labor requirements45
                                h Aesthetics appearance59
                                j Noise50
                                b Equipment maintenance660
                                f Tenant satisfaction60
                                d Equipment lifetimes718
                                e Productivity724
                                k Building safety[2]73
                                g Comfort933
                                i Lighting Quality of Light916
                                c Equipment performance[1]1019
                                a Operating costs1165
                                o Doing good for the Environment[3]1132
                                Other03
                                [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                m Product losses
                                l Ease of selling leasing building
                                n Labor requirements
                                h Aesthetics appearance
                                j Noise
                                b Equipment maintenance
                                f Tenant satisfaction
                                d Equipment lifetimes
                                e Productivity
                                k Building safety[2]
                                g Comfort
                                i Lighting Quality of Light
                                c Equipment performance[1]
                                a Operating costs
                                o Doing good for the Environment[3]
                                Share of ZALEH NEBS from various measures
                                Each presented as multiple of the (overall) energy savings
                                Super Insulation037
                                Double Glazing026
                                Solar Design025
                                Solar Water Heating005
                                EnergySmart (insul Retrofit)Dunedin (insul)
                                Other124
                                Noise24
                                Budgetfinances25
                                Environmental65
                                Health78
                                Reduced moves86
                                Ease of sellingrenting130
                                DampnessMold1316
                                Comfort-related4933
                                101101
                                Operational NEBs44
                                DesignConstruction NEBs31
                                Occupant NEBs25
                                1
                                OtherOther
                                NoiseNoise
                                BudgetfinancesBudgetfinances
                                EnvironmentalEnvironmental
                                HealthHealth
                                Reduced movesReduced moves
                                Ease of sellingrentingEase of sellingrenting
                                DampnessMoldDampnessMold
                                Comfort-relatedComfort-related
                                Boiler
                                VendorNPPartic
                                Perform0280202
                                Footprint028013017
                                ControlFeatures031022016
                                Tenant compl006008014
                                Noise005006014
                                Maint004009007
                                Other004003007
                                Lifetime-006018006
                                MF New construction energy star
                                Operating cost014
                                Environment012
                                Maintenance01
                                Lifetime01
                                Performance009
                                Occupant Satisf009
                                Comfort008
                                Noise006
                                Sellingleasing006
                                Aesthetics005
                                Light Quality005
                                Power QualReliab004
                                Safety002
                                RIGHTLIGHTS
                                Environment023
                                Quality016
                                Quantity014
                                PgmAvail01
                                Flicker008
                                Maintenance006
                                Lifetime006
                                SalesProductivity006
                                Safety005
                                Noise005
                                Control003
                                Illness0
                                PerformPerformPerform
                                FootprintFootprintFootprint
                                ControlFeaturesControlFeaturesControlFeatures
                                Tenant complTenant complTenant compl
                                NoiseNoiseNoise
                                MaintMaintMaint
                                OtherOtherOther
                                LifetimeLifetimeLifetime
                                Environment
                                Quality
                                Quantity
                                PgmAvail
                                Flicker
                                Maintenance
                                Lifetime
                                SalesProductivity
                                Safety
                                Noise
                                Control
                                Illness

                                Variability

                                Relevant NEB Categories

                                Program measure invariant (suitable for ldquoadderrdquo)

                                middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                                Program measure dependent

                                middot Economic ndash societal (depends on measures and local manufacture installation)

                                middot Health and safety health care illnesses ndash societal and participant (measure)

                                middot Water wastewater infrastructure and water bill savings ndash societal and participant

                                middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                Climate dependent

                                middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                Residential Target dependent (low income or MF vs SF)

                                middot Payment related ndash utility (arrearages etc stronger for low income targets)

                                middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

                                middot Participant benefits related to hardship and payments

                                middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                                Used for

                                Major Advantages

                                Major Disadvantages

                                Direct measurement (specific studies of changes on-site)

                                Comrsquol labor productivity etc

                                Direct precise attributable

                                Small estimation sample sizes specialized cases poor transferability expensive

                                Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

                                Insurance water health others

                                Long history easy secondary sources

                                Credible to reviewers vetted inputs

                                Not available for all NEBs

                                Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

                                Emissions economics

                                Third party peer-vetted models available for economics jobs and emissions

                                Not available for all NEBs

                                Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

                                Wide variety of Participant NEBs

                                Large sample sizes amp statistical properties

                                Affordable

                                Multiple estimates leading to similar ranges

                                Direct method of measuring some key NEBs

                                Concerns about surveys as a source of quantitative values amp reliability

                                Recall from survey respondents

                                Proper attribution to programs measures

                                Sheet2

                                Sheet3

                                SERA

                                PROS AND CONS OF NEBS MEASUREMENT METHODS

                                13

                                Source Skumatz SERA research

                                Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

                                Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

                                Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

                                Long history easy secondary sources Credible to reviewers vetted inputs

                                Not available for all NEBs

                                Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

                                Third party peer-vetted models available for economics jobs and emissions

                                Not available for all NEBs

                                Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

                                Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

                                Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

                                SERA

                                PROGRESS TRANSFERABILITY OF NEBS

                                14

                                Source Skumatz SERA research

                                Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

                                bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                                Program measure dependent

                                bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

                                education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                Residential Target dependent (low income or MF vs SF)

                                bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

                                vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                                Cost savings consistency if transferable vs risk

                                SERA

                                RANGES TYPICAL NEB VALUES FOR WxRETROFI

                                15

                                SERA

                                PROGRESS AND GAPS Consistency Transferability

                                Some programs types others less studied Transferability research

                                Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

                                measures until then Across the board Savings share Regression

                                SERA

                                PROGRESS AND GAPS Gas vs Electric

                                Some researchndashusually program-wide not broken down International research found participant NEBs have

                                similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

                                MF Less-commonly-studied complicated by poor response

                                and complexity of sector (decision-maker some measures in home some central) separate from low income not common

                                Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

                                Commercial gaps in some measures

                                SERA

                                USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                                Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                                18

                                0

                                005

                                01

                                015

                                02

                                025

                                Environment

                                Quality

                                Quantity

                                PgmAvailFlicker

                                Mainten

                                ance

                                Lifetime

                                SalesP

                                roductivity

                                Safety

                                Noise

                                Control

                                Illness

                                Total NEBs Multiplier=X

                                $

                                Savings value=$X

                                Chart3

                                023
                                016
                                014
                                01
                                008
                                006
                                006
                                006
                                005
                                005
                                003
                                0

                                Chart1

                                Vendor
                                NP
                                Partic
                                028
                                02
                                02
                                028
                                013
                                017
                                031
                                022
                                016
                                006
                                008
                                014
                                005
                                006
                                014
                                004
                                009
                                007
                                004
                                003
                                007
                                -006
                                018
                                006

                                Sheet1

                                Sheet1

                                Sheet2

                                Sheet3

                                SERA

                                NEB VALUES

                                19

                                SERA

                                ARE NEBS HIGH VALUE Energy savings are often the minority of

                                benefits from program measuresndash Donrsquot ignore that

                                20

                                0 05 1 15 2 25 3

                                Energy Sav

                                NEBs

                                NEB vs Energy Savings ValueIncluding all NEBs

                                Source (SkumatzSERA 2010 amp others)

                                Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                SERA

                                RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                MF Many others

                                Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                Variety of end uses

                                SERA

                                TOP NEBS FOR WEATHERIZATION PROGRAM

                                Weatherization program multipliergt1

                                Source SERA Research

                                SERA

                                RESULTS FOR ENERGY STARreg APPLIANCES

                                0002004006008

                                01012014016018

                                Maint

                                Appear

                                Perf LifeNoise Satis

                                Comfort

                                LiteSafe

                                ty SellMove

                                Water

                                Enviro

                                Refrig DW CW

                                Source SERA Research

                                SERA

                                INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                5 increments to 50

                                Source SERA research

                                Chart5

                                EnergySmart (insul Retrofit)
                                Dunedin (insul)
                                001
                                024
                                002
                                004
                                002
                                005
                                006
                                005
                                007
                                008
                                008
                                006
                                013
                                0
                                013
                                016
                                049
                                033

                                cx

                                cx

                                nz

                                Sheet3

                                Sheet3

                                Super Insulation
                                Double Glazing
                                Solar Design
                                Solar Water Heating
                                Super Insulation
                                Double Glazing
                                Solar Design
                                Solar Water Heating
                                EnergySmart (insul Retrofit)
                                Dunedin (insul)

                                SERA

                                ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                0

                                002

                                004

                                006

                                008

                                01

                                012

                                Maint

                                Perf LifeSatis

                                f

                                ComfortAesth

                                NoiseSafe

                                ty

                                LightingSelli

                                ng

                                Stay in home

                                Enviro

                                Sick Days

                                Calls

                                ES Homes HP wES Energy savings multiplier=1

                                Source SERA Research

                                SERA

                                CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                DG CHP HVAC Equipment rebate Other

                                Building codes incentives by cities

                                Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                Variety of end uses

                                SERA

                                AUDIT FINANCIAL PROGRAM

                                0

                                01

                                02

                                03

                                04

                                05

                                06

                                07

                                Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                Source SERA Research

                                SERA

                                CampI NEW CONSTRUCTION

                                -005

                                0

                                005

                                01

                                015

                                02

                                AampE Owner(Pct of Total Partic NEBs)

                                Source SERA Research

                                SERA

                                CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                0 2 4 6 8 10 12

                                m Product losses

                                l Ease of selling leasing building

                                n Labor requirements

                                h Aesthetics appearance

                                j Noise

                                b Equipment maintenance

                                f Tenant satisfaction

                                d Equipment lifetimes

                                e Productivity

                                k Building safety[2]

                                g Comfort

                                i Lighting Quality of Light

                                c Equipment performance[1]

                                a Operating costs

                                o Doing good for the Environment[3]

                                Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                vcv

                                Source SERA Research

                                Chart1

                                CampI Technical Assistance NEBs
                                002
                                003
                                004
                                005
                                005
                                006
                                006
                                007
                                007
                                007
                                009
                                009
                                01
                                011
                                011

                                Sheet1

                                Sheet1

                                CampI Technical Assistance Program NEBs (Participant)

                                Sheet2

                                Sheet3

                                SERA

                                EXPRESSING NEBS VALUEndashCxe

                                Strong value from RetroCxSource SERA Research

                                SERA

                                SOCIETAL NEB EXAMPLES

                                31

                                SERA

                                JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                32

                                0

                                02

                                04

                                06

                                08

                                1

                                12

                                HPWxRetrof Appliance

                                CA

                                WI

                                Natl

                                (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                Jobs Economic

                                SERA

                                EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                33

                                SERA

                                UTILITY NEBSUtility NEBs for Template Program

                                Debt WriteOff (util)13

                                Rate Subsidy(util)61

                                HealthSafety(util)0

                                Colln Costs (util)0

                                Gas Calls (util)0

                                Calls to CSRs(util)2

                                TampD (util)16

                                Arrears (util)0

                                Reconnects (util)0

                                Notices (util)7

                                Shutoffs (util)1

                                Rate subsidy TampD

                                Payment-related

                                Source Skumatz Economic Research Associates research

                                Example Low Income Weatherization

                                SERA

                                ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                Value of 100 HALVES Payback doubles ROI

                                Source Skumatz Economics (SERA)

                                35

                                ( for NEBs Multiplier of 125)

                                SERA

                                NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                36

                                hellip2001 ON

                                SERA

                                37

                                NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                For true representation of B amp C elements of NEBs address missing factor bias

                                Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                BC applications were considered early on then pulled back awaiting more quantitative evidence

                                SERA

                                CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                38

                                Test Benefits Costs States Using Traditionally

                                Improved treat-ment with NEBs

                                Utility Cost (or Program Administrator Test) (UCT or PAC)

                                bull Avoided supply costs for transmission distribution and generation (TDampG)

                                bull Avoided gas and water supply costs

                                bull Program administration

                                bull Participant incentives

                                bull Increased supply cost

                                CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                Use cost only paid by the utility

                                Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                Same as above plus bull increased revenue

                                Same as above plus bull Decreased revenue

                                AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                Participant cost bull Utility bill reductions bull Participant incentives

                                bull Participant direct costs

                                AR CA FL HI IA IN MI MN NY VA

                                Participant NEBs

                                Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                SERA

                                APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                Expansion Concern with risk calculation accuracy

                                39

                                Source Skumatz SERA research

                                SERA

                                40

                                METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                Maximize DSM opportunities amp feedback Accuracy tailoring

                                Minimize Regulatory amp Implementer Risk

                                Minimize Evaluation Cost

                                Adder

                                Readily MeasurableHybrid

                                All NEBsSource SERA Research

                                SERA

                                41

                                STATE TREATMENT OF NEBS NEBsRegulatory

                                Screening Application Utilities regions

                                Program Marketing Fairly widespread use in utilities states across the country

                                MO

                                RE AGGRESSIVE ==gt

                                Test Pgm Screen - adder

                                IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                Test Pgm Screen - readily measured

                                MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                Test Hybrid (potential adder amp measured)

                                CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                Test Pgm screen - Broad

                                With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                SERA

                                42

                                Key

                                States with NEBs in CE tests for At least one type of program

                                copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                STATES WITH NEBS IN CE TESTS

                                NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                SERA

                                SELECTED STATE PROGRESS CA

                                2001 ndash LIPPT model NY

                                Mid-2000s measurement amp scenarios not included CO

                                Adders (10 electric 25 LI 5 gas) VT

                                15 adder helped by previous research DC

                                10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                43

                                Source Skumatz SERA research

                                SERA

                                SELECTED STATE PROGRESS Dominos ongoing

                                Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                44

                                Source Skumatz SERA research

                                SERA

                                APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                Expansion Concern with risk calculation accuracy

                                45

                                Source Skumatz SERA research

                                SERA

                                46

                                NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                SourceSkumatz SERA Independent research

                                NTG ndash accuracy measurement incomplete

                                Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                Cost Complicated expensive local changes

                                RELATIVE Risk from NEBs

                                Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                NEBs as accurate as many inputs (all could improve)

                                SERA

                                DISCOUNT RATE MATTERS

                                47

                                Source Skumatz SERA Research

                                0

                                20

                                40

                                60

                                80

                                100

                                120

                                1 2 3 4 5 6 7 8 9 10

                                PV Ratios vs 1 Discount Rate

                                10 yr PV 20 yr PV

                                Discount rate - percent

                                WACCSocialLTT

                                Source Skumatz Electricity Journal IEPEC paper independent research

                                SERA

                                EUL VARIATIONS-ADOPTED VALUES

                                Source Skumatz Economic Research Associates SERA independent research

                                0

                                50

                                100

                                150

                                200

                                250

                                300

                                350

                                400

                                450

                                0 10 20 30 40 50 60 70

                                EUL Variation by Measure

                                67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                Poorly supportedOldTechnology change

                                SERA

                                KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                change the BC conclusion

                                On the cusphellip

                                If NO Yoursquore doneAnd bias addressedsufficiently

                                IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                Source SERA all rights reserved

                                SERArsquos lsquoNEB-ItrdquoModel ndash

                                SERA

                                IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                Utility Soc Part Conserv Recrsquom

                                Rationale

                                Base Percent X X X Program-invariant

                                Low Income X X X X Multiple sources

                                Weatherization X X X Substantial Participantimpacts

                                Measure Program-specific

                                X Varies by measure sector

                                Other Recomrsquos Local Research

                                50

                                Source Skumatz SERA

                                Developing values for multiple states amp utilities

                                SERA

                                TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                Other sources of bias to research (EUL NTG etc) bring high variation

                                5) States are incorporating NEBs dominos input in deliberations in multiple states

                                6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                THANK YOU

                                Questions

                                Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                SERA

                                POWER OF NEBS ndashBARRIERS DISCONNECTS

                                53

                                SERA

                                NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                Negative NEB values cost of barrier

                                Solar Water Heat NZ$ Euros

                                Solar Design NZ$ Euros

                                Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                -23 -12 (079)

                                -11 -6 (06)

                                Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                Source Skumatz Economic Research Associates research

                                SERA

                                CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                -5-4-3-2-1012345

                                Op Co

                                st

                                Init C

                                ost

                                Maint

                                Perfo

                                rm

                                Prod

                                uctiv

                                Tena

                                nt Sa

                                t

                                Comf

                                ort

                                Appe

                                ar

                                Light

                                Qual

                                SellL

                                ease

                                Devp Ownr AampE

                                Note AgtE (~4 vs ~2) for these NEBs

                                Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                Source Skumatz Economic Research Associates Inc 2002

                                Owners more positive ndashUnderinvestment train

                                • Soup to nuts on non-energy benefits NEIs
                                • TOPICS
                                • BACKGROUND NEBS
                                • 20+ YEARS OF NEBS PROGRESShellip
                                • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                • KEY APPLICATIONS OF NEBS
                                • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                • Neb measurement
                                • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                • MEASURING PARTICIPANTNEBs
                                • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                • PROGRESS TRANSFERABILITY OF NEBS
                                • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                • PROGRESS AND GAPS
                                • PROGRESS AND GAPS
                                • USING NEB NEI VALUES
                                • Slide Number 19
                                • ARE NEBS HIGH VALUE
                                • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                • TOP NEBS FOR WEATHERIZATION PROGRAM
                                • RESULTS FOR ENERGY STARreg APPLIANCES
                                • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                • CampI PROGRAMS ldquoNEBBEDrdquo
                                • AUDIT FINANCIAL PROGRAM
                                • CampI NEW CONSTRUCTION
                                • CampI TECHNICAL ASSISTANCE
                                • EXPRESSING NEBS VALUEndashCxe
                                • Slide Number 31
                                • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                • EMISSIONS (SOCIETAL NEBS)
                                • UTILITY NEBS
                                • ADJUSTED PAYBACKS
                                • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                • NEBS IN BC
                                • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                • APPLICATION TO BC
                                • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                • STATE TREATMENT OF NEBS NEBs
                                • Slide Number 42
                                • SELECTED STATE PROGRESS
                                • SELECTED STATE PROGRESS
                                • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                • DISCOUNT RATE MATTERS
                                • EUL VARIATIONS- ADOPTED VALUES
                                • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                • TAKEAWAYSON NEBSNEIS
                                • THANK YOUQuestions
                                • Slide Number 53
                                • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                  Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                  m Product losses20
                                  l Ease of selling leasing building33
                                  n Labor requirements45
                                  h Aesthetics appearance59
                                  j Noise50
                                  b Equipment maintenance660
                                  f Tenant satisfaction60
                                  d Equipment lifetimes718
                                  e Productivity724
                                  k Building safety[2]73
                                  g Comfort933
                                  i Lighting Quality of Light916
                                  c Equipment performance[1]1019
                                  a Operating costs1165
                                  o Doing good for the Environment[3]1132
                                  Other03
                                  [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                  [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                  [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                  m Product losses
                                  l Ease of selling leasing building
                                  n Labor requirements
                                  h Aesthetics appearance
                                  j Noise
                                  b Equipment maintenance
                                  f Tenant satisfaction
                                  d Equipment lifetimes
                                  e Productivity
                                  k Building safety[2]
                                  g Comfort
                                  i Lighting Quality of Light
                                  c Equipment performance[1]
                                  a Operating costs
                                  o Doing good for the Environment[3]
                                  Share of ZALEH NEBS from various measures
                                  Each presented as multiple of the (overall) energy savings
                                  Super Insulation037
                                  Double Glazing026
                                  Solar Design025
                                  Solar Water Heating005
                                  EnergySmart (insul Retrofit)Dunedin (insul)
                                  Other124
                                  Noise24
                                  Budgetfinances25
                                  Environmental65
                                  Health78
                                  Reduced moves86
                                  Ease of sellingrenting130
                                  DampnessMold1316
                                  Comfort-related4933
                                  101101
                                  Operational NEBs44
                                  DesignConstruction NEBs31
                                  Occupant NEBs25
                                  1
                                  OtherOther
                                  NoiseNoise
                                  BudgetfinancesBudgetfinances
                                  EnvironmentalEnvironmental
                                  HealthHealth
                                  Reduced movesReduced moves
                                  Ease of sellingrentingEase of sellingrenting
                                  DampnessMoldDampnessMold
                                  Comfort-relatedComfort-related
                                  Boiler
                                  VendorNPPartic
                                  Perform0280202
                                  Footprint028013017
                                  ControlFeatures031022016
                                  Tenant compl006008014
                                  Noise005006014
                                  Maint004009007
                                  Other004003007
                                  Lifetime-006018006
                                  MF New construction energy star
                                  Operating cost014
                                  Environment012
                                  Maintenance01
                                  Lifetime01
                                  Performance009
                                  Occupant Satisf009
                                  Comfort008
                                  Noise006
                                  Sellingleasing006
                                  Aesthetics005
                                  Light Quality005
                                  Power QualReliab004
                                  Safety002
                                  RIGHTLIGHTS
                                  Environment023
                                  Quality016
                                  Quantity014
                                  PgmAvail01
                                  Flicker008
                                  Maintenance006
                                  Lifetime006
                                  SalesProductivity006
                                  Safety005
                                  Noise005
                                  Control003
                                  Illness0
                                  PerformPerformPerform
                                  FootprintFootprintFootprint
                                  ControlFeaturesControlFeaturesControlFeatures
                                  Tenant complTenant complTenant compl
                                  NoiseNoiseNoise
                                  MaintMaintMaint
                                  OtherOtherOther
                                  LifetimeLifetimeLifetime
                                  Environment
                                  Quality
                                  Quantity
                                  PgmAvail
                                  Flicker
                                  Maintenance
                                  Lifetime
                                  SalesProductivity
                                  Safety
                                  Noise
                                  Control
                                  Illness

                                  Variability

                                  Relevant NEB Categories

                                  Program measure invariant (suitable for ldquoadderrdquo)

                                  middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                                  Program measure dependent

                                  middot Economic ndash societal (depends on measures and local manufacture installation)

                                  middot Health and safety health care illnesses ndash societal and participant (measure)

                                  middot Water wastewater infrastructure and water bill savings ndash societal and participant

                                  middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                  Climate dependent

                                  middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                  Residential Target dependent (low income or MF vs SF)

                                  middot Payment related ndash utility (arrearages etc stronger for low income targets)

                                  middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

                                  middot Participant benefits related to hardship and payments

                                  middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                                  Used for

                                  Major Advantages

                                  Major Disadvantages

                                  Direct measurement (specific studies of changes on-site)

                                  Comrsquol labor productivity etc

                                  Direct precise attributable

                                  Small estimation sample sizes specialized cases poor transferability expensive

                                  Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

                                  Insurance water health others

                                  Long history easy secondary sources

                                  Credible to reviewers vetted inputs

                                  Not available for all NEBs

                                  Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

                                  Emissions economics

                                  Third party peer-vetted models available for economics jobs and emissions

                                  Not available for all NEBs

                                  Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

                                  Wide variety of Participant NEBs

                                  Large sample sizes amp statistical properties

                                  Affordable

                                  Multiple estimates leading to similar ranges

                                  Direct method of measuring some key NEBs

                                  Concerns about surveys as a source of quantitative values amp reliability

                                  Recall from survey respondents

                                  Proper attribution to programs measures

                                  Sheet3

                                  SERA

                                  PROS AND CONS OF NEBS MEASUREMENT METHODS

                                  13

                                  Source Skumatz SERA research

                                  Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

                                  Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

                                  Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

                                  Long history easy secondary sources Credible to reviewers vetted inputs

                                  Not available for all NEBs

                                  Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

                                  Third party peer-vetted models available for economics jobs and emissions

                                  Not available for all NEBs

                                  Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

                                  Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

                                  Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

                                  SERA

                                  PROGRESS TRANSFERABILITY OF NEBS

                                  14

                                  Source Skumatz SERA research

                                  Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

                                  bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                                  Program measure dependent

                                  bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

                                  education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                  Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                  Residential Target dependent (low income or MF vs SF)

                                  bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

                                  vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                                  Cost savings consistency if transferable vs risk

                                  SERA

                                  RANGES TYPICAL NEB VALUES FOR WxRETROFI

                                  15

                                  SERA

                                  PROGRESS AND GAPS Consistency Transferability

                                  Some programs types others less studied Transferability research

                                  Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

                                  measures until then Across the board Savings share Regression

                                  SERA

                                  PROGRESS AND GAPS Gas vs Electric

                                  Some researchndashusually program-wide not broken down International research found participant NEBs have

                                  similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

                                  MF Less-commonly-studied complicated by poor response

                                  and complexity of sector (decision-maker some measures in home some central) separate from low income not common

                                  Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

                                  Commercial gaps in some measures

                                  SERA

                                  USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                                  Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                                  18

                                  0

                                  005

                                  01

                                  015

                                  02

                                  025

                                  Environment

                                  Quality

                                  Quantity

                                  PgmAvailFlicker

                                  Mainten

                                  ance

                                  Lifetime

                                  SalesP

                                  roductivity

                                  Safety

                                  Noise

                                  Control

                                  Illness

                                  Total NEBs Multiplier=X

                                  $

                                  Savings value=$X

                                  Chart3

                                  023
                                  016
                                  014
                                  01
                                  008
                                  006
                                  006
                                  006
                                  005
                                  005
                                  003
                                  0

                                  Chart1

                                  Vendor
                                  NP
                                  Partic
                                  028
                                  02
                                  02
                                  028
                                  013
                                  017
                                  031
                                  022
                                  016
                                  006
                                  008
                                  014
                                  005
                                  006
                                  014
                                  004
                                  009
                                  007
                                  004
                                  003
                                  007
                                  -006
                                  018
                                  006

                                  Sheet1

                                  Sheet1

                                  Sheet2

                                  Sheet3

                                  SERA

                                  NEB VALUES

                                  19

                                  SERA

                                  ARE NEBS HIGH VALUE Energy savings are often the minority of

                                  benefits from program measuresndash Donrsquot ignore that

                                  20

                                  0 05 1 15 2 25 3

                                  Energy Sav

                                  NEBs

                                  NEB vs Energy Savings ValueIncluding all NEBs

                                  Source (SkumatzSERA 2010 amp others)

                                  Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                  SERA

                                  RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                  MF Many others

                                  Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                  Variety of end uses

                                  SERA

                                  TOP NEBS FOR WEATHERIZATION PROGRAM

                                  Weatherization program multipliergt1

                                  Source SERA Research

                                  SERA

                                  RESULTS FOR ENERGY STARreg APPLIANCES

                                  0002004006008

                                  01012014016018

                                  Maint

                                  Appear

                                  Perf LifeNoise Satis

                                  Comfort

                                  LiteSafe

                                  ty SellMove

                                  Water

                                  Enviro

                                  Refrig DW CW

                                  Source SERA Research

                                  SERA

                                  INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                  5 increments to 50

                                  Source SERA research

                                  Chart5

                                  EnergySmart (insul Retrofit)
                                  Dunedin (insul)
                                  001
                                  024
                                  002
                                  004
                                  002
                                  005
                                  006
                                  005
                                  007
                                  008
                                  008
                                  006
                                  013
                                  0
                                  013
                                  016
                                  049
                                  033

                                  cx

                                  cx

                                  nz

                                  Sheet3

                                  Sheet3

                                  Super Insulation
                                  Double Glazing
                                  Solar Design
                                  Solar Water Heating
                                  Super Insulation
                                  Double Glazing
                                  Solar Design
                                  Solar Water Heating
                                  EnergySmart (insul Retrofit)
                                  Dunedin (insul)

                                  SERA

                                  ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                  0

                                  002

                                  004

                                  006

                                  008

                                  01

                                  012

                                  Maint

                                  Perf LifeSatis

                                  f

                                  ComfortAesth

                                  NoiseSafe

                                  ty

                                  LightingSelli

                                  ng

                                  Stay in home

                                  Enviro

                                  Sick Days

                                  Calls

                                  ES Homes HP wES Energy savings multiplier=1

                                  Source SERA Research

                                  SERA

                                  CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                  DG CHP HVAC Equipment rebate Other

                                  Building codes incentives by cities

                                  Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                  Variety of end uses

                                  SERA

                                  AUDIT FINANCIAL PROGRAM

                                  0

                                  01

                                  02

                                  03

                                  04

                                  05

                                  06

                                  07

                                  Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                  Source SERA Research

                                  SERA

                                  CampI NEW CONSTRUCTION

                                  -005

                                  0

                                  005

                                  01

                                  015

                                  02

                                  AampE Owner(Pct of Total Partic NEBs)

                                  Source SERA Research

                                  SERA

                                  CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                  0 2 4 6 8 10 12

                                  m Product losses

                                  l Ease of selling leasing building

                                  n Labor requirements

                                  h Aesthetics appearance

                                  j Noise

                                  b Equipment maintenance

                                  f Tenant satisfaction

                                  d Equipment lifetimes

                                  e Productivity

                                  k Building safety[2]

                                  g Comfort

                                  i Lighting Quality of Light

                                  c Equipment performance[1]

                                  a Operating costs

                                  o Doing good for the Environment[3]

                                  Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                  vcv

                                  Source SERA Research

                                  Chart1

                                  CampI Technical Assistance NEBs
                                  002
                                  003
                                  004
                                  005
                                  005
                                  006
                                  006
                                  007
                                  007
                                  007
                                  009
                                  009
                                  01
                                  011
                                  011

                                  Sheet1

                                  Sheet1

                                  CampI Technical Assistance Program NEBs (Participant)

                                  Sheet2

                                  Sheet3

                                  SERA

                                  EXPRESSING NEBS VALUEndashCxe

                                  Strong value from RetroCxSource SERA Research

                                  SERA

                                  SOCIETAL NEB EXAMPLES

                                  31

                                  SERA

                                  JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                  32

                                  0

                                  02

                                  04

                                  06

                                  08

                                  1

                                  12

                                  HPWxRetrof Appliance

                                  CA

                                  WI

                                  Natl

                                  (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                  Jobs Economic

                                  SERA

                                  EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                  33

                                  SERA

                                  UTILITY NEBSUtility NEBs for Template Program

                                  Debt WriteOff (util)13

                                  Rate Subsidy(util)61

                                  HealthSafety(util)0

                                  Colln Costs (util)0

                                  Gas Calls (util)0

                                  Calls to CSRs(util)2

                                  TampD (util)16

                                  Arrears (util)0

                                  Reconnects (util)0

                                  Notices (util)7

                                  Shutoffs (util)1

                                  Rate subsidy TampD

                                  Payment-related

                                  Source Skumatz Economic Research Associates research

                                  Example Low Income Weatherization

                                  SERA

                                  ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                  be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                  Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                  Value of 100 HALVES Payback doubles ROI

                                  Source Skumatz Economics (SERA)

                                  35

                                  ( for NEBs Multiplier of 125)

                                  SERA

                                  NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                  36

                                  hellip2001 ON

                                  SERA

                                  37

                                  NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                  For true representation of B amp C elements of NEBs address missing factor bias

                                  Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                  BC applications were considered early on then pulled back awaiting more quantitative evidence

                                  SERA

                                  CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                  38

                                  Test Benefits Costs States Using Traditionally

                                  Improved treat-ment with NEBs

                                  Utility Cost (or Program Administrator Test) (UCT or PAC)

                                  bull Avoided supply costs for transmission distribution and generation (TDampG)

                                  bull Avoided gas and water supply costs

                                  bull Program administration

                                  bull Participant incentives

                                  bull Increased supply cost

                                  CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                  Use cost only paid by the utility

                                  Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                  Same as above plus bull increased revenue

                                  Same as above plus bull Decreased revenue

                                  AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                  Participant cost bull Utility bill reductions bull Participant incentives

                                  bull Participant direct costs

                                  AR CA FL HI IA IN MI MN NY VA

                                  Participant NEBs

                                  Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                  Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                  SERA

                                  APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                  Expansion Concern with risk calculation accuracy

                                  39

                                  Source Skumatz SERA research

                                  SERA

                                  40

                                  METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                  Maximize DSM opportunities amp feedback Accuracy tailoring

                                  Minimize Regulatory amp Implementer Risk

                                  Minimize Evaluation Cost

                                  Adder

                                  Readily MeasurableHybrid

                                  All NEBsSource SERA Research

                                  SERA

                                  41

                                  STATE TREATMENT OF NEBS NEBsRegulatory

                                  Screening Application Utilities regions

                                  Program Marketing Fairly widespread use in utilities states across the country

                                  MO

                                  RE AGGRESSIVE ==gt

                                  Test Pgm Screen - adder

                                  IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                  Test Pgm Screen - readily measured

                                  MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                  Test Hybrid (potential adder amp measured)

                                  CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                  Test Pgm screen - Broad

                                  With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                  Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                  SERA

                                  42

                                  Key

                                  States with NEBs in CE tests for At least one type of program

                                  copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                  STATES WITH NEBS IN CE TESTS

                                  NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                  groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                  SERA

                                  SELECTED STATE PROGRESS CA

                                  2001 ndash LIPPT model NY

                                  Mid-2000s measurement amp scenarios not included CO

                                  Adders (10 electric 25 LI 5 gas) VT

                                  15 adder helped by previous research DC

                                  10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                  43

                                  Source Skumatz SERA research

                                  SERA

                                  SELECTED STATE PROGRESS Dominos ongoing

                                  Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                  Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                  Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                  Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                  44

                                  Source Skumatz SERA research

                                  SERA

                                  APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                  Expansion Concern with risk calculation accuracy

                                  45

                                  Source Skumatz SERA research

                                  SERA

                                  46

                                  NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                  [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                  SourceSkumatz SERA Independent research

                                  NTG ndash accuracy measurement incomplete

                                  Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                  NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                  EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                  Cost Complicated expensive local changes

                                  RELATIVE Risk from NEBs

                                  Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                  Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                  NEBs as accurate as many inputs (all could improve)

                                  SERA

                                  DISCOUNT RATE MATTERS

                                  47

                                  Source Skumatz SERA Research

                                  0

                                  20

                                  40

                                  60

                                  80

                                  100

                                  120

                                  1 2 3 4 5 6 7 8 9 10

                                  PV Ratios vs 1 Discount Rate

                                  10 yr PV 20 yr PV

                                  Discount rate - percent

                                  WACCSocialLTT

                                  Source Skumatz Electricity Journal IEPEC paper independent research

                                  SERA

                                  EUL VARIATIONS-ADOPTED VALUES

                                  Source Skumatz Economic Research Associates SERA independent research

                                  0

                                  50

                                  100

                                  150

                                  200

                                  250

                                  300

                                  350

                                  400

                                  450

                                  0 10 20 30 40 50 60 70

                                  EUL Variation by Measure

                                  67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                  Poorly supportedOldTechnology change

                                  SERA

                                  KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                  Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                  measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                  change the BC conclusion

                                  On the cusphellip

                                  If NO Yoursquore doneAnd bias addressedsufficiently

                                  IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                  Source SERA all rights reserved

                                  SERArsquos lsquoNEB-ItrdquoModel ndash

                                  SERA

                                  IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                  Utility Soc Part Conserv Recrsquom

                                  Rationale

                                  Base Percent X X X Program-invariant

                                  Low Income X X X X Multiple sources

                                  Weatherization X X X Substantial Participantimpacts

                                  Measure Program-specific

                                  X Varies by measure sector

                                  Other Recomrsquos Local Research

                                  50

                                  Source Skumatz SERA

                                  Developing values for multiple states amp utilities

                                  SERA

                                  TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                  Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                  Other sources of bias to research (EUL NTG etc) bring high variation

                                  5) States are incorporating NEBs dominos input in deliberations in multiple states

                                  6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                  THANK YOU

                                  Questions

                                  Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                  SERA

                                  POWER OF NEBS ndashBARRIERS DISCONNECTS

                                  53

                                  SERA

                                  NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                  Negative NEB values cost of barrier

                                  Solar Water Heat NZ$ Euros

                                  Solar Design NZ$ Euros

                                  Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                  -23 -12 (079)

                                  -11 -6 (06)

                                  Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                  Source Skumatz Economic Research Associates research

                                  SERA

                                  CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                  NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                  -5-4-3-2-1012345

                                  Op Co

                                  st

                                  Init C

                                  ost

                                  Maint

                                  Perfo

                                  rm

                                  Prod

                                  uctiv

                                  Tena

                                  nt Sa

                                  t

                                  Comf

                                  ort

                                  Appe

                                  ar

                                  Light

                                  Qual

                                  SellL

                                  ease

                                  Devp Ownr AampE

                                  Note AgtE (~4 vs ~2) for these NEBs

                                  Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                  Source Skumatz Economic Research Associates Inc 2002

                                  Owners more positive ndashUnderinvestment train

                                  • Soup to nuts on non-energy benefits NEIs
                                  • TOPICS
                                  • BACKGROUND NEBS
                                  • 20+ YEARS OF NEBS PROGRESShellip
                                  • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                  • KEY APPLICATIONS OF NEBS
                                  • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                  • Neb measurement
                                  • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                  • MEASURING PARTICIPANTNEBs
                                  • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                  • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                  • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                  • PROGRESS TRANSFERABILITY OF NEBS
                                  • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                  • PROGRESS AND GAPS
                                  • PROGRESS AND GAPS
                                  • USING NEB NEI VALUES
                                  • Slide Number 19
                                  • ARE NEBS HIGH VALUE
                                  • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                  • TOP NEBS FOR WEATHERIZATION PROGRAM
                                  • RESULTS FOR ENERGY STARreg APPLIANCES
                                  • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                  • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                  • CampI PROGRAMS ldquoNEBBEDrdquo
                                  • AUDIT FINANCIAL PROGRAM
                                  • CampI NEW CONSTRUCTION
                                  • CampI TECHNICAL ASSISTANCE
                                  • EXPRESSING NEBS VALUEndashCxe
                                  • Slide Number 31
                                  • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                  • EMISSIONS (SOCIETAL NEBS)
                                  • UTILITY NEBS
                                  • ADJUSTED PAYBACKS
                                  • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                  • NEBS IN BC
                                  • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                  • APPLICATION TO BC
                                  • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                  • STATE TREATMENT OF NEBS NEBs
                                  • Slide Number 42
                                  • SELECTED STATE PROGRESS
                                  • SELECTED STATE PROGRESS
                                  • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                  • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                  • DISCOUNT RATE MATTERS
                                  • EUL VARIATIONS- ADOPTED VALUES
                                  • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                  • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                  • TAKEAWAYSON NEBSNEIS
                                  • THANK YOUQuestions
                                  • Slide Number 53
                                  • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                  • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                    Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                    m Product losses20
                                    l Ease of selling leasing building33
                                    n Labor requirements45
                                    h Aesthetics appearance59
                                    j Noise50
                                    b Equipment maintenance660
                                    f Tenant satisfaction60
                                    d Equipment lifetimes718
                                    e Productivity724
                                    k Building safety[2]73
                                    g Comfort933
                                    i Lighting Quality of Light916
                                    c Equipment performance[1]1019
                                    a Operating costs1165
                                    o Doing good for the Environment[3]1132
                                    Other03
                                    [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                    [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                    [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                    m Product losses
                                    l Ease of selling leasing building
                                    n Labor requirements
                                    h Aesthetics appearance
                                    j Noise
                                    b Equipment maintenance
                                    f Tenant satisfaction
                                    d Equipment lifetimes
                                    e Productivity
                                    k Building safety[2]
                                    g Comfort
                                    i Lighting Quality of Light
                                    c Equipment performance[1]
                                    a Operating costs
                                    o Doing good for the Environment[3]
                                    Share of ZALEH NEBS from various measures
                                    Each presented as multiple of the (overall) energy savings
                                    Super Insulation037
                                    Double Glazing026
                                    Solar Design025
                                    Solar Water Heating005
                                    EnergySmart (insul Retrofit)Dunedin (insul)
                                    Other124
                                    Noise24
                                    Budgetfinances25
                                    Environmental65
                                    Health78
                                    Reduced moves86
                                    Ease of sellingrenting130
                                    DampnessMold1316
                                    Comfort-related4933
                                    101101
                                    Operational NEBs44
                                    DesignConstruction NEBs31
                                    Occupant NEBs25
                                    1
                                    OtherOther
                                    NoiseNoise
                                    BudgetfinancesBudgetfinances
                                    EnvironmentalEnvironmental
                                    HealthHealth
                                    Reduced movesReduced moves
                                    Ease of sellingrentingEase of sellingrenting
                                    DampnessMoldDampnessMold
                                    Comfort-relatedComfort-related
                                    Boiler
                                    VendorNPPartic
                                    Perform0280202
                                    Footprint028013017
                                    ControlFeatures031022016
                                    Tenant compl006008014
                                    Noise005006014
                                    Maint004009007
                                    Other004003007
                                    Lifetime-006018006
                                    MF New construction energy star
                                    Operating cost014
                                    Environment012
                                    Maintenance01
                                    Lifetime01
                                    Performance009
                                    Occupant Satisf009
                                    Comfort008
                                    Noise006
                                    Sellingleasing006
                                    Aesthetics005
                                    Light Quality005
                                    Power QualReliab004
                                    Safety002
                                    RIGHTLIGHTS
                                    Environment023
                                    Quality016
                                    Quantity014
                                    PgmAvail01
                                    Flicker008
                                    Maintenance006
                                    Lifetime006
                                    SalesProductivity006
                                    Safety005
                                    Noise005
                                    Control003
                                    Illness0
                                    PerformPerformPerform
                                    FootprintFootprintFootprint
                                    ControlFeaturesControlFeaturesControlFeatures
                                    Tenant complTenant complTenant compl
                                    NoiseNoiseNoise
                                    MaintMaintMaint
                                    OtherOtherOther
                                    LifetimeLifetimeLifetime
                                    Environment
                                    Quality
                                    Quantity
                                    PgmAvail
                                    Flicker
                                    Maintenance
                                    Lifetime
                                    SalesProductivity
                                    Safety
                                    Noise
                                    Control
                                    Illness

                                    Variability

                                    Relevant NEB Categories

                                    Program measure invariant (suitable for ldquoadderrdquo)

                                    middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                                    Program measure dependent

                                    middot Economic ndash societal (depends on measures and local manufacture installation)

                                    middot Health and safety health care illnesses ndash societal and participant (measure)

                                    middot Water wastewater infrastructure and water bill savings ndash societal and participant

                                    middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                    Climate dependent

                                    middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                    Residential Target dependent (low income or MF vs SF)

                                    middot Payment related ndash utility (arrearages etc stronger for low income targets)

                                    middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

                                    middot Participant benefits related to hardship and payments

                                    middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                                    Used for

                                    Major Advantages

                                    Major Disadvantages

                                    Direct measurement (specific studies of changes on-site)

                                    Comrsquol labor productivity etc

                                    Direct precise attributable

                                    Small estimation sample sizes specialized cases poor transferability expensive

                                    Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

                                    Insurance water health others

                                    Long history easy secondary sources

                                    Credible to reviewers vetted inputs

                                    Not available for all NEBs

                                    Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

                                    Emissions economics

                                    Third party peer-vetted models available for economics jobs and emissions

                                    Not available for all NEBs

                                    Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

                                    Wide variety of Participant NEBs

                                    Large sample sizes amp statistical properties

                                    Affordable

                                    Multiple estimates leading to similar ranges

                                    Direct method of measuring some key NEBs

                                    Concerns about surveys as a source of quantitative values amp reliability

                                    Recall from survey respondents

                                    Proper attribution to programs measures

                                    SERA

                                    PROS AND CONS OF NEBS MEASUREMENT METHODS

                                    13

                                    Source Skumatz SERA research

                                    Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc

                                    Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive

                                    Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others

                                    Long history easy secondary sources Credible to reviewers vetted inputs

                                    Not available for all NEBs

                                    Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics

                                    Third party peer-vetted models available for economics jobs and emissions

                                    Not available for all NEBs

                                    Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs

                                    Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs

                                    Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures

                                    SERA

                                    PROGRESS TRANSFERABILITY OF NEBS

                                    14

                                    Source Skumatz SERA research

                                    Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

                                    bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                                    Program measure dependent

                                    bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

                                    education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                    Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                    Residential Target dependent (low income or MF vs SF)

                                    bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

                                    vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                                    Cost savings consistency if transferable vs risk

                                    SERA

                                    RANGES TYPICAL NEB VALUES FOR WxRETROFI

                                    15

                                    SERA

                                    PROGRESS AND GAPS Consistency Transferability

                                    Some programs types others less studied Transferability research

                                    Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

                                    measures until then Across the board Savings share Regression

                                    SERA

                                    PROGRESS AND GAPS Gas vs Electric

                                    Some researchndashusually program-wide not broken down International research found participant NEBs have

                                    similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

                                    MF Less-commonly-studied complicated by poor response

                                    and complexity of sector (decision-maker some measures in home some central) separate from low income not common

                                    Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

                                    Commercial gaps in some measures

                                    SERA

                                    USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                                    Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                                    18

                                    0

                                    005

                                    01

                                    015

                                    02

                                    025

                                    Environment

                                    Quality

                                    Quantity

                                    PgmAvailFlicker

                                    Mainten

                                    ance

                                    Lifetime

                                    SalesP

                                    roductivity

                                    Safety

                                    Noise

                                    Control

                                    Illness

                                    Total NEBs Multiplier=X

                                    $

                                    Savings value=$X

                                    Chart3

                                    023
                                    016
                                    014
                                    01
                                    008
                                    006
                                    006
                                    006
                                    005
                                    005
                                    003
                                    0

                                    Chart1

                                    Vendor
                                    NP
                                    Partic
                                    028
                                    02
                                    02
                                    028
                                    013
                                    017
                                    031
                                    022
                                    016
                                    006
                                    008
                                    014
                                    005
                                    006
                                    014
                                    004
                                    009
                                    007
                                    004
                                    003
                                    007
                                    -006
                                    018
                                    006

                                    Sheet1

                                    Sheet1

                                    Sheet2

                                    Sheet3

                                    SERA

                                    NEB VALUES

                                    19

                                    SERA

                                    ARE NEBS HIGH VALUE Energy savings are often the minority of

                                    benefits from program measuresndash Donrsquot ignore that

                                    20

                                    0 05 1 15 2 25 3

                                    Energy Sav

                                    NEBs

                                    NEB vs Energy Savings ValueIncluding all NEBs

                                    Source (SkumatzSERA 2010 amp others)

                                    Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                    SERA

                                    RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                    MF Many others

                                    Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                    Variety of end uses

                                    SERA

                                    TOP NEBS FOR WEATHERIZATION PROGRAM

                                    Weatherization program multipliergt1

                                    Source SERA Research

                                    SERA

                                    RESULTS FOR ENERGY STARreg APPLIANCES

                                    0002004006008

                                    01012014016018

                                    Maint

                                    Appear

                                    Perf LifeNoise Satis

                                    Comfort

                                    LiteSafe

                                    ty SellMove

                                    Water

                                    Enviro

                                    Refrig DW CW

                                    Source SERA Research

                                    SERA

                                    INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                    5 increments to 50

                                    Source SERA research

                                    Chart5

                                    EnergySmart (insul Retrofit)
                                    Dunedin (insul)
                                    001
                                    024
                                    002
                                    004
                                    002
                                    005
                                    006
                                    005
                                    007
                                    008
                                    008
                                    006
                                    013
                                    0
                                    013
                                    016
                                    049
                                    033

                                    cx

                                    cx

                                    nz

                                    Sheet3

                                    Sheet3

                                    Super Insulation
                                    Double Glazing
                                    Solar Design
                                    Solar Water Heating
                                    Super Insulation
                                    Double Glazing
                                    Solar Design
                                    Solar Water Heating
                                    EnergySmart (insul Retrofit)
                                    Dunedin (insul)

                                    SERA

                                    ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                    0

                                    002

                                    004

                                    006

                                    008

                                    01

                                    012

                                    Maint

                                    Perf LifeSatis

                                    f

                                    ComfortAesth

                                    NoiseSafe

                                    ty

                                    LightingSelli

                                    ng

                                    Stay in home

                                    Enviro

                                    Sick Days

                                    Calls

                                    ES Homes HP wES Energy savings multiplier=1

                                    Source SERA Research

                                    SERA

                                    CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                    DG CHP HVAC Equipment rebate Other

                                    Building codes incentives by cities

                                    Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                    Variety of end uses

                                    SERA

                                    AUDIT FINANCIAL PROGRAM

                                    0

                                    01

                                    02

                                    03

                                    04

                                    05

                                    06

                                    07

                                    Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                    Source SERA Research

                                    SERA

                                    CampI NEW CONSTRUCTION

                                    -005

                                    0

                                    005

                                    01

                                    015

                                    02

                                    AampE Owner(Pct of Total Partic NEBs)

                                    Source SERA Research

                                    SERA

                                    CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                    0 2 4 6 8 10 12

                                    m Product losses

                                    l Ease of selling leasing building

                                    n Labor requirements

                                    h Aesthetics appearance

                                    j Noise

                                    b Equipment maintenance

                                    f Tenant satisfaction

                                    d Equipment lifetimes

                                    e Productivity

                                    k Building safety[2]

                                    g Comfort

                                    i Lighting Quality of Light

                                    c Equipment performance[1]

                                    a Operating costs

                                    o Doing good for the Environment[3]

                                    Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                    vcv

                                    Source SERA Research

                                    Chart1

                                    CampI Technical Assistance NEBs
                                    002
                                    003
                                    004
                                    005
                                    005
                                    006
                                    006
                                    007
                                    007
                                    007
                                    009
                                    009
                                    01
                                    011
                                    011

                                    Sheet1

                                    Sheet1

                                    CampI Technical Assistance Program NEBs (Participant)

                                    Sheet2

                                    Sheet3

                                    SERA

                                    EXPRESSING NEBS VALUEndashCxe

                                    Strong value from RetroCxSource SERA Research

                                    SERA

                                    SOCIETAL NEB EXAMPLES

                                    31

                                    SERA

                                    JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                    32

                                    0

                                    02

                                    04

                                    06

                                    08

                                    1

                                    12

                                    HPWxRetrof Appliance

                                    CA

                                    WI

                                    Natl

                                    (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                    Jobs Economic

                                    SERA

                                    EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                    33

                                    SERA

                                    UTILITY NEBSUtility NEBs for Template Program

                                    Debt WriteOff (util)13

                                    Rate Subsidy(util)61

                                    HealthSafety(util)0

                                    Colln Costs (util)0

                                    Gas Calls (util)0

                                    Calls to CSRs(util)2

                                    TampD (util)16

                                    Arrears (util)0

                                    Reconnects (util)0

                                    Notices (util)7

                                    Shutoffs (util)1

                                    Rate subsidy TampD

                                    Payment-related

                                    Source Skumatz Economic Research Associates research

                                    Example Low Income Weatherization

                                    SERA

                                    ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                    be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                    Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                    Value of 100 HALVES Payback doubles ROI

                                    Source Skumatz Economics (SERA)

                                    35

                                    ( for NEBs Multiplier of 125)

                                    SERA

                                    NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                    36

                                    hellip2001 ON

                                    SERA

                                    37

                                    NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                    For true representation of B amp C elements of NEBs address missing factor bias

                                    Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                    BC applications were considered early on then pulled back awaiting more quantitative evidence

                                    SERA

                                    CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                    38

                                    Test Benefits Costs States Using Traditionally

                                    Improved treat-ment with NEBs

                                    Utility Cost (or Program Administrator Test) (UCT or PAC)

                                    bull Avoided supply costs for transmission distribution and generation (TDampG)

                                    bull Avoided gas and water supply costs

                                    bull Program administration

                                    bull Participant incentives

                                    bull Increased supply cost

                                    CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                    Use cost only paid by the utility

                                    Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                    Same as above plus bull increased revenue

                                    Same as above plus bull Decreased revenue

                                    AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                    Participant cost bull Utility bill reductions bull Participant incentives

                                    bull Participant direct costs

                                    AR CA FL HI IA IN MI MN NY VA

                                    Participant NEBs

                                    Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                    Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                    SERA

                                    APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                    Expansion Concern with risk calculation accuracy

                                    39

                                    Source Skumatz SERA research

                                    SERA

                                    40

                                    METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                    Maximize DSM opportunities amp feedback Accuracy tailoring

                                    Minimize Regulatory amp Implementer Risk

                                    Minimize Evaluation Cost

                                    Adder

                                    Readily MeasurableHybrid

                                    All NEBsSource SERA Research

                                    SERA

                                    41

                                    STATE TREATMENT OF NEBS NEBsRegulatory

                                    Screening Application Utilities regions

                                    Program Marketing Fairly widespread use in utilities states across the country

                                    MO

                                    RE AGGRESSIVE ==gt

                                    Test Pgm Screen - adder

                                    IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                    Test Pgm Screen - readily measured

                                    MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                    Test Hybrid (potential adder amp measured)

                                    CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                    Test Pgm screen - Broad

                                    With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                    Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                    SERA

                                    42

                                    Key

                                    States with NEBs in CE tests for At least one type of program

                                    copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                    STATES WITH NEBS IN CE TESTS

                                    NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                    groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                    SERA

                                    SELECTED STATE PROGRESS CA

                                    2001 ndash LIPPT model NY

                                    Mid-2000s measurement amp scenarios not included CO

                                    Adders (10 electric 25 LI 5 gas) VT

                                    15 adder helped by previous research DC

                                    10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                    43

                                    Source Skumatz SERA research

                                    SERA

                                    SELECTED STATE PROGRESS Dominos ongoing

                                    Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                    Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                    Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                    Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                    44

                                    Source Skumatz SERA research

                                    SERA

                                    APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                    Expansion Concern with risk calculation accuracy

                                    45

                                    Source Skumatz SERA research

                                    SERA

                                    46

                                    NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                    [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                    SourceSkumatz SERA Independent research

                                    NTG ndash accuracy measurement incomplete

                                    Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                    NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                    EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                    Cost Complicated expensive local changes

                                    RELATIVE Risk from NEBs

                                    Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                    Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                    NEBs as accurate as many inputs (all could improve)

                                    SERA

                                    DISCOUNT RATE MATTERS

                                    47

                                    Source Skumatz SERA Research

                                    0

                                    20

                                    40

                                    60

                                    80

                                    100

                                    120

                                    1 2 3 4 5 6 7 8 9 10

                                    PV Ratios vs 1 Discount Rate

                                    10 yr PV 20 yr PV

                                    Discount rate - percent

                                    WACCSocialLTT

                                    Source Skumatz Electricity Journal IEPEC paper independent research

                                    SERA

                                    EUL VARIATIONS-ADOPTED VALUES

                                    Source Skumatz Economic Research Associates SERA independent research

                                    0

                                    50

                                    100

                                    150

                                    200

                                    250

                                    300

                                    350

                                    400

                                    450

                                    0 10 20 30 40 50 60 70

                                    EUL Variation by Measure

                                    67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                    Poorly supportedOldTechnology change

                                    SERA

                                    KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                    Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                    measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                    change the BC conclusion

                                    On the cusphellip

                                    If NO Yoursquore doneAnd bias addressedsufficiently

                                    IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                    Source SERA all rights reserved

                                    SERArsquos lsquoNEB-ItrdquoModel ndash

                                    SERA

                                    IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                    Utility Soc Part Conserv Recrsquom

                                    Rationale

                                    Base Percent X X X Program-invariant

                                    Low Income X X X X Multiple sources

                                    Weatherization X X X Substantial Participantimpacts

                                    Measure Program-specific

                                    X Varies by measure sector

                                    Other Recomrsquos Local Research

                                    50

                                    Source Skumatz SERA

                                    Developing values for multiple states amp utilities

                                    SERA

                                    TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                    Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                    Other sources of bias to research (EUL NTG etc) bring high variation

                                    5) States are incorporating NEBs dominos input in deliberations in multiple states

                                    6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                    THANK YOU

                                    Questions

                                    Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                    SERA

                                    POWER OF NEBS ndashBARRIERS DISCONNECTS

                                    53

                                    SERA

                                    NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                    Negative NEB values cost of barrier

                                    Solar Water Heat NZ$ Euros

                                    Solar Design NZ$ Euros

                                    Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                    -23 -12 (079)

                                    -11 -6 (06)

                                    Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                    Source Skumatz Economic Research Associates research

                                    SERA

                                    CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                    NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                    -5-4-3-2-1012345

                                    Op Co

                                    st

                                    Init C

                                    ost

                                    Maint

                                    Perfo

                                    rm

                                    Prod

                                    uctiv

                                    Tena

                                    nt Sa

                                    t

                                    Comf

                                    ort

                                    Appe

                                    ar

                                    Light

                                    Qual

                                    SellL

                                    ease

                                    Devp Ownr AampE

                                    Note AgtE (~4 vs ~2) for these NEBs

                                    Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                    Source Skumatz Economic Research Associates Inc 2002

                                    Owners more positive ndashUnderinvestment train

                                    • Soup to nuts on non-energy benefits NEIs
                                    • TOPICS
                                    • BACKGROUND NEBS
                                    • 20+ YEARS OF NEBS PROGRESShellip
                                    • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                    • KEY APPLICATIONS OF NEBS
                                    • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                    • Neb measurement
                                    • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                    • MEASURING PARTICIPANTNEBs
                                    • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                    • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                    • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                    • PROGRESS TRANSFERABILITY OF NEBS
                                    • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                    • PROGRESS AND GAPS
                                    • PROGRESS AND GAPS
                                    • USING NEB NEI VALUES
                                    • Slide Number 19
                                    • ARE NEBS HIGH VALUE
                                    • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                    • TOP NEBS FOR WEATHERIZATION PROGRAM
                                    • RESULTS FOR ENERGY STARreg APPLIANCES
                                    • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                    • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                    • CampI PROGRAMS ldquoNEBBEDrdquo
                                    • AUDIT FINANCIAL PROGRAM
                                    • CampI NEW CONSTRUCTION
                                    • CampI TECHNICAL ASSISTANCE
                                    • EXPRESSING NEBS VALUEndashCxe
                                    • Slide Number 31
                                    • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                    • EMISSIONS (SOCIETAL NEBS)
                                    • UTILITY NEBS
                                    • ADJUSTED PAYBACKS
                                    • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                    • NEBS IN BC
                                    • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                    • APPLICATION TO BC
                                    • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                    • STATE TREATMENT OF NEBS NEBs
                                    • Slide Number 42
                                    • SELECTED STATE PROGRESS
                                    • SELECTED STATE PROGRESS
                                    • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                    • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                    • DISCOUNT RATE MATTERS
                                    • EUL VARIATIONS- ADOPTED VALUES
                                    • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                    • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                    • TAKEAWAYSON NEBSNEIS
                                    • THANK YOUQuestions
                                    • Slide Number 53
                                    • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                    • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                      Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                      m Product losses20
                                      l Ease of selling leasing building33
                                      n Labor requirements45
                                      h Aesthetics appearance59
                                      j Noise50
                                      b Equipment maintenance660
                                      f Tenant satisfaction60
                                      d Equipment lifetimes718
                                      e Productivity724
                                      k Building safety[2]73
                                      g Comfort933
                                      i Lighting Quality of Light916
                                      c Equipment performance[1]1019
                                      a Operating costs1165
                                      o Doing good for the Environment[3]1132
                                      Other03
                                      [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                      [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                      [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                      m Product losses
                                      l Ease of selling leasing building
                                      n Labor requirements
                                      h Aesthetics appearance
                                      j Noise
                                      b Equipment maintenance
                                      f Tenant satisfaction
                                      d Equipment lifetimes
                                      e Productivity
                                      k Building safety[2]
                                      g Comfort
                                      i Lighting Quality of Light
                                      c Equipment performance[1]
                                      a Operating costs
                                      o Doing good for the Environment[3]
                                      Share of ZALEH NEBS from various measures
                                      Each presented as multiple of the (overall) energy savings
                                      Super Insulation037
                                      Double Glazing026
                                      Solar Design025
                                      Solar Water Heating005
                                      EnergySmart (insul Retrofit)Dunedin (insul)
                                      Other124
                                      Noise24
                                      Budgetfinances25
                                      Environmental65
                                      Health78
                                      Reduced moves86
                                      Ease of sellingrenting130
                                      DampnessMold1316
                                      Comfort-related4933
                                      101101
                                      Operational NEBs44
                                      DesignConstruction NEBs31
                                      Occupant NEBs25
                                      1
                                      OtherOther
                                      NoiseNoise
                                      BudgetfinancesBudgetfinances
                                      EnvironmentalEnvironmental
                                      HealthHealth
                                      Reduced movesReduced moves
                                      Ease of sellingrentingEase of sellingrenting
                                      DampnessMoldDampnessMold
                                      Comfort-relatedComfort-related
                                      Boiler
                                      VendorNPPartic
                                      Perform0280202
                                      Footprint028013017
                                      ControlFeatures031022016
                                      Tenant compl006008014
                                      Noise005006014
                                      Maint004009007
                                      Other004003007
                                      Lifetime-006018006
                                      MF New construction energy star
                                      Operating cost014
                                      Environment012
                                      Maintenance01
                                      Lifetime01
                                      Performance009
                                      Occupant Satisf009
                                      Comfort008
                                      Noise006
                                      Sellingleasing006
                                      Aesthetics005
                                      Light Quality005
                                      Power QualReliab004
                                      Safety002
                                      RIGHTLIGHTS
                                      Environment023
                                      Quality016
                                      Quantity014
                                      PgmAvail01
                                      Flicker008
                                      Maintenance006
                                      Lifetime006
                                      SalesProductivity006
                                      Safety005
                                      Noise005
                                      Control003
                                      Illness0
                                      PerformPerformPerform
                                      FootprintFootprintFootprint
                                      ControlFeaturesControlFeaturesControlFeatures
                                      Tenant complTenant complTenant compl
                                      NoiseNoiseNoise
                                      MaintMaintMaint
                                      OtherOtherOther
                                      LifetimeLifetimeLifetime
                                      Environment
                                      Quality
                                      Quantity
                                      PgmAvail
                                      Flicker
                                      Maintenance
                                      Lifetime
                                      SalesProductivity
                                      Safety
                                      Noise
                                      Control
                                      Illness

                                      Variability

                                      Relevant NEB Categories

                                      Program measure invariant (suitable for ldquoadderrdquo)

                                      middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                                      Program measure dependent

                                      middot Economic ndash societal (depends on measures and local manufacture installation)

                                      middot Health and safety health care illnesses ndash societal and participant (measure)

                                      middot Water wastewater infrastructure and water bill savings ndash societal and participant

                                      middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                      Climate dependent

                                      middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                      Residential Target dependent (low income or MF vs SF)

                                      middot Payment related ndash utility (arrearages etc stronger for low income targets)

                                      middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

                                      middot Participant benefits related to hardship and payments

                                      middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                                      Used for

                                      Major Advantages

                                      Major Disadvantages

                                      Direct measurement (specific studies of changes on-site)

                                      Comrsquol labor productivity etc

                                      Direct precise attributable

                                      Small estimation sample sizes specialized cases poor transferability expensive

                                      Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)

                                      Insurance water health others

                                      Long history easy secondary sources

                                      Credible to reviewers vetted inputs

                                      Not available for all NEBs

                                      Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)

                                      Emissions economics

                                      Third party peer-vetted models available for economics jobs and emissions

                                      Not available for all NEBs

                                      Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)

                                      Wide variety of Participant NEBs

                                      Large sample sizes amp statistical properties

                                      Affordable

                                      Multiple estimates leading to similar ranges

                                      Direct method of measuring some key NEBs

                                      Concerns about surveys as a source of quantitative values amp reliability

                                      Recall from survey respondents

                                      Proper attribution to programs measures

                                      SERA

                                      PROGRESS TRANSFERABILITY OF NEBS

                                      14

                                      Source Skumatz SERA research

                                      Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)

                                      bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                                      Program measure dependent

                                      bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control

                                      education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                      Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                      Residential Target dependent (low income or MF vs SF)

                                      bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill

                                      vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                                      Cost savings consistency if transferable vs risk

                                      SERA

                                      RANGES TYPICAL NEB VALUES FOR WxRETROFI

                                      15

                                      SERA

                                      PROGRESS AND GAPS Consistency Transferability

                                      Some programs types others less studied Transferability research

                                      Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

                                      measures until then Across the board Savings share Regression

                                      SERA

                                      PROGRESS AND GAPS Gas vs Electric

                                      Some researchndashusually program-wide not broken down International research found participant NEBs have

                                      similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

                                      MF Less-commonly-studied complicated by poor response

                                      and complexity of sector (decision-maker some measures in home some central) separate from low income not common

                                      Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

                                      Commercial gaps in some measures

                                      SERA

                                      USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                                      Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                                      18

                                      0

                                      005

                                      01

                                      015

                                      02

                                      025

                                      Environment

                                      Quality

                                      Quantity

                                      PgmAvailFlicker

                                      Mainten

                                      ance

                                      Lifetime

                                      SalesP

                                      roductivity

                                      Safety

                                      Noise

                                      Control

                                      Illness

                                      Total NEBs Multiplier=X

                                      $

                                      Savings value=$X

                                      Chart3

                                      023
                                      016
                                      014
                                      01
                                      008
                                      006
                                      006
                                      006
                                      005
                                      005
                                      003
                                      0

                                      Chart1

                                      Vendor
                                      NP
                                      Partic
                                      028
                                      02
                                      02
                                      028
                                      013
                                      017
                                      031
                                      022
                                      016
                                      006
                                      008
                                      014
                                      005
                                      006
                                      014
                                      004
                                      009
                                      007
                                      004
                                      003
                                      007
                                      -006
                                      018
                                      006

                                      Sheet1

                                      Sheet1

                                      Sheet2

                                      Sheet3

                                      SERA

                                      NEB VALUES

                                      19

                                      SERA

                                      ARE NEBS HIGH VALUE Energy savings are often the minority of

                                      benefits from program measuresndash Donrsquot ignore that

                                      20

                                      0 05 1 15 2 25 3

                                      Energy Sav

                                      NEBs

                                      NEB vs Energy Savings ValueIncluding all NEBs

                                      Source (SkumatzSERA 2010 amp others)

                                      Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                      SERA

                                      RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                      MF Many others

                                      Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                      Variety of end uses

                                      SERA

                                      TOP NEBS FOR WEATHERIZATION PROGRAM

                                      Weatherization program multipliergt1

                                      Source SERA Research

                                      SERA

                                      RESULTS FOR ENERGY STARreg APPLIANCES

                                      0002004006008

                                      01012014016018

                                      Maint

                                      Appear

                                      Perf LifeNoise Satis

                                      Comfort

                                      LiteSafe

                                      ty SellMove

                                      Water

                                      Enviro

                                      Refrig DW CW

                                      Source SERA Research

                                      SERA

                                      INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                      5 increments to 50

                                      Source SERA research

                                      Chart5

                                      EnergySmart (insul Retrofit)
                                      Dunedin (insul)
                                      001
                                      024
                                      002
                                      004
                                      002
                                      005
                                      006
                                      005
                                      007
                                      008
                                      008
                                      006
                                      013
                                      0
                                      013
                                      016
                                      049
                                      033

                                      cx

                                      cx

                                      nz

                                      Sheet3

                                      Sheet3

                                      Super Insulation
                                      Double Glazing
                                      Solar Design
                                      Solar Water Heating
                                      Super Insulation
                                      Double Glazing
                                      Solar Design
                                      Solar Water Heating
                                      EnergySmart (insul Retrofit)
                                      Dunedin (insul)

                                      SERA

                                      ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                      0

                                      002

                                      004

                                      006

                                      008

                                      01

                                      012

                                      Maint

                                      Perf LifeSatis

                                      f

                                      ComfortAesth

                                      NoiseSafe

                                      ty

                                      LightingSelli

                                      ng

                                      Stay in home

                                      Enviro

                                      Sick Days

                                      Calls

                                      ES Homes HP wES Energy savings multiplier=1

                                      Source SERA Research

                                      SERA

                                      CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                      DG CHP HVAC Equipment rebate Other

                                      Building codes incentives by cities

                                      Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                      Variety of end uses

                                      SERA

                                      AUDIT FINANCIAL PROGRAM

                                      0

                                      01

                                      02

                                      03

                                      04

                                      05

                                      06

                                      07

                                      Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                      Source SERA Research

                                      SERA

                                      CampI NEW CONSTRUCTION

                                      -005

                                      0

                                      005

                                      01

                                      015

                                      02

                                      AampE Owner(Pct of Total Partic NEBs)

                                      Source SERA Research

                                      SERA

                                      CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                      0 2 4 6 8 10 12

                                      m Product losses

                                      l Ease of selling leasing building

                                      n Labor requirements

                                      h Aesthetics appearance

                                      j Noise

                                      b Equipment maintenance

                                      f Tenant satisfaction

                                      d Equipment lifetimes

                                      e Productivity

                                      k Building safety[2]

                                      g Comfort

                                      i Lighting Quality of Light

                                      c Equipment performance[1]

                                      a Operating costs

                                      o Doing good for the Environment[3]

                                      Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                      vcv

                                      Source SERA Research

                                      Chart1

                                      CampI Technical Assistance NEBs
                                      002
                                      003
                                      004
                                      005
                                      005
                                      006
                                      006
                                      007
                                      007
                                      007
                                      009
                                      009
                                      01
                                      011
                                      011

                                      Sheet1

                                      Sheet1

                                      CampI Technical Assistance Program NEBs (Participant)

                                      Sheet2

                                      Sheet3

                                      SERA

                                      EXPRESSING NEBS VALUEndashCxe

                                      Strong value from RetroCxSource SERA Research

                                      SERA

                                      SOCIETAL NEB EXAMPLES

                                      31

                                      SERA

                                      JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                      32

                                      0

                                      02

                                      04

                                      06

                                      08

                                      1

                                      12

                                      HPWxRetrof Appliance

                                      CA

                                      WI

                                      Natl

                                      (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                      Jobs Economic

                                      SERA

                                      EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                      33

                                      SERA

                                      UTILITY NEBSUtility NEBs for Template Program

                                      Debt WriteOff (util)13

                                      Rate Subsidy(util)61

                                      HealthSafety(util)0

                                      Colln Costs (util)0

                                      Gas Calls (util)0

                                      Calls to CSRs(util)2

                                      TampD (util)16

                                      Arrears (util)0

                                      Reconnects (util)0

                                      Notices (util)7

                                      Shutoffs (util)1

                                      Rate subsidy TampD

                                      Payment-related

                                      Source Skumatz Economic Research Associates research

                                      Example Low Income Weatherization

                                      SERA

                                      ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                      be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                      Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                      Value of 100 HALVES Payback doubles ROI

                                      Source Skumatz Economics (SERA)

                                      35

                                      ( for NEBs Multiplier of 125)

                                      SERA

                                      NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                      36

                                      hellip2001 ON

                                      SERA

                                      37

                                      NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                      For true representation of B amp C elements of NEBs address missing factor bias

                                      Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                      BC applications were considered early on then pulled back awaiting more quantitative evidence

                                      SERA

                                      CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                      38

                                      Test Benefits Costs States Using Traditionally

                                      Improved treat-ment with NEBs

                                      Utility Cost (or Program Administrator Test) (UCT or PAC)

                                      bull Avoided supply costs for transmission distribution and generation (TDampG)

                                      bull Avoided gas and water supply costs

                                      bull Program administration

                                      bull Participant incentives

                                      bull Increased supply cost

                                      CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                      Use cost only paid by the utility

                                      Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                      Same as above plus bull increased revenue

                                      Same as above plus bull Decreased revenue

                                      AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                      Participant cost bull Utility bill reductions bull Participant incentives

                                      bull Participant direct costs

                                      AR CA FL HI IA IN MI MN NY VA

                                      Participant NEBs

                                      Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                      Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                      SERA

                                      APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                      Expansion Concern with risk calculation accuracy

                                      39

                                      Source Skumatz SERA research

                                      SERA

                                      40

                                      METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                      Maximize DSM opportunities amp feedback Accuracy tailoring

                                      Minimize Regulatory amp Implementer Risk

                                      Minimize Evaluation Cost

                                      Adder

                                      Readily MeasurableHybrid

                                      All NEBsSource SERA Research

                                      SERA

                                      41

                                      STATE TREATMENT OF NEBS NEBsRegulatory

                                      Screening Application Utilities regions

                                      Program Marketing Fairly widespread use in utilities states across the country

                                      MO

                                      RE AGGRESSIVE ==gt

                                      Test Pgm Screen - adder

                                      IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                      Test Pgm Screen - readily measured

                                      MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                      Test Hybrid (potential adder amp measured)

                                      CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                      Test Pgm screen - Broad

                                      With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                      Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                      SERA

                                      42

                                      Key

                                      States with NEBs in CE tests for At least one type of program

                                      copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                      STATES WITH NEBS IN CE TESTS

                                      NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                      groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                      SERA

                                      SELECTED STATE PROGRESS CA

                                      2001 ndash LIPPT model NY

                                      Mid-2000s measurement amp scenarios not included CO

                                      Adders (10 electric 25 LI 5 gas) VT

                                      15 adder helped by previous research DC

                                      10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                      43

                                      Source Skumatz SERA research

                                      SERA

                                      SELECTED STATE PROGRESS Dominos ongoing

                                      Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                      Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                      Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                      Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                      44

                                      Source Skumatz SERA research

                                      SERA

                                      APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                      Expansion Concern with risk calculation accuracy

                                      45

                                      Source Skumatz SERA research

                                      SERA

                                      46

                                      NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                      [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                      SourceSkumatz SERA Independent research

                                      NTG ndash accuracy measurement incomplete

                                      Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                      NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                      EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                      Cost Complicated expensive local changes

                                      RELATIVE Risk from NEBs

                                      Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                      Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                      NEBs as accurate as many inputs (all could improve)

                                      SERA

                                      DISCOUNT RATE MATTERS

                                      47

                                      Source Skumatz SERA Research

                                      0

                                      20

                                      40

                                      60

                                      80

                                      100

                                      120

                                      1 2 3 4 5 6 7 8 9 10

                                      PV Ratios vs 1 Discount Rate

                                      10 yr PV 20 yr PV

                                      Discount rate - percent

                                      WACCSocialLTT

                                      Source Skumatz Electricity Journal IEPEC paper independent research

                                      SERA

                                      EUL VARIATIONS-ADOPTED VALUES

                                      Source Skumatz Economic Research Associates SERA independent research

                                      0

                                      50

                                      100

                                      150

                                      200

                                      250

                                      300

                                      350

                                      400

                                      450

                                      0 10 20 30 40 50 60 70

                                      EUL Variation by Measure

                                      67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                      Poorly supportedOldTechnology change

                                      SERA

                                      KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                      Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                      measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                      change the BC conclusion

                                      On the cusphellip

                                      If NO Yoursquore doneAnd bias addressedsufficiently

                                      IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                      Source SERA all rights reserved

                                      SERArsquos lsquoNEB-ItrdquoModel ndash

                                      SERA

                                      IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                      Utility Soc Part Conserv Recrsquom

                                      Rationale

                                      Base Percent X X X Program-invariant

                                      Low Income X X X X Multiple sources

                                      Weatherization X X X Substantial Participantimpacts

                                      Measure Program-specific

                                      X Varies by measure sector

                                      Other Recomrsquos Local Research

                                      50

                                      Source Skumatz SERA

                                      Developing values for multiple states amp utilities

                                      SERA

                                      TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                      Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                      Other sources of bias to research (EUL NTG etc) bring high variation

                                      5) States are incorporating NEBs dominos input in deliberations in multiple states

                                      6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                      THANK YOU

                                      Questions

                                      Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                      SERA

                                      POWER OF NEBS ndashBARRIERS DISCONNECTS

                                      53

                                      SERA

                                      NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                      Negative NEB values cost of barrier

                                      Solar Water Heat NZ$ Euros

                                      Solar Design NZ$ Euros

                                      Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                      -23 -12 (079)

                                      -11 -6 (06)

                                      Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                      Source Skumatz Economic Research Associates research

                                      SERA

                                      CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                      NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                      -5-4-3-2-1012345

                                      Op Co

                                      st

                                      Init C

                                      ost

                                      Maint

                                      Perfo

                                      rm

                                      Prod

                                      uctiv

                                      Tena

                                      nt Sa

                                      t

                                      Comf

                                      ort

                                      Appe

                                      ar

                                      Light

                                      Qual

                                      SellL

                                      ease

                                      Devp Ownr AampE

                                      Note AgtE (~4 vs ~2) for these NEBs

                                      Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                      Source Skumatz Economic Research Associates Inc 2002

                                      Owners more positive ndashUnderinvestment train

                                      • Soup to nuts on non-energy benefits NEIs
                                      • TOPICS
                                      • BACKGROUND NEBS
                                      • 20+ YEARS OF NEBS PROGRESShellip
                                      • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                      • KEY APPLICATIONS OF NEBS
                                      • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                      • Neb measurement
                                      • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                      • MEASURING PARTICIPANTNEBs
                                      • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                      • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                      • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                      • PROGRESS TRANSFERABILITY OF NEBS
                                      • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                      • PROGRESS AND GAPS
                                      • PROGRESS AND GAPS
                                      • USING NEB NEI VALUES
                                      • Slide Number 19
                                      • ARE NEBS HIGH VALUE
                                      • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                      • TOP NEBS FOR WEATHERIZATION PROGRAM
                                      • RESULTS FOR ENERGY STARreg APPLIANCES
                                      • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                      • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                      • CampI PROGRAMS ldquoNEBBEDrdquo
                                      • AUDIT FINANCIAL PROGRAM
                                      • CampI NEW CONSTRUCTION
                                      • CampI TECHNICAL ASSISTANCE
                                      • EXPRESSING NEBS VALUEndashCxe
                                      • Slide Number 31
                                      • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                      • EMISSIONS (SOCIETAL NEBS)
                                      • UTILITY NEBS
                                      • ADJUSTED PAYBACKS
                                      • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                      • NEBS IN BC
                                      • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                      • APPLICATION TO BC
                                      • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                      • STATE TREATMENT OF NEBS NEBs
                                      • Slide Number 42
                                      • SELECTED STATE PROGRESS
                                      • SELECTED STATE PROGRESS
                                      • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                      • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                      • DISCOUNT RATE MATTERS
                                      • EUL VARIATIONS- ADOPTED VALUES
                                      • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                      • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                      • TAKEAWAYSON NEBSNEIS
                                      • THANK YOUQuestions
                                      • Slide Number 53
                                      • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                      • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                        Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                        m Product losses20
                                        l Ease of selling leasing building33
                                        n Labor requirements45
                                        h Aesthetics appearance59
                                        j Noise50
                                        b Equipment maintenance660
                                        f Tenant satisfaction60
                                        d Equipment lifetimes718
                                        e Productivity724
                                        k Building safety[2]73
                                        g Comfort933
                                        i Lighting Quality of Light916
                                        c Equipment performance[1]1019
                                        a Operating costs1165
                                        o Doing good for the Environment[3]1132
                                        Other03
                                        [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                        [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                        [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                        m Product losses
                                        l Ease of selling leasing building
                                        n Labor requirements
                                        h Aesthetics appearance
                                        j Noise
                                        b Equipment maintenance
                                        f Tenant satisfaction
                                        d Equipment lifetimes
                                        e Productivity
                                        k Building safety[2]
                                        g Comfort
                                        i Lighting Quality of Light
                                        c Equipment performance[1]
                                        a Operating costs
                                        o Doing good for the Environment[3]
                                        Share of ZALEH NEBS from various measures
                                        Each presented as multiple of the (overall) energy savings
                                        Super Insulation037
                                        Double Glazing026
                                        Solar Design025
                                        Solar Water Heating005
                                        EnergySmart (insul Retrofit)Dunedin (insul)
                                        Other124
                                        Noise24
                                        Budgetfinances25
                                        Environmental65
                                        Health78
                                        Reduced moves86
                                        Ease of sellingrenting130
                                        DampnessMold1316
                                        Comfort-related4933
                                        101101
                                        Operational NEBs44
                                        DesignConstruction NEBs31
                                        Occupant NEBs25
                                        1
                                        OtherOther
                                        NoiseNoise
                                        BudgetfinancesBudgetfinances
                                        EnvironmentalEnvironmental
                                        HealthHealth
                                        Reduced movesReduced moves
                                        Ease of sellingrentingEase of sellingrenting
                                        DampnessMoldDampnessMold
                                        Comfort-relatedComfort-related
                                        Boiler
                                        VendorNPPartic
                                        Perform0280202
                                        Footprint028013017
                                        ControlFeatures031022016
                                        Tenant compl006008014
                                        Noise005006014
                                        Maint004009007
                                        Other004003007
                                        Lifetime-006018006
                                        MF New construction energy star
                                        Operating cost014
                                        Environment012
                                        Maintenance01
                                        Lifetime01
                                        Performance009
                                        Occupant Satisf009
                                        Comfort008
                                        Noise006
                                        Sellingleasing006
                                        Aesthetics005
                                        Light Quality005
                                        Power QualReliab004
                                        Safety002
                                        RIGHTLIGHTS
                                        Environment023
                                        Quality016
                                        Quantity014
                                        PgmAvail01
                                        Flicker008
                                        Maintenance006
                                        Lifetime006
                                        SalesProductivity006
                                        Safety005
                                        Noise005
                                        Control003
                                        Illness0
                                        PerformPerformPerform
                                        FootprintFootprintFootprint
                                        ControlFeaturesControlFeaturesControlFeatures
                                        Tenant complTenant complTenant compl
                                        NoiseNoiseNoise
                                        MaintMaintMaint
                                        OtherOtherOther
                                        LifetimeLifetimeLifetime
                                        Environment
                                        Quality
                                        Quantity
                                        PgmAvail
                                        Flicker
                                        Maintenance
                                        Lifetime
                                        SalesProductivity
                                        Safety
                                        Noise
                                        Control
                                        Illness

                                        Variability

                                        Relevant NEB Categories

                                        Program measure invariant (suitable for ldquoadderrdquo)

                                        middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)

                                        Program measure dependent

                                        middot Economic ndash societal (depends on measures and local manufacture installation)

                                        middot Health and safety health care illnesses ndash societal and participant (measure)

                                        middot Water wastewater infrastructure and water bill savings ndash societal and participant

                                        middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                        Climate dependent

                                        middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country

                                        Residential Target dependent (low income or MF vs SF)

                                        middot Payment related ndash utility (arrearages etc stronger for low income targets)

                                        middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)

                                        middot Participant benefits related to hardship and payments

                                        middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF

                                        SERA

                                        RANGES TYPICAL NEB VALUES FOR WxRETROFI

                                        15

                                        SERA

                                        PROGRESS AND GAPS Consistency Transferability

                                        Some programs types others less studied Transferability research

                                        Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

                                        measures until then Across the board Savings share Regression

                                        SERA

                                        PROGRESS AND GAPS Gas vs Electric

                                        Some researchndashusually program-wide not broken down International research found participant NEBs have

                                        similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

                                        MF Less-commonly-studied complicated by poor response

                                        and complexity of sector (decision-maker some measures in home some central) separate from low income not common

                                        Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

                                        Commercial gaps in some measures

                                        SERA

                                        USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                                        Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                                        18

                                        0

                                        005

                                        01

                                        015

                                        02

                                        025

                                        Environment

                                        Quality

                                        Quantity

                                        PgmAvailFlicker

                                        Mainten

                                        ance

                                        Lifetime

                                        SalesP

                                        roductivity

                                        Safety

                                        Noise

                                        Control

                                        Illness

                                        Total NEBs Multiplier=X

                                        $

                                        Savings value=$X

                                        Chart3

                                        023
                                        016
                                        014
                                        01
                                        008
                                        006
                                        006
                                        006
                                        005
                                        005
                                        003
                                        0

                                        Chart1

                                        Vendor
                                        NP
                                        Partic
                                        028
                                        02
                                        02
                                        028
                                        013
                                        017
                                        031
                                        022
                                        016
                                        006
                                        008
                                        014
                                        005
                                        006
                                        014
                                        004
                                        009
                                        007
                                        004
                                        003
                                        007
                                        -006
                                        018
                                        006

                                        Sheet1

                                        Sheet1

                                        Sheet2

                                        Sheet3

                                        SERA

                                        NEB VALUES

                                        19

                                        SERA

                                        ARE NEBS HIGH VALUE Energy savings are often the minority of

                                        benefits from program measuresndash Donrsquot ignore that

                                        20

                                        0 05 1 15 2 25 3

                                        Energy Sav

                                        NEBs

                                        NEB vs Energy Savings ValueIncluding all NEBs

                                        Source (SkumatzSERA 2010 amp others)

                                        Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                        SERA

                                        RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                        MF Many others

                                        Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                        Variety of end uses

                                        SERA

                                        TOP NEBS FOR WEATHERIZATION PROGRAM

                                        Weatherization program multipliergt1

                                        Source SERA Research

                                        SERA

                                        RESULTS FOR ENERGY STARreg APPLIANCES

                                        0002004006008

                                        01012014016018

                                        Maint

                                        Appear

                                        Perf LifeNoise Satis

                                        Comfort

                                        LiteSafe

                                        ty SellMove

                                        Water

                                        Enviro

                                        Refrig DW CW

                                        Source SERA Research

                                        SERA

                                        INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                        5 increments to 50

                                        Source SERA research

                                        Chart5

                                        EnergySmart (insul Retrofit)
                                        Dunedin (insul)
                                        001
                                        024
                                        002
                                        004
                                        002
                                        005
                                        006
                                        005
                                        007
                                        008
                                        008
                                        006
                                        013
                                        0
                                        013
                                        016
                                        049
                                        033

                                        cx

                                        cx

                                        nz

                                        Sheet3

                                        Sheet3

                                        Super Insulation
                                        Double Glazing
                                        Solar Design
                                        Solar Water Heating
                                        Super Insulation
                                        Double Glazing
                                        Solar Design
                                        Solar Water Heating
                                        EnergySmart (insul Retrofit)
                                        Dunedin (insul)

                                        SERA

                                        ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                        0

                                        002

                                        004

                                        006

                                        008

                                        01

                                        012

                                        Maint

                                        Perf LifeSatis

                                        f

                                        ComfortAesth

                                        NoiseSafe

                                        ty

                                        LightingSelli

                                        ng

                                        Stay in home

                                        Enviro

                                        Sick Days

                                        Calls

                                        ES Homes HP wES Energy savings multiplier=1

                                        Source SERA Research

                                        SERA

                                        CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                        DG CHP HVAC Equipment rebate Other

                                        Building codes incentives by cities

                                        Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                        Variety of end uses

                                        SERA

                                        AUDIT FINANCIAL PROGRAM

                                        0

                                        01

                                        02

                                        03

                                        04

                                        05

                                        06

                                        07

                                        Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                        Source SERA Research

                                        SERA

                                        CampI NEW CONSTRUCTION

                                        -005

                                        0

                                        005

                                        01

                                        015

                                        02

                                        AampE Owner(Pct of Total Partic NEBs)

                                        Source SERA Research

                                        SERA

                                        CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                        0 2 4 6 8 10 12

                                        m Product losses

                                        l Ease of selling leasing building

                                        n Labor requirements

                                        h Aesthetics appearance

                                        j Noise

                                        b Equipment maintenance

                                        f Tenant satisfaction

                                        d Equipment lifetimes

                                        e Productivity

                                        k Building safety[2]

                                        g Comfort

                                        i Lighting Quality of Light

                                        c Equipment performance[1]

                                        a Operating costs

                                        o Doing good for the Environment[3]

                                        Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                        vcv

                                        Source SERA Research

                                        Chart1

                                        CampI Technical Assistance NEBs
                                        002
                                        003
                                        004
                                        005
                                        005
                                        006
                                        006
                                        007
                                        007
                                        007
                                        009
                                        009
                                        01
                                        011
                                        011

                                        Sheet1

                                        Sheet1

                                        CampI Technical Assistance Program NEBs (Participant)

                                        Sheet2

                                        Sheet3

                                        SERA

                                        EXPRESSING NEBS VALUEndashCxe

                                        Strong value from RetroCxSource SERA Research

                                        SERA

                                        SOCIETAL NEB EXAMPLES

                                        31

                                        SERA

                                        JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                        32

                                        0

                                        02

                                        04

                                        06

                                        08

                                        1

                                        12

                                        HPWxRetrof Appliance

                                        CA

                                        WI

                                        Natl

                                        (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                        Jobs Economic

                                        SERA

                                        EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                        33

                                        SERA

                                        UTILITY NEBSUtility NEBs for Template Program

                                        Debt WriteOff (util)13

                                        Rate Subsidy(util)61

                                        HealthSafety(util)0

                                        Colln Costs (util)0

                                        Gas Calls (util)0

                                        Calls to CSRs(util)2

                                        TampD (util)16

                                        Arrears (util)0

                                        Reconnects (util)0

                                        Notices (util)7

                                        Shutoffs (util)1

                                        Rate subsidy TampD

                                        Payment-related

                                        Source Skumatz Economic Research Associates research

                                        Example Low Income Weatherization

                                        SERA

                                        ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                        be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                        Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                        Value of 100 HALVES Payback doubles ROI

                                        Source Skumatz Economics (SERA)

                                        35

                                        ( for NEBs Multiplier of 125)

                                        SERA

                                        NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                        36

                                        hellip2001 ON

                                        SERA

                                        37

                                        NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                        For true representation of B amp C elements of NEBs address missing factor bias

                                        Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                        BC applications were considered early on then pulled back awaiting more quantitative evidence

                                        SERA

                                        CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                        38

                                        Test Benefits Costs States Using Traditionally

                                        Improved treat-ment with NEBs

                                        Utility Cost (or Program Administrator Test) (UCT or PAC)

                                        bull Avoided supply costs for transmission distribution and generation (TDampG)

                                        bull Avoided gas and water supply costs

                                        bull Program administration

                                        bull Participant incentives

                                        bull Increased supply cost

                                        CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                        Use cost only paid by the utility

                                        Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                        Same as above plus bull increased revenue

                                        Same as above plus bull Decreased revenue

                                        AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                        Participant cost bull Utility bill reductions bull Participant incentives

                                        bull Participant direct costs

                                        AR CA FL HI IA IN MI MN NY VA

                                        Participant NEBs

                                        Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                        Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                        SERA

                                        APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                        Expansion Concern with risk calculation accuracy

                                        39

                                        Source Skumatz SERA research

                                        SERA

                                        40

                                        METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                        Maximize DSM opportunities amp feedback Accuracy tailoring

                                        Minimize Regulatory amp Implementer Risk

                                        Minimize Evaluation Cost

                                        Adder

                                        Readily MeasurableHybrid

                                        All NEBsSource SERA Research

                                        SERA

                                        41

                                        STATE TREATMENT OF NEBS NEBsRegulatory

                                        Screening Application Utilities regions

                                        Program Marketing Fairly widespread use in utilities states across the country

                                        MO

                                        RE AGGRESSIVE ==gt

                                        Test Pgm Screen - adder

                                        IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                        Test Pgm Screen - readily measured

                                        MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                        Test Hybrid (potential adder amp measured)

                                        CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                        Test Pgm screen - Broad

                                        With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                        Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                        SERA

                                        42

                                        Key

                                        States with NEBs in CE tests for At least one type of program

                                        copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                        STATES WITH NEBS IN CE TESTS

                                        NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                        groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                        SERA

                                        SELECTED STATE PROGRESS CA

                                        2001 ndash LIPPT model NY

                                        Mid-2000s measurement amp scenarios not included CO

                                        Adders (10 electric 25 LI 5 gas) VT

                                        15 adder helped by previous research DC

                                        10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                        43

                                        Source Skumatz SERA research

                                        SERA

                                        SELECTED STATE PROGRESS Dominos ongoing

                                        Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                        Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                        Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                        Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                        44

                                        Source Skumatz SERA research

                                        SERA

                                        APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                        Expansion Concern with risk calculation accuracy

                                        45

                                        Source Skumatz SERA research

                                        SERA

                                        46

                                        NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                        [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                        SourceSkumatz SERA Independent research

                                        NTG ndash accuracy measurement incomplete

                                        Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                        NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                        EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                        Cost Complicated expensive local changes

                                        RELATIVE Risk from NEBs

                                        Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                        Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                        NEBs as accurate as many inputs (all could improve)

                                        SERA

                                        DISCOUNT RATE MATTERS

                                        47

                                        Source Skumatz SERA Research

                                        0

                                        20

                                        40

                                        60

                                        80

                                        100

                                        120

                                        1 2 3 4 5 6 7 8 9 10

                                        PV Ratios vs 1 Discount Rate

                                        10 yr PV 20 yr PV

                                        Discount rate - percent

                                        WACCSocialLTT

                                        Source Skumatz Electricity Journal IEPEC paper independent research

                                        SERA

                                        EUL VARIATIONS-ADOPTED VALUES

                                        Source Skumatz Economic Research Associates SERA independent research

                                        0

                                        50

                                        100

                                        150

                                        200

                                        250

                                        300

                                        350

                                        400

                                        450

                                        0 10 20 30 40 50 60 70

                                        EUL Variation by Measure

                                        67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                        Poorly supportedOldTechnology change

                                        SERA

                                        KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                        Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                        measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                        change the BC conclusion

                                        On the cusphellip

                                        If NO Yoursquore doneAnd bias addressedsufficiently

                                        IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                        Source SERA all rights reserved

                                        SERArsquos lsquoNEB-ItrdquoModel ndash

                                        SERA

                                        IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                        Utility Soc Part Conserv Recrsquom

                                        Rationale

                                        Base Percent X X X Program-invariant

                                        Low Income X X X X Multiple sources

                                        Weatherization X X X Substantial Participantimpacts

                                        Measure Program-specific

                                        X Varies by measure sector

                                        Other Recomrsquos Local Research

                                        50

                                        Source Skumatz SERA

                                        Developing values for multiple states amp utilities

                                        SERA

                                        TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                        Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                        Other sources of bias to research (EUL NTG etc) bring high variation

                                        5) States are incorporating NEBs dominos input in deliberations in multiple states

                                        6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                        THANK YOU

                                        Questions

                                        Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                        SERA

                                        POWER OF NEBS ndashBARRIERS DISCONNECTS

                                        53

                                        SERA

                                        NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                        Negative NEB values cost of barrier

                                        Solar Water Heat NZ$ Euros

                                        Solar Design NZ$ Euros

                                        Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                        -23 -12 (079)

                                        -11 -6 (06)

                                        Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                        Source Skumatz Economic Research Associates research

                                        SERA

                                        CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                        NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                        -5-4-3-2-1012345

                                        Op Co

                                        st

                                        Init C

                                        ost

                                        Maint

                                        Perfo

                                        rm

                                        Prod

                                        uctiv

                                        Tena

                                        nt Sa

                                        t

                                        Comf

                                        ort

                                        Appe

                                        ar

                                        Light

                                        Qual

                                        SellL

                                        ease

                                        Devp Ownr AampE

                                        Note AgtE (~4 vs ~2) for these NEBs

                                        Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                        Source Skumatz Economic Research Associates Inc 2002

                                        Owners more positive ndashUnderinvestment train

                                        • Soup to nuts on non-energy benefits NEIs
                                        • TOPICS
                                        • BACKGROUND NEBS
                                        • 20+ YEARS OF NEBS PROGRESShellip
                                        • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                        • KEY APPLICATIONS OF NEBS
                                        • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                        • Neb measurement
                                        • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                        • MEASURING PARTICIPANTNEBs
                                        • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                        • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                        • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                        • PROGRESS TRANSFERABILITY OF NEBS
                                        • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                        • PROGRESS AND GAPS
                                        • PROGRESS AND GAPS
                                        • USING NEB NEI VALUES
                                        • Slide Number 19
                                        • ARE NEBS HIGH VALUE
                                        • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                        • TOP NEBS FOR WEATHERIZATION PROGRAM
                                        • RESULTS FOR ENERGY STARreg APPLIANCES
                                        • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                        • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                        • CampI PROGRAMS ldquoNEBBEDrdquo
                                        • AUDIT FINANCIAL PROGRAM
                                        • CampI NEW CONSTRUCTION
                                        • CampI TECHNICAL ASSISTANCE
                                        • EXPRESSING NEBS VALUEndashCxe
                                        • Slide Number 31
                                        • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                        • EMISSIONS (SOCIETAL NEBS)
                                        • UTILITY NEBS
                                        • ADJUSTED PAYBACKS
                                        • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                        • NEBS IN BC
                                        • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                        • APPLICATION TO BC
                                        • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                        • STATE TREATMENT OF NEBS NEBs
                                        • Slide Number 42
                                        • SELECTED STATE PROGRESS
                                        • SELECTED STATE PROGRESS
                                        • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                        • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                        • DISCOUNT RATE MATTERS
                                        • EUL VARIATIONS- ADOPTED VALUES
                                        • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                        • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                        • TAKEAWAYSON NEBSNEIS
                                        • THANK YOUQuestions
                                        • Slide Number 53
                                        • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                        • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                          Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                          m Product losses20
                                          l Ease of selling leasing building33
                                          n Labor requirements45
                                          h Aesthetics appearance59
                                          j Noise50
                                          b Equipment maintenance660
                                          f Tenant satisfaction60
                                          d Equipment lifetimes718
                                          e Productivity724
                                          k Building safety[2]73
                                          g Comfort933
                                          i Lighting Quality of Light916
                                          c Equipment performance[1]1019
                                          a Operating costs1165
                                          o Doing good for the Environment[3]1132
                                          Other03
                                          [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                          [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                          [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                          m Product losses
                                          l Ease of selling leasing building
                                          n Labor requirements
                                          h Aesthetics appearance
                                          j Noise
                                          b Equipment maintenance
                                          f Tenant satisfaction
                                          d Equipment lifetimes
                                          e Productivity
                                          k Building safety[2]
                                          g Comfort
                                          i Lighting Quality of Light
                                          c Equipment performance[1]
                                          a Operating costs
                                          o Doing good for the Environment[3]
                                          Share of ZALEH NEBS from various measures
                                          Each presented as multiple of the (overall) energy savings
                                          Super Insulation037
                                          Double Glazing026
                                          Solar Design025
                                          Solar Water Heating005
                                          EnergySmart (insul Retrofit)Dunedin (insul)
                                          Other124
                                          Noise24
                                          Budgetfinances25
                                          Environmental65
                                          Health78
                                          Reduced moves86
                                          Ease of sellingrenting130
                                          DampnessMold1316
                                          Comfort-related4933
                                          101101
                                          Operational NEBs44
                                          DesignConstruction NEBs31
                                          Occupant NEBs25
                                          1
                                          OtherOther
                                          NoiseNoise
                                          BudgetfinancesBudgetfinances
                                          EnvironmentalEnvironmental
                                          HealthHealth
                                          Reduced movesReduced moves
                                          Ease of sellingrentingEase of sellingrenting
                                          DampnessMoldDampnessMold
                                          Comfort-relatedComfort-related
                                          Boiler
                                          VendorNPPartic
                                          Perform0280202
                                          Footprint028013017
                                          ControlFeatures031022016
                                          Tenant compl006008014
                                          Noise005006014
                                          Maint004009007
                                          Other004003007
                                          Lifetime-006018006
                                          MF New construction energy star
                                          Operating cost014
                                          Environment012
                                          Maintenance01
                                          Lifetime01
                                          Performance009
                                          Occupant Satisf009
                                          Comfort008
                                          Noise006
                                          Sellingleasing006
                                          Aesthetics005
                                          Light Quality005
                                          Power QualReliab004
                                          Safety002
                                          RIGHTLIGHTS
                                          Environment023
                                          Quality016
                                          Quantity014
                                          PgmAvail01
                                          Flicker008
                                          Maintenance006
                                          Lifetime006
                                          SalesProductivity006
                                          Safety005
                                          Noise005
                                          Control003
                                          Illness0
                                          PerformPerformPerform
                                          FootprintFootprintFootprint
                                          ControlFeaturesControlFeaturesControlFeatures
                                          Tenant complTenant complTenant compl
                                          NoiseNoiseNoise
                                          MaintMaintMaint
                                          OtherOtherOther
                                          LifetimeLifetimeLifetime
                                          Environment
                                          Quality
                                          Quantity
                                          PgmAvail
                                          Flicker
                                          Maintenance
                                          Lifetime
                                          SalesProductivity
                                          Safety
                                          Noise
                                          Control
                                          Illness

                                          SERA

                                          PROGRESS AND GAPS Consistency Transferability

                                          Some programs types others less studied Transferability research

                                          Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for

                                          measures until then Across the board Savings share Regression

                                          SERA

                                          PROGRESS AND GAPS Gas vs Electric

                                          Some researchndashusually program-wide not broken down International research found participant NEBs have

                                          similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

                                          MF Less-commonly-studied complicated by poor response

                                          and complexity of sector (decision-maker some measures in home some central) separate from low income not common

                                          Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

                                          Commercial gaps in some measures

                                          SERA

                                          USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                                          Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                                          18

                                          0

                                          005

                                          01

                                          015

                                          02

                                          025

                                          Environment

                                          Quality

                                          Quantity

                                          PgmAvailFlicker

                                          Mainten

                                          ance

                                          Lifetime

                                          SalesP

                                          roductivity

                                          Safety

                                          Noise

                                          Control

                                          Illness

                                          Total NEBs Multiplier=X

                                          $

                                          Savings value=$X

                                          Chart3

                                          023
                                          016
                                          014
                                          01
                                          008
                                          006
                                          006
                                          006
                                          005
                                          005
                                          003
                                          0

                                          Chart1

                                          Vendor
                                          NP
                                          Partic
                                          028
                                          02
                                          02
                                          028
                                          013
                                          017
                                          031
                                          022
                                          016
                                          006
                                          008
                                          014
                                          005
                                          006
                                          014
                                          004
                                          009
                                          007
                                          004
                                          003
                                          007
                                          -006
                                          018
                                          006

                                          Sheet1

                                          Sheet1

                                          Sheet2

                                          Sheet3

                                          SERA

                                          NEB VALUES

                                          19

                                          SERA

                                          ARE NEBS HIGH VALUE Energy savings are often the minority of

                                          benefits from program measuresndash Donrsquot ignore that

                                          20

                                          0 05 1 15 2 25 3

                                          Energy Sav

                                          NEBs

                                          NEB vs Energy Savings ValueIncluding all NEBs

                                          Source (SkumatzSERA 2010 amp others)

                                          Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                          SERA

                                          RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                          MF Many others

                                          Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                          Variety of end uses

                                          SERA

                                          TOP NEBS FOR WEATHERIZATION PROGRAM

                                          Weatherization program multipliergt1

                                          Source SERA Research

                                          SERA

                                          RESULTS FOR ENERGY STARreg APPLIANCES

                                          0002004006008

                                          01012014016018

                                          Maint

                                          Appear

                                          Perf LifeNoise Satis

                                          Comfort

                                          LiteSafe

                                          ty SellMove

                                          Water

                                          Enviro

                                          Refrig DW CW

                                          Source SERA Research

                                          SERA

                                          INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                          5 increments to 50

                                          Source SERA research

                                          Chart5

                                          EnergySmart (insul Retrofit)
                                          Dunedin (insul)
                                          001
                                          024
                                          002
                                          004
                                          002
                                          005
                                          006
                                          005
                                          007
                                          008
                                          008
                                          006
                                          013
                                          0
                                          013
                                          016
                                          049
                                          033

                                          cx

                                          cx

                                          nz

                                          Sheet3

                                          Sheet3

                                          Super Insulation
                                          Double Glazing
                                          Solar Design
                                          Solar Water Heating
                                          Super Insulation
                                          Double Glazing
                                          Solar Design
                                          Solar Water Heating
                                          EnergySmart (insul Retrofit)
                                          Dunedin (insul)

                                          SERA

                                          ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                          0

                                          002

                                          004

                                          006

                                          008

                                          01

                                          012

                                          Maint

                                          Perf LifeSatis

                                          f

                                          ComfortAesth

                                          NoiseSafe

                                          ty

                                          LightingSelli

                                          ng

                                          Stay in home

                                          Enviro

                                          Sick Days

                                          Calls

                                          ES Homes HP wES Energy savings multiplier=1

                                          Source SERA Research

                                          SERA

                                          CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                          DG CHP HVAC Equipment rebate Other

                                          Building codes incentives by cities

                                          Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                          Variety of end uses

                                          SERA

                                          AUDIT FINANCIAL PROGRAM

                                          0

                                          01

                                          02

                                          03

                                          04

                                          05

                                          06

                                          07

                                          Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                          Source SERA Research

                                          SERA

                                          CampI NEW CONSTRUCTION

                                          -005

                                          0

                                          005

                                          01

                                          015

                                          02

                                          AampE Owner(Pct of Total Partic NEBs)

                                          Source SERA Research

                                          SERA

                                          CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                          0 2 4 6 8 10 12

                                          m Product losses

                                          l Ease of selling leasing building

                                          n Labor requirements

                                          h Aesthetics appearance

                                          j Noise

                                          b Equipment maintenance

                                          f Tenant satisfaction

                                          d Equipment lifetimes

                                          e Productivity

                                          k Building safety[2]

                                          g Comfort

                                          i Lighting Quality of Light

                                          c Equipment performance[1]

                                          a Operating costs

                                          o Doing good for the Environment[3]

                                          Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                          vcv

                                          Source SERA Research

                                          Chart1

                                          CampI Technical Assistance NEBs
                                          002
                                          003
                                          004
                                          005
                                          005
                                          006
                                          006
                                          007
                                          007
                                          007
                                          009
                                          009
                                          01
                                          011
                                          011

                                          Sheet1

                                          Sheet1

                                          CampI Technical Assistance Program NEBs (Participant)

                                          Sheet2

                                          Sheet3

                                          SERA

                                          EXPRESSING NEBS VALUEndashCxe

                                          Strong value from RetroCxSource SERA Research

                                          SERA

                                          SOCIETAL NEB EXAMPLES

                                          31

                                          SERA

                                          JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                          32

                                          0

                                          02

                                          04

                                          06

                                          08

                                          1

                                          12

                                          HPWxRetrof Appliance

                                          CA

                                          WI

                                          Natl

                                          (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                          Jobs Economic

                                          SERA

                                          EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                          33

                                          SERA

                                          UTILITY NEBSUtility NEBs for Template Program

                                          Debt WriteOff (util)13

                                          Rate Subsidy(util)61

                                          HealthSafety(util)0

                                          Colln Costs (util)0

                                          Gas Calls (util)0

                                          Calls to CSRs(util)2

                                          TampD (util)16

                                          Arrears (util)0

                                          Reconnects (util)0

                                          Notices (util)7

                                          Shutoffs (util)1

                                          Rate subsidy TampD

                                          Payment-related

                                          Source Skumatz Economic Research Associates research

                                          Example Low Income Weatherization

                                          SERA

                                          ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                          be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                          Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                          Value of 100 HALVES Payback doubles ROI

                                          Source Skumatz Economics (SERA)

                                          35

                                          ( for NEBs Multiplier of 125)

                                          SERA

                                          NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                          36

                                          hellip2001 ON

                                          SERA

                                          37

                                          NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                          For true representation of B amp C elements of NEBs address missing factor bias

                                          Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                          BC applications were considered early on then pulled back awaiting more quantitative evidence

                                          SERA

                                          CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                          38

                                          Test Benefits Costs States Using Traditionally

                                          Improved treat-ment with NEBs

                                          Utility Cost (or Program Administrator Test) (UCT or PAC)

                                          bull Avoided supply costs for transmission distribution and generation (TDampG)

                                          bull Avoided gas and water supply costs

                                          bull Program administration

                                          bull Participant incentives

                                          bull Increased supply cost

                                          CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                          Use cost only paid by the utility

                                          Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                          Same as above plus bull increased revenue

                                          Same as above plus bull Decreased revenue

                                          AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                          Participant cost bull Utility bill reductions bull Participant incentives

                                          bull Participant direct costs

                                          AR CA FL HI IA IN MI MN NY VA

                                          Participant NEBs

                                          Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                          Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                          SERA

                                          APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                          Expansion Concern with risk calculation accuracy

                                          39

                                          Source Skumatz SERA research

                                          SERA

                                          40

                                          METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                          Maximize DSM opportunities amp feedback Accuracy tailoring

                                          Minimize Regulatory amp Implementer Risk

                                          Minimize Evaluation Cost

                                          Adder

                                          Readily MeasurableHybrid

                                          All NEBsSource SERA Research

                                          SERA

                                          41

                                          STATE TREATMENT OF NEBS NEBsRegulatory

                                          Screening Application Utilities regions

                                          Program Marketing Fairly widespread use in utilities states across the country

                                          MO

                                          RE AGGRESSIVE ==gt

                                          Test Pgm Screen - adder

                                          IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                          Test Pgm Screen - readily measured

                                          MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                          Test Hybrid (potential adder amp measured)

                                          CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                          Test Pgm screen - Broad

                                          With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                          Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                          SERA

                                          42

                                          Key

                                          States with NEBs in CE tests for At least one type of program

                                          copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                          STATES WITH NEBS IN CE TESTS

                                          NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                          groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                          SERA

                                          SELECTED STATE PROGRESS CA

                                          2001 ndash LIPPT model NY

                                          Mid-2000s measurement amp scenarios not included CO

                                          Adders (10 electric 25 LI 5 gas) VT

                                          15 adder helped by previous research DC

                                          10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                          43

                                          Source Skumatz SERA research

                                          SERA

                                          SELECTED STATE PROGRESS Dominos ongoing

                                          Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                          Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                          Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                          Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                          44

                                          Source Skumatz SERA research

                                          SERA

                                          APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                          Expansion Concern with risk calculation accuracy

                                          45

                                          Source Skumatz SERA research

                                          SERA

                                          46

                                          NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                          [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                          SourceSkumatz SERA Independent research

                                          NTG ndash accuracy measurement incomplete

                                          Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                          NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                          EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                          Cost Complicated expensive local changes

                                          RELATIVE Risk from NEBs

                                          Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                          Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                          NEBs as accurate as many inputs (all could improve)

                                          SERA

                                          DISCOUNT RATE MATTERS

                                          47

                                          Source Skumatz SERA Research

                                          0

                                          20

                                          40

                                          60

                                          80

                                          100

                                          120

                                          1 2 3 4 5 6 7 8 9 10

                                          PV Ratios vs 1 Discount Rate

                                          10 yr PV 20 yr PV

                                          Discount rate - percent

                                          WACCSocialLTT

                                          Source Skumatz Electricity Journal IEPEC paper independent research

                                          SERA

                                          EUL VARIATIONS-ADOPTED VALUES

                                          Source Skumatz Economic Research Associates SERA independent research

                                          0

                                          50

                                          100

                                          150

                                          200

                                          250

                                          300

                                          350

                                          400

                                          450

                                          0 10 20 30 40 50 60 70

                                          EUL Variation by Measure

                                          67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                          Poorly supportedOldTechnology change

                                          SERA

                                          KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                          Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                          measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                          change the BC conclusion

                                          On the cusphellip

                                          If NO Yoursquore doneAnd bias addressedsufficiently

                                          IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                          Source SERA all rights reserved

                                          SERArsquos lsquoNEB-ItrdquoModel ndash

                                          SERA

                                          IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                          Utility Soc Part Conserv Recrsquom

                                          Rationale

                                          Base Percent X X X Program-invariant

                                          Low Income X X X X Multiple sources

                                          Weatherization X X X Substantial Participantimpacts

                                          Measure Program-specific

                                          X Varies by measure sector

                                          Other Recomrsquos Local Research

                                          50

                                          Source Skumatz SERA

                                          Developing values for multiple states amp utilities

                                          SERA

                                          TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                          Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                          Other sources of bias to research (EUL NTG etc) bring high variation

                                          5) States are incorporating NEBs dominos input in deliberations in multiple states

                                          6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                          THANK YOU

                                          Questions

                                          Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                          SERA

                                          POWER OF NEBS ndashBARRIERS DISCONNECTS

                                          53

                                          SERA

                                          NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                          Negative NEB values cost of barrier

                                          Solar Water Heat NZ$ Euros

                                          Solar Design NZ$ Euros

                                          Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                          -23 -12 (079)

                                          -11 -6 (06)

                                          Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                          Source Skumatz Economic Research Associates research

                                          SERA

                                          CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                          NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                          -5-4-3-2-1012345

                                          Op Co

                                          st

                                          Init C

                                          ost

                                          Maint

                                          Perfo

                                          rm

                                          Prod

                                          uctiv

                                          Tena

                                          nt Sa

                                          t

                                          Comf

                                          ort

                                          Appe

                                          ar

                                          Light

                                          Qual

                                          SellL

                                          ease

                                          Devp Ownr AampE

                                          Note AgtE (~4 vs ~2) for these NEBs

                                          Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                          Source Skumatz Economic Research Associates Inc 2002

                                          Owners more positive ndashUnderinvestment train

                                          • Soup to nuts on non-energy benefits NEIs
                                          • TOPICS
                                          • BACKGROUND NEBS
                                          • 20+ YEARS OF NEBS PROGRESShellip
                                          • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                          • KEY APPLICATIONS OF NEBS
                                          • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                          • Neb measurement
                                          • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                          • MEASURING PARTICIPANTNEBs
                                          • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                          • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                          • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                          • PROGRESS TRANSFERABILITY OF NEBS
                                          • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                          • PROGRESS AND GAPS
                                          • PROGRESS AND GAPS
                                          • USING NEB NEI VALUES
                                          • Slide Number 19
                                          • ARE NEBS HIGH VALUE
                                          • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                          • TOP NEBS FOR WEATHERIZATION PROGRAM
                                          • RESULTS FOR ENERGY STARreg APPLIANCES
                                          • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                          • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                          • CampI PROGRAMS ldquoNEBBEDrdquo
                                          • AUDIT FINANCIAL PROGRAM
                                          • CampI NEW CONSTRUCTION
                                          • CampI TECHNICAL ASSISTANCE
                                          • EXPRESSING NEBS VALUEndashCxe
                                          • Slide Number 31
                                          • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                          • EMISSIONS (SOCIETAL NEBS)
                                          • UTILITY NEBS
                                          • ADJUSTED PAYBACKS
                                          • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                          • NEBS IN BC
                                          • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                          • APPLICATION TO BC
                                          • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                          • STATE TREATMENT OF NEBS NEBs
                                          • Slide Number 42
                                          • SELECTED STATE PROGRESS
                                          • SELECTED STATE PROGRESS
                                          • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                          • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                          • DISCOUNT RATE MATTERS
                                          • EUL VARIATIONS- ADOPTED VALUES
                                          • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                          • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                          • TAKEAWAYSON NEBSNEIS
                                          • THANK YOUQuestions
                                          • Slide Number 53
                                          • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                          • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                            Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                            m Product losses20
                                            l Ease of selling leasing building33
                                            n Labor requirements45
                                            h Aesthetics appearance59
                                            j Noise50
                                            b Equipment maintenance660
                                            f Tenant satisfaction60
                                            d Equipment lifetimes718
                                            e Productivity724
                                            k Building safety[2]73
                                            g Comfort933
                                            i Lighting Quality of Light916
                                            c Equipment performance[1]1019
                                            a Operating costs1165
                                            o Doing good for the Environment[3]1132
                                            Other03
                                            [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                            [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                            [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                            m Product losses
                                            l Ease of selling leasing building
                                            n Labor requirements
                                            h Aesthetics appearance
                                            j Noise
                                            b Equipment maintenance
                                            f Tenant satisfaction
                                            d Equipment lifetimes
                                            e Productivity
                                            k Building safety[2]
                                            g Comfort
                                            i Lighting Quality of Light
                                            c Equipment performance[1]
                                            a Operating costs
                                            o Doing good for the Environment[3]
                                            Share of ZALEH NEBS from various measures
                                            Each presented as multiple of the (overall) energy savings
                                            Super Insulation037
                                            Double Glazing026
                                            Solar Design025
                                            Solar Water Heating005
                                            EnergySmart (insul Retrofit)Dunedin (insul)
                                            Other124
                                            Noise24
                                            Budgetfinances25
                                            Environmental65
                                            Health78
                                            Reduced moves86
                                            Ease of sellingrenting130
                                            DampnessMold1316
                                            Comfort-related4933
                                            101101
                                            Operational NEBs44
                                            DesignConstruction NEBs31
                                            Occupant NEBs25
                                            1
                                            OtherOther
                                            NoiseNoise
                                            BudgetfinancesBudgetfinances
                                            EnvironmentalEnvironmental
                                            HealthHealth
                                            Reduced movesReduced moves
                                            Ease of sellingrentingEase of sellingrenting
                                            DampnessMoldDampnessMold
                                            Comfort-relatedComfort-related
                                            Boiler
                                            VendorNPPartic
                                            Perform0280202
                                            Footprint028013017
                                            ControlFeatures031022016
                                            Tenant compl006008014
                                            Noise005006014
                                            Maint004009007
                                            Other004003007
                                            Lifetime-006018006
                                            MF New construction energy star
                                            Operating cost014
                                            Environment012
                                            Maintenance01
                                            Lifetime01
                                            Performance009
                                            Occupant Satisf009
                                            Comfort008
                                            Noise006
                                            Sellingleasing006
                                            Aesthetics005
                                            Light Quality005
                                            Power QualReliab004
                                            Safety002
                                            RIGHTLIGHTS
                                            Environment023
                                            Quality016
                                            Quantity014
                                            PgmAvail01
                                            Flicker008
                                            Maintenance006
                                            Lifetime006
                                            SalesProductivity006
                                            Safety005
                                            Noise005
                                            Control003
                                            Illness0
                                            PerformPerformPerform
                                            FootprintFootprintFootprint
                                            ControlFeaturesControlFeaturesControlFeatures
                                            Tenant complTenant complTenant compl
                                            NoiseNoiseNoise
                                            MaintMaintMaint
                                            OtherOtherOther
                                            LifetimeLifetimeLifetime
                                            Environment
                                            Quality
                                            Quantity
                                            PgmAvail
                                            Flicker
                                            Maintenance
                                            Lifetime
                                            SalesProductivity
                                            Safety
                                            Noise
                                            Control
                                            Illness

                                            SERA

                                            PROGRESS AND GAPS Gas vs Electric

                                            Some researchndashusually program-wide not broken down International research found participant NEBs have

                                            similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin

                                            MF Less-commonly-studied complicated by poor response

                                            and complexity of sector (decision-maker some measures in home some central) separate from low income not common

                                            Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap

                                            Commercial gaps in some measures

                                            SERA

                                            USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                                            Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                                            18

                                            0

                                            005

                                            01

                                            015

                                            02

                                            025

                                            Environment

                                            Quality

                                            Quantity

                                            PgmAvailFlicker

                                            Mainten

                                            ance

                                            Lifetime

                                            SalesP

                                            roductivity

                                            Safety

                                            Noise

                                            Control

                                            Illness

                                            Total NEBs Multiplier=X

                                            $

                                            Savings value=$X

                                            Chart3

                                            023
                                            016
                                            014
                                            01
                                            008
                                            006
                                            006
                                            006
                                            005
                                            005
                                            003
                                            0

                                            Chart1

                                            Vendor
                                            NP
                                            Partic
                                            028
                                            02
                                            02
                                            028
                                            013
                                            017
                                            031
                                            022
                                            016
                                            006
                                            008
                                            014
                                            005
                                            006
                                            014
                                            004
                                            009
                                            007
                                            004
                                            003
                                            007
                                            -006
                                            018
                                            006

                                            Sheet1

                                            Sheet1

                                            Sheet2

                                            Sheet3

                                            SERA

                                            NEB VALUES

                                            19

                                            SERA

                                            ARE NEBS HIGH VALUE Energy savings are often the minority of

                                            benefits from program measuresndash Donrsquot ignore that

                                            20

                                            0 05 1 15 2 25 3

                                            Energy Sav

                                            NEBs

                                            NEB vs Energy Savings ValueIncluding all NEBs

                                            Source (SkumatzSERA 2010 amp others)

                                            Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                            SERA

                                            RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                            MF Many others

                                            Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                            Variety of end uses

                                            SERA

                                            TOP NEBS FOR WEATHERIZATION PROGRAM

                                            Weatherization program multipliergt1

                                            Source SERA Research

                                            SERA

                                            RESULTS FOR ENERGY STARreg APPLIANCES

                                            0002004006008

                                            01012014016018

                                            Maint

                                            Appear

                                            Perf LifeNoise Satis

                                            Comfort

                                            LiteSafe

                                            ty SellMove

                                            Water

                                            Enviro

                                            Refrig DW CW

                                            Source SERA Research

                                            SERA

                                            INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                            5 increments to 50

                                            Source SERA research

                                            Chart5

                                            EnergySmart (insul Retrofit)
                                            Dunedin (insul)
                                            001
                                            024
                                            002
                                            004
                                            002
                                            005
                                            006
                                            005
                                            007
                                            008
                                            008
                                            006
                                            013
                                            0
                                            013
                                            016
                                            049
                                            033

                                            cx

                                            cx

                                            nz

                                            Sheet3

                                            Sheet3

                                            Super Insulation
                                            Double Glazing
                                            Solar Design
                                            Solar Water Heating
                                            Super Insulation
                                            Double Glazing
                                            Solar Design
                                            Solar Water Heating
                                            EnergySmart (insul Retrofit)
                                            Dunedin (insul)

                                            SERA

                                            ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                            0

                                            002

                                            004

                                            006

                                            008

                                            01

                                            012

                                            Maint

                                            Perf LifeSatis

                                            f

                                            ComfortAesth

                                            NoiseSafe

                                            ty

                                            LightingSelli

                                            ng

                                            Stay in home

                                            Enviro

                                            Sick Days

                                            Calls

                                            ES Homes HP wES Energy savings multiplier=1

                                            Source SERA Research

                                            SERA

                                            CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                            DG CHP HVAC Equipment rebate Other

                                            Building codes incentives by cities

                                            Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                            Variety of end uses

                                            SERA

                                            AUDIT FINANCIAL PROGRAM

                                            0

                                            01

                                            02

                                            03

                                            04

                                            05

                                            06

                                            07

                                            Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                            Source SERA Research

                                            SERA

                                            CampI NEW CONSTRUCTION

                                            -005

                                            0

                                            005

                                            01

                                            015

                                            02

                                            AampE Owner(Pct of Total Partic NEBs)

                                            Source SERA Research

                                            SERA

                                            CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                            0 2 4 6 8 10 12

                                            m Product losses

                                            l Ease of selling leasing building

                                            n Labor requirements

                                            h Aesthetics appearance

                                            j Noise

                                            b Equipment maintenance

                                            f Tenant satisfaction

                                            d Equipment lifetimes

                                            e Productivity

                                            k Building safety[2]

                                            g Comfort

                                            i Lighting Quality of Light

                                            c Equipment performance[1]

                                            a Operating costs

                                            o Doing good for the Environment[3]

                                            Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                            vcv

                                            Source SERA Research

                                            Chart1

                                            CampI Technical Assistance NEBs
                                            002
                                            003
                                            004
                                            005
                                            005
                                            006
                                            006
                                            007
                                            007
                                            007
                                            009
                                            009
                                            01
                                            011
                                            011

                                            Sheet1

                                            Sheet1

                                            CampI Technical Assistance Program NEBs (Participant)

                                            Sheet2

                                            Sheet3

                                            SERA

                                            EXPRESSING NEBS VALUEndashCxe

                                            Strong value from RetroCxSource SERA Research

                                            SERA

                                            SOCIETAL NEB EXAMPLES

                                            31

                                            SERA

                                            JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                            32

                                            0

                                            02

                                            04

                                            06

                                            08

                                            1

                                            12

                                            HPWxRetrof Appliance

                                            CA

                                            WI

                                            Natl

                                            (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                            Jobs Economic

                                            SERA

                                            EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                            33

                                            SERA

                                            UTILITY NEBSUtility NEBs for Template Program

                                            Debt WriteOff (util)13

                                            Rate Subsidy(util)61

                                            HealthSafety(util)0

                                            Colln Costs (util)0

                                            Gas Calls (util)0

                                            Calls to CSRs(util)2

                                            TampD (util)16

                                            Arrears (util)0

                                            Reconnects (util)0

                                            Notices (util)7

                                            Shutoffs (util)1

                                            Rate subsidy TampD

                                            Payment-related

                                            Source Skumatz Economic Research Associates research

                                            Example Low Income Weatherization

                                            SERA

                                            ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                            be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                            Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                            Value of 100 HALVES Payback doubles ROI

                                            Source Skumatz Economics (SERA)

                                            35

                                            ( for NEBs Multiplier of 125)

                                            SERA

                                            NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                            36

                                            hellip2001 ON

                                            SERA

                                            37

                                            NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                            For true representation of B amp C elements of NEBs address missing factor bias

                                            Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                            BC applications were considered early on then pulled back awaiting more quantitative evidence

                                            SERA

                                            CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                            38

                                            Test Benefits Costs States Using Traditionally

                                            Improved treat-ment with NEBs

                                            Utility Cost (or Program Administrator Test) (UCT or PAC)

                                            bull Avoided supply costs for transmission distribution and generation (TDampG)

                                            bull Avoided gas and water supply costs

                                            bull Program administration

                                            bull Participant incentives

                                            bull Increased supply cost

                                            CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                            Use cost only paid by the utility

                                            Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                            Same as above plus bull increased revenue

                                            Same as above plus bull Decreased revenue

                                            AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                            Participant cost bull Utility bill reductions bull Participant incentives

                                            bull Participant direct costs

                                            AR CA FL HI IA IN MI MN NY VA

                                            Participant NEBs

                                            Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                            Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                            SERA

                                            APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                            Expansion Concern with risk calculation accuracy

                                            39

                                            Source Skumatz SERA research

                                            SERA

                                            40

                                            METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                            Maximize DSM opportunities amp feedback Accuracy tailoring

                                            Minimize Regulatory amp Implementer Risk

                                            Minimize Evaluation Cost

                                            Adder

                                            Readily MeasurableHybrid

                                            All NEBsSource SERA Research

                                            SERA

                                            41

                                            STATE TREATMENT OF NEBS NEBsRegulatory

                                            Screening Application Utilities regions

                                            Program Marketing Fairly widespread use in utilities states across the country

                                            MO

                                            RE AGGRESSIVE ==gt

                                            Test Pgm Screen - adder

                                            IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                            Test Pgm Screen - readily measured

                                            MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                            Test Hybrid (potential adder amp measured)

                                            CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                            Test Pgm screen - Broad

                                            With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                            Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                            SERA

                                            42

                                            Key

                                            States with NEBs in CE tests for At least one type of program

                                            copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                            STATES WITH NEBS IN CE TESTS

                                            NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                            groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                            SERA

                                            SELECTED STATE PROGRESS CA

                                            2001 ndash LIPPT model NY

                                            Mid-2000s measurement amp scenarios not included CO

                                            Adders (10 electric 25 LI 5 gas) VT

                                            15 adder helped by previous research DC

                                            10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                            43

                                            Source Skumatz SERA research

                                            SERA

                                            SELECTED STATE PROGRESS Dominos ongoing

                                            Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                            Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                            Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                            Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                            44

                                            Source Skumatz SERA research

                                            SERA

                                            APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                            Expansion Concern with risk calculation accuracy

                                            45

                                            Source Skumatz SERA research

                                            SERA

                                            46

                                            NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                            [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                            SourceSkumatz SERA Independent research

                                            NTG ndash accuracy measurement incomplete

                                            Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                            NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                            EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                            Cost Complicated expensive local changes

                                            RELATIVE Risk from NEBs

                                            Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                            Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                            NEBs as accurate as many inputs (all could improve)

                                            SERA

                                            DISCOUNT RATE MATTERS

                                            47

                                            Source Skumatz SERA Research

                                            0

                                            20

                                            40

                                            60

                                            80

                                            100

                                            120

                                            1 2 3 4 5 6 7 8 9 10

                                            PV Ratios vs 1 Discount Rate

                                            10 yr PV 20 yr PV

                                            Discount rate - percent

                                            WACCSocialLTT

                                            Source Skumatz Electricity Journal IEPEC paper independent research

                                            SERA

                                            EUL VARIATIONS-ADOPTED VALUES

                                            Source Skumatz Economic Research Associates SERA independent research

                                            0

                                            50

                                            100

                                            150

                                            200

                                            250

                                            300

                                            350

                                            400

                                            450

                                            0 10 20 30 40 50 60 70

                                            EUL Variation by Measure

                                            67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                            Poorly supportedOldTechnology change

                                            SERA

                                            KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                            Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                            measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                            change the BC conclusion

                                            On the cusphellip

                                            If NO Yoursquore doneAnd bias addressedsufficiently

                                            IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                            Source SERA all rights reserved

                                            SERArsquos lsquoNEB-ItrdquoModel ndash

                                            SERA

                                            IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                            Utility Soc Part Conserv Recrsquom

                                            Rationale

                                            Base Percent X X X Program-invariant

                                            Low Income X X X X Multiple sources

                                            Weatherization X X X Substantial Participantimpacts

                                            Measure Program-specific

                                            X Varies by measure sector

                                            Other Recomrsquos Local Research

                                            50

                                            Source Skumatz SERA

                                            Developing values for multiple states amp utilities

                                            SERA

                                            TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                            Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                            Other sources of bias to research (EUL NTG etc) bring high variation

                                            5) States are incorporating NEBs dominos input in deliberations in multiple states

                                            6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                            THANK YOU

                                            Questions

                                            Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                            SERA

                                            POWER OF NEBS ndashBARRIERS DISCONNECTS

                                            53

                                            SERA

                                            NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                            Negative NEB values cost of barrier

                                            Solar Water Heat NZ$ Euros

                                            Solar Design NZ$ Euros

                                            Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                            -23 -12 (079)

                                            -11 -6 (06)

                                            Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                            Source Skumatz Economic Research Associates research

                                            SERA

                                            CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                            NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                            -5-4-3-2-1012345

                                            Op Co

                                            st

                                            Init C

                                            ost

                                            Maint

                                            Perfo

                                            rm

                                            Prod

                                            uctiv

                                            Tena

                                            nt Sa

                                            t

                                            Comf

                                            ort

                                            Appe

                                            ar

                                            Light

                                            Qual

                                            SellL

                                            ease

                                            Devp Ownr AampE

                                            Note AgtE (~4 vs ~2) for these NEBs

                                            Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                            Source Skumatz Economic Research Associates Inc 2002

                                            Owners more positive ndashUnderinvestment train

                                            • Soup to nuts on non-energy benefits NEIs
                                            • TOPICS
                                            • BACKGROUND NEBS
                                            • 20+ YEARS OF NEBS PROGRESShellip
                                            • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                            • KEY APPLICATIONS OF NEBS
                                            • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                            • Neb measurement
                                            • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                            • MEASURING PARTICIPANTNEBs
                                            • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                            • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                            • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                            • PROGRESS TRANSFERABILITY OF NEBS
                                            • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                            • PROGRESS AND GAPS
                                            • PROGRESS AND GAPS
                                            • USING NEB NEI VALUES
                                            • Slide Number 19
                                            • ARE NEBS HIGH VALUE
                                            • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                            • TOP NEBS FOR WEATHERIZATION PROGRAM
                                            • RESULTS FOR ENERGY STARreg APPLIANCES
                                            • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                            • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                            • CampI PROGRAMS ldquoNEBBEDrdquo
                                            • AUDIT FINANCIAL PROGRAM
                                            • CampI NEW CONSTRUCTION
                                            • CampI TECHNICAL ASSISTANCE
                                            • EXPRESSING NEBS VALUEndashCxe
                                            • Slide Number 31
                                            • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                            • EMISSIONS (SOCIETAL NEBS)
                                            • UTILITY NEBS
                                            • ADJUSTED PAYBACKS
                                            • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                            • NEBS IN BC
                                            • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                            • APPLICATION TO BC
                                            • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                            • STATE TREATMENT OF NEBS NEBs
                                            • Slide Number 42
                                            • SELECTED STATE PROGRESS
                                            • SELECTED STATE PROGRESS
                                            • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                            • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                            • DISCOUNT RATE MATTERS
                                            • EUL VARIATIONS- ADOPTED VALUES
                                            • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                            • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                            • TAKEAWAYSON NEBSNEIS
                                            • THANK YOUQuestions
                                            • Slide Number 53
                                            • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                            • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                              Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                              m Product losses20
                                              l Ease of selling leasing building33
                                              n Labor requirements45
                                              h Aesthetics appearance59
                                              j Noise50
                                              b Equipment maintenance660
                                              f Tenant satisfaction60
                                              d Equipment lifetimes718
                                              e Productivity724
                                              k Building safety[2]73
                                              g Comfort933
                                              i Lighting Quality of Light916
                                              c Equipment performance[1]1019
                                              a Operating costs1165
                                              o Doing good for the Environment[3]1132
                                              Other03
                                              [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                              [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                              [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                              m Product losses
                                              l Ease of selling leasing building
                                              n Labor requirements
                                              h Aesthetics appearance
                                              j Noise
                                              b Equipment maintenance
                                              f Tenant satisfaction
                                              d Equipment lifetimes
                                              e Productivity
                                              k Building safety[2]
                                              g Comfort
                                              i Lighting Quality of Light
                                              c Equipment performance[1]
                                              a Operating costs
                                              o Doing good for the Environment[3]
                                              Share of ZALEH NEBS from various measures
                                              Each presented as multiple of the (overall) energy savings
                                              Super Insulation037
                                              Double Glazing026
                                              Solar Design025
                                              Solar Water Heating005
                                              EnergySmart (insul Retrofit)Dunedin (insul)
                                              Other124
                                              Noise24
                                              Budgetfinances25
                                              Environmental65
                                              Health78
                                              Reduced moves86
                                              Ease of sellingrenting130
                                              DampnessMold1316
                                              Comfort-related4933
                                              101101
                                              Operational NEBs44
                                              DesignConstruction NEBs31
                                              Occupant NEBs25
                                              1
                                              OtherOther
                                              NoiseNoise
                                              BudgetfinancesBudgetfinances
                                              EnvironmentalEnvironmental
                                              HealthHealth
                                              Reduced movesReduced moves
                                              Ease of sellingrentingEase of sellingrenting
                                              DampnessMoldDampnessMold
                                              Comfort-relatedComfort-related
                                              Boiler
                                              VendorNPPartic
                                              Perform0280202
                                              Footprint028013017
                                              ControlFeatures031022016
                                              Tenant compl006008014
                                              Noise005006014
                                              Maint004009007
                                              Other004003007
                                              Lifetime-006018006
                                              MF New construction energy star
                                              Operating cost014
                                              Environment012
                                              Maintenance01
                                              Lifetime01
                                              Performance009
                                              Occupant Satisf009
                                              Comfort008
                                              Noise006
                                              Sellingleasing006
                                              Aesthetics005
                                              Light Quality005
                                              Power QualReliab004
                                              Safety002
                                              RIGHTLIGHTS
                                              Environment023
                                              Quality016
                                              Quantity014
                                              PgmAvail01
                                              Flicker008
                                              Maintenance006
                                              Lifetime006
                                              SalesProductivity006
                                              Safety005
                                              Noise005
                                              Control003
                                              Illness0
                                              PerformPerformPerform
                                              FootprintFootprintFootprint
                                              ControlFeaturesControlFeaturesControlFeatures
                                              Tenant complTenant complTenant compl
                                              NoiseNoiseNoise
                                              MaintMaintMaint
                                              OtherOtherOther
                                              LifetimeLifetimeLifetime
                                              Environment
                                              Quality
                                              Quantity
                                              PgmAvail
                                              Flicker
                                              Maintenance
                                              Lifetime
                                              SalesProductivity
                                              Safety
                                              Noise
                                              Control
                                              Illness

                                              SERA

                                              USING NEB NEI VALUES Improve Cost-effectiveness (CE) of

                                              Marketing (better uptakecontact) ROI amp Payback calculations Deeper Measures ndash beyond lighting

                                              18

                                              0

                                              005

                                              01

                                              015

                                              02

                                              025

                                              Environment

                                              Quality

                                              Quantity

                                              PgmAvailFlicker

                                              Mainten

                                              ance

                                              Lifetime

                                              SalesP

                                              roductivity

                                              Safety

                                              Noise

                                              Control

                                              Illness

                                              Total NEBs Multiplier=X

                                              $

                                              Savings value=$X

                                              Chart3

                                              023
                                              016
                                              014
                                              01
                                              008
                                              006
                                              006
                                              006
                                              005
                                              005
                                              003
                                              0

                                              Chart1

                                              Vendor
                                              NP
                                              Partic
                                              028
                                              02
                                              02
                                              028
                                              013
                                              017
                                              031
                                              022
                                              016
                                              006
                                              008
                                              014
                                              005
                                              006
                                              014
                                              004
                                              009
                                              007
                                              004
                                              003
                                              007
                                              -006
                                              018
                                              006

                                              Sheet1

                                              Sheet1

                                              Sheet2

                                              Sheet3

                                              SERA

                                              NEB VALUES

                                              19

                                              SERA

                                              ARE NEBS HIGH VALUE Energy savings are often the minority of

                                              benefits from program measuresndash Donrsquot ignore that

                                              20

                                              0 05 1 15 2 25 3

                                              Energy Sav

                                              NEBs

                                              NEB vs Energy Savings ValueIncluding all NEBs

                                              Source (SkumatzSERA 2010 amp others)

                                              Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                              SERA

                                              RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                              MF Many others

                                              Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                              Variety of end uses

                                              SERA

                                              TOP NEBS FOR WEATHERIZATION PROGRAM

                                              Weatherization program multipliergt1

                                              Source SERA Research

                                              SERA

                                              RESULTS FOR ENERGY STARreg APPLIANCES

                                              0002004006008

                                              01012014016018

                                              Maint

                                              Appear

                                              Perf LifeNoise Satis

                                              Comfort

                                              LiteSafe

                                              ty SellMove

                                              Water

                                              Enviro

                                              Refrig DW CW

                                              Source SERA Research

                                              SERA

                                              INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                              5 increments to 50

                                              Source SERA research

                                              Chart5

                                              EnergySmart (insul Retrofit)
                                              Dunedin (insul)
                                              001
                                              024
                                              002
                                              004
                                              002
                                              005
                                              006
                                              005
                                              007
                                              008
                                              008
                                              006
                                              013
                                              0
                                              013
                                              016
                                              049
                                              033

                                              cx

                                              cx

                                              nz

                                              Sheet3

                                              Sheet3

                                              Super Insulation
                                              Double Glazing
                                              Solar Design
                                              Solar Water Heating
                                              Super Insulation
                                              Double Glazing
                                              Solar Design
                                              Solar Water Heating
                                              EnergySmart (insul Retrofit)
                                              Dunedin (insul)

                                              SERA

                                              ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                              0

                                              002

                                              004

                                              006

                                              008

                                              01

                                              012

                                              Maint

                                              Perf LifeSatis

                                              f

                                              ComfortAesth

                                              NoiseSafe

                                              ty

                                              LightingSelli

                                              ng

                                              Stay in home

                                              Enviro

                                              Sick Days

                                              Calls

                                              ES Homes HP wES Energy savings multiplier=1

                                              Source SERA Research

                                              SERA

                                              CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                              DG CHP HVAC Equipment rebate Other

                                              Building codes incentives by cities

                                              Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                              Variety of end uses

                                              SERA

                                              AUDIT FINANCIAL PROGRAM

                                              0

                                              01

                                              02

                                              03

                                              04

                                              05

                                              06

                                              07

                                              Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                              Source SERA Research

                                              SERA

                                              CampI NEW CONSTRUCTION

                                              -005

                                              0

                                              005

                                              01

                                              015

                                              02

                                              AampE Owner(Pct of Total Partic NEBs)

                                              Source SERA Research

                                              SERA

                                              CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                              0 2 4 6 8 10 12

                                              m Product losses

                                              l Ease of selling leasing building

                                              n Labor requirements

                                              h Aesthetics appearance

                                              j Noise

                                              b Equipment maintenance

                                              f Tenant satisfaction

                                              d Equipment lifetimes

                                              e Productivity

                                              k Building safety[2]

                                              g Comfort

                                              i Lighting Quality of Light

                                              c Equipment performance[1]

                                              a Operating costs

                                              o Doing good for the Environment[3]

                                              Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                              vcv

                                              Source SERA Research

                                              Chart1

                                              CampI Technical Assistance NEBs
                                              002
                                              003
                                              004
                                              005
                                              005
                                              006
                                              006
                                              007
                                              007
                                              007
                                              009
                                              009
                                              01
                                              011
                                              011

                                              Sheet1

                                              Sheet1

                                              CampI Technical Assistance Program NEBs (Participant)

                                              Sheet2

                                              Sheet3

                                              SERA

                                              EXPRESSING NEBS VALUEndashCxe

                                              Strong value from RetroCxSource SERA Research

                                              SERA

                                              SOCIETAL NEB EXAMPLES

                                              31

                                              SERA

                                              JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                              32

                                              0

                                              02

                                              04

                                              06

                                              08

                                              1

                                              12

                                              HPWxRetrof Appliance

                                              CA

                                              WI

                                              Natl

                                              (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                              Jobs Economic

                                              SERA

                                              EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                              33

                                              SERA

                                              UTILITY NEBSUtility NEBs for Template Program

                                              Debt WriteOff (util)13

                                              Rate Subsidy(util)61

                                              HealthSafety(util)0

                                              Colln Costs (util)0

                                              Gas Calls (util)0

                                              Calls to CSRs(util)2

                                              TampD (util)16

                                              Arrears (util)0

                                              Reconnects (util)0

                                              Notices (util)7

                                              Shutoffs (util)1

                                              Rate subsidy TampD

                                              Payment-related

                                              Source Skumatz Economic Research Associates research

                                              Example Low Income Weatherization

                                              SERA

                                              ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                              be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                              Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                              Value of 100 HALVES Payback doubles ROI

                                              Source Skumatz Economics (SERA)

                                              35

                                              ( for NEBs Multiplier of 125)

                                              SERA

                                              NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                              36

                                              hellip2001 ON

                                              SERA

                                              37

                                              NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                              For true representation of B amp C elements of NEBs address missing factor bias

                                              Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                              BC applications were considered early on then pulled back awaiting more quantitative evidence

                                              SERA

                                              CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                              38

                                              Test Benefits Costs States Using Traditionally

                                              Improved treat-ment with NEBs

                                              Utility Cost (or Program Administrator Test) (UCT or PAC)

                                              bull Avoided supply costs for transmission distribution and generation (TDampG)

                                              bull Avoided gas and water supply costs

                                              bull Program administration

                                              bull Participant incentives

                                              bull Increased supply cost

                                              CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                              Use cost only paid by the utility

                                              Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                              Same as above plus bull increased revenue

                                              Same as above plus bull Decreased revenue

                                              AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                              Participant cost bull Utility bill reductions bull Participant incentives

                                              bull Participant direct costs

                                              AR CA FL HI IA IN MI MN NY VA

                                              Participant NEBs

                                              Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                              Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                              SERA

                                              APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                              Expansion Concern with risk calculation accuracy

                                              39

                                              Source Skumatz SERA research

                                              SERA

                                              40

                                              METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                              Maximize DSM opportunities amp feedback Accuracy tailoring

                                              Minimize Regulatory amp Implementer Risk

                                              Minimize Evaluation Cost

                                              Adder

                                              Readily MeasurableHybrid

                                              All NEBsSource SERA Research

                                              SERA

                                              41

                                              STATE TREATMENT OF NEBS NEBsRegulatory

                                              Screening Application Utilities regions

                                              Program Marketing Fairly widespread use in utilities states across the country

                                              MO

                                              RE AGGRESSIVE ==gt

                                              Test Pgm Screen - adder

                                              IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                              Test Pgm Screen - readily measured

                                              MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                              Test Hybrid (potential adder amp measured)

                                              CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                              Test Pgm screen - Broad

                                              With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                              Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                              SERA

                                              42

                                              Key

                                              States with NEBs in CE tests for At least one type of program

                                              copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                              STATES WITH NEBS IN CE TESTS

                                              NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                              groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                              SERA

                                              SELECTED STATE PROGRESS CA

                                              2001 ndash LIPPT model NY

                                              Mid-2000s measurement amp scenarios not included CO

                                              Adders (10 electric 25 LI 5 gas) VT

                                              15 adder helped by previous research DC

                                              10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                              43

                                              Source Skumatz SERA research

                                              SERA

                                              SELECTED STATE PROGRESS Dominos ongoing

                                              Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                              Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                              Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                              Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                              44

                                              Source Skumatz SERA research

                                              SERA

                                              APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                              Expansion Concern with risk calculation accuracy

                                              45

                                              Source Skumatz SERA research

                                              SERA

                                              46

                                              NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                              [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                              SourceSkumatz SERA Independent research

                                              NTG ndash accuracy measurement incomplete

                                              Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                              NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                              EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                              Cost Complicated expensive local changes

                                              RELATIVE Risk from NEBs

                                              Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                              Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                              NEBs as accurate as many inputs (all could improve)

                                              SERA

                                              DISCOUNT RATE MATTERS

                                              47

                                              Source Skumatz SERA Research

                                              0

                                              20

                                              40

                                              60

                                              80

                                              100

                                              120

                                              1 2 3 4 5 6 7 8 9 10

                                              PV Ratios vs 1 Discount Rate

                                              10 yr PV 20 yr PV

                                              Discount rate - percent

                                              WACCSocialLTT

                                              Source Skumatz Electricity Journal IEPEC paper independent research

                                              SERA

                                              EUL VARIATIONS-ADOPTED VALUES

                                              Source Skumatz Economic Research Associates SERA independent research

                                              0

                                              50

                                              100

                                              150

                                              200

                                              250

                                              300

                                              350

                                              400

                                              450

                                              0 10 20 30 40 50 60 70

                                              EUL Variation by Measure

                                              67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                              Poorly supportedOldTechnology change

                                              SERA

                                              KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                              Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                              measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                              change the BC conclusion

                                              On the cusphellip

                                              If NO Yoursquore doneAnd bias addressedsufficiently

                                              IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                              Source SERA all rights reserved

                                              SERArsquos lsquoNEB-ItrdquoModel ndash

                                              SERA

                                              IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                              Utility Soc Part Conserv Recrsquom

                                              Rationale

                                              Base Percent X X X Program-invariant

                                              Low Income X X X X Multiple sources

                                              Weatherization X X X Substantial Participantimpacts

                                              Measure Program-specific

                                              X Varies by measure sector

                                              Other Recomrsquos Local Research

                                              50

                                              Source Skumatz SERA

                                              Developing values for multiple states amp utilities

                                              SERA

                                              TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                              Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                              Other sources of bias to research (EUL NTG etc) bring high variation

                                              5) States are incorporating NEBs dominos input in deliberations in multiple states

                                              6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                              THANK YOU

                                              Questions

                                              Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                              SERA

                                              POWER OF NEBS ndashBARRIERS DISCONNECTS

                                              53

                                              SERA

                                              NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                              Negative NEB values cost of barrier

                                              Solar Water Heat NZ$ Euros

                                              Solar Design NZ$ Euros

                                              Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                              -23 -12 (079)

                                              -11 -6 (06)

                                              Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                              Source Skumatz Economic Research Associates research

                                              SERA

                                              CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                              NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                              -5-4-3-2-1012345

                                              Op Co

                                              st

                                              Init C

                                              ost

                                              Maint

                                              Perfo

                                              rm

                                              Prod

                                              uctiv

                                              Tena

                                              nt Sa

                                              t

                                              Comf

                                              ort

                                              Appe

                                              ar

                                              Light

                                              Qual

                                              SellL

                                              ease

                                              Devp Ownr AampE

                                              Note AgtE (~4 vs ~2) for these NEBs

                                              Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                              Source Skumatz Economic Research Associates Inc 2002

                                              Owners more positive ndashUnderinvestment train

                                              • Soup to nuts on non-energy benefits NEIs
                                              • TOPICS
                                              • BACKGROUND NEBS
                                              • 20+ YEARS OF NEBS PROGRESShellip
                                              • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                              • KEY APPLICATIONS OF NEBS
                                              • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                              • Neb measurement
                                              • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                              • MEASURING PARTICIPANTNEBs
                                              • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                              • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                              • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                              • PROGRESS TRANSFERABILITY OF NEBS
                                              • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                              • PROGRESS AND GAPS
                                              • PROGRESS AND GAPS
                                              • USING NEB NEI VALUES
                                              • Slide Number 19
                                              • ARE NEBS HIGH VALUE
                                              • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                              • TOP NEBS FOR WEATHERIZATION PROGRAM
                                              • RESULTS FOR ENERGY STARreg APPLIANCES
                                              • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                              • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                              • CampI PROGRAMS ldquoNEBBEDrdquo
                                              • AUDIT FINANCIAL PROGRAM
                                              • CampI NEW CONSTRUCTION
                                              • CampI TECHNICAL ASSISTANCE
                                              • EXPRESSING NEBS VALUEndashCxe
                                              • Slide Number 31
                                              • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                              • EMISSIONS (SOCIETAL NEBS)
                                              • UTILITY NEBS
                                              • ADJUSTED PAYBACKS
                                              • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                              • NEBS IN BC
                                              • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                              • APPLICATION TO BC
                                              • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                              • STATE TREATMENT OF NEBS NEBs
                                              • Slide Number 42
                                              • SELECTED STATE PROGRESS
                                              • SELECTED STATE PROGRESS
                                              • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                              • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                              • DISCOUNT RATE MATTERS
                                              • EUL VARIATIONS- ADOPTED VALUES
                                              • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                              • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                              • TAKEAWAYSON NEBSNEIS
                                              • THANK YOUQuestions
                                              • Slide Number 53
                                              • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                              • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                m Product losses20
                                                l Ease of selling leasing building33
                                                n Labor requirements45
                                                h Aesthetics appearance59
                                                j Noise50
                                                b Equipment maintenance660
                                                f Tenant satisfaction60
                                                d Equipment lifetimes718
                                                e Productivity724
                                                k Building safety[2]73
                                                g Comfort933
                                                i Lighting Quality of Light916
                                                c Equipment performance[1]1019
                                                a Operating costs1165
                                                o Doing good for the Environment[3]1132
                                                Other03
                                                [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                m Product losses
                                                l Ease of selling leasing building
                                                n Labor requirements
                                                h Aesthetics appearance
                                                j Noise
                                                b Equipment maintenance
                                                f Tenant satisfaction
                                                d Equipment lifetimes
                                                e Productivity
                                                k Building safety[2]
                                                g Comfort
                                                i Lighting Quality of Light
                                                c Equipment performance[1]
                                                a Operating costs
                                                o Doing good for the Environment[3]
                                                Share of ZALEH NEBS from various measures
                                                Each presented as multiple of the (overall) energy savings
                                                Super Insulation037
                                                Double Glazing026
                                                Solar Design025
                                                Solar Water Heating005
                                                EnergySmart (insul Retrofit)Dunedin (insul)
                                                Other124
                                                Noise24
                                                Budgetfinances25
                                                Environmental65
                                                Health78
                                                Reduced moves86
                                                Ease of sellingrenting130
                                                DampnessMold1316
                                                Comfort-related4933
                                                101101
                                                Operational NEBs44
                                                DesignConstruction NEBs31
                                                Occupant NEBs25
                                                1
                                                OtherOther
                                                NoiseNoise
                                                BudgetfinancesBudgetfinances
                                                EnvironmentalEnvironmental
                                                HealthHealth
                                                Reduced movesReduced moves
                                                Ease of sellingrentingEase of sellingrenting
                                                DampnessMoldDampnessMold
                                                Comfort-relatedComfort-related
                                                Boiler
                                                VendorNPPartic
                                                Perform0280202
                                                Footprint028013017
                                                ControlFeatures031022016
                                                Tenant compl006008014
                                                Noise005006014
                                                Maint004009007
                                                Other004003007
                                                Lifetime-006018006
                                                MF New construction energy star
                                                Operating cost014
                                                Environment012
                                                Maintenance01
                                                Lifetime01
                                                Performance009
                                                Occupant Satisf009
                                                Comfort008
                                                Noise006
                                                Sellingleasing006
                                                Aesthetics005
                                                Light Quality005
                                                Power QualReliab004
                                                Safety002
                                                RIGHTLIGHTS
                                                Environment023
                                                Quality016
                                                Quantity014
                                                PgmAvail01
                                                Flicker008
                                                Maintenance006
                                                Lifetime006
                                                SalesProductivity006
                                                Safety005
                                                Noise005
                                                Control003
                                                Illness0
                                                PerformPerformPerform
                                                FootprintFootprintFootprint
                                                ControlFeaturesControlFeaturesControlFeatures
                                                Tenant complTenant complTenant compl
                                                NoiseNoiseNoise
                                                MaintMaintMaint
                                                OtherOtherOther
                                                LifetimeLifetimeLifetime
                                                Environment
                                                Quality
                                                Quantity
                                                PgmAvail
                                                Flicker
                                                Maintenance
                                                Lifetime
                                                SalesProductivity
                                                Safety
                                                Noise
                                                Control
                                                Illness

                                                Chart3

                                                023
                                                016
                                                014
                                                01
                                                008
                                                006
                                                006
                                                006
                                                005
                                                005
                                                003
                                                0

                                                Chart1

                                                Vendor
                                                NP
                                                Partic
                                                028
                                                02
                                                02
                                                028
                                                013
                                                017
                                                031
                                                022
                                                016
                                                006
                                                008
                                                014
                                                005
                                                006
                                                014
                                                004
                                                009
                                                007
                                                004
                                                003
                                                007
                                                -006
                                                018
                                                006

                                                Sheet1

                                                Sheet1

                                                Sheet2

                                                Sheet3

                                                SERA

                                                NEB VALUES

                                                19

                                                SERA

                                                ARE NEBS HIGH VALUE Energy savings are often the minority of

                                                benefits from program measuresndash Donrsquot ignore that

                                                20

                                                0 05 1 15 2 25 3

                                                Energy Sav

                                                NEBs

                                                NEB vs Energy Savings ValueIncluding all NEBs

                                                Source (SkumatzSERA 2010 amp others)

                                                Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                                SERA

                                                RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                                MF Many others

                                                Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                                Variety of end uses

                                                SERA

                                                TOP NEBS FOR WEATHERIZATION PROGRAM

                                                Weatherization program multipliergt1

                                                Source SERA Research

                                                SERA

                                                RESULTS FOR ENERGY STARreg APPLIANCES

                                                0002004006008

                                                01012014016018

                                                Maint

                                                Appear

                                                Perf LifeNoise Satis

                                                Comfort

                                                LiteSafe

                                                ty SellMove

                                                Water

                                                Enviro

                                                Refrig DW CW

                                                Source SERA Research

                                                SERA

                                                INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                                5 increments to 50

                                                Source SERA research

                                                Chart5

                                                EnergySmart (insul Retrofit)
                                                Dunedin (insul)
                                                001
                                                024
                                                002
                                                004
                                                002
                                                005
                                                006
                                                005
                                                007
                                                008
                                                008
                                                006
                                                013
                                                0
                                                013
                                                016
                                                049
                                                033

                                                cx

                                                cx

                                                nz

                                                Sheet3

                                                Sheet3

                                                Super Insulation
                                                Double Glazing
                                                Solar Design
                                                Solar Water Heating
                                                Super Insulation
                                                Double Glazing
                                                Solar Design
                                                Solar Water Heating
                                                EnergySmart (insul Retrofit)
                                                Dunedin (insul)

                                                SERA

                                                ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                0

                                                002

                                                004

                                                006

                                                008

                                                01

                                                012

                                                Maint

                                                Perf LifeSatis

                                                f

                                                ComfortAesth

                                                NoiseSafe

                                                ty

                                                LightingSelli

                                                ng

                                                Stay in home

                                                Enviro

                                                Sick Days

                                                Calls

                                                ES Homes HP wES Energy savings multiplier=1

                                                Source SERA Research

                                                SERA

                                                CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                DG CHP HVAC Equipment rebate Other

                                                Building codes incentives by cities

                                                Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                Variety of end uses

                                                SERA

                                                AUDIT FINANCIAL PROGRAM

                                                0

                                                01

                                                02

                                                03

                                                04

                                                05

                                                06

                                                07

                                                Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                Source SERA Research

                                                SERA

                                                CampI NEW CONSTRUCTION

                                                -005

                                                0

                                                005

                                                01

                                                015

                                                02

                                                AampE Owner(Pct of Total Partic NEBs)

                                                Source SERA Research

                                                SERA

                                                CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                0 2 4 6 8 10 12

                                                m Product losses

                                                l Ease of selling leasing building

                                                n Labor requirements

                                                h Aesthetics appearance

                                                j Noise

                                                b Equipment maintenance

                                                f Tenant satisfaction

                                                d Equipment lifetimes

                                                e Productivity

                                                k Building safety[2]

                                                g Comfort

                                                i Lighting Quality of Light

                                                c Equipment performance[1]

                                                a Operating costs

                                                o Doing good for the Environment[3]

                                                Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                vcv

                                                Source SERA Research

                                                Chart1

                                                CampI Technical Assistance NEBs
                                                002
                                                003
                                                004
                                                005
                                                005
                                                006
                                                006
                                                007
                                                007
                                                007
                                                009
                                                009
                                                01
                                                011
                                                011

                                                Sheet1

                                                Sheet1

                                                CampI Technical Assistance Program NEBs (Participant)

                                                Sheet2

                                                Sheet3

                                                SERA

                                                EXPRESSING NEBS VALUEndashCxe

                                                Strong value from RetroCxSource SERA Research

                                                SERA

                                                SOCIETAL NEB EXAMPLES

                                                31

                                                SERA

                                                JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                32

                                                0

                                                02

                                                04

                                                06

                                                08

                                                1

                                                12

                                                HPWxRetrof Appliance

                                                CA

                                                WI

                                                Natl

                                                (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                Jobs Economic

                                                SERA

                                                EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                33

                                                SERA

                                                UTILITY NEBSUtility NEBs for Template Program

                                                Debt WriteOff (util)13

                                                Rate Subsidy(util)61

                                                HealthSafety(util)0

                                                Colln Costs (util)0

                                                Gas Calls (util)0

                                                Calls to CSRs(util)2

                                                TampD (util)16

                                                Arrears (util)0

                                                Reconnects (util)0

                                                Notices (util)7

                                                Shutoffs (util)1

                                                Rate subsidy TampD

                                                Payment-related

                                                Source Skumatz Economic Research Associates research

                                                Example Low Income Weatherization

                                                SERA

                                                ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                Value of 100 HALVES Payback doubles ROI

                                                Source Skumatz Economics (SERA)

                                                35

                                                ( for NEBs Multiplier of 125)

                                                SERA

                                                NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                36

                                                hellip2001 ON

                                                SERA

                                                37

                                                NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                For true representation of B amp C elements of NEBs address missing factor bias

                                                Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                SERA

                                                CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                38

                                                Test Benefits Costs States Using Traditionally

                                                Improved treat-ment with NEBs

                                                Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                bull Avoided gas and water supply costs

                                                bull Program administration

                                                bull Participant incentives

                                                bull Increased supply cost

                                                CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                Use cost only paid by the utility

                                                Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                Same as above plus bull increased revenue

                                                Same as above plus bull Decreased revenue

                                                AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                Participant cost bull Utility bill reductions bull Participant incentives

                                                bull Participant direct costs

                                                AR CA FL HI IA IN MI MN NY VA

                                                Participant NEBs

                                                Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                SERA

                                                APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                Expansion Concern with risk calculation accuracy

                                                39

                                                Source Skumatz SERA research

                                                SERA

                                                40

                                                METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                Maximize DSM opportunities amp feedback Accuracy tailoring

                                                Minimize Regulatory amp Implementer Risk

                                                Minimize Evaluation Cost

                                                Adder

                                                Readily MeasurableHybrid

                                                All NEBsSource SERA Research

                                                SERA

                                                41

                                                STATE TREATMENT OF NEBS NEBsRegulatory

                                                Screening Application Utilities regions

                                                Program Marketing Fairly widespread use in utilities states across the country

                                                MO

                                                RE AGGRESSIVE ==gt

                                                Test Pgm Screen - adder

                                                IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                Test Pgm Screen - readily measured

                                                MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                Test Hybrid (potential adder amp measured)

                                                CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                Test Pgm screen - Broad

                                                With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                SERA

                                                42

                                                Key

                                                States with NEBs in CE tests for At least one type of program

                                                copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                STATES WITH NEBS IN CE TESTS

                                                NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                SERA

                                                SELECTED STATE PROGRESS CA

                                                2001 ndash LIPPT model NY

                                                Mid-2000s measurement amp scenarios not included CO

                                                Adders (10 electric 25 LI 5 gas) VT

                                                15 adder helped by previous research DC

                                                10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                43

                                                Source Skumatz SERA research

                                                SERA

                                                SELECTED STATE PROGRESS Dominos ongoing

                                                Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                44

                                                Source Skumatz SERA research

                                                SERA

                                                APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                Expansion Concern with risk calculation accuracy

                                                45

                                                Source Skumatz SERA research

                                                SERA

                                                46

                                                NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                SourceSkumatz SERA Independent research

                                                NTG ndash accuracy measurement incomplete

                                                Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                Cost Complicated expensive local changes

                                                RELATIVE Risk from NEBs

                                                Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                NEBs as accurate as many inputs (all could improve)

                                                SERA

                                                DISCOUNT RATE MATTERS

                                                47

                                                Source Skumatz SERA Research

                                                0

                                                20

                                                40

                                                60

                                                80

                                                100

                                                120

                                                1 2 3 4 5 6 7 8 9 10

                                                PV Ratios vs 1 Discount Rate

                                                10 yr PV 20 yr PV

                                                Discount rate - percent

                                                WACCSocialLTT

                                                Source Skumatz Electricity Journal IEPEC paper independent research

                                                SERA

                                                EUL VARIATIONS-ADOPTED VALUES

                                                Source Skumatz Economic Research Associates SERA independent research

                                                0

                                                50

                                                100

                                                150

                                                200

                                                250

                                                300

                                                350

                                                400

                                                450

                                                0 10 20 30 40 50 60 70

                                                EUL Variation by Measure

                                                67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                Poorly supportedOldTechnology change

                                                SERA

                                                KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                change the BC conclusion

                                                On the cusphellip

                                                If NO Yoursquore doneAnd bias addressedsufficiently

                                                IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                Source SERA all rights reserved

                                                SERArsquos lsquoNEB-ItrdquoModel ndash

                                                SERA

                                                IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                Utility Soc Part Conserv Recrsquom

                                                Rationale

                                                Base Percent X X X Program-invariant

                                                Low Income X X X X Multiple sources

                                                Weatherization X X X Substantial Participantimpacts

                                                Measure Program-specific

                                                X Varies by measure sector

                                                Other Recomrsquos Local Research

                                                50

                                                Source Skumatz SERA

                                                Developing values for multiple states amp utilities

                                                SERA

                                                TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                Other sources of bias to research (EUL NTG etc) bring high variation

                                                5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                THANK YOU

                                                Questions

                                                Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                SERA

                                                POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                53

                                                SERA

                                                NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                Negative NEB values cost of barrier

                                                Solar Water Heat NZ$ Euros

                                                Solar Design NZ$ Euros

                                                Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                -23 -12 (079)

                                                -11 -6 (06)

                                                Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                Source Skumatz Economic Research Associates research

                                                SERA

                                                CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                -5-4-3-2-1012345

                                                Op Co

                                                st

                                                Init C

                                                ost

                                                Maint

                                                Perfo

                                                rm

                                                Prod

                                                uctiv

                                                Tena

                                                nt Sa

                                                t

                                                Comf

                                                ort

                                                Appe

                                                ar

                                                Light

                                                Qual

                                                SellL

                                                ease

                                                Devp Ownr AampE

                                                Note AgtE (~4 vs ~2) for these NEBs

                                                Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                Source Skumatz Economic Research Associates Inc 2002

                                                Owners more positive ndashUnderinvestment train

                                                • Soup to nuts on non-energy benefits NEIs
                                                • TOPICS
                                                • BACKGROUND NEBS
                                                • 20+ YEARS OF NEBS PROGRESShellip
                                                • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                • KEY APPLICATIONS OF NEBS
                                                • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                • Neb measurement
                                                • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                • MEASURING PARTICIPANTNEBs
                                                • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                • PROGRESS TRANSFERABILITY OF NEBS
                                                • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                • PROGRESS AND GAPS
                                                • PROGRESS AND GAPS
                                                • USING NEB NEI VALUES
                                                • Slide Number 19
                                                • ARE NEBS HIGH VALUE
                                                • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                • RESULTS FOR ENERGY STARreg APPLIANCES
                                                • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                • CampI PROGRAMS ldquoNEBBEDrdquo
                                                • AUDIT FINANCIAL PROGRAM
                                                • CampI NEW CONSTRUCTION
                                                • CampI TECHNICAL ASSISTANCE
                                                • EXPRESSING NEBS VALUEndashCxe
                                                • Slide Number 31
                                                • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                • EMISSIONS (SOCIETAL NEBS)
                                                • UTILITY NEBS
                                                • ADJUSTED PAYBACKS
                                                • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                • NEBS IN BC
                                                • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                • APPLICATION TO BC
                                                • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                • STATE TREATMENT OF NEBS NEBs
                                                • Slide Number 42
                                                • SELECTED STATE PROGRESS
                                                • SELECTED STATE PROGRESS
                                                • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                • DISCOUNT RATE MATTERS
                                                • EUL VARIATIONS- ADOPTED VALUES
                                                • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                • TAKEAWAYSON NEBSNEIS
                                                • THANK YOUQuestions
                                                • Slide Number 53
                                                • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                  Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                  m Product losses20
                                                  l Ease of selling leasing building33
                                                  n Labor requirements45
                                                  h Aesthetics appearance59
                                                  j Noise50
                                                  b Equipment maintenance660
                                                  f Tenant satisfaction60
                                                  d Equipment lifetimes718
                                                  e Productivity724
                                                  k Building safety[2]73
                                                  g Comfort933
                                                  i Lighting Quality of Light916
                                                  c Equipment performance[1]1019
                                                  a Operating costs1165
                                                  o Doing good for the Environment[3]1132
                                                  Other03
                                                  [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                  [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                  [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                  m Product losses
                                                  l Ease of selling leasing building
                                                  n Labor requirements
                                                  h Aesthetics appearance
                                                  j Noise
                                                  b Equipment maintenance
                                                  f Tenant satisfaction
                                                  d Equipment lifetimes
                                                  e Productivity
                                                  k Building safety[2]
                                                  g Comfort
                                                  i Lighting Quality of Light
                                                  c Equipment performance[1]
                                                  a Operating costs
                                                  o Doing good for the Environment[3]
                                                  Share of ZALEH NEBS from various measures
                                                  Each presented as multiple of the (overall) energy savings
                                                  Super Insulation037
                                                  Double Glazing026
                                                  Solar Design025
                                                  Solar Water Heating005
                                                  EnergySmart (insul Retrofit)Dunedin (insul)
                                                  Other124
                                                  Noise24
                                                  Budgetfinances25
                                                  Environmental65
                                                  Health78
                                                  Reduced moves86
                                                  Ease of sellingrenting130
                                                  DampnessMold1316
                                                  Comfort-related4933
                                                  101101
                                                  Operational NEBs44
                                                  DesignConstruction NEBs31
                                                  Occupant NEBs25
                                                  1
                                                  OtherOther
                                                  NoiseNoise
                                                  BudgetfinancesBudgetfinances
                                                  EnvironmentalEnvironmental
                                                  HealthHealth
                                                  Reduced movesReduced moves
                                                  Ease of sellingrentingEase of sellingrenting
                                                  DampnessMoldDampnessMold
                                                  Comfort-relatedComfort-related
                                                  Boiler
                                                  VendorNPPartic
                                                  Perform0280202
                                                  Footprint028013017
                                                  ControlFeatures031022016
                                                  Tenant compl006008014
                                                  Noise005006014
                                                  Maint004009007
                                                  Other004003007
                                                  Lifetime-006018006
                                                  MF New construction energy star
                                                  Operating cost014
                                                  Environment012
                                                  Maintenance01
                                                  Lifetime01
                                                  Performance009
                                                  Occupant Satisf009
                                                  Comfort008
                                                  Noise006
                                                  Sellingleasing006
                                                  Aesthetics005
                                                  Light Quality005
                                                  Power QualReliab004
                                                  Safety002
                                                  RIGHTLIGHTS
                                                  Environment023
                                                  Quality016
                                                  Quantity014
                                                  PgmAvail01
                                                  Flicker008
                                                  Maintenance006
                                                  Lifetime006
                                                  SalesProductivity006
                                                  Safety005
                                                  Noise005
                                                  Control003
                                                  Illness0
                                                  PerformPerformPerform
                                                  FootprintFootprintFootprint
                                                  ControlFeaturesControlFeaturesControlFeatures
                                                  Tenant complTenant complTenant compl
                                                  NoiseNoiseNoise
                                                  MaintMaintMaint
                                                  OtherOtherOther
                                                  LifetimeLifetimeLifetime
                                                  Environment
                                                  Quality
                                                  Quantity
                                                  PgmAvail
                                                  Flicker
                                                  Maintenance
                                                  Lifetime
                                                  SalesProductivity
                                                  Safety
                                                  Noise
                                                  Control
                                                  Illness

                                                  Chart1

                                                  Vendor
                                                  NP
                                                  Partic
                                                  028
                                                  02
                                                  02
                                                  028
                                                  013
                                                  017
                                                  031
                                                  022
                                                  016
                                                  006
                                                  008
                                                  014
                                                  005
                                                  006
                                                  014
                                                  004
                                                  009
                                                  007
                                                  004
                                                  003
                                                  007
                                                  -006
                                                  018
                                                  006

                                                  Sheet1

                                                  Sheet1

                                                  Sheet2

                                                  Sheet3

                                                  SERA

                                                  NEB VALUES

                                                  19

                                                  SERA

                                                  ARE NEBS HIGH VALUE Energy savings are often the minority of

                                                  benefits from program measuresndash Donrsquot ignore that

                                                  20

                                                  0 05 1 15 2 25 3

                                                  Energy Sav

                                                  NEBs

                                                  NEB vs Energy Savings ValueIncluding all NEBs

                                                  Source (SkumatzSERA 2010 amp others)

                                                  Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                                  SERA

                                                  RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                                  MF Many others

                                                  Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                                  Variety of end uses

                                                  SERA

                                                  TOP NEBS FOR WEATHERIZATION PROGRAM

                                                  Weatherization program multipliergt1

                                                  Source SERA Research

                                                  SERA

                                                  RESULTS FOR ENERGY STARreg APPLIANCES

                                                  0002004006008

                                                  01012014016018

                                                  Maint

                                                  Appear

                                                  Perf LifeNoise Satis

                                                  Comfort

                                                  LiteSafe

                                                  ty SellMove

                                                  Water

                                                  Enviro

                                                  Refrig DW CW

                                                  Source SERA Research

                                                  SERA

                                                  INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                                  5 increments to 50

                                                  Source SERA research

                                                  Chart5

                                                  EnergySmart (insul Retrofit)
                                                  Dunedin (insul)
                                                  001
                                                  024
                                                  002
                                                  004
                                                  002
                                                  005
                                                  006
                                                  005
                                                  007
                                                  008
                                                  008
                                                  006
                                                  013
                                                  0
                                                  013
                                                  016
                                                  049
                                                  033

                                                  cx

                                                  cx

                                                  nz

                                                  Sheet3

                                                  Sheet3

                                                  Super Insulation
                                                  Double Glazing
                                                  Solar Design
                                                  Solar Water Heating
                                                  Super Insulation
                                                  Double Glazing
                                                  Solar Design
                                                  Solar Water Heating
                                                  EnergySmart (insul Retrofit)
                                                  Dunedin (insul)

                                                  SERA

                                                  ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                  0

                                                  002

                                                  004

                                                  006

                                                  008

                                                  01

                                                  012

                                                  Maint

                                                  Perf LifeSatis

                                                  f

                                                  ComfortAesth

                                                  NoiseSafe

                                                  ty

                                                  LightingSelli

                                                  ng

                                                  Stay in home

                                                  Enviro

                                                  Sick Days

                                                  Calls

                                                  ES Homes HP wES Energy savings multiplier=1

                                                  Source SERA Research

                                                  SERA

                                                  CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                  DG CHP HVAC Equipment rebate Other

                                                  Building codes incentives by cities

                                                  Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                  Variety of end uses

                                                  SERA

                                                  AUDIT FINANCIAL PROGRAM

                                                  0

                                                  01

                                                  02

                                                  03

                                                  04

                                                  05

                                                  06

                                                  07

                                                  Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                  Source SERA Research

                                                  SERA

                                                  CampI NEW CONSTRUCTION

                                                  -005

                                                  0

                                                  005

                                                  01

                                                  015

                                                  02

                                                  AampE Owner(Pct of Total Partic NEBs)

                                                  Source SERA Research

                                                  SERA

                                                  CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                  0 2 4 6 8 10 12

                                                  m Product losses

                                                  l Ease of selling leasing building

                                                  n Labor requirements

                                                  h Aesthetics appearance

                                                  j Noise

                                                  b Equipment maintenance

                                                  f Tenant satisfaction

                                                  d Equipment lifetimes

                                                  e Productivity

                                                  k Building safety[2]

                                                  g Comfort

                                                  i Lighting Quality of Light

                                                  c Equipment performance[1]

                                                  a Operating costs

                                                  o Doing good for the Environment[3]

                                                  Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                  vcv

                                                  Source SERA Research

                                                  Chart1

                                                  CampI Technical Assistance NEBs
                                                  002
                                                  003
                                                  004
                                                  005
                                                  005
                                                  006
                                                  006
                                                  007
                                                  007
                                                  007
                                                  009
                                                  009
                                                  01
                                                  011
                                                  011

                                                  Sheet1

                                                  Sheet1

                                                  CampI Technical Assistance Program NEBs (Participant)

                                                  Sheet2

                                                  Sheet3

                                                  SERA

                                                  EXPRESSING NEBS VALUEndashCxe

                                                  Strong value from RetroCxSource SERA Research

                                                  SERA

                                                  SOCIETAL NEB EXAMPLES

                                                  31

                                                  SERA

                                                  JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                  32

                                                  0

                                                  02

                                                  04

                                                  06

                                                  08

                                                  1

                                                  12

                                                  HPWxRetrof Appliance

                                                  CA

                                                  WI

                                                  Natl

                                                  (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                  Jobs Economic

                                                  SERA

                                                  EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                  33

                                                  SERA

                                                  UTILITY NEBSUtility NEBs for Template Program

                                                  Debt WriteOff (util)13

                                                  Rate Subsidy(util)61

                                                  HealthSafety(util)0

                                                  Colln Costs (util)0

                                                  Gas Calls (util)0

                                                  Calls to CSRs(util)2

                                                  TampD (util)16

                                                  Arrears (util)0

                                                  Reconnects (util)0

                                                  Notices (util)7

                                                  Shutoffs (util)1

                                                  Rate subsidy TampD

                                                  Payment-related

                                                  Source Skumatz Economic Research Associates research

                                                  Example Low Income Weatherization

                                                  SERA

                                                  ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                  be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                  Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                  Value of 100 HALVES Payback doubles ROI

                                                  Source Skumatz Economics (SERA)

                                                  35

                                                  ( for NEBs Multiplier of 125)

                                                  SERA

                                                  NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                  36

                                                  hellip2001 ON

                                                  SERA

                                                  37

                                                  NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                  For true representation of B amp C elements of NEBs address missing factor bias

                                                  Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                  BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                  SERA

                                                  CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                  38

                                                  Test Benefits Costs States Using Traditionally

                                                  Improved treat-ment with NEBs

                                                  Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                  bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                  bull Avoided gas and water supply costs

                                                  bull Program administration

                                                  bull Participant incentives

                                                  bull Increased supply cost

                                                  CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                  Use cost only paid by the utility

                                                  Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                  Same as above plus bull increased revenue

                                                  Same as above plus bull Decreased revenue

                                                  AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                  Participant cost bull Utility bill reductions bull Participant incentives

                                                  bull Participant direct costs

                                                  AR CA FL HI IA IN MI MN NY VA

                                                  Participant NEBs

                                                  Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                  Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                  SERA

                                                  APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                  Expansion Concern with risk calculation accuracy

                                                  39

                                                  Source Skumatz SERA research

                                                  SERA

                                                  40

                                                  METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                  Maximize DSM opportunities amp feedback Accuracy tailoring

                                                  Minimize Regulatory amp Implementer Risk

                                                  Minimize Evaluation Cost

                                                  Adder

                                                  Readily MeasurableHybrid

                                                  All NEBsSource SERA Research

                                                  SERA

                                                  41

                                                  STATE TREATMENT OF NEBS NEBsRegulatory

                                                  Screening Application Utilities regions

                                                  Program Marketing Fairly widespread use in utilities states across the country

                                                  MO

                                                  RE AGGRESSIVE ==gt

                                                  Test Pgm Screen - adder

                                                  IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                  Test Pgm Screen - readily measured

                                                  MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                  Test Hybrid (potential adder amp measured)

                                                  CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                  Test Pgm screen - Broad

                                                  With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                  Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                  SERA

                                                  42

                                                  Key

                                                  States with NEBs in CE tests for At least one type of program

                                                  copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                  STATES WITH NEBS IN CE TESTS

                                                  NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                  groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                  SERA

                                                  SELECTED STATE PROGRESS CA

                                                  2001 ndash LIPPT model NY

                                                  Mid-2000s measurement amp scenarios not included CO

                                                  Adders (10 electric 25 LI 5 gas) VT

                                                  15 adder helped by previous research DC

                                                  10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                  43

                                                  Source Skumatz SERA research

                                                  SERA

                                                  SELECTED STATE PROGRESS Dominos ongoing

                                                  Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                  Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                  Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                  Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                  44

                                                  Source Skumatz SERA research

                                                  SERA

                                                  APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                  Expansion Concern with risk calculation accuracy

                                                  45

                                                  Source Skumatz SERA research

                                                  SERA

                                                  46

                                                  NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                  [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                  SourceSkumatz SERA Independent research

                                                  NTG ndash accuracy measurement incomplete

                                                  Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                  NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                  EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                  Cost Complicated expensive local changes

                                                  RELATIVE Risk from NEBs

                                                  Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                  Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                  NEBs as accurate as many inputs (all could improve)

                                                  SERA

                                                  DISCOUNT RATE MATTERS

                                                  47

                                                  Source Skumatz SERA Research

                                                  0

                                                  20

                                                  40

                                                  60

                                                  80

                                                  100

                                                  120

                                                  1 2 3 4 5 6 7 8 9 10

                                                  PV Ratios vs 1 Discount Rate

                                                  10 yr PV 20 yr PV

                                                  Discount rate - percent

                                                  WACCSocialLTT

                                                  Source Skumatz Electricity Journal IEPEC paper independent research

                                                  SERA

                                                  EUL VARIATIONS-ADOPTED VALUES

                                                  Source Skumatz Economic Research Associates SERA independent research

                                                  0

                                                  50

                                                  100

                                                  150

                                                  200

                                                  250

                                                  300

                                                  350

                                                  400

                                                  450

                                                  0 10 20 30 40 50 60 70

                                                  EUL Variation by Measure

                                                  67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                  Poorly supportedOldTechnology change

                                                  SERA

                                                  KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                  Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                  measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                  change the BC conclusion

                                                  On the cusphellip

                                                  If NO Yoursquore doneAnd bias addressedsufficiently

                                                  IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                  Source SERA all rights reserved

                                                  SERArsquos lsquoNEB-ItrdquoModel ndash

                                                  SERA

                                                  IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                  Utility Soc Part Conserv Recrsquom

                                                  Rationale

                                                  Base Percent X X X Program-invariant

                                                  Low Income X X X X Multiple sources

                                                  Weatherization X X X Substantial Participantimpacts

                                                  Measure Program-specific

                                                  X Varies by measure sector

                                                  Other Recomrsquos Local Research

                                                  50

                                                  Source Skumatz SERA

                                                  Developing values for multiple states amp utilities

                                                  SERA

                                                  TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                  Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                  Other sources of bias to research (EUL NTG etc) bring high variation

                                                  5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                  6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                  THANK YOU

                                                  Questions

                                                  Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                  SERA

                                                  POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                  53

                                                  SERA

                                                  NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                  Negative NEB values cost of barrier

                                                  Solar Water Heat NZ$ Euros

                                                  Solar Design NZ$ Euros

                                                  Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                  -23 -12 (079)

                                                  -11 -6 (06)

                                                  Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                  Source Skumatz Economic Research Associates research

                                                  SERA

                                                  CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                  NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                  -5-4-3-2-1012345

                                                  Op Co

                                                  st

                                                  Init C

                                                  ost

                                                  Maint

                                                  Perfo

                                                  rm

                                                  Prod

                                                  uctiv

                                                  Tena

                                                  nt Sa

                                                  t

                                                  Comf

                                                  ort

                                                  Appe

                                                  ar

                                                  Light

                                                  Qual

                                                  SellL

                                                  ease

                                                  Devp Ownr AampE

                                                  Note AgtE (~4 vs ~2) for these NEBs

                                                  Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                  Source Skumatz Economic Research Associates Inc 2002

                                                  Owners more positive ndashUnderinvestment train

                                                  • Soup to nuts on non-energy benefits NEIs
                                                  • TOPICS
                                                  • BACKGROUND NEBS
                                                  • 20+ YEARS OF NEBS PROGRESShellip
                                                  • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                  • KEY APPLICATIONS OF NEBS
                                                  • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                  • Neb measurement
                                                  • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                  • MEASURING PARTICIPANTNEBs
                                                  • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                  • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                  • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                  • PROGRESS TRANSFERABILITY OF NEBS
                                                  • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                  • PROGRESS AND GAPS
                                                  • PROGRESS AND GAPS
                                                  • USING NEB NEI VALUES
                                                  • Slide Number 19
                                                  • ARE NEBS HIGH VALUE
                                                  • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                  • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                  • RESULTS FOR ENERGY STARreg APPLIANCES
                                                  • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                  • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                  • CampI PROGRAMS ldquoNEBBEDrdquo
                                                  • AUDIT FINANCIAL PROGRAM
                                                  • CampI NEW CONSTRUCTION
                                                  • CampI TECHNICAL ASSISTANCE
                                                  • EXPRESSING NEBS VALUEndashCxe
                                                  • Slide Number 31
                                                  • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                  • EMISSIONS (SOCIETAL NEBS)
                                                  • UTILITY NEBS
                                                  • ADJUSTED PAYBACKS
                                                  • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                  • NEBS IN BC
                                                  • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                  • APPLICATION TO BC
                                                  • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                  • STATE TREATMENT OF NEBS NEBs
                                                  • Slide Number 42
                                                  • SELECTED STATE PROGRESS
                                                  • SELECTED STATE PROGRESS
                                                  • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                  • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                  • DISCOUNT RATE MATTERS
                                                  • EUL VARIATIONS- ADOPTED VALUES
                                                  • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                  • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                  • TAKEAWAYSON NEBSNEIS
                                                  • THANK YOUQuestions
                                                  • Slide Number 53
                                                  • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                  • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                    Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                    m Product losses20
                                                    l Ease of selling leasing building33
                                                    n Labor requirements45
                                                    h Aesthetics appearance59
                                                    j Noise50
                                                    b Equipment maintenance660
                                                    f Tenant satisfaction60
                                                    d Equipment lifetimes718
                                                    e Productivity724
                                                    k Building safety[2]73
                                                    g Comfort933
                                                    i Lighting Quality of Light916
                                                    c Equipment performance[1]1019
                                                    a Operating costs1165
                                                    o Doing good for the Environment[3]1132
                                                    Other03
                                                    [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                    [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                    [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                    m Product losses
                                                    l Ease of selling leasing building
                                                    n Labor requirements
                                                    h Aesthetics appearance
                                                    j Noise
                                                    b Equipment maintenance
                                                    f Tenant satisfaction
                                                    d Equipment lifetimes
                                                    e Productivity
                                                    k Building safety[2]
                                                    g Comfort
                                                    i Lighting Quality of Light
                                                    c Equipment performance[1]
                                                    a Operating costs
                                                    o Doing good for the Environment[3]
                                                    Share of ZALEH NEBS from various measures
                                                    Each presented as multiple of the (overall) energy savings
                                                    Super Insulation037
                                                    Double Glazing026
                                                    Solar Design025
                                                    Solar Water Heating005
                                                    EnergySmart (insul Retrofit)Dunedin (insul)
                                                    Other124
                                                    Noise24
                                                    Budgetfinances25
                                                    Environmental65
                                                    Health78
                                                    Reduced moves86
                                                    Ease of sellingrenting130
                                                    DampnessMold1316
                                                    Comfort-related4933
                                                    101101
                                                    Operational NEBs44
                                                    DesignConstruction NEBs31
                                                    Occupant NEBs25
                                                    1
                                                    OtherOther
                                                    NoiseNoise
                                                    BudgetfinancesBudgetfinances
                                                    EnvironmentalEnvironmental
                                                    HealthHealth
                                                    Reduced movesReduced moves
                                                    Ease of sellingrentingEase of sellingrenting
                                                    DampnessMoldDampnessMold
                                                    Comfort-relatedComfort-related
                                                    Boiler
                                                    VendorNPPartic
                                                    Perform0280202
                                                    Footprint028013017
                                                    ControlFeatures031022016
                                                    Tenant compl006008014
                                                    Noise005006014
                                                    Maint004009007
                                                    Other004003007
                                                    Lifetime-006018006
                                                    MF New construction energy star
                                                    Operating cost014
                                                    Environment012
                                                    Maintenance01
                                                    Lifetime01
                                                    Performance009
                                                    Occupant Satisf009
                                                    Comfort008
                                                    Noise006
                                                    Sellingleasing006
                                                    Aesthetics005
                                                    Light Quality005
                                                    Power QualReliab004
                                                    Safety002
                                                    RIGHTLIGHTS
                                                    Environment023
                                                    Quality016
                                                    Quantity014
                                                    PgmAvail01
                                                    Flicker008
                                                    Maintenance006
                                                    Lifetime006
                                                    SalesProductivity006
                                                    Safety005
                                                    Noise005
                                                    Control003
                                                    Illness0
                                                    PerformPerformPerform
                                                    FootprintFootprintFootprint
                                                    ControlFeaturesControlFeaturesControlFeatures
                                                    Tenant complTenant complTenant compl
                                                    NoiseNoiseNoise
                                                    MaintMaintMaint
                                                    OtherOtherOther
                                                    LifetimeLifetimeLifetime

                                                    Sheet1

                                                    Sheet1

                                                    Sheet2

                                                    Sheet3

                                                    SERA

                                                    NEB VALUES

                                                    19

                                                    SERA

                                                    ARE NEBS HIGH VALUE Energy savings are often the minority of

                                                    benefits from program measuresndash Donrsquot ignore that

                                                    20

                                                    0 05 1 15 2 25 3

                                                    Energy Sav

                                                    NEBs

                                                    NEB vs Energy Savings ValueIncluding all NEBs

                                                    Source (SkumatzSERA 2010 amp others)

                                                    Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                                    SERA

                                                    RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                                    MF Many others

                                                    Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                                    Variety of end uses

                                                    SERA

                                                    TOP NEBS FOR WEATHERIZATION PROGRAM

                                                    Weatherization program multipliergt1

                                                    Source SERA Research

                                                    SERA

                                                    RESULTS FOR ENERGY STARreg APPLIANCES

                                                    0002004006008

                                                    01012014016018

                                                    Maint

                                                    Appear

                                                    Perf LifeNoise Satis

                                                    Comfort

                                                    LiteSafe

                                                    ty SellMove

                                                    Water

                                                    Enviro

                                                    Refrig DW CW

                                                    Source SERA Research

                                                    SERA

                                                    INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                                    5 increments to 50

                                                    Source SERA research

                                                    Chart5

                                                    EnergySmart (insul Retrofit)
                                                    Dunedin (insul)
                                                    001
                                                    024
                                                    002
                                                    004
                                                    002
                                                    005
                                                    006
                                                    005
                                                    007
                                                    008
                                                    008
                                                    006
                                                    013
                                                    0
                                                    013
                                                    016
                                                    049
                                                    033

                                                    cx

                                                    cx

                                                    nz

                                                    Sheet3

                                                    Sheet3

                                                    Super Insulation
                                                    Double Glazing
                                                    Solar Design
                                                    Solar Water Heating
                                                    Super Insulation
                                                    Double Glazing
                                                    Solar Design
                                                    Solar Water Heating
                                                    EnergySmart (insul Retrofit)
                                                    Dunedin (insul)

                                                    SERA

                                                    ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                    0

                                                    002

                                                    004

                                                    006

                                                    008

                                                    01

                                                    012

                                                    Maint

                                                    Perf LifeSatis

                                                    f

                                                    ComfortAesth

                                                    NoiseSafe

                                                    ty

                                                    LightingSelli

                                                    ng

                                                    Stay in home

                                                    Enviro

                                                    Sick Days

                                                    Calls

                                                    ES Homes HP wES Energy savings multiplier=1

                                                    Source SERA Research

                                                    SERA

                                                    CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                    DG CHP HVAC Equipment rebate Other

                                                    Building codes incentives by cities

                                                    Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                    Variety of end uses

                                                    SERA

                                                    AUDIT FINANCIAL PROGRAM

                                                    0

                                                    01

                                                    02

                                                    03

                                                    04

                                                    05

                                                    06

                                                    07

                                                    Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                    Source SERA Research

                                                    SERA

                                                    CampI NEW CONSTRUCTION

                                                    -005

                                                    0

                                                    005

                                                    01

                                                    015

                                                    02

                                                    AampE Owner(Pct of Total Partic NEBs)

                                                    Source SERA Research

                                                    SERA

                                                    CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                    0 2 4 6 8 10 12

                                                    m Product losses

                                                    l Ease of selling leasing building

                                                    n Labor requirements

                                                    h Aesthetics appearance

                                                    j Noise

                                                    b Equipment maintenance

                                                    f Tenant satisfaction

                                                    d Equipment lifetimes

                                                    e Productivity

                                                    k Building safety[2]

                                                    g Comfort

                                                    i Lighting Quality of Light

                                                    c Equipment performance[1]

                                                    a Operating costs

                                                    o Doing good for the Environment[3]

                                                    Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                    vcv

                                                    Source SERA Research

                                                    Chart1

                                                    CampI Technical Assistance NEBs
                                                    002
                                                    003
                                                    004
                                                    005
                                                    005
                                                    006
                                                    006
                                                    007
                                                    007
                                                    007
                                                    009
                                                    009
                                                    01
                                                    011
                                                    011

                                                    Sheet1

                                                    Sheet1

                                                    CampI Technical Assistance Program NEBs (Participant)

                                                    Sheet2

                                                    Sheet3

                                                    SERA

                                                    EXPRESSING NEBS VALUEndashCxe

                                                    Strong value from RetroCxSource SERA Research

                                                    SERA

                                                    SOCIETAL NEB EXAMPLES

                                                    31

                                                    SERA

                                                    JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                    32

                                                    0

                                                    02

                                                    04

                                                    06

                                                    08

                                                    1

                                                    12

                                                    HPWxRetrof Appliance

                                                    CA

                                                    WI

                                                    Natl

                                                    (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                    Jobs Economic

                                                    SERA

                                                    EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                    33

                                                    SERA

                                                    UTILITY NEBSUtility NEBs for Template Program

                                                    Debt WriteOff (util)13

                                                    Rate Subsidy(util)61

                                                    HealthSafety(util)0

                                                    Colln Costs (util)0

                                                    Gas Calls (util)0

                                                    Calls to CSRs(util)2

                                                    TampD (util)16

                                                    Arrears (util)0

                                                    Reconnects (util)0

                                                    Notices (util)7

                                                    Shutoffs (util)1

                                                    Rate subsidy TampD

                                                    Payment-related

                                                    Source Skumatz Economic Research Associates research

                                                    Example Low Income Weatherization

                                                    SERA

                                                    ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                    be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                    Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                    Value of 100 HALVES Payback doubles ROI

                                                    Source Skumatz Economics (SERA)

                                                    35

                                                    ( for NEBs Multiplier of 125)

                                                    SERA

                                                    NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                    36

                                                    hellip2001 ON

                                                    SERA

                                                    37

                                                    NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                    For true representation of B amp C elements of NEBs address missing factor bias

                                                    Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                    BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                    SERA

                                                    CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                    38

                                                    Test Benefits Costs States Using Traditionally

                                                    Improved treat-ment with NEBs

                                                    Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                    bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                    bull Avoided gas and water supply costs

                                                    bull Program administration

                                                    bull Participant incentives

                                                    bull Increased supply cost

                                                    CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                    Use cost only paid by the utility

                                                    Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                    Same as above plus bull increased revenue

                                                    Same as above plus bull Decreased revenue

                                                    AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                    Participant cost bull Utility bill reductions bull Participant incentives

                                                    bull Participant direct costs

                                                    AR CA FL HI IA IN MI MN NY VA

                                                    Participant NEBs

                                                    Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                    Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                    SERA

                                                    APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                    Expansion Concern with risk calculation accuracy

                                                    39

                                                    Source Skumatz SERA research

                                                    SERA

                                                    40

                                                    METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                    Maximize DSM opportunities amp feedback Accuracy tailoring

                                                    Minimize Regulatory amp Implementer Risk

                                                    Minimize Evaluation Cost

                                                    Adder

                                                    Readily MeasurableHybrid

                                                    All NEBsSource SERA Research

                                                    SERA

                                                    41

                                                    STATE TREATMENT OF NEBS NEBsRegulatory

                                                    Screening Application Utilities regions

                                                    Program Marketing Fairly widespread use in utilities states across the country

                                                    MO

                                                    RE AGGRESSIVE ==gt

                                                    Test Pgm Screen - adder

                                                    IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                    Test Pgm Screen - readily measured

                                                    MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                    Test Hybrid (potential adder amp measured)

                                                    CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                    Test Pgm screen - Broad

                                                    With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                    Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                    SERA

                                                    42

                                                    Key

                                                    States with NEBs in CE tests for At least one type of program

                                                    copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                    STATES WITH NEBS IN CE TESTS

                                                    NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                    groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                    SERA

                                                    SELECTED STATE PROGRESS CA

                                                    2001 ndash LIPPT model NY

                                                    Mid-2000s measurement amp scenarios not included CO

                                                    Adders (10 electric 25 LI 5 gas) VT

                                                    15 adder helped by previous research DC

                                                    10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                    43

                                                    Source Skumatz SERA research

                                                    SERA

                                                    SELECTED STATE PROGRESS Dominos ongoing

                                                    Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                    Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                    Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                    Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                    44

                                                    Source Skumatz SERA research

                                                    SERA

                                                    APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                    Expansion Concern with risk calculation accuracy

                                                    45

                                                    Source Skumatz SERA research

                                                    SERA

                                                    46

                                                    NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                    [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                    SourceSkumatz SERA Independent research

                                                    NTG ndash accuracy measurement incomplete

                                                    Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                    NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                    EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                    Cost Complicated expensive local changes

                                                    RELATIVE Risk from NEBs

                                                    Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                    Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                    NEBs as accurate as many inputs (all could improve)

                                                    SERA

                                                    DISCOUNT RATE MATTERS

                                                    47

                                                    Source Skumatz SERA Research

                                                    0

                                                    20

                                                    40

                                                    60

                                                    80

                                                    100

                                                    120

                                                    1 2 3 4 5 6 7 8 9 10

                                                    PV Ratios vs 1 Discount Rate

                                                    10 yr PV 20 yr PV

                                                    Discount rate - percent

                                                    WACCSocialLTT

                                                    Source Skumatz Electricity Journal IEPEC paper independent research

                                                    SERA

                                                    EUL VARIATIONS-ADOPTED VALUES

                                                    Source Skumatz Economic Research Associates SERA independent research

                                                    0

                                                    50

                                                    100

                                                    150

                                                    200

                                                    250

                                                    300

                                                    350

                                                    400

                                                    450

                                                    0 10 20 30 40 50 60 70

                                                    EUL Variation by Measure

                                                    67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                    Poorly supportedOldTechnology change

                                                    SERA

                                                    KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                    Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                    measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                    change the BC conclusion

                                                    On the cusphellip

                                                    If NO Yoursquore doneAnd bias addressedsufficiently

                                                    IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                    Source SERA all rights reserved

                                                    SERArsquos lsquoNEB-ItrdquoModel ndash

                                                    SERA

                                                    IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                    Utility Soc Part Conserv Recrsquom

                                                    Rationale

                                                    Base Percent X X X Program-invariant

                                                    Low Income X X X X Multiple sources

                                                    Weatherization X X X Substantial Participantimpacts

                                                    Measure Program-specific

                                                    X Varies by measure sector

                                                    Other Recomrsquos Local Research

                                                    50

                                                    Source Skumatz SERA

                                                    Developing values for multiple states amp utilities

                                                    SERA

                                                    TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                    Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                    Other sources of bias to research (EUL NTG etc) bring high variation

                                                    5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                    6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                    THANK YOU

                                                    Questions

                                                    Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                    SERA

                                                    POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                    53

                                                    SERA

                                                    NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                    Negative NEB values cost of barrier

                                                    Solar Water Heat NZ$ Euros

                                                    Solar Design NZ$ Euros

                                                    Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                    -23 -12 (079)

                                                    -11 -6 (06)

                                                    Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                    Source Skumatz Economic Research Associates research

                                                    SERA

                                                    CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                    NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                    -5-4-3-2-1012345

                                                    Op Co

                                                    st

                                                    Init C

                                                    ost

                                                    Maint

                                                    Perfo

                                                    rm

                                                    Prod

                                                    uctiv

                                                    Tena

                                                    nt Sa

                                                    t

                                                    Comf

                                                    ort

                                                    Appe

                                                    ar

                                                    Light

                                                    Qual

                                                    SellL

                                                    ease

                                                    Devp Ownr AampE

                                                    Note AgtE (~4 vs ~2) for these NEBs

                                                    Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                    Source Skumatz Economic Research Associates Inc 2002

                                                    Owners more positive ndashUnderinvestment train

                                                    • Soup to nuts on non-energy benefits NEIs
                                                    • TOPICS
                                                    • BACKGROUND NEBS
                                                    • 20+ YEARS OF NEBS PROGRESShellip
                                                    • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                    • KEY APPLICATIONS OF NEBS
                                                    • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                    • Neb measurement
                                                    • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                    • MEASURING PARTICIPANTNEBs
                                                    • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                    • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                    • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                    • PROGRESS TRANSFERABILITY OF NEBS
                                                    • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                    • PROGRESS AND GAPS
                                                    • PROGRESS AND GAPS
                                                    • USING NEB NEI VALUES
                                                    • Slide Number 19
                                                    • ARE NEBS HIGH VALUE
                                                    • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                    • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                    • RESULTS FOR ENERGY STARreg APPLIANCES
                                                    • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                    • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                    • CampI PROGRAMS ldquoNEBBEDrdquo
                                                    • AUDIT FINANCIAL PROGRAM
                                                    • CampI NEW CONSTRUCTION
                                                    • CampI TECHNICAL ASSISTANCE
                                                    • EXPRESSING NEBS VALUEndashCxe
                                                    • Slide Number 31
                                                    • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                    • EMISSIONS (SOCIETAL NEBS)
                                                    • UTILITY NEBS
                                                    • ADJUSTED PAYBACKS
                                                    • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                    • NEBS IN BC
                                                    • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                    • APPLICATION TO BC
                                                    • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                    • STATE TREATMENT OF NEBS NEBs
                                                    • Slide Number 42
                                                    • SELECTED STATE PROGRESS
                                                    • SELECTED STATE PROGRESS
                                                    • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                    • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                    • DISCOUNT RATE MATTERS
                                                    • EUL VARIATIONS- ADOPTED VALUES
                                                    • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                    • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                    • TAKEAWAYSON NEBSNEIS
                                                    • THANK YOUQuestions
                                                    • Slide Number 53
                                                    • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                    • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                      Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                      m Product losses20
                                                      l Ease of selling leasing building33
                                                      n Labor requirements45
                                                      h Aesthetics appearance59
                                                      j Noise50
                                                      b Equipment maintenance660
                                                      f Tenant satisfaction60
                                                      d Equipment lifetimes718
                                                      e Productivity724
                                                      k Building safety[2]73
                                                      g Comfort933
                                                      i Lighting Quality of Light916
                                                      c Equipment performance[1]1019
                                                      a Operating costs1165
                                                      o Doing good for the Environment[3]1132
                                                      Other03
                                                      [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                      [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                      [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                      m Product losses
                                                      l Ease of selling leasing building
                                                      n Labor requirements
                                                      h Aesthetics appearance
                                                      j Noise
                                                      b Equipment maintenance
                                                      f Tenant satisfaction
                                                      d Equipment lifetimes
                                                      e Productivity
                                                      k Building safety[2]
                                                      g Comfort
                                                      i Lighting Quality of Light
                                                      c Equipment performance[1]
                                                      a Operating costs
                                                      o Doing good for the Environment[3]
                                                      Share of ZALEH NEBS from various measures
                                                      Each presented as multiple of the (overall) energy savings
                                                      Super Insulation037
                                                      Double Glazing026
                                                      Solar Design025
                                                      Solar Water Heating005
                                                      EnergySmart (insul Retrofit)Dunedin (insul)
                                                      Other124
                                                      Noise24
                                                      Budgetfinances25
                                                      Environmental65
                                                      Health78
                                                      Reduced moves86
                                                      Ease of sellingrenting130
                                                      DampnessMold1316
                                                      Comfort-related4933
                                                      101101
                                                      Operational NEBs44
                                                      DesignConstruction NEBs31
                                                      Occupant NEBs25
                                                      1
                                                      OtherOther
                                                      NoiseNoise
                                                      BudgetfinancesBudgetfinances
                                                      EnvironmentalEnvironmental
                                                      HealthHealth
                                                      Reduced movesReduced moves
                                                      Ease of sellingrentingEase of sellingrenting
                                                      DampnessMoldDampnessMold
                                                      Comfort-relatedComfort-related
                                                      Boiler
                                                      VendorNPPartic
                                                      Perform0280202
                                                      Footprint028013017
                                                      ControlFeatures031022016
                                                      Tenant compl006008014
                                                      Noise005006014
                                                      Maint004009007
                                                      Other004003007
                                                      Lifetime-006018006
                                                      MF New construction energy star
                                                      Operating cost014
                                                      Environment012
                                                      Maintenance01
                                                      Lifetime01
                                                      Performance009
                                                      Occupant Satisf009
                                                      Comfort008
                                                      Noise006
                                                      Sellingleasing006
                                                      Aesthetics005
                                                      Light Quality005
                                                      Power QualReliab004
                                                      Safety002
                                                      RIGHTLIGHTS
                                                      Environment023
                                                      Quality016
                                                      Quantity014
                                                      PgmAvail01
                                                      Flicker008
                                                      Maintenance006
                                                      Lifetime006
                                                      SalesProductivity006
                                                      Safety005
                                                      Noise005
                                                      Control003
                                                      Illness0

                                                      Sheet1

                                                      Sheet2

                                                      Sheet3

                                                      SERA

                                                      NEB VALUES

                                                      19

                                                      SERA

                                                      ARE NEBS HIGH VALUE Energy savings are often the minority of

                                                      benefits from program measuresndash Donrsquot ignore that

                                                      20

                                                      0 05 1 15 2 25 3

                                                      Energy Sav

                                                      NEBs

                                                      NEB vs Energy Savings ValueIncluding all NEBs

                                                      Source (SkumatzSERA 2010 amp others)

                                                      Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                                      SERA

                                                      RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                                      MF Many others

                                                      Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                                      Variety of end uses

                                                      SERA

                                                      TOP NEBS FOR WEATHERIZATION PROGRAM

                                                      Weatherization program multipliergt1

                                                      Source SERA Research

                                                      SERA

                                                      RESULTS FOR ENERGY STARreg APPLIANCES

                                                      0002004006008

                                                      01012014016018

                                                      Maint

                                                      Appear

                                                      Perf LifeNoise Satis

                                                      Comfort

                                                      LiteSafe

                                                      ty SellMove

                                                      Water

                                                      Enviro

                                                      Refrig DW CW

                                                      Source SERA Research

                                                      SERA

                                                      INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                                      5 increments to 50

                                                      Source SERA research

                                                      Chart5

                                                      EnergySmart (insul Retrofit)
                                                      Dunedin (insul)
                                                      001
                                                      024
                                                      002
                                                      004
                                                      002
                                                      005
                                                      006
                                                      005
                                                      007
                                                      008
                                                      008
                                                      006
                                                      013
                                                      0
                                                      013
                                                      016
                                                      049
                                                      033

                                                      cx

                                                      cx

                                                      nz

                                                      Sheet3

                                                      Sheet3

                                                      Super Insulation
                                                      Double Glazing
                                                      Solar Design
                                                      Solar Water Heating
                                                      Super Insulation
                                                      Double Glazing
                                                      Solar Design
                                                      Solar Water Heating
                                                      EnergySmart (insul Retrofit)
                                                      Dunedin (insul)

                                                      SERA

                                                      ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                      0

                                                      002

                                                      004

                                                      006

                                                      008

                                                      01

                                                      012

                                                      Maint

                                                      Perf LifeSatis

                                                      f

                                                      ComfortAesth

                                                      NoiseSafe

                                                      ty

                                                      LightingSelli

                                                      ng

                                                      Stay in home

                                                      Enviro

                                                      Sick Days

                                                      Calls

                                                      ES Homes HP wES Energy savings multiplier=1

                                                      Source SERA Research

                                                      SERA

                                                      CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                      DG CHP HVAC Equipment rebate Other

                                                      Building codes incentives by cities

                                                      Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                      Variety of end uses

                                                      SERA

                                                      AUDIT FINANCIAL PROGRAM

                                                      0

                                                      01

                                                      02

                                                      03

                                                      04

                                                      05

                                                      06

                                                      07

                                                      Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                      Source SERA Research

                                                      SERA

                                                      CampI NEW CONSTRUCTION

                                                      -005

                                                      0

                                                      005

                                                      01

                                                      015

                                                      02

                                                      AampE Owner(Pct of Total Partic NEBs)

                                                      Source SERA Research

                                                      SERA

                                                      CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                      0 2 4 6 8 10 12

                                                      m Product losses

                                                      l Ease of selling leasing building

                                                      n Labor requirements

                                                      h Aesthetics appearance

                                                      j Noise

                                                      b Equipment maintenance

                                                      f Tenant satisfaction

                                                      d Equipment lifetimes

                                                      e Productivity

                                                      k Building safety[2]

                                                      g Comfort

                                                      i Lighting Quality of Light

                                                      c Equipment performance[1]

                                                      a Operating costs

                                                      o Doing good for the Environment[3]

                                                      Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                      vcv

                                                      Source SERA Research

                                                      Chart1

                                                      CampI Technical Assistance NEBs
                                                      002
                                                      003
                                                      004
                                                      005
                                                      005
                                                      006
                                                      006
                                                      007
                                                      007
                                                      007
                                                      009
                                                      009
                                                      01
                                                      011
                                                      011

                                                      Sheet1

                                                      Sheet1

                                                      CampI Technical Assistance Program NEBs (Participant)

                                                      Sheet2

                                                      Sheet3

                                                      SERA

                                                      EXPRESSING NEBS VALUEndashCxe

                                                      Strong value from RetroCxSource SERA Research

                                                      SERA

                                                      SOCIETAL NEB EXAMPLES

                                                      31

                                                      SERA

                                                      JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                      32

                                                      0

                                                      02

                                                      04

                                                      06

                                                      08

                                                      1

                                                      12

                                                      HPWxRetrof Appliance

                                                      CA

                                                      WI

                                                      Natl

                                                      (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                      Jobs Economic

                                                      SERA

                                                      EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                      33

                                                      SERA

                                                      UTILITY NEBSUtility NEBs for Template Program

                                                      Debt WriteOff (util)13

                                                      Rate Subsidy(util)61

                                                      HealthSafety(util)0

                                                      Colln Costs (util)0

                                                      Gas Calls (util)0

                                                      Calls to CSRs(util)2

                                                      TampD (util)16

                                                      Arrears (util)0

                                                      Reconnects (util)0

                                                      Notices (util)7

                                                      Shutoffs (util)1

                                                      Rate subsidy TampD

                                                      Payment-related

                                                      Source Skumatz Economic Research Associates research

                                                      Example Low Income Weatherization

                                                      SERA

                                                      ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                      be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                      Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                      Value of 100 HALVES Payback doubles ROI

                                                      Source Skumatz Economics (SERA)

                                                      35

                                                      ( for NEBs Multiplier of 125)

                                                      SERA

                                                      NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                      36

                                                      hellip2001 ON

                                                      SERA

                                                      37

                                                      NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                      For true representation of B amp C elements of NEBs address missing factor bias

                                                      Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                      BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                      SERA

                                                      CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                      38

                                                      Test Benefits Costs States Using Traditionally

                                                      Improved treat-ment with NEBs

                                                      Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                      bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                      bull Avoided gas and water supply costs

                                                      bull Program administration

                                                      bull Participant incentives

                                                      bull Increased supply cost

                                                      CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                      Use cost only paid by the utility

                                                      Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                      Same as above plus bull increased revenue

                                                      Same as above plus bull Decreased revenue

                                                      AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                      Participant cost bull Utility bill reductions bull Participant incentives

                                                      bull Participant direct costs

                                                      AR CA FL HI IA IN MI MN NY VA

                                                      Participant NEBs

                                                      Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                      Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                      SERA

                                                      APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                      Expansion Concern with risk calculation accuracy

                                                      39

                                                      Source Skumatz SERA research

                                                      SERA

                                                      40

                                                      METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                      Maximize DSM opportunities amp feedback Accuracy tailoring

                                                      Minimize Regulatory amp Implementer Risk

                                                      Minimize Evaluation Cost

                                                      Adder

                                                      Readily MeasurableHybrid

                                                      All NEBsSource SERA Research

                                                      SERA

                                                      41

                                                      STATE TREATMENT OF NEBS NEBsRegulatory

                                                      Screening Application Utilities regions

                                                      Program Marketing Fairly widespread use in utilities states across the country

                                                      MO

                                                      RE AGGRESSIVE ==gt

                                                      Test Pgm Screen - adder

                                                      IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                      Test Pgm Screen - readily measured

                                                      MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                      Test Hybrid (potential adder amp measured)

                                                      CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                      Test Pgm screen - Broad

                                                      With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                      Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                      SERA

                                                      42

                                                      Key

                                                      States with NEBs in CE tests for At least one type of program

                                                      copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                      STATES WITH NEBS IN CE TESTS

                                                      NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                      groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                      SERA

                                                      SELECTED STATE PROGRESS CA

                                                      2001 ndash LIPPT model NY

                                                      Mid-2000s measurement amp scenarios not included CO

                                                      Adders (10 electric 25 LI 5 gas) VT

                                                      15 adder helped by previous research DC

                                                      10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                      43

                                                      Source Skumatz SERA research

                                                      SERA

                                                      SELECTED STATE PROGRESS Dominos ongoing

                                                      Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                      Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                      Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                      Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                      44

                                                      Source Skumatz SERA research

                                                      SERA

                                                      APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                      Expansion Concern with risk calculation accuracy

                                                      45

                                                      Source Skumatz SERA research

                                                      SERA

                                                      46

                                                      NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                      [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                      SourceSkumatz SERA Independent research

                                                      NTG ndash accuracy measurement incomplete

                                                      Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                      NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                      EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                      Cost Complicated expensive local changes

                                                      RELATIVE Risk from NEBs

                                                      Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                      Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                      NEBs as accurate as many inputs (all could improve)

                                                      SERA

                                                      DISCOUNT RATE MATTERS

                                                      47

                                                      Source Skumatz SERA Research

                                                      0

                                                      20

                                                      40

                                                      60

                                                      80

                                                      100

                                                      120

                                                      1 2 3 4 5 6 7 8 9 10

                                                      PV Ratios vs 1 Discount Rate

                                                      10 yr PV 20 yr PV

                                                      Discount rate - percent

                                                      WACCSocialLTT

                                                      Source Skumatz Electricity Journal IEPEC paper independent research

                                                      SERA

                                                      EUL VARIATIONS-ADOPTED VALUES

                                                      Source Skumatz Economic Research Associates SERA independent research

                                                      0

                                                      50

                                                      100

                                                      150

                                                      200

                                                      250

                                                      300

                                                      350

                                                      400

                                                      450

                                                      0 10 20 30 40 50 60 70

                                                      EUL Variation by Measure

                                                      67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                      Poorly supportedOldTechnology change

                                                      SERA

                                                      KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                      Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                      measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                      change the BC conclusion

                                                      On the cusphellip

                                                      If NO Yoursquore doneAnd bias addressedsufficiently

                                                      IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                      Source SERA all rights reserved

                                                      SERArsquos lsquoNEB-ItrdquoModel ndash

                                                      SERA

                                                      IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                      Utility Soc Part Conserv Recrsquom

                                                      Rationale

                                                      Base Percent X X X Program-invariant

                                                      Low Income X X X X Multiple sources

                                                      Weatherization X X X Substantial Participantimpacts

                                                      Measure Program-specific

                                                      X Varies by measure sector

                                                      Other Recomrsquos Local Research

                                                      50

                                                      Source Skumatz SERA

                                                      Developing values for multiple states amp utilities

                                                      SERA

                                                      TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                      Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                      Other sources of bias to research (EUL NTG etc) bring high variation

                                                      5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                      6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                      THANK YOU

                                                      Questions

                                                      Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                      SERA

                                                      POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                      53

                                                      SERA

                                                      NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                      Negative NEB values cost of barrier

                                                      Solar Water Heat NZ$ Euros

                                                      Solar Design NZ$ Euros

                                                      Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                      -23 -12 (079)

                                                      -11 -6 (06)

                                                      Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                      Source Skumatz Economic Research Associates research

                                                      SERA

                                                      CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                      NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                      -5-4-3-2-1012345

                                                      Op Co

                                                      st

                                                      Init C

                                                      ost

                                                      Maint

                                                      Perfo

                                                      rm

                                                      Prod

                                                      uctiv

                                                      Tena

                                                      nt Sa

                                                      t

                                                      Comf

                                                      ort

                                                      Appe

                                                      ar

                                                      Light

                                                      Qual

                                                      SellL

                                                      ease

                                                      Devp Ownr AampE

                                                      Note AgtE (~4 vs ~2) for these NEBs

                                                      Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                      Source Skumatz Economic Research Associates Inc 2002

                                                      Owners more positive ndashUnderinvestment train

                                                      • Soup to nuts on non-energy benefits NEIs
                                                      • TOPICS
                                                      • BACKGROUND NEBS
                                                      • 20+ YEARS OF NEBS PROGRESShellip
                                                      • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                      • KEY APPLICATIONS OF NEBS
                                                      • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                      • Neb measurement
                                                      • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                      • MEASURING PARTICIPANTNEBs
                                                      • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                      • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                      • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                      • PROGRESS TRANSFERABILITY OF NEBS
                                                      • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                      • PROGRESS AND GAPS
                                                      • PROGRESS AND GAPS
                                                      • USING NEB NEI VALUES
                                                      • Slide Number 19
                                                      • ARE NEBS HIGH VALUE
                                                      • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                      • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                      • RESULTS FOR ENERGY STARreg APPLIANCES
                                                      • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                      • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                      • CampI PROGRAMS ldquoNEBBEDrdquo
                                                      • AUDIT FINANCIAL PROGRAM
                                                      • CampI NEW CONSTRUCTION
                                                      • CampI TECHNICAL ASSISTANCE
                                                      • EXPRESSING NEBS VALUEndashCxe
                                                      • Slide Number 31
                                                      • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                      • EMISSIONS (SOCIETAL NEBS)
                                                      • UTILITY NEBS
                                                      • ADJUSTED PAYBACKS
                                                      • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                      • NEBS IN BC
                                                      • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                      • APPLICATION TO BC
                                                      • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                      • STATE TREATMENT OF NEBS NEBs
                                                      • Slide Number 42
                                                      • SELECTED STATE PROGRESS
                                                      • SELECTED STATE PROGRESS
                                                      • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                      • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                      • DISCOUNT RATE MATTERS
                                                      • EUL VARIATIONS- ADOPTED VALUES
                                                      • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                      • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                      • TAKEAWAYSON NEBSNEIS
                                                      • THANK YOUQuestions
                                                      • Slide Number 53
                                                      • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                      • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                        Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                        m Product losses20
                                                        l Ease of selling leasing building33
                                                        n Labor requirements45
                                                        h Aesthetics appearance59
                                                        j Noise50
                                                        b Equipment maintenance660
                                                        f Tenant satisfaction60
                                                        d Equipment lifetimes718
                                                        e Productivity724
                                                        k Building safety[2]73
                                                        g Comfort933
                                                        i Lighting Quality of Light916
                                                        c Equipment performance[1]1019
                                                        a Operating costs1165
                                                        o Doing good for the Environment[3]1132
                                                        Other03
                                                        [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                        [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                        [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                        m Product losses
                                                        l Ease of selling leasing building
                                                        n Labor requirements
                                                        h Aesthetics appearance
                                                        j Noise
                                                        b Equipment maintenance
                                                        f Tenant satisfaction
                                                        d Equipment lifetimes
                                                        e Productivity
                                                        k Building safety[2]
                                                        g Comfort
                                                        i Lighting Quality of Light
                                                        c Equipment performance[1]
                                                        a Operating costs
                                                        o Doing good for the Environment[3]
                                                        Share of ZALEH NEBS from various measures
                                                        Each presented as multiple of the (overall) energy savings
                                                        Super Insulation037
                                                        Double Glazing026
                                                        Solar Design025
                                                        Solar Water Heating005
                                                        EnergySmart (insul Retrofit)Dunedin (insul)
                                                        Other124
                                                        Noise24
                                                        Budgetfinances25
                                                        Environmental65
                                                        Health78
                                                        Reduced moves86
                                                        Ease of sellingrenting130
                                                        DampnessMold1316
                                                        Comfort-related4933
                                                        101101
                                                        Operational NEBs44
                                                        DesignConstruction NEBs31
                                                        Occupant NEBs25
                                                        1
                                                        OtherOther
                                                        NoiseNoise
                                                        BudgetfinancesBudgetfinances
                                                        EnvironmentalEnvironmental
                                                        HealthHealth
                                                        Reduced movesReduced moves
                                                        Ease of sellingrentingEase of sellingrenting
                                                        DampnessMoldDampnessMold
                                                        Comfort-relatedComfort-related

                                                        Sheet2

                                                        Sheet3

                                                        SERA

                                                        NEB VALUES

                                                        19

                                                        SERA

                                                        ARE NEBS HIGH VALUE Energy savings are often the minority of

                                                        benefits from program measuresndash Donrsquot ignore that

                                                        20

                                                        0 05 1 15 2 25 3

                                                        Energy Sav

                                                        NEBs

                                                        NEB vs Energy Savings ValueIncluding all NEBs

                                                        Source (SkumatzSERA 2010 amp others)

                                                        Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                                        SERA

                                                        RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                                        MF Many others

                                                        Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                                        Variety of end uses

                                                        SERA

                                                        TOP NEBS FOR WEATHERIZATION PROGRAM

                                                        Weatherization program multipliergt1

                                                        Source SERA Research

                                                        SERA

                                                        RESULTS FOR ENERGY STARreg APPLIANCES

                                                        0002004006008

                                                        01012014016018

                                                        Maint

                                                        Appear

                                                        Perf LifeNoise Satis

                                                        Comfort

                                                        LiteSafe

                                                        ty SellMove

                                                        Water

                                                        Enviro

                                                        Refrig DW CW

                                                        Source SERA Research

                                                        SERA

                                                        INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                                        5 increments to 50

                                                        Source SERA research

                                                        Chart5

                                                        EnergySmart (insul Retrofit)
                                                        Dunedin (insul)
                                                        001
                                                        024
                                                        002
                                                        004
                                                        002
                                                        005
                                                        006
                                                        005
                                                        007
                                                        008
                                                        008
                                                        006
                                                        013
                                                        0
                                                        013
                                                        016
                                                        049
                                                        033

                                                        cx

                                                        cx

                                                        nz

                                                        Sheet3

                                                        Sheet3

                                                        Super Insulation
                                                        Double Glazing
                                                        Solar Design
                                                        Solar Water Heating
                                                        Super Insulation
                                                        Double Glazing
                                                        Solar Design
                                                        Solar Water Heating
                                                        EnergySmart (insul Retrofit)
                                                        Dunedin (insul)

                                                        SERA

                                                        ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                        0

                                                        002

                                                        004

                                                        006

                                                        008

                                                        01

                                                        012

                                                        Maint

                                                        Perf LifeSatis

                                                        f

                                                        ComfortAesth

                                                        NoiseSafe

                                                        ty

                                                        LightingSelli

                                                        ng

                                                        Stay in home

                                                        Enviro

                                                        Sick Days

                                                        Calls

                                                        ES Homes HP wES Energy savings multiplier=1

                                                        Source SERA Research

                                                        SERA

                                                        CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                        DG CHP HVAC Equipment rebate Other

                                                        Building codes incentives by cities

                                                        Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                        Variety of end uses

                                                        SERA

                                                        AUDIT FINANCIAL PROGRAM

                                                        0

                                                        01

                                                        02

                                                        03

                                                        04

                                                        05

                                                        06

                                                        07

                                                        Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                        Source SERA Research

                                                        SERA

                                                        CampI NEW CONSTRUCTION

                                                        -005

                                                        0

                                                        005

                                                        01

                                                        015

                                                        02

                                                        AampE Owner(Pct of Total Partic NEBs)

                                                        Source SERA Research

                                                        SERA

                                                        CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                        0 2 4 6 8 10 12

                                                        m Product losses

                                                        l Ease of selling leasing building

                                                        n Labor requirements

                                                        h Aesthetics appearance

                                                        j Noise

                                                        b Equipment maintenance

                                                        f Tenant satisfaction

                                                        d Equipment lifetimes

                                                        e Productivity

                                                        k Building safety[2]

                                                        g Comfort

                                                        i Lighting Quality of Light

                                                        c Equipment performance[1]

                                                        a Operating costs

                                                        o Doing good for the Environment[3]

                                                        Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                        vcv

                                                        Source SERA Research

                                                        Chart1

                                                        CampI Technical Assistance NEBs
                                                        002
                                                        003
                                                        004
                                                        005
                                                        005
                                                        006
                                                        006
                                                        007
                                                        007
                                                        007
                                                        009
                                                        009
                                                        01
                                                        011
                                                        011

                                                        Sheet1

                                                        Sheet1

                                                        CampI Technical Assistance Program NEBs (Participant)

                                                        Sheet2

                                                        Sheet3

                                                        SERA

                                                        EXPRESSING NEBS VALUEndashCxe

                                                        Strong value from RetroCxSource SERA Research

                                                        SERA

                                                        SOCIETAL NEB EXAMPLES

                                                        31

                                                        SERA

                                                        JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                        32

                                                        0

                                                        02

                                                        04

                                                        06

                                                        08

                                                        1

                                                        12

                                                        HPWxRetrof Appliance

                                                        CA

                                                        WI

                                                        Natl

                                                        (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                        Jobs Economic

                                                        SERA

                                                        EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                        33

                                                        SERA

                                                        UTILITY NEBSUtility NEBs for Template Program

                                                        Debt WriteOff (util)13

                                                        Rate Subsidy(util)61

                                                        HealthSafety(util)0

                                                        Colln Costs (util)0

                                                        Gas Calls (util)0

                                                        Calls to CSRs(util)2

                                                        TampD (util)16

                                                        Arrears (util)0

                                                        Reconnects (util)0

                                                        Notices (util)7

                                                        Shutoffs (util)1

                                                        Rate subsidy TampD

                                                        Payment-related

                                                        Source Skumatz Economic Research Associates research

                                                        Example Low Income Weatherization

                                                        SERA

                                                        ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                        be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                        Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                        Value of 100 HALVES Payback doubles ROI

                                                        Source Skumatz Economics (SERA)

                                                        35

                                                        ( for NEBs Multiplier of 125)

                                                        SERA

                                                        NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                        36

                                                        hellip2001 ON

                                                        SERA

                                                        37

                                                        NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                        For true representation of B amp C elements of NEBs address missing factor bias

                                                        Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                        BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                        SERA

                                                        CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                        38

                                                        Test Benefits Costs States Using Traditionally

                                                        Improved treat-ment with NEBs

                                                        Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                        bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                        bull Avoided gas and water supply costs

                                                        bull Program administration

                                                        bull Participant incentives

                                                        bull Increased supply cost

                                                        CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                        Use cost only paid by the utility

                                                        Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                        Same as above plus bull increased revenue

                                                        Same as above plus bull Decreased revenue

                                                        AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                        Participant cost bull Utility bill reductions bull Participant incentives

                                                        bull Participant direct costs

                                                        AR CA FL HI IA IN MI MN NY VA

                                                        Participant NEBs

                                                        Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                        Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                        SERA

                                                        APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                        Expansion Concern with risk calculation accuracy

                                                        39

                                                        Source Skumatz SERA research

                                                        SERA

                                                        40

                                                        METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                        Maximize DSM opportunities amp feedback Accuracy tailoring

                                                        Minimize Regulatory amp Implementer Risk

                                                        Minimize Evaluation Cost

                                                        Adder

                                                        Readily MeasurableHybrid

                                                        All NEBsSource SERA Research

                                                        SERA

                                                        41

                                                        STATE TREATMENT OF NEBS NEBsRegulatory

                                                        Screening Application Utilities regions

                                                        Program Marketing Fairly widespread use in utilities states across the country

                                                        MO

                                                        RE AGGRESSIVE ==gt

                                                        Test Pgm Screen - adder

                                                        IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                        Test Pgm Screen - readily measured

                                                        MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                        Test Hybrid (potential adder amp measured)

                                                        CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                        Test Pgm screen - Broad

                                                        With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                        Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                        SERA

                                                        42

                                                        Key

                                                        States with NEBs in CE tests for At least one type of program

                                                        copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                        STATES WITH NEBS IN CE TESTS

                                                        NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                        groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                        SERA

                                                        SELECTED STATE PROGRESS CA

                                                        2001 ndash LIPPT model NY

                                                        Mid-2000s measurement amp scenarios not included CO

                                                        Adders (10 electric 25 LI 5 gas) VT

                                                        15 adder helped by previous research DC

                                                        10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                        43

                                                        Source Skumatz SERA research

                                                        SERA

                                                        SELECTED STATE PROGRESS Dominos ongoing

                                                        Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                        Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                        Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                        Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                        44

                                                        Source Skumatz SERA research

                                                        SERA

                                                        APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                        Expansion Concern with risk calculation accuracy

                                                        45

                                                        Source Skumatz SERA research

                                                        SERA

                                                        46

                                                        NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                        [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                        SourceSkumatz SERA Independent research

                                                        NTG ndash accuracy measurement incomplete

                                                        Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                        NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                        EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                        Cost Complicated expensive local changes

                                                        RELATIVE Risk from NEBs

                                                        Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                        Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                        NEBs as accurate as many inputs (all could improve)

                                                        SERA

                                                        DISCOUNT RATE MATTERS

                                                        47

                                                        Source Skumatz SERA Research

                                                        0

                                                        20

                                                        40

                                                        60

                                                        80

                                                        100

                                                        120

                                                        1 2 3 4 5 6 7 8 9 10

                                                        PV Ratios vs 1 Discount Rate

                                                        10 yr PV 20 yr PV

                                                        Discount rate - percent

                                                        WACCSocialLTT

                                                        Source Skumatz Electricity Journal IEPEC paper independent research

                                                        SERA

                                                        EUL VARIATIONS-ADOPTED VALUES

                                                        Source Skumatz Economic Research Associates SERA independent research

                                                        0

                                                        50

                                                        100

                                                        150

                                                        200

                                                        250

                                                        300

                                                        350

                                                        400

                                                        450

                                                        0 10 20 30 40 50 60 70

                                                        EUL Variation by Measure

                                                        67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                        Poorly supportedOldTechnology change

                                                        SERA

                                                        KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                        Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                        measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                        change the BC conclusion

                                                        On the cusphellip

                                                        If NO Yoursquore doneAnd bias addressedsufficiently

                                                        IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                        Source SERA all rights reserved

                                                        SERArsquos lsquoNEB-ItrdquoModel ndash

                                                        SERA

                                                        IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                        Utility Soc Part Conserv Recrsquom

                                                        Rationale

                                                        Base Percent X X X Program-invariant

                                                        Low Income X X X X Multiple sources

                                                        Weatherization X X X Substantial Participantimpacts

                                                        Measure Program-specific

                                                        X Varies by measure sector

                                                        Other Recomrsquos Local Research

                                                        50

                                                        Source Skumatz SERA

                                                        Developing values for multiple states amp utilities

                                                        SERA

                                                        TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                        Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                        Other sources of bias to research (EUL NTG etc) bring high variation

                                                        5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                        6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                        THANK YOU

                                                        Questions

                                                        Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                        SERA

                                                        POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                        53

                                                        SERA

                                                        NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                        Negative NEB values cost of barrier

                                                        Solar Water Heat NZ$ Euros

                                                        Solar Design NZ$ Euros

                                                        Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                        -23 -12 (079)

                                                        -11 -6 (06)

                                                        Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                        Source Skumatz Economic Research Associates research

                                                        SERA

                                                        CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                        NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                        -5-4-3-2-1012345

                                                        Op Co

                                                        st

                                                        Init C

                                                        ost

                                                        Maint

                                                        Perfo

                                                        rm

                                                        Prod

                                                        uctiv

                                                        Tena

                                                        nt Sa

                                                        t

                                                        Comf

                                                        ort

                                                        Appe

                                                        ar

                                                        Light

                                                        Qual

                                                        SellL

                                                        ease

                                                        Devp Ownr AampE

                                                        Note AgtE (~4 vs ~2) for these NEBs

                                                        Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                        Source Skumatz Economic Research Associates Inc 2002

                                                        Owners more positive ndashUnderinvestment train

                                                        • Soup to nuts on non-energy benefits NEIs
                                                        • TOPICS
                                                        • BACKGROUND NEBS
                                                        • 20+ YEARS OF NEBS PROGRESShellip
                                                        • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                        • KEY APPLICATIONS OF NEBS
                                                        • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                        • Neb measurement
                                                        • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                        • MEASURING PARTICIPANTNEBs
                                                        • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                        • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                        • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                        • PROGRESS TRANSFERABILITY OF NEBS
                                                        • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                        • PROGRESS AND GAPS
                                                        • PROGRESS AND GAPS
                                                        • USING NEB NEI VALUES
                                                        • Slide Number 19
                                                        • ARE NEBS HIGH VALUE
                                                        • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                        • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                        • RESULTS FOR ENERGY STARreg APPLIANCES
                                                        • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                        • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                        • CampI PROGRAMS ldquoNEBBEDrdquo
                                                        • AUDIT FINANCIAL PROGRAM
                                                        • CampI NEW CONSTRUCTION
                                                        • CampI TECHNICAL ASSISTANCE
                                                        • EXPRESSING NEBS VALUEndashCxe
                                                        • Slide Number 31
                                                        • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                        • EMISSIONS (SOCIETAL NEBS)
                                                        • UTILITY NEBS
                                                        • ADJUSTED PAYBACKS
                                                        • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                        • NEBS IN BC
                                                        • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                        • APPLICATION TO BC
                                                        • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                        • STATE TREATMENT OF NEBS NEBs
                                                        • Slide Number 42
                                                        • SELECTED STATE PROGRESS
                                                        • SELECTED STATE PROGRESS
                                                        • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                        • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                        • DISCOUNT RATE MATTERS
                                                        • EUL VARIATIONS- ADOPTED VALUES
                                                        • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                        • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                        • TAKEAWAYSON NEBSNEIS
                                                        • THANK YOUQuestions
                                                        • Slide Number 53
                                                        • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                        • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                          Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                          m Product losses20
                                                          l Ease of selling leasing building33
                                                          n Labor requirements45
                                                          h Aesthetics appearance59
                                                          j Noise50
                                                          b Equipment maintenance660
                                                          f Tenant satisfaction60
                                                          d Equipment lifetimes718
                                                          e Productivity724
                                                          k Building safety[2]73
                                                          g Comfort933
                                                          i Lighting Quality of Light916
                                                          c Equipment performance[1]1019
                                                          a Operating costs1165
                                                          o Doing good for the Environment[3]1132
                                                          Other03
                                                          [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                          [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                          [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                          m Product losses
                                                          l Ease of selling leasing building
                                                          n Labor requirements
                                                          h Aesthetics appearance
                                                          j Noise
                                                          b Equipment maintenance
                                                          f Tenant satisfaction
                                                          d Equipment lifetimes
                                                          e Productivity
                                                          k Building safety[2]
                                                          g Comfort
                                                          i Lighting Quality of Light
                                                          c Equipment performance[1]
                                                          a Operating costs
                                                          o Doing good for the Environment[3]
                                                          Share of ZALEH NEBS from various measures
                                                          Each presented as multiple of the (overall) energy savings
                                                          Super Insulation037
                                                          Double Glazing026
                                                          Solar Design025
                                                          Solar Water Heating005
                                                          EnergySmart (insul Retrofit)Dunedin (insul)
                                                          Other124
                                                          Noise24
                                                          Budgetfinances25
                                                          Environmental65
                                                          Health78
                                                          Reduced moves86
                                                          Ease of sellingrenting130
                                                          DampnessMold1316
                                                          Comfort-related4933
                                                          101101
                                                          Operational NEBs44
                                                          DesignConstruction NEBs31
                                                          Occupant NEBs25
                                                          1
                                                          OtherOther
                                                          NoiseNoise
                                                          BudgetfinancesBudgetfinances
                                                          EnvironmentalEnvironmental
                                                          HealthHealth
                                                          Reduced movesReduced moves
                                                          Ease of sellingrentingEase of sellingrenting
                                                          DampnessMoldDampnessMold
                                                          Comfort-relatedComfort-related

                                                          Sheet3

                                                          SERA

                                                          NEB VALUES

                                                          19

                                                          SERA

                                                          ARE NEBS HIGH VALUE Energy savings are often the minority of

                                                          benefits from program measuresndash Donrsquot ignore that

                                                          20

                                                          0 05 1 15 2 25 3

                                                          Energy Sav

                                                          NEBs

                                                          NEB vs Energy Savings ValueIncluding all NEBs

                                                          Source (SkumatzSERA 2010 amp others)

                                                          Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                                          SERA

                                                          RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                                          MF Many others

                                                          Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                                          Variety of end uses

                                                          SERA

                                                          TOP NEBS FOR WEATHERIZATION PROGRAM

                                                          Weatherization program multipliergt1

                                                          Source SERA Research

                                                          SERA

                                                          RESULTS FOR ENERGY STARreg APPLIANCES

                                                          0002004006008

                                                          01012014016018

                                                          Maint

                                                          Appear

                                                          Perf LifeNoise Satis

                                                          Comfort

                                                          LiteSafe

                                                          ty SellMove

                                                          Water

                                                          Enviro

                                                          Refrig DW CW

                                                          Source SERA Research

                                                          SERA

                                                          INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                                          5 increments to 50

                                                          Source SERA research

                                                          Chart5

                                                          EnergySmart (insul Retrofit)
                                                          Dunedin (insul)
                                                          001
                                                          024
                                                          002
                                                          004
                                                          002
                                                          005
                                                          006
                                                          005
                                                          007
                                                          008
                                                          008
                                                          006
                                                          013
                                                          0
                                                          013
                                                          016
                                                          049
                                                          033

                                                          cx

                                                          cx

                                                          nz

                                                          Sheet3

                                                          Sheet3

                                                          Super Insulation
                                                          Double Glazing
                                                          Solar Design
                                                          Solar Water Heating
                                                          Super Insulation
                                                          Double Glazing
                                                          Solar Design
                                                          Solar Water Heating
                                                          EnergySmart (insul Retrofit)
                                                          Dunedin (insul)

                                                          SERA

                                                          ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                          0

                                                          002

                                                          004

                                                          006

                                                          008

                                                          01

                                                          012

                                                          Maint

                                                          Perf LifeSatis

                                                          f

                                                          ComfortAesth

                                                          NoiseSafe

                                                          ty

                                                          LightingSelli

                                                          ng

                                                          Stay in home

                                                          Enviro

                                                          Sick Days

                                                          Calls

                                                          ES Homes HP wES Energy savings multiplier=1

                                                          Source SERA Research

                                                          SERA

                                                          CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                          DG CHP HVAC Equipment rebate Other

                                                          Building codes incentives by cities

                                                          Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                          Variety of end uses

                                                          SERA

                                                          AUDIT FINANCIAL PROGRAM

                                                          0

                                                          01

                                                          02

                                                          03

                                                          04

                                                          05

                                                          06

                                                          07

                                                          Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                          Source SERA Research

                                                          SERA

                                                          CampI NEW CONSTRUCTION

                                                          -005

                                                          0

                                                          005

                                                          01

                                                          015

                                                          02

                                                          AampE Owner(Pct of Total Partic NEBs)

                                                          Source SERA Research

                                                          SERA

                                                          CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                          0 2 4 6 8 10 12

                                                          m Product losses

                                                          l Ease of selling leasing building

                                                          n Labor requirements

                                                          h Aesthetics appearance

                                                          j Noise

                                                          b Equipment maintenance

                                                          f Tenant satisfaction

                                                          d Equipment lifetimes

                                                          e Productivity

                                                          k Building safety[2]

                                                          g Comfort

                                                          i Lighting Quality of Light

                                                          c Equipment performance[1]

                                                          a Operating costs

                                                          o Doing good for the Environment[3]

                                                          Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                          vcv

                                                          Source SERA Research

                                                          Chart1

                                                          CampI Technical Assistance NEBs
                                                          002
                                                          003
                                                          004
                                                          005
                                                          005
                                                          006
                                                          006
                                                          007
                                                          007
                                                          007
                                                          009
                                                          009
                                                          01
                                                          011
                                                          011

                                                          Sheet1

                                                          Sheet1

                                                          CampI Technical Assistance Program NEBs (Participant)

                                                          Sheet2

                                                          Sheet3

                                                          SERA

                                                          EXPRESSING NEBS VALUEndashCxe

                                                          Strong value from RetroCxSource SERA Research

                                                          SERA

                                                          SOCIETAL NEB EXAMPLES

                                                          31

                                                          SERA

                                                          JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                          32

                                                          0

                                                          02

                                                          04

                                                          06

                                                          08

                                                          1

                                                          12

                                                          HPWxRetrof Appliance

                                                          CA

                                                          WI

                                                          Natl

                                                          (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                          Jobs Economic

                                                          SERA

                                                          EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                          33

                                                          SERA

                                                          UTILITY NEBSUtility NEBs for Template Program

                                                          Debt WriteOff (util)13

                                                          Rate Subsidy(util)61

                                                          HealthSafety(util)0

                                                          Colln Costs (util)0

                                                          Gas Calls (util)0

                                                          Calls to CSRs(util)2

                                                          TampD (util)16

                                                          Arrears (util)0

                                                          Reconnects (util)0

                                                          Notices (util)7

                                                          Shutoffs (util)1

                                                          Rate subsidy TampD

                                                          Payment-related

                                                          Source Skumatz Economic Research Associates research

                                                          Example Low Income Weatherization

                                                          SERA

                                                          ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                          be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                          Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                          Value of 100 HALVES Payback doubles ROI

                                                          Source Skumatz Economics (SERA)

                                                          35

                                                          ( for NEBs Multiplier of 125)

                                                          SERA

                                                          NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                          36

                                                          hellip2001 ON

                                                          SERA

                                                          37

                                                          NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                          For true representation of B amp C elements of NEBs address missing factor bias

                                                          Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                          BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                          SERA

                                                          CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                          38

                                                          Test Benefits Costs States Using Traditionally

                                                          Improved treat-ment with NEBs

                                                          Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                          bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                          bull Avoided gas and water supply costs

                                                          bull Program administration

                                                          bull Participant incentives

                                                          bull Increased supply cost

                                                          CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                          Use cost only paid by the utility

                                                          Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                          Same as above plus bull increased revenue

                                                          Same as above plus bull Decreased revenue

                                                          AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                          Participant cost bull Utility bill reductions bull Participant incentives

                                                          bull Participant direct costs

                                                          AR CA FL HI IA IN MI MN NY VA

                                                          Participant NEBs

                                                          Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                          Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                          SERA

                                                          APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                          Expansion Concern with risk calculation accuracy

                                                          39

                                                          Source Skumatz SERA research

                                                          SERA

                                                          40

                                                          METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                          Maximize DSM opportunities amp feedback Accuracy tailoring

                                                          Minimize Regulatory amp Implementer Risk

                                                          Minimize Evaluation Cost

                                                          Adder

                                                          Readily MeasurableHybrid

                                                          All NEBsSource SERA Research

                                                          SERA

                                                          41

                                                          STATE TREATMENT OF NEBS NEBsRegulatory

                                                          Screening Application Utilities regions

                                                          Program Marketing Fairly widespread use in utilities states across the country

                                                          MO

                                                          RE AGGRESSIVE ==gt

                                                          Test Pgm Screen - adder

                                                          IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                          Test Pgm Screen - readily measured

                                                          MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                          Test Hybrid (potential adder amp measured)

                                                          CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                          Test Pgm screen - Broad

                                                          With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                          Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                          SERA

                                                          42

                                                          Key

                                                          States with NEBs in CE tests for At least one type of program

                                                          copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                          STATES WITH NEBS IN CE TESTS

                                                          NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                          groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                          SERA

                                                          SELECTED STATE PROGRESS CA

                                                          2001 ndash LIPPT model NY

                                                          Mid-2000s measurement amp scenarios not included CO

                                                          Adders (10 electric 25 LI 5 gas) VT

                                                          15 adder helped by previous research DC

                                                          10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                          43

                                                          Source Skumatz SERA research

                                                          SERA

                                                          SELECTED STATE PROGRESS Dominos ongoing

                                                          Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                          Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                          Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                          Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                          44

                                                          Source Skumatz SERA research

                                                          SERA

                                                          APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                          Expansion Concern with risk calculation accuracy

                                                          45

                                                          Source Skumatz SERA research

                                                          SERA

                                                          46

                                                          NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                          [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                          SourceSkumatz SERA Independent research

                                                          NTG ndash accuracy measurement incomplete

                                                          Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                          NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                          EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                          Cost Complicated expensive local changes

                                                          RELATIVE Risk from NEBs

                                                          Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                          Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                          NEBs as accurate as many inputs (all could improve)

                                                          SERA

                                                          DISCOUNT RATE MATTERS

                                                          47

                                                          Source Skumatz SERA Research

                                                          0

                                                          20

                                                          40

                                                          60

                                                          80

                                                          100

                                                          120

                                                          1 2 3 4 5 6 7 8 9 10

                                                          PV Ratios vs 1 Discount Rate

                                                          10 yr PV 20 yr PV

                                                          Discount rate - percent

                                                          WACCSocialLTT

                                                          Source Skumatz Electricity Journal IEPEC paper independent research

                                                          SERA

                                                          EUL VARIATIONS-ADOPTED VALUES

                                                          Source Skumatz Economic Research Associates SERA independent research

                                                          0

                                                          50

                                                          100

                                                          150

                                                          200

                                                          250

                                                          300

                                                          350

                                                          400

                                                          450

                                                          0 10 20 30 40 50 60 70

                                                          EUL Variation by Measure

                                                          67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                          Poorly supportedOldTechnology change

                                                          SERA

                                                          KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                          Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                          measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                          change the BC conclusion

                                                          On the cusphellip

                                                          If NO Yoursquore doneAnd bias addressedsufficiently

                                                          IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                          Source SERA all rights reserved

                                                          SERArsquos lsquoNEB-ItrdquoModel ndash

                                                          SERA

                                                          IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                          Utility Soc Part Conserv Recrsquom

                                                          Rationale

                                                          Base Percent X X X Program-invariant

                                                          Low Income X X X X Multiple sources

                                                          Weatherization X X X Substantial Participantimpacts

                                                          Measure Program-specific

                                                          X Varies by measure sector

                                                          Other Recomrsquos Local Research

                                                          50

                                                          Source Skumatz SERA

                                                          Developing values for multiple states amp utilities

                                                          SERA

                                                          TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                          Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                          Other sources of bias to research (EUL NTG etc) bring high variation

                                                          5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                          6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                          THANK YOU

                                                          Questions

                                                          Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                          SERA

                                                          POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                          53

                                                          SERA

                                                          NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                          Negative NEB values cost of barrier

                                                          Solar Water Heat NZ$ Euros

                                                          Solar Design NZ$ Euros

                                                          Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                          -23 -12 (079)

                                                          -11 -6 (06)

                                                          Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                          Source Skumatz Economic Research Associates research

                                                          SERA

                                                          CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                          NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                          -5-4-3-2-1012345

                                                          Op Co

                                                          st

                                                          Init C

                                                          ost

                                                          Maint

                                                          Perfo

                                                          rm

                                                          Prod

                                                          uctiv

                                                          Tena

                                                          nt Sa

                                                          t

                                                          Comf

                                                          ort

                                                          Appe

                                                          ar

                                                          Light

                                                          Qual

                                                          SellL

                                                          ease

                                                          Devp Ownr AampE

                                                          Note AgtE (~4 vs ~2) for these NEBs

                                                          Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                          Source Skumatz Economic Research Associates Inc 2002

                                                          Owners more positive ndashUnderinvestment train

                                                          • Soup to nuts on non-energy benefits NEIs
                                                          • TOPICS
                                                          • BACKGROUND NEBS
                                                          • 20+ YEARS OF NEBS PROGRESShellip
                                                          • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                          • KEY APPLICATIONS OF NEBS
                                                          • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                          • Neb measurement
                                                          • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                          • MEASURING PARTICIPANTNEBs
                                                          • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                          • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                          • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                          • PROGRESS TRANSFERABILITY OF NEBS
                                                          • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                          • PROGRESS AND GAPS
                                                          • PROGRESS AND GAPS
                                                          • USING NEB NEI VALUES
                                                          • Slide Number 19
                                                          • ARE NEBS HIGH VALUE
                                                          • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                          • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                          • RESULTS FOR ENERGY STARreg APPLIANCES
                                                          • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                          • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                          • CampI PROGRAMS ldquoNEBBEDrdquo
                                                          • AUDIT FINANCIAL PROGRAM
                                                          • CampI NEW CONSTRUCTION
                                                          • CampI TECHNICAL ASSISTANCE
                                                          • EXPRESSING NEBS VALUEndashCxe
                                                          • Slide Number 31
                                                          • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                          • EMISSIONS (SOCIETAL NEBS)
                                                          • UTILITY NEBS
                                                          • ADJUSTED PAYBACKS
                                                          • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                          • NEBS IN BC
                                                          • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                          • APPLICATION TO BC
                                                          • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                          • STATE TREATMENT OF NEBS NEBs
                                                          • Slide Number 42
                                                          • SELECTED STATE PROGRESS
                                                          • SELECTED STATE PROGRESS
                                                          • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                          • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                          • DISCOUNT RATE MATTERS
                                                          • EUL VARIATIONS- ADOPTED VALUES
                                                          • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                          • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                          • TAKEAWAYSON NEBSNEIS
                                                          • THANK YOUQuestions
                                                          • Slide Number 53
                                                          • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                          • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                            Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                            m Product losses20
                                                            l Ease of selling leasing building33
                                                            n Labor requirements45
                                                            h Aesthetics appearance59
                                                            j Noise50
                                                            b Equipment maintenance660
                                                            f Tenant satisfaction60
                                                            d Equipment lifetimes718
                                                            e Productivity724
                                                            k Building safety[2]73
                                                            g Comfort933
                                                            i Lighting Quality of Light916
                                                            c Equipment performance[1]1019
                                                            a Operating costs1165
                                                            o Doing good for the Environment[3]1132
                                                            Other03
                                                            [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                            [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                            [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                            m Product losses
                                                            l Ease of selling leasing building
                                                            n Labor requirements
                                                            h Aesthetics appearance
                                                            j Noise
                                                            b Equipment maintenance
                                                            f Tenant satisfaction
                                                            d Equipment lifetimes
                                                            e Productivity
                                                            k Building safety[2]
                                                            g Comfort
                                                            i Lighting Quality of Light
                                                            c Equipment performance[1]
                                                            a Operating costs
                                                            o Doing good for the Environment[3]
                                                            Share of ZALEH NEBS from various measures
                                                            Each presented as multiple of the (overall) energy savings
                                                            Super Insulation037
                                                            Double Glazing026
                                                            Solar Design025
                                                            Solar Water Heating005
                                                            EnergySmart (insul Retrofit)Dunedin (insul)
                                                            Other124
                                                            Noise24
                                                            Budgetfinances25
                                                            Environmental65
                                                            Health78
                                                            Reduced moves86
                                                            Ease of sellingrenting130
                                                            DampnessMold1316
                                                            Comfort-related4933
                                                            101101
                                                            Operational NEBs44
                                                            DesignConstruction NEBs31
                                                            Occupant NEBs25
                                                            1
                                                            OtherOther
                                                            NoiseNoise
                                                            BudgetfinancesBudgetfinances
                                                            EnvironmentalEnvironmental
                                                            HealthHealth
                                                            Reduced movesReduced moves
                                                            Ease of sellingrentingEase of sellingrenting
                                                            DampnessMoldDampnessMold
                                                            Comfort-relatedComfort-related

                                                            SERA

                                                            NEB VALUES

                                                            19

                                                            SERA

                                                            ARE NEBS HIGH VALUE Energy savings are often the minority of

                                                            benefits from program measuresndash Donrsquot ignore that

                                                            20

                                                            0 05 1 15 2 25 3

                                                            Energy Sav

                                                            NEBs

                                                            NEB vs Energy Savings ValueIncluding all NEBs

                                                            Source (SkumatzSERA 2010 amp others)

                                                            Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                                            SERA

                                                            RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                                            MF Many others

                                                            Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                                            Variety of end uses

                                                            SERA

                                                            TOP NEBS FOR WEATHERIZATION PROGRAM

                                                            Weatherization program multipliergt1

                                                            Source SERA Research

                                                            SERA

                                                            RESULTS FOR ENERGY STARreg APPLIANCES

                                                            0002004006008

                                                            01012014016018

                                                            Maint

                                                            Appear

                                                            Perf LifeNoise Satis

                                                            Comfort

                                                            LiteSafe

                                                            ty SellMove

                                                            Water

                                                            Enviro

                                                            Refrig DW CW

                                                            Source SERA Research

                                                            SERA

                                                            INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                                            5 increments to 50

                                                            Source SERA research

                                                            Chart5

                                                            EnergySmart (insul Retrofit)
                                                            Dunedin (insul)
                                                            001
                                                            024
                                                            002
                                                            004
                                                            002
                                                            005
                                                            006
                                                            005
                                                            007
                                                            008
                                                            008
                                                            006
                                                            013
                                                            0
                                                            013
                                                            016
                                                            049
                                                            033

                                                            cx

                                                            cx

                                                            nz

                                                            Sheet3

                                                            Sheet3

                                                            Super Insulation
                                                            Double Glazing
                                                            Solar Design
                                                            Solar Water Heating
                                                            Super Insulation
                                                            Double Glazing
                                                            Solar Design
                                                            Solar Water Heating
                                                            EnergySmart (insul Retrofit)
                                                            Dunedin (insul)

                                                            SERA

                                                            ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                            0

                                                            002

                                                            004

                                                            006

                                                            008

                                                            01

                                                            012

                                                            Maint

                                                            Perf LifeSatis

                                                            f

                                                            ComfortAesth

                                                            NoiseSafe

                                                            ty

                                                            LightingSelli

                                                            ng

                                                            Stay in home

                                                            Enviro

                                                            Sick Days

                                                            Calls

                                                            ES Homes HP wES Energy savings multiplier=1

                                                            Source SERA Research

                                                            SERA

                                                            CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                            DG CHP HVAC Equipment rebate Other

                                                            Building codes incentives by cities

                                                            Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                            Variety of end uses

                                                            SERA

                                                            AUDIT FINANCIAL PROGRAM

                                                            0

                                                            01

                                                            02

                                                            03

                                                            04

                                                            05

                                                            06

                                                            07

                                                            Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                            Source SERA Research

                                                            SERA

                                                            CampI NEW CONSTRUCTION

                                                            -005

                                                            0

                                                            005

                                                            01

                                                            015

                                                            02

                                                            AampE Owner(Pct of Total Partic NEBs)

                                                            Source SERA Research

                                                            SERA

                                                            CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                            0 2 4 6 8 10 12

                                                            m Product losses

                                                            l Ease of selling leasing building

                                                            n Labor requirements

                                                            h Aesthetics appearance

                                                            j Noise

                                                            b Equipment maintenance

                                                            f Tenant satisfaction

                                                            d Equipment lifetimes

                                                            e Productivity

                                                            k Building safety[2]

                                                            g Comfort

                                                            i Lighting Quality of Light

                                                            c Equipment performance[1]

                                                            a Operating costs

                                                            o Doing good for the Environment[3]

                                                            Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                            vcv

                                                            Source SERA Research

                                                            Chart1

                                                            CampI Technical Assistance NEBs
                                                            002
                                                            003
                                                            004
                                                            005
                                                            005
                                                            006
                                                            006
                                                            007
                                                            007
                                                            007
                                                            009
                                                            009
                                                            01
                                                            011
                                                            011

                                                            Sheet1

                                                            Sheet1

                                                            CampI Technical Assistance Program NEBs (Participant)

                                                            Sheet2

                                                            Sheet3

                                                            SERA

                                                            EXPRESSING NEBS VALUEndashCxe

                                                            Strong value from RetroCxSource SERA Research

                                                            SERA

                                                            SOCIETAL NEB EXAMPLES

                                                            31

                                                            SERA

                                                            JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                            32

                                                            0

                                                            02

                                                            04

                                                            06

                                                            08

                                                            1

                                                            12

                                                            HPWxRetrof Appliance

                                                            CA

                                                            WI

                                                            Natl

                                                            (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                            Jobs Economic

                                                            SERA

                                                            EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                            33

                                                            SERA

                                                            UTILITY NEBSUtility NEBs for Template Program

                                                            Debt WriteOff (util)13

                                                            Rate Subsidy(util)61

                                                            HealthSafety(util)0

                                                            Colln Costs (util)0

                                                            Gas Calls (util)0

                                                            Calls to CSRs(util)2

                                                            TampD (util)16

                                                            Arrears (util)0

                                                            Reconnects (util)0

                                                            Notices (util)7

                                                            Shutoffs (util)1

                                                            Rate subsidy TampD

                                                            Payment-related

                                                            Source Skumatz Economic Research Associates research

                                                            Example Low Income Weatherization

                                                            SERA

                                                            ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                            be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                            Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                            Value of 100 HALVES Payback doubles ROI

                                                            Source Skumatz Economics (SERA)

                                                            35

                                                            ( for NEBs Multiplier of 125)

                                                            SERA

                                                            NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                            36

                                                            hellip2001 ON

                                                            SERA

                                                            37

                                                            NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                            For true representation of B amp C elements of NEBs address missing factor bias

                                                            Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                            BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                            SERA

                                                            CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                            38

                                                            Test Benefits Costs States Using Traditionally

                                                            Improved treat-ment with NEBs

                                                            Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                            bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                            bull Avoided gas and water supply costs

                                                            bull Program administration

                                                            bull Participant incentives

                                                            bull Increased supply cost

                                                            CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                            Use cost only paid by the utility

                                                            Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                            Same as above plus bull increased revenue

                                                            Same as above plus bull Decreased revenue

                                                            AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                            Participant cost bull Utility bill reductions bull Participant incentives

                                                            bull Participant direct costs

                                                            AR CA FL HI IA IN MI MN NY VA

                                                            Participant NEBs

                                                            Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                            Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                            SERA

                                                            APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                            Expansion Concern with risk calculation accuracy

                                                            39

                                                            Source Skumatz SERA research

                                                            SERA

                                                            40

                                                            METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                            Maximize DSM opportunities amp feedback Accuracy tailoring

                                                            Minimize Regulatory amp Implementer Risk

                                                            Minimize Evaluation Cost

                                                            Adder

                                                            Readily MeasurableHybrid

                                                            All NEBsSource SERA Research

                                                            SERA

                                                            41

                                                            STATE TREATMENT OF NEBS NEBsRegulatory

                                                            Screening Application Utilities regions

                                                            Program Marketing Fairly widespread use in utilities states across the country

                                                            MO

                                                            RE AGGRESSIVE ==gt

                                                            Test Pgm Screen - adder

                                                            IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                            Test Pgm Screen - readily measured

                                                            MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                            Test Hybrid (potential adder amp measured)

                                                            CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                            Test Pgm screen - Broad

                                                            With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                            Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                            SERA

                                                            42

                                                            Key

                                                            States with NEBs in CE tests for At least one type of program

                                                            copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                            STATES WITH NEBS IN CE TESTS

                                                            NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                            groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                            SERA

                                                            SELECTED STATE PROGRESS CA

                                                            2001 ndash LIPPT model NY

                                                            Mid-2000s measurement amp scenarios not included CO

                                                            Adders (10 electric 25 LI 5 gas) VT

                                                            15 adder helped by previous research DC

                                                            10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                            43

                                                            Source Skumatz SERA research

                                                            SERA

                                                            SELECTED STATE PROGRESS Dominos ongoing

                                                            Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                            Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                            Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                            Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                            44

                                                            Source Skumatz SERA research

                                                            SERA

                                                            APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                            Expansion Concern with risk calculation accuracy

                                                            45

                                                            Source Skumatz SERA research

                                                            SERA

                                                            46

                                                            NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                            [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                            SourceSkumatz SERA Independent research

                                                            NTG ndash accuracy measurement incomplete

                                                            Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                            NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                            EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                            Cost Complicated expensive local changes

                                                            RELATIVE Risk from NEBs

                                                            Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                            Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                            NEBs as accurate as many inputs (all could improve)

                                                            SERA

                                                            DISCOUNT RATE MATTERS

                                                            47

                                                            Source Skumatz SERA Research

                                                            0

                                                            20

                                                            40

                                                            60

                                                            80

                                                            100

                                                            120

                                                            1 2 3 4 5 6 7 8 9 10

                                                            PV Ratios vs 1 Discount Rate

                                                            10 yr PV 20 yr PV

                                                            Discount rate - percent

                                                            WACCSocialLTT

                                                            Source Skumatz Electricity Journal IEPEC paper independent research

                                                            SERA

                                                            EUL VARIATIONS-ADOPTED VALUES

                                                            Source Skumatz Economic Research Associates SERA independent research

                                                            0

                                                            50

                                                            100

                                                            150

                                                            200

                                                            250

                                                            300

                                                            350

                                                            400

                                                            450

                                                            0 10 20 30 40 50 60 70

                                                            EUL Variation by Measure

                                                            67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                            Poorly supportedOldTechnology change

                                                            SERA

                                                            KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                            Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                            measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                            change the BC conclusion

                                                            On the cusphellip

                                                            If NO Yoursquore doneAnd bias addressedsufficiently

                                                            IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                            Source SERA all rights reserved

                                                            SERArsquos lsquoNEB-ItrdquoModel ndash

                                                            SERA

                                                            IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                            Utility Soc Part Conserv Recrsquom

                                                            Rationale

                                                            Base Percent X X X Program-invariant

                                                            Low Income X X X X Multiple sources

                                                            Weatherization X X X Substantial Participantimpacts

                                                            Measure Program-specific

                                                            X Varies by measure sector

                                                            Other Recomrsquos Local Research

                                                            50

                                                            Source Skumatz SERA

                                                            Developing values for multiple states amp utilities

                                                            SERA

                                                            TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                            Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                            Other sources of bias to research (EUL NTG etc) bring high variation

                                                            5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                            6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                            THANK YOU

                                                            Questions

                                                            Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                            SERA

                                                            POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                            53

                                                            SERA

                                                            NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                            Negative NEB values cost of barrier

                                                            Solar Water Heat NZ$ Euros

                                                            Solar Design NZ$ Euros

                                                            Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                            -23 -12 (079)

                                                            -11 -6 (06)

                                                            Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                            Source Skumatz Economic Research Associates research

                                                            SERA

                                                            CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                            NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                            -5-4-3-2-1012345

                                                            Op Co

                                                            st

                                                            Init C

                                                            ost

                                                            Maint

                                                            Perfo

                                                            rm

                                                            Prod

                                                            uctiv

                                                            Tena

                                                            nt Sa

                                                            t

                                                            Comf

                                                            ort

                                                            Appe

                                                            ar

                                                            Light

                                                            Qual

                                                            SellL

                                                            ease

                                                            Devp Ownr AampE

                                                            Note AgtE (~4 vs ~2) for these NEBs

                                                            Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                            Source Skumatz Economic Research Associates Inc 2002

                                                            Owners more positive ndashUnderinvestment train

                                                            • Soup to nuts on non-energy benefits NEIs
                                                            • TOPICS
                                                            • BACKGROUND NEBS
                                                            • 20+ YEARS OF NEBS PROGRESShellip
                                                            • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                            • KEY APPLICATIONS OF NEBS
                                                            • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                            • Neb measurement
                                                            • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                            • MEASURING PARTICIPANTNEBs
                                                            • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                            • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                            • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                            • PROGRESS TRANSFERABILITY OF NEBS
                                                            • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                            • PROGRESS AND GAPS
                                                            • PROGRESS AND GAPS
                                                            • USING NEB NEI VALUES
                                                            • Slide Number 19
                                                            • ARE NEBS HIGH VALUE
                                                            • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                            • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                            • RESULTS FOR ENERGY STARreg APPLIANCES
                                                            • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                            • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                            • CampI PROGRAMS ldquoNEBBEDrdquo
                                                            • AUDIT FINANCIAL PROGRAM
                                                            • CampI NEW CONSTRUCTION
                                                            • CampI TECHNICAL ASSISTANCE
                                                            • EXPRESSING NEBS VALUEndashCxe
                                                            • Slide Number 31
                                                            • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                            • EMISSIONS (SOCIETAL NEBS)
                                                            • UTILITY NEBS
                                                            • ADJUSTED PAYBACKS
                                                            • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                            • NEBS IN BC
                                                            • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                            • APPLICATION TO BC
                                                            • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                            • STATE TREATMENT OF NEBS NEBs
                                                            • Slide Number 42
                                                            • SELECTED STATE PROGRESS
                                                            • SELECTED STATE PROGRESS
                                                            • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                            • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                            • DISCOUNT RATE MATTERS
                                                            • EUL VARIATIONS- ADOPTED VALUES
                                                            • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                            • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                            • TAKEAWAYSON NEBSNEIS
                                                            • THANK YOUQuestions
                                                            • Slide Number 53
                                                            • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                            • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                              Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                              m Product losses20
                                                              l Ease of selling leasing building33
                                                              n Labor requirements45
                                                              h Aesthetics appearance59
                                                              j Noise50
                                                              b Equipment maintenance660
                                                              f Tenant satisfaction60
                                                              d Equipment lifetimes718
                                                              e Productivity724
                                                              k Building safety[2]73
                                                              g Comfort933
                                                              i Lighting Quality of Light916
                                                              c Equipment performance[1]1019
                                                              a Operating costs1165
                                                              o Doing good for the Environment[3]1132
                                                              Other03
                                                              [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                              [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                              [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                              m Product losses
                                                              l Ease of selling leasing building
                                                              n Labor requirements
                                                              h Aesthetics appearance
                                                              j Noise
                                                              b Equipment maintenance
                                                              f Tenant satisfaction
                                                              d Equipment lifetimes
                                                              e Productivity
                                                              k Building safety[2]
                                                              g Comfort
                                                              i Lighting Quality of Light
                                                              c Equipment performance[1]
                                                              a Operating costs
                                                              o Doing good for the Environment[3]
                                                              Share of ZALEH NEBS from various measures
                                                              Each presented as multiple of the (overall) energy savings
                                                              Super Insulation037
                                                              Double Glazing026
                                                              Solar Design025
                                                              Solar Water Heating005
                                                              EnergySmart (insul Retrofit)Dunedin (insul)
                                                              Other124
                                                              Noise24
                                                              Budgetfinances25
                                                              Environmental65
                                                              Health78
                                                              Reduced moves86
                                                              Ease of sellingrenting130
                                                              DampnessMold1316
                                                              Comfort-related4933
                                                              101101
                                                              Operational NEBs44
                                                              DesignConstruction NEBs31
                                                              Occupant NEBs25
                                                              1
                                                              OtherOther
                                                              NoiseNoise
                                                              BudgetfinancesBudgetfinances
                                                              EnvironmentalEnvironmental
                                                              HealthHealth
                                                              Reduced movesReduced moves
                                                              Ease of sellingrentingEase of sellingrenting
                                                              DampnessMoldDampnessMold
                                                              Comfort-relatedComfort-related

                                                              SERA

                                                              NEB VALUES

                                                              19

                                                              SERA

                                                              ARE NEBS HIGH VALUE Energy savings are often the minority of

                                                              benefits from program measuresndash Donrsquot ignore that

                                                              20

                                                              0 05 1 15 2 25 3

                                                              Energy Sav

                                                              NEBs

                                                              NEB vs Energy Savings ValueIncluding all NEBs

                                                              Source (SkumatzSERA 2010 amp others)

                                                              Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                                              SERA

                                                              RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                                              MF Many others

                                                              Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                                              Variety of end uses

                                                              SERA

                                                              TOP NEBS FOR WEATHERIZATION PROGRAM

                                                              Weatherization program multipliergt1

                                                              Source SERA Research

                                                              SERA

                                                              RESULTS FOR ENERGY STARreg APPLIANCES

                                                              0002004006008

                                                              01012014016018

                                                              Maint

                                                              Appear

                                                              Perf LifeNoise Satis

                                                              Comfort

                                                              LiteSafe

                                                              ty SellMove

                                                              Water

                                                              Enviro

                                                              Refrig DW CW

                                                              Source SERA Research

                                                              SERA

                                                              INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                                              5 increments to 50

                                                              Source SERA research

                                                              Chart5

                                                              EnergySmart (insul Retrofit)
                                                              Dunedin (insul)
                                                              001
                                                              024
                                                              002
                                                              004
                                                              002
                                                              005
                                                              006
                                                              005
                                                              007
                                                              008
                                                              008
                                                              006
                                                              013
                                                              0
                                                              013
                                                              016
                                                              049
                                                              033

                                                              cx

                                                              cx

                                                              nz

                                                              Sheet3

                                                              Sheet3

                                                              Super Insulation
                                                              Double Glazing
                                                              Solar Design
                                                              Solar Water Heating
                                                              Super Insulation
                                                              Double Glazing
                                                              Solar Design
                                                              Solar Water Heating
                                                              EnergySmart (insul Retrofit)
                                                              Dunedin (insul)

                                                              SERA

                                                              ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                              0

                                                              002

                                                              004

                                                              006

                                                              008

                                                              01

                                                              012

                                                              Maint

                                                              Perf LifeSatis

                                                              f

                                                              ComfortAesth

                                                              NoiseSafe

                                                              ty

                                                              LightingSelli

                                                              ng

                                                              Stay in home

                                                              Enviro

                                                              Sick Days

                                                              Calls

                                                              ES Homes HP wES Energy savings multiplier=1

                                                              Source SERA Research

                                                              SERA

                                                              CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                              DG CHP HVAC Equipment rebate Other

                                                              Building codes incentives by cities

                                                              Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                              Variety of end uses

                                                              SERA

                                                              AUDIT FINANCIAL PROGRAM

                                                              0

                                                              01

                                                              02

                                                              03

                                                              04

                                                              05

                                                              06

                                                              07

                                                              Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                              Source SERA Research

                                                              SERA

                                                              CampI NEW CONSTRUCTION

                                                              -005

                                                              0

                                                              005

                                                              01

                                                              015

                                                              02

                                                              AampE Owner(Pct of Total Partic NEBs)

                                                              Source SERA Research

                                                              SERA

                                                              CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                              0 2 4 6 8 10 12

                                                              m Product losses

                                                              l Ease of selling leasing building

                                                              n Labor requirements

                                                              h Aesthetics appearance

                                                              j Noise

                                                              b Equipment maintenance

                                                              f Tenant satisfaction

                                                              d Equipment lifetimes

                                                              e Productivity

                                                              k Building safety[2]

                                                              g Comfort

                                                              i Lighting Quality of Light

                                                              c Equipment performance[1]

                                                              a Operating costs

                                                              o Doing good for the Environment[3]

                                                              Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                              vcv

                                                              Source SERA Research

                                                              Chart1

                                                              CampI Technical Assistance NEBs
                                                              002
                                                              003
                                                              004
                                                              005
                                                              005
                                                              006
                                                              006
                                                              007
                                                              007
                                                              007
                                                              009
                                                              009
                                                              01
                                                              011
                                                              011

                                                              Sheet1

                                                              Sheet1

                                                              CampI Technical Assistance Program NEBs (Participant)

                                                              Sheet2

                                                              Sheet3

                                                              SERA

                                                              EXPRESSING NEBS VALUEndashCxe

                                                              Strong value from RetroCxSource SERA Research

                                                              SERA

                                                              SOCIETAL NEB EXAMPLES

                                                              31

                                                              SERA

                                                              JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                              32

                                                              0

                                                              02

                                                              04

                                                              06

                                                              08

                                                              1

                                                              12

                                                              HPWxRetrof Appliance

                                                              CA

                                                              WI

                                                              Natl

                                                              (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                              Jobs Economic

                                                              SERA

                                                              EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                              33

                                                              SERA

                                                              UTILITY NEBSUtility NEBs for Template Program

                                                              Debt WriteOff (util)13

                                                              Rate Subsidy(util)61

                                                              HealthSafety(util)0

                                                              Colln Costs (util)0

                                                              Gas Calls (util)0

                                                              Calls to CSRs(util)2

                                                              TampD (util)16

                                                              Arrears (util)0

                                                              Reconnects (util)0

                                                              Notices (util)7

                                                              Shutoffs (util)1

                                                              Rate subsidy TampD

                                                              Payment-related

                                                              Source Skumatz Economic Research Associates research

                                                              Example Low Income Weatherization

                                                              SERA

                                                              ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                              be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                              Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                              Value of 100 HALVES Payback doubles ROI

                                                              Source Skumatz Economics (SERA)

                                                              35

                                                              ( for NEBs Multiplier of 125)

                                                              SERA

                                                              NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                              36

                                                              hellip2001 ON

                                                              SERA

                                                              37

                                                              NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                              For true representation of B amp C elements of NEBs address missing factor bias

                                                              Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                              BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                              SERA

                                                              CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                              38

                                                              Test Benefits Costs States Using Traditionally

                                                              Improved treat-ment with NEBs

                                                              Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                              bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                              bull Avoided gas and water supply costs

                                                              bull Program administration

                                                              bull Participant incentives

                                                              bull Increased supply cost

                                                              CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                              Use cost only paid by the utility

                                                              Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                              Same as above plus bull increased revenue

                                                              Same as above plus bull Decreased revenue

                                                              AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                              Participant cost bull Utility bill reductions bull Participant incentives

                                                              bull Participant direct costs

                                                              AR CA FL HI IA IN MI MN NY VA

                                                              Participant NEBs

                                                              Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                              Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                              SERA

                                                              APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                              Expansion Concern with risk calculation accuracy

                                                              39

                                                              Source Skumatz SERA research

                                                              SERA

                                                              40

                                                              METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                              Maximize DSM opportunities amp feedback Accuracy tailoring

                                                              Minimize Regulatory amp Implementer Risk

                                                              Minimize Evaluation Cost

                                                              Adder

                                                              Readily MeasurableHybrid

                                                              All NEBsSource SERA Research

                                                              SERA

                                                              41

                                                              STATE TREATMENT OF NEBS NEBsRegulatory

                                                              Screening Application Utilities regions

                                                              Program Marketing Fairly widespread use in utilities states across the country

                                                              MO

                                                              RE AGGRESSIVE ==gt

                                                              Test Pgm Screen - adder

                                                              IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                              Test Pgm Screen - readily measured

                                                              MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                              Test Hybrid (potential adder amp measured)

                                                              CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                              Test Pgm screen - Broad

                                                              With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                              Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                              SERA

                                                              42

                                                              Key

                                                              States with NEBs in CE tests for At least one type of program

                                                              copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                              STATES WITH NEBS IN CE TESTS

                                                              NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                              groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                              SERA

                                                              SELECTED STATE PROGRESS CA

                                                              2001 ndash LIPPT model NY

                                                              Mid-2000s measurement amp scenarios not included CO

                                                              Adders (10 electric 25 LI 5 gas) VT

                                                              15 adder helped by previous research DC

                                                              10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                              43

                                                              Source Skumatz SERA research

                                                              SERA

                                                              SELECTED STATE PROGRESS Dominos ongoing

                                                              Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                              Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                              Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                              Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                              44

                                                              Source Skumatz SERA research

                                                              SERA

                                                              APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                              Expansion Concern with risk calculation accuracy

                                                              45

                                                              Source Skumatz SERA research

                                                              SERA

                                                              46

                                                              NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                              [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                              SourceSkumatz SERA Independent research

                                                              NTG ndash accuracy measurement incomplete

                                                              Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                              NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                              EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                              Cost Complicated expensive local changes

                                                              RELATIVE Risk from NEBs

                                                              Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                              Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                              NEBs as accurate as many inputs (all could improve)

                                                              SERA

                                                              DISCOUNT RATE MATTERS

                                                              47

                                                              Source Skumatz SERA Research

                                                              0

                                                              20

                                                              40

                                                              60

                                                              80

                                                              100

                                                              120

                                                              1 2 3 4 5 6 7 8 9 10

                                                              PV Ratios vs 1 Discount Rate

                                                              10 yr PV 20 yr PV

                                                              Discount rate - percent

                                                              WACCSocialLTT

                                                              Source Skumatz Electricity Journal IEPEC paper independent research

                                                              SERA

                                                              EUL VARIATIONS-ADOPTED VALUES

                                                              Source Skumatz Economic Research Associates SERA independent research

                                                              0

                                                              50

                                                              100

                                                              150

                                                              200

                                                              250

                                                              300

                                                              350

                                                              400

                                                              450

                                                              0 10 20 30 40 50 60 70

                                                              EUL Variation by Measure

                                                              67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                              Poorly supportedOldTechnology change

                                                              SERA

                                                              KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                              Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                              measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                              change the BC conclusion

                                                              On the cusphellip

                                                              If NO Yoursquore doneAnd bias addressedsufficiently

                                                              IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                              Source SERA all rights reserved

                                                              SERArsquos lsquoNEB-ItrdquoModel ndash

                                                              SERA

                                                              IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                              Utility Soc Part Conserv Recrsquom

                                                              Rationale

                                                              Base Percent X X X Program-invariant

                                                              Low Income X X X X Multiple sources

                                                              Weatherization X X X Substantial Participantimpacts

                                                              Measure Program-specific

                                                              X Varies by measure sector

                                                              Other Recomrsquos Local Research

                                                              50

                                                              Source Skumatz SERA

                                                              Developing values for multiple states amp utilities

                                                              SERA

                                                              TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                              Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                              Other sources of bias to research (EUL NTG etc) bring high variation

                                                              5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                              6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                              THANK YOU

                                                              Questions

                                                              Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                              SERA

                                                              POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                              53

                                                              SERA

                                                              NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                              Negative NEB values cost of barrier

                                                              Solar Water Heat NZ$ Euros

                                                              Solar Design NZ$ Euros

                                                              Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                              -23 -12 (079)

                                                              -11 -6 (06)

                                                              Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                              Source Skumatz Economic Research Associates research

                                                              SERA

                                                              CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                              NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                              -5-4-3-2-1012345

                                                              Op Co

                                                              st

                                                              Init C

                                                              ost

                                                              Maint

                                                              Perfo

                                                              rm

                                                              Prod

                                                              uctiv

                                                              Tena

                                                              nt Sa

                                                              t

                                                              Comf

                                                              ort

                                                              Appe

                                                              ar

                                                              Light

                                                              Qual

                                                              SellL

                                                              ease

                                                              Devp Ownr AampE

                                                              Note AgtE (~4 vs ~2) for these NEBs

                                                              Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                              Source Skumatz Economic Research Associates Inc 2002

                                                              Owners more positive ndashUnderinvestment train

                                                              • Soup to nuts on non-energy benefits NEIs
                                                              • TOPICS
                                                              • BACKGROUND NEBS
                                                              • 20+ YEARS OF NEBS PROGRESShellip
                                                              • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                              • KEY APPLICATIONS OF NEBS
                                                              • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                              • Neb measurement
                                                              • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                              • MEASURING PARTICIPANTNEBs
                                                              • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                              • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                              • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                              • PROGRESS TRANSFERABILITY OF NEBS
                                                              • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                              • PROGRESS AND GAPS
                                                              • PROGRESS AND GAPS
                                                              • USING NEB NEI VALUES
                                                              • Slide Number 19
                                                              • ARE NEBS HIGH VALUE
                                                              • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                              • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                              • RESULTS FOR ENERGY STARreg APPLIANCES
                                                              • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                              • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                              • CampI PROGRAMS ldquoNEBBEDrdquo
                                                              • AUDIT FINANCIAL PROGRAM
                                                              • CampI NEW CONSTRUCTION
                                                              • CampI TECHNICAL ASSISTANCE
                                                              • EXPRESSING NEBS VALUEndashCxe
                                                              • Slide Number 31
                                                              • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                              • EMISSIONS (SOCIETAL NEBS)
                                                              • UTILITY NEBS
                                                              • ADJUSTED PAYBACKS
                                                              • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                              • NEBS IN BC
                                                              • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                              • APPLICATION TO BC
                                                              • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                              • STATE TREATMENT OF NEBS NEBs
                                                              • Slide Number 42
                                                              • SELECTED STATE PROGRESS
                                                              • SELECTED STATE PROGRESS
                                                              • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                              • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                              • DISCOUNT RATE MATTERS
                                                              • EUL VARIATIONS- ADOPTED VALUES
                                                              • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                              • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                              • TAKEAWAYSON NEBSNEIS
                                                              • THANK YOUQuestions
                                                              • Slide Number 53
                                                              • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                              • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                m Product losses20
                                                                l Ease of selling leasing building33
                                                                n Labor requirements45
                                                                h Aesthetics appearance59
                                                                j Noise50
                                                                b Equipment maintenance660
                                                                f Tenant satisfaction60
                                                                d Equipment lifetimes718
                                                                e Productivity724
                                                                k Building safety[2]73
                                                                g Comfort933
                                                                i Lighting Quality of Light916
                                                                c Equipment performance[1]1019
                                                                a Operating costs1165
                                                                o Doing good for the Environment[3]1132
                                                                Other03
                                                                [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                m Product losses
                                                                l Ease of selling leasing building
                                                                n Labor requirements
                                                                h Aesthetics appearance
                                                                j Noise
                                                                b Equipment maintenance
                                                                f Tenant satisfaction
                                                                d Equipment lifetimes
                                                                e Productivity
                                                                k Building safety[2]
                                                                g Comfort
                                                                i Lighting Quality of Light
                                                                c Equipment performance[1]
                                                                a Operating costs
                                                                o Doing good for the Environment[3]
                                                                Share of ZALEH NEBS from various measures
                                                                Each presented as multiple of the (overall) energy savings
                                                                Super Insulation037
                                                                Double Glazing026
                                                                Solar Design025
                                                                Solar Water Heating005
                                                                EnergySmart (insul Retrofit)Dunedin (insul)
                                                                Other124
                                                                Noise24
                                                                Budgetfinances25
                                                                Environmental65
                                                                Health78
                                                                Reduced moves86
                                                                Ease of sellingrenting130
                                                                DampnessMold1316
                                                                Comfort-related4933
                                                                101101
                                                                Operational NEBs44
                                                                DesignConstruction NEBs31
                                                                Occupant NEBs25
                                                                1
                                                                OtherOther
                                                                NoiseNoise
                                                                BudgetfinancesBudgetfinances
                                                                EnvironmentalEnvironmental
                                                                HealthHealth
                                                                Reduced movesReduced moves
                                                                Ease of sellingrentingEase of sellingrenting
                                                                DampnessMoldDampnessMold
                                                                Comfort-relatedComfort-related

                                                                SERA

                                                                ARE NEBS HIGH VALUE Energy savings are often the minority of

                                                                benefits from program measuresndash Donrsquot ignore that

                                                                20

                                                                0 05 1 15 2 25 3

                                                                Energy Sav

                                                                NEBs

                                                                NEB vs Energy Savings ValueIncluding all NEBs

                                                                Source (SkumatzSERA 2010 amp others)

                                                                Omitting canmisrepresent decisionmaking ampmpactshellip withimplications

                                                                SERA

                                                                RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                                                MF Many others

                                                                Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                                                Variety of end uses

                                                                SERA

                                                                TOP NEBS FOR WEATHERIZATION PROGRAM

                                                                Weatherization program multipliergt1

                                                                Source SERA Research

                                                                SERA

                                                                RESULTS FOR ENERGY STARreg APPLIANCES

                                                                0002004006008

                                                                01012014016018

                                                                Maint

                                                                Appear

                                                                Perf LifeNoise Satis

                                                                Comfort

                                                                LiteSafe

                                                                ty SellMove

                                                                Water

                                                                Enviro

                                                                Refrig DW CW

                                                                Source SERA Research

                                                                SERA

                                                                INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                                                5 increments to 50

                                                                Source SERA research

                                                                Chart5

                                                                EnergySmart (insul Retrofit)
                                                                Dunedin (insul)
                                                                001
                                                                024
                                                                002
                                                                004
                                                                002
                                                                005
                                                                006
                                                                005
                                                                007
                                                                008
                                                                008
                                                                006
                                                                013
                                                                0
                                                                013
                                                                016
                                                                049
                                                                033

                                                                cx

                                                                cx

                                                                nz

                                                                Sheet3

                                                                Sheet3

                                                                Super Insulation
                                                                Double Glazing
                                                                Solar Design
                                                                Solar Water Heating
                                                                Super Insulation
                                                                Double Glazing
                                                                Solar Design
                                                                Solar Water Heating
                                                                EnergySmart (insul Retrofit)
                                                                Dunedin (insul)

                                                                SERA

                                                                ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                                0

                                                                002

                                                                004

                                                                006

                                                                008

                                                                01

                                                                012

                                                                Maint

                                                                Perf LifeSatis

                                                                f

                                                                ComfortAesth

                                                                NoiseSafe

                                                                ty

                                                                LightingSelli

                                                                ng

                                                                Stay in home

                                                                Enviro

                                                                Sick Days

                                                                Calls

                                                                ES Homes HP wES Energy savings multiplier=1

                                                                Source SERA Research

                                                                SERA

                                                                CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                DG CHP HVAC Equipment rebate Other

                                                                Building codes incentives by cities

                                                                Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                Variety of end uses

                                                                SERA

                                                                AUDIT FINANCIAL PROGRAM

                                                                0

                                                                01

                                                                02

                                                                03

                                                                04

                                                                05

                                                                06

                                                                07

                                                                Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                Source SERA Research

                                                                SERA

                                                                CampI NEW CONSTRUCTION

                                                                -005

                                                                0

                                                                005

                                                                01

                                                                015

                                                                02

                                                                AampE Owner(Pct of Total Partic NEBs)

                                                                Source SERA Research

                                                                SERA

                                                                CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                0 2 4 6 8 10 12

                                                                m Product losses

                                                                l Ease of selling leasing building

                                                                n Labor requirements

                                                                h Aesthetics appearance

                                                                j Noise

                                                                b Equipment maintenance

                                                                f Tenant satisfaction

                                                                d Equipment lifetimes

                                                                e Productivity

                                                                k Building safety[2]

                                                                g Comfort

                                                                i Lighting Quality of Light

                                                                c Equipment performance[1]

                                                                a Operating costs

                                                                o Doing good for the Environment[3]

                                                                Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                vcv

                                                                Source SERA Research

                                                                Chart1

                                                                CampI Technical Assistance NEBs
                                                                002
                                                                003
                                                                004
                                                                005
                                                                005
                                                                006
                                                                006
                                                                007
                                                                007
                                                                007
                                                                009
                                                                009
                                                                01
                                                                011
                                                                011

                                                                Sheet1

                                                                Sheet1

                                                                CampI Technical Assistance Program NEBs (Participant)

                                                                Sheet2

                                                                Sheet3

                                                                SERA

                                                                EXPRESSING NEBS VALUEndashCxe

                                                                Strong value from RetroCxSource SERA Research

                                                                SERA

                                                                SOCIETAL NEB EXAMPLES

                                                                31

                                                                SERA

                                                                JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                32

                                                                0

                                                                02

                                                                04

                                                                06

                                                                08

                                                                1

                                                                12

                                                                HPWxRetrof Appliance

                                                                CA

                                                                WI

                                                                Natl

                                                                (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                Jobs Economic

                                                                SERA

                                                                EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                33

                                                                SERA

                                                                UTILITY NEBSUtility NEBs for Template Program

                                                                Debt WriteOff (util)13

                                                                Rate Subsidy(util)61

                                                                HealthSafety(util)0

                                                                Colln Costs (util)0

                                                                Gas Calls (util)0

                                                                Calls to CSRs(util)2

                                                                TampD (util)16

                                                                Arrears (util)0

                                                                Reconnects (util)0

                                                                Notices (util)7

                                                                Shutoffs (util)1

                                                                Rate subsidy TampD

                                                                Payment-related

                                                                Source Skumatz Economic Research Associates research

                                                                Example Low Income Weatherization

                                                                SERA

                                                                ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                Value of 100 HALVES Payback doubles ROI

                                                                Source Skumatz Economics (SERA)

                                                                35

                                                                ( for NEBs Multiplier of 125)

                                                                SERA

                                                                NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                36

                                                                hellip2001 ON

                                                                SERA

                                                                37

                                                                NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                For true representation of B amp C elements of NEBs address missing factor bias

                                                                Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                SERA

                                                                CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                38

                                                                Test Benefits Costs States Using Traditionally

                                                                Improved treat-ment with NEBs

                                                                Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                bull Avoided gas and water supply costs

                                                                bull Program administration

                                                                bull Participant incentives

                                                                bull Increased supply cost

                                                                CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                Use cost only paid by the utility

                                                                Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                Same as above plus bull increased revenue

                                                                Same as above plus bull Decreased revenue

                                                                AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                Participant cost bull Utility bill reductions bull Participant incentives

                                                                bull Participant direct costs

                                                                AR CA FL HI IA IN MI MN NY VA

                                                                Participant NEBs

                                                                Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                SERA

                                                                APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                Expansion Concern with risk calculation accuracy

                                                                39

                                                                Source Skumatz SERA research

                                                                SERA

                                                                40

                                                                METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                Minimize Regulatory amp Implementer Risk

                                                                Minimize Evaluation Cost

                                                                Adder

                                                                Readily MeasurableHybrid

                                                                All NEBsSource SERA Research

                                                                SERA

                                                                41

                                                                STATE TREATMENT OF NEBS NEBsRegulatory

                                                                Screening Application Utilities regions

                                                                Program Marketing Fairly widespread use in utilities states across the country

                                                                MO

                                                                RE AGGRESSIVE ==gt

                                                                Test Pgm Screen - adder

                                                                IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                Test Pgm Screen - readily measured

                                                                MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                Test Hybrid (potential adder amp measured)

                                                                CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                Test Pgm screen - Broad

                                                                With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                SERA

                                                                42

                                                                Key

                                                                States with NEBs in CE tests for At least one type of program

                                                                copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                STATES WITH NEBS IN CE TESTS

                                                                NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                SERA

                                                                SELECTED STATE PROGRESS CA

                                                                2001 ndash LIPPT model NY

                                                                Mid-2000s measurement amp scenarios not included CO

                                                                Adders (10 electric 25 LI 5 gas) VT

                                                                15 adder helped by previous research DC

                                                                10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                43

                                                                Source Skumatz SERA research

                                                                SERA

                                                                SELECTED STATE PROGRESS Dominos ongoing

                                                                Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                44

                                                                Source Skumatz SERA research

                                                                SERA

                                                                APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                Expansion Concern with risk calculation accuracy

                                                                45

                                                                Source Skumatz SERA research

                                                                SERA

                                                                46

                                                                NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                SourceSkumatz SERA Independent research

                                                                NTG ndash accuracy measurement incomplete

                                                                Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                Cost Complicated expensive local changes

                                                                RELATIVE Risk from NEBs

                                                                Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                NEBs as accurate as many inputs (all could improve)

                                                                SERA

                                                                DISCOUNT RATE MATTERS

                                                                47

                                                                Source Skumatz SERA Research

                                                                0

                                                                20

                                                                40

                                                                60

                                                                80

                                                                100

                                                                120

                                                                1 2 3 4 5 6 7 8 9 10

                                                                PV Ratios vs 1 Discount Rate

                                                                10 yr PV 20 yr PV

                                                                Discount rate - percent

                                                                WACCSocialLTT

                                                                Source Skumatz Electricity Journal IEPEC paper independent research

                                                                SERA

                                                                EUL VARIATIONS-ADOPTED VALUES

                                                                Source Skumatz Economic Research Associates SERA independent research

                                                                0

                                                                50

                                                                100

                                                                150

                                                                200

                                                                250

                                                                300

                                                                350

                                                                400

                                                                450

                                                                0 10 20 30 40 50 60 70

                                                                EUL Variation by Measure

                                                                67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                Poorly supportedOldTechnology change

                                                                SERA

                                                                KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                change the BC conclusion

                                                                On the cusphellip

                                                                If NO Yoursquore doneAnd bias addressedsufficiently

                                                                IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                Source SERA all rights reserved

                                                                SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                SERA

                                                                IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                Utility Soc Part Conserv Recrsquom

                                                                Rationale

                                                                Base Percent X X X Program-invariant

                                                                Low Income X X X X Multiple sources

                                                                Weatherization X X X Substantial Participantimpacts

                                                                Measure Program-specific

                                                                X Varies by measure sector

                                                                Other Recomrsquos Local Research

                                                                50

                                                                Source Skumatz SERA

                                                                Developing values for multiple states amp utilities

                                                                SERA

                                                                TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                Other sources of bias to research (EUL NTG etc) bring high variation

                                                                5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                THANK YOU

                                                                Questions

                                                                Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                SERA

                                                                POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                53

                                                                SERA

                                                                NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                Negative NEB values cost of barrier

                                                                Solar Water Heat NZ$ Euros

                                                                Solar Design NZ$ Euros

                                                                Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                -23 -12 (079)

                                                                -11 -6 (06)

                                                                Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                Source Skumatz Economic Research Associates research

                                                                SERA

                                                                CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                -5-4-3-2-1012345

                                                                Op Co

                                                                st

                                                                Init C

                                                                ost

                                                                Maint

                                                                Perfo

                                                                rm

                                                                Prod

                                                                uctiv

                                                                Tena

                                                                nt Sa

                                                                t

                                                                Comf

                                                                ort

                                                                Appe

                                                                ar

                                                                Light

                                                                Qual

                                                                SellL

                                                                ease

                                                                Devp Ownr AampE

                                                                Note AgtE (~4 vs ~2) for these NEBs

                                                                Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                Source Skumatz Economic Research Associates Inc 2002

                                                                Owners more positive ndashUnderinvestment train

                                                                • Soup to nuts on non-energy benefits NEIs
                                                                • TOPICS
                                                                • BACKGROUND NEBS
                                                                • 20+ YEARS OF NEBS PROGRESShellip
                                                                • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                • KEY APPLICATIONS OF NEBS
                                                                • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                • Neb measurement
                                                                • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                • MEASURING PARTICIPANTNEBs
                                                                • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                • PROGRESS TRANSFERABILITY OF NEBS
                                                                • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                • PROGRESS AND GAPS
                                                                • PROGRESS AND GAPS
                                                                • USING NEB NEI VALUES
                                                                • Slide Number 19
                                                                • ARE NEBS HIGH VALUE
                                                                • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                • AUDIT FINANCIAL PROGRAM
                                                                • CampI NEW CONSTRUCTION
                                                                • CampI TECHNICAL ASSISTANCE
                                                                • EXPRESSING NEBS VALUEndashCxe
                                                                • Slide Number 31
                                                                • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                • EMISSIONS (SOCIETAL NEBS)
                                                                • UTILITY NEBS
                                                                • ADJUSTED PAYBACKS
                                                                • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                • NEBS IN BC
                                                                • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                • APPLICATION TO BC
                                                                • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                • STATE TREATMENT OF NEBS NEBs
                                                                • Slide Number 42
                                                                • SELECTED STATE PROGRESS
                                                                • SELECTED STATE PROGRESS
                                                                • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                • DISCOUNT RATE MATTERS
                                                                • EUL VARIATIONS- ADOPTED VALUES
                                                                • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                • TAKEAWAYSON NEBSNEIS
                                                                • THANK YOUQuestions
                                                                • Slide Number 53
                                                                • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                  Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                  m Product losses20
                                                                  l Ease of selling leasing building33
                                                                  n Labor requirements45
                                                                  h Aesthetics appearance59
                                                                  j Noise50
                                                                  b Equipment maintenance660
                                                                  f Tenant satisfaction60
                                                                  d Equipment lifetimes718
                                                                  e Productivity724
                                                                  k Building safety[2]73
                                                                  g Comfort933
                                                                  i Lighting Quality of Light916
                                                                  c Equipment performance[1]1019
                                                                  a Operating costs1165
                                                                  o Doing good for the Environment[3]1132
                                                                  Other03
                                                                  [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                  [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                  [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                  m Product losses
                                                                  l Ease of selling leasing building
                                                                  n Labor requirements
                                                                  h Aesthetics appearance
                                                                  j Noise
                                                                  b Equipment maintenance
                                                                  f Tenant satisfaction
                                                                  d Equipment lifetimes
                                                                  e Productivity
                                                                  k Building safety[2]
                                                                  g Comfort
                                                                  i Lighting Quality of Light
                                                                  c Equipment performance[1]
                                                                  a Operating costs
                                                                  o Doing good for the Environment[3]
                                                                  Share of ZALEH NEBS from various measures
                                                                  Each presented as multiple of the (overall) energy savings
                                                                  Super Insulation037
                                                                  Double Glazing026
                                                                  Solar Design025
                                                                  Solar Water Heating005
                                                                  EnergySmart (insul Retrofit)Dunedin (insul)
                                                                  Other124
                                                                  Noise24
                                                                  Budgetfinances25
                                                                  Environmental65
                                                                  Health78
                                                                  Reduced moves86
                                                                  Ease of sellingrenting130
                                                                  DampnessMold1316
                                                                  Comfort-related4933
                                                                  101101
                                                                  Operational NEBs44
                                                                  DesignConstruction NEBs31
                                                                  Occupant NEBs25
                                                                  1
                                                                  OtherOther
                                                                  NoiseNoise
                                                                  BudgetfinancesBudgetfinances
                                                                  EnvironmentalEnvironmental
                                                                  HealthHealth
                                                                  Reduced movesReduced moves
                                                                  Ease of sellingrentingEase of sellingrenting
                                                                  DampnessMoldDampnessMold
                                                                  Comfort-relatedComfort-related

                                                                  SERA

                                                                  RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables

                                                                  MF Many others

                                                                  Thousands of surveys results By measures By program types By sectors By stakeholders By geography

                                                                  Variety of end uses

                                                                  SERA

                                                                  TOP NEBS FOR WEATHERIZATION PROGRAM

                                                                  Weatherization program multipliergt1

                                                                  Source SERA Research

                                                                  SERA

                                                                  RESULTS FOR ENERGY STARreg APPLIANCES

                                                                  0002004006008

                                                                  01012014016018

                                                                  Maint

                                                                  Appear

                                                                  Perf LifeNoise Satis

                                                                  Comfort

                                                                  LiteSafe

                                                                  ty SellMove

                                                                  Water

                                                                  Enviro

                                                                  Refrig DW CW

                                                                  Source SERA Research

                                                                  SERA

                                                                  INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                                                  5 increments to 50

                                                                  Source SERA research

                                                                  Chart5

                                                                  EnergySmart (insul Retrofit)
                                                                  Dunedin (insul)
                                                                  001
                                                                  024
                                                                  002
                                                                  004
                                                                  002
                                                                  005
                                                                  006
                                                                  005
                                                                  007
                                                                  008
                                                                  008
                                                                  006
                                                                  013
                                                                  0
                                                                  013
                                                                  016
                                                                  049
                                                                  033

                                                                  cx

                                                                  cx

                                                                  nz

                                                                  Sheet3

                                                                  Sheet3

                                                                  Super Insulation
                                                                  Double Glazing
                                                                  Solar Design
                                                                  Solar Water Heating
                                                                  Super Insulation
                                                                  Double Glazing
                                                                  Solar Design
                                                                  Solar Water Heating
                                                                  EnergySmart (insul Retrofit)
                                                                  Dunedin (insul)

                                                                  SERA

                                                                  ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                                  0

                                                                  002

                                                                  004

                                                                  006

                                                                  008

                                                                  01

                                                                  012

                                                                  Maint

                                                                  Perf LifeSatis

                                                                  f

                                                                  ComfortAesth

                                                                  NoiseSafe

                                                                  ty

                                                                  LightingSelli

                                                                  ng

                                                                  Stay in home

                                                                  Enviro

                                                                  Sick Days

                                                                  Calls

                                                                  ES Homes HP wES Energy savings multiplier=1

                                                                  Source SERA Research

                                                                  SERA

                                                                  CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                  DG CHP HVAC Equipment rebate Other

                                                                  Building codes incentives by cities

                                                                  Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                  Variety of end uses

                                                                  SERA

                                                                  AUDIT FINANCIAL PROGRAM

                                                                  0

                                                                  01

                                                                  02

                                                                  03

                                                                  04

                                                                  05

                                                                  06

                                                                  07

                                                                  Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                  Source SERA Research

                                                                  SERA

                                                                  CampI NEW CONSTRUCTION

                                                                  -005

                                                                  0

                                                                  005

                                                                  01

                                                                  015

                                                                  02

                                                                  AampE Owner(Pct of Total Partic NEBs)

                                                                  Source SERA Research

                                                                  SERA

                                                                  CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                  0 2 4 6 8 10 12

                                                                  m Product losses

                                                                  l Ease of selling leasing building

                                                                  n Labor requirements

                                                                  h Aesthetics appearance

                                                                  j Noise

                                                                  b Equipment maintenance

                                                                  f Tenant satisfaction

                                                                  d Equipment lifetimes

                                                                  e Productivity

                                                                  k Building safety[2]

                                                                  g Comfort

                                                                  i Lighting Quality of Light

                                                                  c Equipment performance[1]

                                                                  a Operating costs

                                                                  o Doing good for the Environment[3]

                                                                  Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                  vcv

                                                                  Source SERA Research

                                                                  Chart1

                                                                  CampI Technical Assistance NEBs
                                                                  002
                                                                  003
                                                                  004
                                                                  005
                                                                  005
                                                                  006
                                                                  006
                                                                  007
                                                                  007
                                                                  007
                                                                  009
                                                                  009
                                                                  01
                                                                  011
                                                                  011

                                                                  Sheet1

                                                                  Sheet1

                                                                  CampI Technical Assistance Program NEBs (Participant)

                                                                  Sheet2

                                                                  Sheet3

                                                                  SERA

                                                                  EXPRESSING NEBS VALUEndashCxe

                                                                  Strong value from RetroCxSource SERA Research

                                                                  SERA

                                                                  SOCIETAL NEB EXAMPLES

                                                                  31

                                                                  SERA

                                                                  JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                  32

                                                                  0

                                                                  02

                                                                  04

                                                                  06

                                                                  08

                                                                  1

                                                                  12

                                                                  HPWxRetrof Appliance

                                                                  CA

                                                                  WI

                                                                  Natl

                                                                  (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                  Jobs Economic

                                                                  SERA

                                                                  EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                  33

                                                                  SERA

                                                                  UTILITY NEBSUtility NEBs for Template Program

                                                                  Debt WriteOff (util)13

                                                                  Rate Subsidy(util)61

                                                                  HealthSafety(util)0

                                                                  Colln Costs (util)0

                                                                  Gas Calls (util)0

                                                                  Calls to CSRs(util)2

                                                                  TampD (util)16

                                                                  Arrears (util)0

                                                                  Reconnects (util)0

                                                                  Notices (util)7

                                                                  Shutoffs (util)1

                                                                  Rate subsidy TampD

                                                                  Payment-related

                                                                  Source Skumatz Economic Research Associates research

                                                                  Example Low Income Weatherization

                                                                  SERA

                                                                  ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                  be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                  Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                  Value of 100 HALVES Payback doubles ROI

                                                                  Source Skumatz Economics (SERA)

                                                                  35

                                                                  ( for NEBs Multiplier of 125)

                                                                  SERA

                                                                  NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                  36

                                                                  hellip2001 ON

                                                                  SERA

                                                                  37

                                                                  NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                  For true representation of B amp C elements of NEBs address missing factor bias

                                                                  Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                  BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                  SERA

                                                                  CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                  38

                                                                  Test Benefits Costs States Using Traditionally

                                                                  Improved treat-ment with NEBs

                                                                  Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                  bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                  bull Avoided gas and water supply costs

                                                                  bull Program administration

                                                                  bull Participant incentives

                                                                  bull Increased supply cost

                                                                  CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                  Use cost only paid by the utility

                                                                  Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                  Same as above plus bull increased revenue

                                                                  Same as above plus bull Decreased revenue

                                                                  AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                  Participant cost bull Utility bill reductions bull Participant incentives

                                                                  bull Participant direct costs

                                                                  AR CA FL HI IA IN MI MN NY VA

                                                                  Participant NEBs

                                                                  Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                  Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                  SERA

                                                                  APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                  Expansion Concern with risk calculation accuracy

                                                                  39

                                                                  Source Skumatz SERA research

                                                                  SERA

                                                                  40

                                                                  METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                  Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                  Minimize Regulatory amp Implementer Risk

                                                                  Minimize Evaluation Cost

                                                                  Adder

                                                                  Readily MeasurableHybrid

                                                                  All NEBsSource SERA Research

                                                                  SERA

                                                                  41

                                                                  STATE TREATMENT OF NEBS NEBsRegulatory

                                                                  Screening Application Utilities regions

                                                                  Program Marketing Fairly widespread use in utilities states across the country

                                                                  MO

                                                                  RE AGGRESSIVE ==gt

                                                                  Test Pgm Screen - adder

                                                                  IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                  Test Pgm Screen - readily measured

                                                                  MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                  Test Hybrid (potential adder amp measured)

                                                                  CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                  Test Pgm screen - Broad

                                                                  With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                  Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                  SERA

                                                                  42

                                                                  Key

                                                                  States with NEBs in CE tests for At least one type of program

                                                                  copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                  STATES WITH NEBS IN CE TESTS

                                                                  NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                  groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                  SERA

                                                                  SELECTED STATE PROGRESS CA

                                                                  2001 ndash LIPPT model NY

                                                                  Mid-2000s measurement amp scenarios not included CO

                                                                  Adders (10 electric 25 LI 5 gas) VT

                                                                  15 adder helped by previous research DC

                                                                  10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                  43

                                                                  Source Skumatz SERA research

                                                                  SERA

                                                                  SELECTED STATE PROGRESS Dominos ongoing

                                                                  Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                  Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                  Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                  Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                  44

                                                                  Source Skumatz SERA research

                                                                  SERA

                                                                  APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                  Expansion Concern with risk calculation accuracy

                                                                  45

                                                                  Source Skumatz SERA research

                                                                  SERA

                                                                  46

                                                                  NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                  [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                  SourceSkumatz SERA Independent research

                                                                  NTG ndash accuracy measurement incomplete

                                                                  Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                  NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                  EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                  Cost Complicated expensive local changes

                                                                  RELATIVE Risk from NEBs

                                                                  Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                  Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                  NEBs as accurate as many inputs (all could improve)

                                                                  SERA

                                                                  DISCOUNT RATE MATTERS

                                                                  47

                                                                  Source Skumatz SERA Research

                                                                  0

                                                                  20

                                                                  40

                                                                  60

                                                                  80

                                                                  100

                                                                  120

                                                                  1 2 3 4 5 6 7 8 9 10

                                                                  PV Ratios vs 1 Discount Rate

                                                                  10 yr PV 20 yr PV

                                                                  Discount rate - percent

                                                                  WACCSocialLTT

                                                                  Source Skumatz Electricity Journal IEPEC paper independent research

                                                                  SERA

                                                                  EUL VARIATIONS-ADOPTED VALUES

                                                                  Source Skumatz Economic Research Associates SERA independent research

                                                                  0

                                                                  50

                                                                  100

                                                                  150

                                                                  200

                                                                  250

                                                                  300

                                                                  350

                                                                  400

                                                                  450

                                                                  0 10 20 30 40 50 60 70

                                                                  EUL Variation by Measure

                                                                  67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                  Poorly supportedOldTechnology change

                                                                  SERA

                                                                  KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                  Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                  measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                  change the BC conclusion

                                                                  On the cusphellip

                                                                  If NO Yoursquore doneAnd bias addressedsufficiently

                                                                  IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                  Source SERA all rights reserved

                                                                  SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                  SERA

                                                                  IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                  Utility Soc Part Conserv Recrsquom

                                                                  Rationale

                                                                  Base Percent X X X Program-invariant

                                                                  Low Income X X X X Multiple sources

                                                                  Weatherization X X X Substantial Participantimpacts

                                                                  Measure Program-specific

                                                                  X Varies by measure sector

                                                                  Other Recomrsquos Local Research

                                                                  50

                                                                  Source Skumatz SERA

                                                                  Developing values for multiple states amp utilities

                                                                  SERA

                                                                  TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                  Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                  Other sources of bias to research (EUL NTG etc) bring high variation

                                                                  5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                  6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                  THANK YOU

                                                                  Questions

                                                                  Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                  SERA

                                                                  POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                  53

                                                                  SERA

                                                                  NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                  Negative NEB values cost of barrier

                                                                  Solar Water Heat NZ$ Euros

                                                                  Solar Design NZ$ Euros

                                                                  Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                  -23 -12 (079)

                                                                  -11 -6 (06)

                                                                  Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                  Source Skumatz Economic Research Associates research

                                                                  SERA

                                                                  CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                  NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                  -5-4-3-2-1012345

                                                                  Op Co

                                                                  st

                                                                  Init C

                                                                  ost

                                                                  Maint

                                                                  Perfo

                                                                  rm

                                                                  Prod

                                                                  uctiv

                                                                  Tena

                                                                  nt Sa

                                                                  t

                                                                  Comf

                                                                  ort

                                                                  Appe

                                                                  ar

                                                                  Light

                                                                  Qual

                                                                  SellL

                                                                  ease

                                                                  Devp Ownr AampE

                                                                  Note AgtE (~4 vs ~2) for these NEBs

                                                                  Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                  Source Skumatz Economic Research Associates Inc 2002

                                                                  Owners more positive ndashUnderinvestment train

                                                                  • Soup to nuts on non-energy benefits NEIs
                                                                  • TOPICS
                                                                  • BACKGROUND NEBS
                                                                  • 20+ YEARS OF NEBS PROGRESShellip
                                                                  • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                  • KEY APPLICATIONS OF NEBS
                                                                  • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                  • Neb measurement
                                                                  • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                  • MEASURING PARTICIPANTNEBs
                                                                  • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                  • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                  • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                  • PROGRESS TRANSFERABILITY OF NEBS
                                                                  • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                  • PROGRESS AND GAPS
                                                                  • PROGRESS AND GAPS
                                                                  • USING NEB NEI VALUES
                                                                  • Slide Number 19
                                                                  • ARE NEBS HIGH VALUE
                                                                  • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                  • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                  • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                  • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                  • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                  • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                  • AUDIT FINANCIAL PROGRAM
                                                                  • CampI NEW CONSTRUCTION
                                                                  • CampI TECHNICAL ASSISTANCE
                                                                  • EXPRESSING NEBS VALUEndashCxe
                                                                  • Slide Number 31
                                                                  • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                  • EMISSIONS (SOCIETAL NEBS)
                                                                  • UTILITY NEBS
                                                                  • ADJUSTED PAYBACKS
                                                                  • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                  • NEBS IN BC
                                                                  • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                  • APPLICATION TO BC
                                                                  • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                  • STATE TREATMENT OF NEBS NEBs
                                                                  • Slide Number 42
                                                                  • SELECTED STATE PROGRESS
                                                                  • SELECTED STATE PROGRESS
                                                                  • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                  • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                  • DISCOUNT RATE MATTERS
                                                                  • EUL VARIATIONS- ADOPTED VALUES
                                                                  • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                  • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                  • TAKEAWAYSON NEBSNEIS
                                                                  • THANK YOUQuestions
                                                                  • Slide Number 53
                                                                  • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                  • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                    Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                    m Product losses20
                                                                    l Ease of selling leasing building33
                                                                    n Labor requirements45
                                                                    h Aesthetics appearance59
                                                                    j Noise50
                                                                    b Equipment maintenance660
                                                                    f Tenant satisfaction60
                                                                    d Equipment lifetimes718
                                                                    e Productivity724
                                                                    k Building safety[2]73
                                                                    g Comfort933
                                                                    i Lighting Quality of Light916
                                                                    c Equipment performance[1]1019
                                                                    a Operating costs1165
                                                                    o Doing good for the Environment[3]1132
                                                                    Other03
                                                                    [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                    [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                    [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                    m Product losses
                                                                    l Ease of selling leasing building
                                                                    n Labor requirements
                                                                    h Aesthetics appearance
                                                                    j Noise
                                                                    b Equipment maintenance
                                                                    f Tenant satisfaction
                                                                    d Equipment lifetimes
                                                                    e Productivity
                                                                    k Building safety[2]
                                                                    g Comfort
                                                                    i Lighting Quality of Light
                                                                    c Equipment performance[1]
                                                                    a Operating costs
                                                                    o Doing good for the Environment[3]
                                                                    Share of ZALEH NEBS from various measures
                                                                    Each presented as multiple of the (overall) energy savings
                                                                    Super Insulation037
                                                                    Double Glazing026
                                                                    Solar Design025
                                                                    Solar Water Heating005
                                                                    EnergySmart (insul Retrofit)Dunedin (insul)
                                                                    Other124
                                                                    Noise24
                                                                    Budgetfinances25
                                                                    Environmental65
                                                                    Health78
                                                                    Reduced moves86
                                                                    Ease of sellingrenting130
                                                                    DampnessMold1316
                                                                    Comfort-related4933
                                                                    101101
                                                                    Operational NEBs44
                                                                    DesignConstruction NEBs31
                                                                    Occupant NEBs25
                                                                    1
                                                                    OtherOther
                                                                    NoiseNoise
                                                                    BudgetfinancesBudgetfinances
                                                                    EnvironmentalEnvironmental
                                                                    HealthHealth
                                                                    Reduced movesReduced moves
                                                                    Ease of sellingrentingEase of sellingrenting
                                                                    DampnessMoldDampnessMold
                                                                    Comfort-relatedComfort-related

                                                                    SERA

                                                                    TOP NEBS FOR WEATHERIZATION PROGRAM

                                                                    Weatherization program multipliergt1

                                                                    Source SERA Research

                                                                    SERA

                                                                    RESULTS FOR ENERGY STARreg APPLIANCES

                                                                    0002004006008

                                                                    01012014016018

                                                                    Maint

                                                                    Appear

                                                                    Perf LifeNoise Satis

                                                                    Comfort

                                                                    LiteSafe

                                                                    ty SellMove

                                                                    Water

                                                                    Enviro

                                                                    Refrig DW CW

                                                                    Source SERA Research

                                                                    SERA

                                                                    INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                                                    5 increments to 50

                                                                    Source SERA research

                                                                    Chart5

                                                                    EnergySmart (insul Retrofit)
                                                                    Dunedin (insul)
                                                                    001
                                                                    024
                                                                    002
                                                                    004
                                                                    002
                                                                    005
                                                                    006
                                                                    005
                                                                    007
                                                                    008
                                                                    008
                                                                    006
                                                                    013
                                                                    0
                                                                    013
                                                                    016
                                                                    049
                                                                    033

                                                                    cx

                                                                    cx

                                                                    nz

                                                                    Sheet3

                                                                    Sheet3

                                                                    Super Insulation
                                                                    Double Glazing
                                                                    Solar Design
                                                                    Solar Water Heating
                                                                    Super Insulation
                                                                    Double Glazing
                                                                    Solar Design
                                                                    Solar Water Heating
                                                                    EnergySmart (insul Retrofit)
                                                                    Dunedin (insul)

                                                                    SERA

                                                                    ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                                    0

                                                                    002

                                                                    004

                                                                    006

                                                                    008

                                                                    01

                                                                    012

                                                                    Maint

                                                                    Perf LifeSatis

                                                                    f

                                                                    ComfortAesth

                                                                    NoiseSafe

                                                                    ty

                                                                    LightingSelli

                                                                    ng

                                                                    Stay in home

                                                                    Enviro

                                                                    Sick Days

                                                                    Calls

                                                                    ES Homes HP wES Energy savings multiplier=1

                                                                    Source SERA Research

                                                                    SERA

                                                                    CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                    DG CHP HVAC Equipment rebate Other

                                                                    Building codes incentives by cities

                                                                    Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                    Variety of end uses

                                                                    SERA

                                                                    AUDIT FINANCIAL PROGRAM

                                                                    0

                                                                    01

                                                                    02

                                                                    03

                                                                    04

                                                                    05

                                                                    06

                                                                    07

                                                                    Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                    Source SERA Research

                                                                    SERA

                                                                    CampI NEW CONSTRUCTION

                                                                    -005

                                                                    0

                                                                    005

                                                                    01

                                                                    015

                                                                    02

                                                                    AampE Owner(Pct of Total Partic NEBs)

                                                                    Source SERA Research

                                                                    SERA

                                                                    CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                    0 2 4 6 8 10 12

                                                                    m Product losses

                                                                    l Ease of selling leasing building

                                                                    n Labor requirements

                                                                    h Aesthetics appearance

                                                                    j Noise

                                                                    b Equipment maintenance

                                                                    f Tenant satisfaction

                                                                    d Equipment lifetimes

                                                                    e Productivity

                                                                    k Building safety[2]

                                                                    g Comfort

                                                                    i Lighting Quality of Light

                                                                    c Equipment performance[1]

                                                                    a Operating costs

                                                                    o Doing good for the Environment[3]

                                                                    Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                    vcv

                                                                    Source SERA Research

                                                                    Chart1

                                                                    CampI Technical Assistance NEBs
                                                                    002
                                                                    003
                                                                    004
                                                                    005
                                                                    005
                                                                    006
                                                                    006
                                                                    007
                                                                    007
                                                                    007
                                                                    009
                                                                    009
                                                                    01
                                                                    011
                                                                    011

                                                                    Sheet1

                                                                    Sheet1

                                                                    CampI Technical Assistance Program NEBs (Participant)

                                                                    Sheet2

                                                                    Sheet3

                                                                    SERA

                                                                    EXPRESSING NEBS VALUEndashCxe

                                                                    Strong value from RetroCxSource SERA Research

                                                                    SERA

                                                                    SOCIETAL NEB EXAMPLES

                                                                    31

                                                                    SERA

                                                                    JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                    32

                                                                    0

                                                                    02

                                                                    04

                                                                    06

                                                                    08

                                                                    1

                                                                    12

                                                                    HPWxRetrof Appliance

                                                                    CA

                                                                    WI

                                                                    Natl

                                                                    (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                    Jobs Economic

                                                                    SERA

                                                                    EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                    33

                                                                    SERA

                                                                    UTILITY NEBSUtility NEBs for Template Program

                                                                    Debt WriteOff (util)13

                                                                    Rate Subsidy(util)61

                                                                    HealthSafety(util)0

                                                                    Colln Costs (util)0

                                                                    Gas Calls (util)0

                                                                    Calls to CSRs(util)2

                                                                    TampD (util)16

                                                                    Arrears (util)0

                                                                    Reconnects (util)0

                                                                    Notices (util)7

                                                                    Shutoffs (util)1

                                                                    Rate subsidy TampD

                                                                    Payment-related

                                                                    Source Skumatz Economic Research Associates research

                                                                    Example Low Income Weatherization

                                                                    SERA

                                                                    ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                    be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                    Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                    Value of 100 HALVES Payback doubles ROI

                                                                    Source Skumatz Economics (SERA)

                                                                    35

                                                                    ( for NEBs Multiplier of 125)

                                                                    SERA

                                                                    NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                    36

                                                                    hellip2001 ON

                                                                    SERA

                                                                    37

                                                                    NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                    For true representation of B amp C elements of NEBs address missing factor bias

                                                                    Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                    BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                    SERA

                                                                    CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                    38

                                                                    Test Benefits Costs States Using Traditionally

                                                                    Improved treat-ment with NEBs

                                                                    Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                    bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                    bull Avoided gas and water supply costs

                                                                    bull Program administration

                                                                    bull Participant incentives

                                                                    bull Increased supply cost

                                                                    CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                    Use cost only paid by the utility

                                                                    Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                    Same as above plus bull increased revenue

                                                                    Same as above plus bull Decreased revenue

                                                                    AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                    Participant cost bull Utility bill reductions bull Participant incentives

                                                                    bull Participant direct costs

                                                                    AR CA FL HI IA IN MI MN NY VA

                                                                    Participant NEBs

                                                                    Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                    Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                    SERA

                                                                    APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                    Expansion Concern with risk calculation accuracy

                                                                    39

                                                                    Source Skumatz SERA research

                                                                    SERA

                                                                    40

                                                                    METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                    Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                    Minimize Regulatory amp Implementer Risk

                                                                    Minimize Evaluation Cost

                                                                    Adder

                                                                    Readily MeasurableHybrid

                                                                    All NEBsSource SERA Research

                                                                    SERA

                                                                    41

                                                                    STATE TREATMENT OF NEBS NEBsRegulatory

                                                                    Screening Application Utilities regions

                                                                    Program Marketing Fairly widespread use in utilities states across the country

                                                                    MO

                                                                    RE AGGRESSIVE ==gt

                                                                    Test Pgm Screen - adder

                                                                    IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                    Test Pgm Screen - readily measured

                                                                    MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                    Test Hybrid (potential adder amp measured)

                                                                    CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                    Test Pgm screen - Broad

                                                                    With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                    Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                    SERA

                                                                    42

                                                                    Key

                                                                    States with NEBs in CE tests for At least one type of program

                                                                    copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                    STATES WITH NEBS IN CE TESTS

                                                                    NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                    groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                    SERA

                                                                    SELECTED STATE PROGRESS CA

                                                                    2001 ndash LIPPT model NY

                                                                    Mid-2000s measurement amp scenarios not included CO

                                                                    Adders (10 electric 25 LI 5 gas) VT

                                                                    15 adder helped by previous research DC

                                                                    10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                    43

                                                                    Source Skumatz SERA research

                                                                    SERA

                                                                    SELECTED STATE PROGRESS Dominos ongoing

                                                                    Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                    Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                    Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                    Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                    44

                                                                    Source Skumatz SERA research

                                                                    SERA

                                                                    APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                    Expansion Concern with risk calculation accuracy

                                                                    45

                                                                    Source Skumatz SERA research

                                                                    SERA

                                                                    46

                                                                    NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                    [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                    SourceSkumatz SERA Independent research

                                                                    NTG ndash accuracy measurement incomplete

                                                                    Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                    NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                    EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                    Cost Complicated expensive local changes

                                                                    RELATIVE Risk from NEBs

                                                                    Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                    Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                    NEBs as accurate as many inputs (all could improve)

                                                                    SERA

                                                                    DISCOUNT RATE MATTERS

                                                                    47

                                                                    Source Skumatz SERA Research

                                                                    0

                                                                    20

                                                                    40

                                                                    60

                                                                    80

                                                                    100

                                                                    120

                                                                    1 2 3 4 5 6 7 8 9 10

                                                                    PV Ratios vs 1 Discount Rate

                                                                    10 yr PV 20 yr PV

                                                                    Discount rate - percent

                                                                    WACCSocialLTT

                                                                    Source Skumatz Electricity Journal IEPEC paper independent research

                                                                    SERA

                                                                    EUL VARIATIONS-ADOPTED VALUES

                                                                    Source Skumatz Economic Research Associates SERA independent research

                                                                    0

                                                                    50

                                                                    100

                                                                    150

                                                                    200

                                                                    250

                                                                    300

                                                                    350

                                                                    400

                                                                    450

                                                                    0 10 20 30 40 50 60 70

                                                                    EUL Variation by Measure

                                                                    67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                    Poorly supportedOldTechnology change

                                                                    SERA

                                                                    KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                    Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                    measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                    change the BC conclusion

                                                                    On the cusphellip

                                                                    If NO Yoursquore doneAnd bias addressedsufficiently

                                                                    IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                    Source SERA all rights reserved

                                                                    SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                    SERA

                                                                    IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                    Utility Soc Part Conserv Recrsquom

                                                                    Rationale

                                                                    Base Percent X X X Program-invariant

                                                                    Low Income X X X X Multiple sources

                                                                    Weatherization X X X Substantial Participantimpacts

                                                                    Measure Program-specific

                                                                    X Varies by measure sector

                                                                    Other Recomrsquos Local Research

                                                                    50

                                                                    Source Skumatz SERA

                                                                    Developing values for multiple states amp utilities

                                                                    SERA

                                                                    TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                    Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                    Other sources of bias to research (EUL NTG etc) bring high variation

                                                                    5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                    6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                    THANK YOU

                                                                    Questions

                                                                    Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                    SERA

                                                                    POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                    53

                                                                    SERA

                                                                    NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                    Negative NEB values cost of barrier

                                                                    Solar Water Heat NZ$ Euros

                                                                    Solar Design NZ$ Euros

                                                                    Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                    -23 -12 (079)

                                                                    -11 -6 (06)

                                                                    Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                    Source Skumatz Economic Research Associates research

                                                                    SERA

                                                                    CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                    NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                    -5-4-3-2-1012345

                                                                    Op Co

                                                                    st

                                                                    Init C

                                                                    ost

                                                                    Maint

                                                                    Perfo

                                                                    rm

                                                                    Prod

                                                                    uctiv

                                                                    Tena

                                                                    nt Sa

                                                                    t

                                                                    Comf

                                                                    ort

                                                                    Appe

                                                                    ar

                                                                    Light

                                                                    Qual

                                                                    SellL

                                                                    ease

                                                                    Devp Ownr AampE

                                                                    Note AgtE (~4 vs ~2) for these NEBs

                                                                    Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                    Source Skumatz Economic Research Associates Inc 2002

                                                                    Owners more positive ndashUnderinvestment train

                                                                    • Soup to nuts on non-energy benefits NEIs
                                                                    • TOPICS
                                                                    • BACKGROUND NEBS
                                                                    • 20+ YEARS OF NEBS PROGRESShellip
                                                                    • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                    • KEY APPLICATIONS OF NEBS
                                                                    • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                    • Neb measurement
                                                                    • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                    • MEASURING PARTICIPANTNEBs
                                                                    • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                    • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                    • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                    • PROGRESS TRANSFERABILITY OF NEBS
                                                                    • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                    • PROGRESS AND GAPS
                                                                    • PROGRESS AND GAPS
                                                                    • USING NEB NEI VALUES
                                                                    • Slide Number 19
                                                                    • ARE NEBS HIGH VALUE
                                                                    • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                    • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                    • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                    • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                    • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                    • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                    • AUDIT FINANCIAL PROGRAM
                                                                    • CampI NEW CONSTRUCTION
                                                                    • CampI TECHNICAL ASSISTANCE
                                                                    • EXPRESSING NEBS VALUEndashCxe
                                                                    • Slide Number 31
                                                                    • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                    • EMISSIONS (SOCIETAL NEBS)
                                                                    • UTILITY NEBS
                                                                    • ADJUSTED PAYBACKS
                                                                    • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                    • NEBS IN BC
                                                                    • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                    • APPLICATION TO BC
                                                                    • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                    • STATE TREATMENT OF NEBS NEBs
                                                                    • Slide Number 42
                                                                    • SELECTED STATE PROGRESS
                                                                    • SELECTED STATE PROGRESS
                                                                    • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                    • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                    • DISCOUNT RATE MATTERS
                                                                    • EUL VARIATIONS- ADOPTED VALUES
                                                                    • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                    • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                    • TAKEAWAYSON NEBSNEIS
                                                                    • THANK YOUQuestions
                                                                    • Slide Number 53
                                                                    • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                    • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                      Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                      m Product losses20
                                                                      l Ease of selling leasing building33
                                                                      n Labor requirements45
                                                                      h Aesthetics appearance59
                                                                      j Noise50
                                                                      b Equipment maintenance660
                                                                      f Tenant satisfaction60
                                                                      d Equipment lifetimes718
                                                                      e Productivity724
                                                                      k Building safety[2]73
                                                                      g Comfort933
                                                                      i Lighting Quality of Light916
                                                                      c Equipment performance[1]1019
                                                                      a Operating costs1165
                                                                      o Doing good for the Environment[3]1132
                                                                      Other03
                                                                      [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                      [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                      [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                      m Product losses
                                                                      l Ease of selling leasing building
                                                                      n Labor requirements
                                                                      h Aesthetics appearance
                                                                      j Noise
                                                                      b Equipment maintenance
                                                                      f Tenant satisfaction
                                                                      d Equipment lifetimes
                                                                      e Productivity
                                                                      k Building safety[2]
                                                                      g Comfort
                                                                      i Lighting Quality of Light
                                                                      c Equipment performance[1]
                                                                      a Operating costs
                                                                      o Doing good for the Environment[3]
                                                                      Share of ZALEH NEBS from various measures
                                                                      Each presented as multiple of the (overall) energy savings
                                                                      Super Insulation037
                                                                      Double Glazing026
                                                                      Solar Design025
                                                                      Solar Water Heating005
                                                                      EnergySmart (insul Retrofit)Dunedin (insul)
                                                                      Other124
                                                                      Noise24
                                                                      Budgetfinances25
                                                                      Environmental65
                                                                      Health78
                                                                      Reduced moves86
                                                                      Ease of sellingrenting130
                                                                      DampnessMold1316
                                                                      Comfort-related4933
                                                                      101101
                                                                      Operational NEBs44
                                                                      DesignConstruction NEBs31
                                                                      Occupant NEBs25
                                                                      1
                                                                      OtherOther
                                                                      NoiseNoise
                                                                      BudgetfinancesBudgetfinances
                                                                      EnvironmentalEnvironmental
                                                                      HealthHealth
                                                                      Reduced movesReduced moves
                                                                      Ease of sellingrentingEase of sellingrenting
                                                                      DampnessMoldDampnessMold
                                                                      Comfort-relatedComfort-related

                                                                      SERA

                                                                      RESULTS FOR ENERGY STARreg APPLIANCES

                                                                      0002004006008

                                                                      01012014016018

                                                                      Maint

                                                                      Appear

                                                                      Perf LifeNoise Satis

                                                                      Comfort

                                                                      LiteSafe

                                                                      ty SellMove

                                                                      Water

                                                                      Enviro

                                                                      Refrig DW CW

                                                                      Source SERA Research

                                                                      SERA

                                                                      INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                                                      5 increments to 50

                                                                      Source SERA research

                                                                      Chart5

                                                                      EnergySmart (insul Retrofit)
                                                                      Dunedin (insul)
                                                                      001
                                                                      024
                                                                      002
                                                                      004
                                                                      002
                                                                      005
                                                                      006
                                                                      005
                                                                      007
                                                                      008
                                                                      008
                                                                      006
                                                                      013
                                                                      0
                                                                      013
                                                                      016
                                                                      049
                                                                      033

                                                                      cx

                                                                      cx

                                                                      nz

                                                                      Sheet3

                                                                      Sheet3

                                                                      Super Insulation
                                                                      Double Glazing
                                                                      Solar Design
                                                                      Solar Water Heating
                                                                      Super Insulation
                                                                      Double Glazing
                                                                      Solar Design
                                                                      Solar Water Heating
                                                                      EnergySmart (insul Retrofit)
                                                                      Dunedin (insul)

                                                                      SERA

                                                                      ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                                      0

                                                                      002

                                                                      004

                                                                      006

                                                                      008

                                                                      01

                                                                      012

                                                                      Maint

                                                                      Perf LifeSatis

                                                                      f

                                                                      ComfortAesth

                                                                      NoiseSafe

                                                                      ty

                                                                      LightingSelli

                                                                      ng

                                                                      Stay in home

                                                                      Enviro

                                                                      Sick Days

                                                                      Calls

                                                                      ES Homes HP wES Energy savings multiplier=1

                                                                      Source SERA Research

                                                                      SERA

                                                                      CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                      DG CHP HVAC Equipment rebate Other

                                                                      Building codes incentives by cities

                                                                      Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                      Variety of end uses

                                                                      SERA

                                                                      AUDIT FINANCIAL PROGRAM

                                                                      0

                                                                      01

                                                                      02

                                                                      03

                                                                      04

                                                                      05

                                                                      06

                                                                      07

                                                                      Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                      Source SERA Research

                                                                      SERA

                                                                      CampI NEW CONSTRUCTION

                                                                      -005

                                                                      0

                                                                      005

                                                                      01

                                                                      015

                                                                      02

                                                                      AampE Owner(Pct of Total Partic NEBs)

                                                                      Source SERA Research

                                                                      SERA

                                                                      CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                      0 2 4 6 8 10 12

                                                                      m Product losses

                                                                      l Ease of selling leasing building

                                                                      n Labor requirements

                                                                      h Aesthetics appearance

                                                                      j Noise

                                                                      b Equipment maintenance

                                                                      f Tenant satisfaction

                                                                      d Equipment lifetimes

                                                                      e Productivity

                                                                      k Building safety[2]

                                                                      g Comfort

                                                                      i Lighting Quality of Light

                                                                      c Equipment performance[1]

                                                                      a Operating costs

                                                                      o Doing good for the Environment[3]

                                                                      Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                      vcv

                                                                      Source SERA Research

                                                                      Chart1

                                                                      CampI Technical Assistance NEBs
                                                                      002
                                                                      003
                                                                      004
                                                                      005
                                                                      005
                                                                      006
                                                                      006
                                                                      007
                                                                      007
                                                                      007
                                                                      009
                                                                      009
                                                                      01
                                                                      011
                                                                      011

                                                                      Sheet1

                                                                      Sheet1

                                                                      CampI Technical Assistance Program NEBs (Participant)

                                                                      Sheet2

                                                                      Sheet3

                                                                      SERA

                                                                      EXPRESSING NEBS VALUEndashCxe

                                                                      Strong value from RetroCxSource SERA Research

                                                                      SERA

                                                                      SOCIETAL NEB EXAMPLES

                                                                      31

                                                                      SERA

                                                                      JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                      32

                                                                      0

                                                                      02

                                                                      04

                                                                      06

                                                                      08

                                                                      1

                                                                      12

                                                                      HPWxRetrof Appliance

                                                                      CA

                                                                      WI

                                                                      Natl

                                                                      (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                      Jobs Economic

                                                                      SERA

                                                                      EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                      33

                                                                      SERA

                                                                      UTILITY NEBSUtility NEBs for Template Program

                                                                      Debt WriteOff (util)13

                                                                      Rate Subsidy(util)61

                                                                      HealthSafety(util)0

                                                                      Colln Costs (util)0

                                                                      Gas Calls (util)0

                                                                      Calls to CSRs(util)2

                                                                      TampD (util)16

                                                                      Arrears (util)0

                                                                      Reconnects (util)0

                                                                      Notices (util)7

                                                                      Shutoffs (util)1

                                                                      Rate subsidy TampD

                                                                      Payment-related

                                                                      Source Skumatz Economic Research Associates research

                                                                      Example Low Income Weatherization

                                                                      SERA

                                                                      ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                      be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                      Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                      Value of 100 HALVES Payback doubles ROI

                                                                      Source Skumatz Economics (SERA)

                                                                      35

                                                                      ( for NEBs Multiplier of 125)

                                                                      SERA

                                                                      NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                      36

                                                                      hellip2001 ON

                                                                      SERA

                                                                      37

                                                                      NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                      For true representation of B amp C elements of NEBs address missing factor bias

                                                                      Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                      BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                      SERA

                                                                      CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                      38

                                                                      Test Benefits Costs States Using Traditionally

                                                                      Improved treat-ment with NEBs

                                                                      Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                      bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                      bull Avoided gas and water supply costs

                                                                      bull Program administration

                                                                      bull Participant incentives

                                                                      bull Increased supply cost

                                                                      CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                      Use cost only paid by the utility

                                                                      Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                      Same as above plus bull increased revenue

                                                                      Same as above plus bull Decreased revenue

                                                                      AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                      Participant cost bull Utility bill reductions bull Participant incentives

                                                                      bull Participant direct costs

                                                                      AR CA FL HI IA IN MI MN NY VA

                                                                      Participant NEBs

                                                                      Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                      Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                      SERA

                                                                      APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                      Expansion Concern with risk calculation accuracy

                                                                      39

                                                                      Source Skumatz SERA research

                                                                      SERA

                                                                      40

                                                                      METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                      Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                      Minimize Regulatory amp Implementer Risk

                                                                      Minimize Evaluation Cost

                                                                      Adder

                                                                      Readily MeasurableHybrid

                                                                      All NEBsSource SERA Research

                                                                      SERA

                                                                      41

                                                                      STATE TREATMENT OF NEBS NEBsRegulatory

                                                                      Screening Application Utilities regions

                                                                      Program Marketing Fairly widespread use in utilities states across the country

                                                                      MO

                                                                      RE AGGRESSIVE ==gt

                                                                      Test Pgm Screen - adder

                                                                      IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                      Test Pgm Screen - readily measured

                                                                      MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                      Test Hybrid (potential adder amp measured)

                                                                      CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                      Test Pgm screen - Broad

                                                                      With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                      Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                      SERA

                                                                      42

                                                                      Key

                                                                      States with NEBs in CE tests for At least one type of program

                                                                      copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                      STATES WITH NEBS IN CE TESTS

                                                                      NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                      groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                      SERA

                                                                      SELECTED STATE PROGRESS CA

                                                                      2001 ndash LIPPT model NY

                                                                      Mid-2000s measurement amp scenarios not included CO

                                                                      Adders (10 electric 25 LI 5 gas) VT

                                                                      15 adder helped by previous research DC

                                                                      10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                      43

                                                                      Source Skumatz SERA research

                                                                      SERA

                                                                      SELECTED STATE PROGRESS Dominos ongoing

                                                                      Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                      Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                      Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                      Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                      44

                                                                      Source Skumatz SERA research

                                                                      SERA

                                                                      APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                      Expansion Concern with risk calculation accuracy

                                                                      45

                                                                      Source Skumatz SERA research

                                                                      SERA

                                                                      46

                                                                      NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                      [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                      SourceSkumatz SERA Independent research

                                                                      NTG ndash accuracy measurement incomplete

                                                                      Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                      NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                      EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                      Cost Complicated expensive local changes

                                                                      RELATIVE Risk from NEBs

                                                                      Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                      Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                      NEBs as accurate as many inputs (all could improve)

                                                                      SERA

                                                                      DISCOUNT RATE MATTERS

                                                                      47

                                                                      Source Skumatz SERA Research

                                                                      0

                                                                      20

                                                                      40

                                                                      60

                                                                      80

                                                                      100

                                                                      120

                                                                      1 2 3 4 5 6 7 8 9 10

                                                                      PV Ratios vs 1 Discount Rate

                                                                      10 yr PV 20 yr PV

                                                                      Discount rate - percent

                                                                      WACCSocialLTT

                                                                      Source Skumatz Electricity Journal IEPEC paper independent research

                                                                      SERA

                                                                      EUL VARIATIONS-ADOPTED VALUES

                                                                      Source Skumatz Economic Research Associates SERA independent research

                                                                      0

                                                                      50

                                                                      100

                                                                      150

                                                                      200

                                                                      250

                                                                      300

                                                                      350

                                                                      400

                                                                      450

                                                                      0 10 20 30 40 50 60 70

                                                                      EUL Variation by Measure

                                                                      67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                      Poorly supportedOldTechnology change

                                                                      SERA

                                                                      KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                      Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                      measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                      change the BC conclusion

                                                                      On the cusphellip

                                                                      If NO Yoursquore doneAnd bias addressedsufficiently

                                                                      IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                      Source SERA all rights reserved

                                                                      SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                      SERA

                                                                      IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                      Utility Soc Part Conserv Recrsquom

                                                                      Rationale

                                                                      Base Percent X X X Program-invariant

                                                                      Low Income X X X X Multiple sources

                                                                      Weatherization X X X Substantial Participantimpacts

                                                                      Measure Program-specific

                                                                      X Varies by measure sector

                                                                      Other Recomrsquos Local Research

                                                                      50

                                                                      Source Skumatz SERA

                                                                      Developing values for multiple states amp utilities

                                                                      SERA

                                                                      TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                      Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                      Other sources of bias to research (EUL NTG etc) bring high variation

                                                                      5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                      6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                      THANK YOU

                                                                      Questions

                                                                      Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                      SERA

                                                                      POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                      53

                                                                      SERA

                                                                      NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                      Negative NEB values cost of barrier

                                                                      Solar Water Heat NZ$ Euros

                                                                      Solar Design NZ$ Euros

                                                                      Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                      -23 -12 (079)

                                                                      -11 -6 (06)

                                                                      Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                      Source Skumatz Economic Research Associates research

                                                                      SERA

                                                                      CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                      NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                      -5-4-3-2-1012345

                                                                      Op Co

                                                                      st

                                                                      Init C

                                                                      ost

                                                                      Maint

                                                                      Perfo

                                                                      rm

                                                                      Prod

                                                                      uctiv

                                                                      Tena

                                                                      nt Sa

                                                                      t

                                                                      Comf

                                                                      ort

                                                                      Appe

                                                                      ar

                                                                      Light

                                                                      Qual

                                                                      SellL

                                                                      ease

                                                                      Devp Ownr AampE

                                                                      Note AgtE (~4 vs ~2) for these NEBs

                                                                      Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                      Source Skumatz Economic Research Associates Inc 2002

                                                                      Owners more positive ndashUnderinvestment train

                                                                      • Soup to nuts on non-energy benefits NEIs
                                                                      • TOPICS
                                                                      • BACKGROUND NEBS
                                                                      • 20+ YEARS OF NEBS PROGRESShellip
                                                                      • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                      • KEY APPLICATIONS OF NEBS
                                                                      • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                      • Neb measurement
                                                                      • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                      • MEASURING PARTICIPANTNEBs
                                                                      • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                      • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                      • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                      • PROGRESS TRANSFERABILITY OF NEBS
                                                                      • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                      • PROGRESS AND GAPS
                                                                      • PROGRESS AND GAPS
                                                                      • USING NEB NEI VALUES
                                                                      • Slide Number 19
                                                                      • ARE NEBS HIGH VALUE
                                                                      • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                      • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                      • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                      • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                      • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                      • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                      • AUDIT FINANCIAL PROGRAM
                                                                      • CampI NEW CONSTRUCTION
                                                                      • CampI TECHNICAL ASSISTANCE
                                                                      • EXPRESSING NEBS VALUEndashCxe
                                                                      • Slide Number 31
                                                                      • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                      • EMISSIONS (SOCIETAL NEBS)
                                                                      • UTILITY NEBS
                                                                      • ADJUSTED PAYBACKS
                                                                      • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                      • NEBS IN BC
                                                                      • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                      • APPLICATION TO BC
                                                                      • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                      • STATE TREATMENT OF NEBS NEBs
                                                                      • Slide Number 42
                                                                      • SELECTED STATE PROGRESS
                                                                      • SELECTED STATE PROGRESS
                                                                      • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                      • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                      • DISCOUNT RATE MATTERS
                                                                      • EUL VARIATIONS- ADOPTED VALUES
                                                                      • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                      • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                      • TAKEAWAYSON NEBSNEIS
                                                                      • THANK YOUQuestions
                                                                      • Slide Number 53
                                                                      • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                      • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                        Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                        m Product losses20
                                                                        l Ease of selling leasing building33
                                                                        n Labor requirements45
                                                                        h Aesthetics appearance59
                                                                        j Noise50
                                                                        b Equipment maintenance660
                                                                        f Tenant satisfaction60
                                                                        d Equipment lifetimes718
                                                                        e Productivity724
                                                                        k Building safety[2]73
                                                                        g Comfort933
                                                                        i Lighting Quality of Light916
                                                                        c Equipment performance[1]1019
                                                                        a Operating costs1165
                                                                        o Doing good for the Environment[3]1132
                                                                        Other03
                                                                        [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                        [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                        [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                        m Product losses
                                                                        l Ease of selling leasing building
                                                                        n Labor requirements
                                                                        h Aesthetics appearance
                                                                        j Noise
                                                                        b Equipment maintenance
                                                                        f Tenant satisfaction
                                                                        d Equipment lifetimes
                                                                        e Productivity
                                                                        k Building safety[2]
                                                                        g Comfort
                                                                        i Lighting Quality of Light
                                                                        c Equipment performance[1]
                                                                        a Operating costs
                                                                        o Doing good for the Environment[3]
                                                                        Share of ZALEH NEBS from various measures
                                                                        Each presented as multiple of the (overall) energy savings
                                                                        Super Insulation037
                                                                        Double Glazing026
                                                                        Solar Design025
                                                                        Solar Water Heating005
                                                                        EnergySmart (insul Retrofit)Dunedin (insul)
                                                                        Other124
                                                                        Noise24
                                                                        Budgetfinances25
                                                                        Environmental65
                                                                        Health78
                                                                        Reduced moves86
                                                                        Ease of sellingrenting130
                                                                        DampnessMold1316
                                                                        Comfort-related4933
                                                                        101101
                                                                        Operational NEBs44
                                                                        DesignConstruction NEBs31
                                                                        Occupant NEBs25
                                                                        1
                                                                        OtherOther
                                                                        NoiseNoise
                                                                        BudgetfinancesBudgetfinances
                                                                        EnvironmentalEnvironmental
                                                                        HealthHealth
                                                                        Reduced movesReduced moves
                                                                        Ease of sellingrentingEase of sellingrenting
                                                                        DampnessMoldDampnessMold
                                                                        Comfort-relatedComfort-related

                                                                        SERA

                                                                        INSULATION RESULTS (DUNEDIN amp ENERGY SMART)

                                                                        5 increments to 50

                                                                        Source SERA research

                                                                        Chart5

                                                                        EnergySmart (insul Retrofit)
                                                                        Dunedin (insul)
                                                                        001
                                                                        024
                                                                        002
                                                                        004
                                                                        002
                                                                        005
                                                                        006
                                                                        005
                                                                        007
                                                                        008
                                                                        008
                                                                        006
                                                                        013
                                                                        0
                                                                        013
                                                                        016
                                                                        049
                                                                        033

                                                                        cx

                                                                        cx

                                                                        nz

                                                                        Sheet3

                                                                        Sheet3

                                                                        Super Insulation
                                                                        Double Glazing
                                                                        Solar Design
                                                                        Solar Water Heating
                                                                        Super Insulation
                                                                        Double Glazing
                                                                        Solar Design
                                                                        Solar Water Heating
                                                                        EnergySmart (insul Retrofit)
                                                                        Dunedin (insul)

                                                                        SERA

                                                                        ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                                        0

                                                                        002

                                                                        004

                                                                        006

                                                                        008

                                                                        01

                                                                        012

                                                                        Maint

                                                                        Perf LifeSatis

                                                                        f

                                                                        ComfortAesth

                                                                        NoiseSafe

                                                                        ty

                                                                        LightingSelli

                                                                        ng

                                                                        Stay in home

                                                                        Enviro

                                                                        Sick Days

                                                                        Calls

                                                                        ES Homes HP wES Energy savings multiplier=1

                                                                        Source SERA Research

                                                                        SERA

                                                                        CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                        DG CHP HVAC Equipment rebate Other

                                                                        Building codes incentives by cities

                                                                        Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                        Variety of end uses

                                                                        SERA

                                                                        AUDIT FINANCIAL PROGRAM

                                                                        0

                                                                        01

                                                                        02

                                                                        03

                                                                        04

                                                                        05

                                                                        06

                                                                        07

                                                                        Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                        Source SERA Research

                                                                        SERA

                                                                        CampI NEW CONSTRUCTION

                                                                        -005

                                                                        0

                                                                        005

                                                                        01

                                                                        015

                                                                        02

                                                                        AampE Owner(Pct of Total Partic NEBs)

                                                                        Source SERA Research

                                                                        SERA

                                                                        CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                        0 2 4 6 8 10 12

                                                                        m Product losses

                                                                        l Ease of selling leasing building

                                                                        n Labor requirements

                                                                        h Aesthetics appearance

                                                                        j Noise

                                                                        b Equipment maintenance

                                                                        f Tenant satisfaction

                                                                        d Equipment lifetimes

                                                                        e Productivity

                                                                        k Building safety[2]

                                                                        g Comfort

                                                                        i Lighting Quality of Light

                                                                        c Equipment performance[1]

                                                                        a Operating costs

                                                                        o Doing good for the Environment[3]

                                                                        Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                        vcv

                                                                        Source SERA Research

                                                                        Chart1

                                                                        CampI Technical Assistance NEBs
                                                                        002
                                                                        003
                                                                        004
                                                                        005
                                                                        005
                                                                        006
                                                                        006
                                                                        007
                                                                        007
                                                                        007
                                                                        009
                                                                        009
                                                                        01
                                                                        011
                                                                        011

                                                                        Sheet1

                                                                        Sheet1

                                                                        CampI Technical Assistance Program NEBs (Participant)

                                                                        Sheet2

                                                                        Sheet3

                                                                        SERA

                                                                        EXPRESSING NEBS VALUEndashCxe

                                                                        Strong value from RetroCxSource SERA Research

                                                                        SERA

                                                                        SOCIETAL NEB EXAMPLES

                                                                        31

                                                                        SERA

                                                                        JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                        32

                                                                        0

                                                                        02

                                                                        04

                                                                        06

                                                                        08

                                                                        1

                                                                        12

                                                                        HPWxRetrof Appliance

                                                                        CA

                                                                        WI

                                                                        Natl

                                                                        (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                        Jobs Economic

                                                                        SERA

                                                                        EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                        33

                                                                        SERA

                                                                        UTILITY NEBSUtility NEBs for Template Program

                                                                        Debt WriteOff (util)13

                                                                        Rate Subsidy(util)61

                                                                        HealthSafety(util)0

                                                                        Colln Costs (util)0

                                                                        Gas Calls (util)0

                                                                        Calls to CSRs(util)2

                                                                        TampD (util)16

                                                                        Arrears (util)0

                                                                        Reconnects (util)0

                                                                        Notices (util)7

                                                                        Shutoffs (util)1

                                                                        Rate subsidy TampD

                                                                        Payment-related

                                                                        Source Skumatz Economic Research Associates research

                                                                        Example Low Income Weatherization

                                                                        SERA

                                                                        ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                        be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                        Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                        Value of 100 HALVES Payback doubles ROI

                                                                        Source Skumatz Economics (SERA)

                                                                        35

                                                                        ( for NEBs Multiplier of 125)

                                                                        SERA

                                                                        NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                        36

                                                                        hellip2001 ON

                                                                        SERA

                                                                        37

                                                                        NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                        For true representation of B amp C elements of NEBs address missing factor bias

                                                                        Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                        BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                        SERA

                                                                        CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                        38

                                                                        Test Benefits Costs States Using Traditionally

                                                                        Improved treat-ment with NEBs

                                                                        Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                        bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                        bull Avoided gas and water supply costs

                                                                        bull Program administration

                                                                        bull Participant incentives

                                                                        bull Increased supply cost

                                                                        CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                        Use cost only paid by the utility

                                                                        Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                        Same as above plus bull increased revenue

                                                                        Same as above plus bull Decreased revenue

                                                                        AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                        Participant cost bull Utility bill reductions bull Participant incentives

                                                                        bull Participant direct costs

                                                                        AR CA FL HI IA IN MI MN NY VA

                                                                        Participant NEBs

                                                                        Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                        Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                        SERA

                                                                        APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                        Expansion Concern with risk calculation accuracy

                                                                        39

                                                                        Source Skumatz SERA research

                                                                        SERA

                                                                        40

                                                                        METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                        Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                        Minimize Regulatory amp Implementer Risk

                                                                        Minimize Evaluation Cost

                                                                        Adder

                                                                        Readily MeasurableHybrid

                                                                        All NEBsSource SERA Research

                                                                        SERA

                                                                        41

                                                                        STATE TREATMENT OF NEBS NEBsRegulatory

                                                                        Screening Application Utilities regions

                                                                        Program Marketing Fairly widespread use in utilities states across the country

                                                                        MO

                                                                        RE AGGRESSIVE ==gt

                                                                        Test Pgm Screen - adder

                                                                        IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                        Test Pgm Screen - readily measured

                                                                        MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                        Test Hybrid (potential adder amp measured)

                                                                        CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                        Test Pgm screen - Broad

                                                                        With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                        Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                        SERA

                                                                        42

                                                                        Key

                                                                        States with NEBs in CE tests for At least one type of program

                                                                        copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                        STATES WITH NEBS IN CE TESTS

                                                                        NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                        groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                        SERA

                                                                        SELECTED STATE PROGRESS CA

                                                                        2001 ndash LIPPT model NY

                                                                        Mid-2000s measurement amp scenarios not included CO

                                                                        Adders (10 electric 25 LI 5 gas) VT

                                                                        15 adder helped by previous research DC

                                                                        10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                        43

                                                                        Source Skumatz SERA research

                                                                        SERA

                                                                        SELECTED STATE PROGRESS Dominos ongoing

                                                                        Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                        Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                        Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                        Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                        44

                                                                        Source Skumatz SERA research

                                                                        SERA

                                                                        APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                        Expansion Concern with risk calculation accuracy

                                                                        45

                                                                        Source Skumatz SERA research

                                                                        SERA

                                                                        46

                                                                        NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                        [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                        SourceSkumatz SERA Independent research

                                                                        NTG ndash accuracy measurement incomplete

                                                                        Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                        NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                        EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                        Cost Complicated expensive local changes

                                                                        RELATIVE Risk from NEBs

                                                                        Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                        Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                        NEBs as accurate as many inputs (all could improve)

                                                                        SERA

                                                                        DISCOUNT RATE MATTERS

                                                                        47

                                                                        Source Skumatz SERA Research

                                                                        0

                                                                        20

                                                                        40

                                                                        60

                                                                        80

                                                                        100

                                                                        120

                                                                        1 2 3 4 5 6 7 8 9 10

                                                                        PV Ratios vs 1 Discount Rate

                                                                        10 yr PV 20 yr PV

                                                                        Discount rate - percent

                                                                        WACCSocialLTT

                                                                        Source Skumatz Electricity Journal IEPEC paper independent research

                                                                        SERA

                                                                        EUL VARIATIONS-ADOPTED VALUES

                                                                        Source Skumatz Economic Research Associates SERA independent research

                                                                        0

                                                                        50

                                                                        100

                                                                        150

                                                                        200

                                                                        250

                                                                        300

                                                                        350

                                                                        400

                                                                        450

                                                                        0 10 20 30 40 50 60 70

                                                                        EUL Variation by Measure

                                                                        67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                        Poorly supportedOldTechnology change

                                                                        SERA

                                                                        KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                        Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                        measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                        change the BC conclusion

                                                                        On the cusphellip

                                                                        If NO Yoursquore doneAnd bias addressedsufficiently

                                                                        IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                        Source SERA all rights reserved

                                                                        SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                        SERA

                                                                        IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                        Utility Soc Part Conserv Recrsquom

                                                                        Rationale

                                                                        Base Percent X X X Program-invariant

                                                                        Low Income X X X X Multiple sources

                                                                        Weatherization X X X Substantial Participantimpacts

                                                                        Measure Program-specific

                                                                        X Varies by measure sector

                                                                        Other Recomrsquos Local Research

                                                                        50

                                                                        Source Skumatz SERA

                                                                        Developing values for multiple states amp utilities

                                                                        SERA

                                                                        TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                        Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                        Other sources of bias to research (EUL NTG etc) bring high variation

                                                                        5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                        6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                        THANK YOU

                                                                        Questions

                                                                        Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                        SERA

                                                                        POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                        53

                                                                        SERA

                                                                        NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                        Negative NEB values cost of barrier

                                                                        Solar Water Heat NZ$ Euros

                                                                        Solar Design NZ$ Euros

                                                                        Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                        -23 -12 (079)

                                                                        -11 -6 (06)

                                                                        Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                        Source Skumatz Economic Research Associates research

                                                                        SERA

                                                                        CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                        NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                        -5-4-3-2-1012345

                                                                        Op Co

                                                                        st

                                                                        Init C

                                                                        ost

                                                                        Maint

                                                                        Perfo

                                                                        rm

                                                                        Prod

                                                                        uctiv

                                                                        Tena

                                                                        nt Sa

                                                                        t

                                                                        Comf

                                                                        ort

                                                                        Appe

                                                                        ar

                                                                        Light

                                                                        Qual

                                                                        SellL

                                                                        ease

                                                                        Devp Ownr AampE

                                                                        Note AgtE (~4 vs ~2) for these NEBs

                                                                        Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                        Source Skumatz Economic Research Associates Inc 2002

                                                                        Owners more positive ndashUnderinvestment train

                                                                        • Soup to nuts on non-energy benefits NEIs
                                                                        • TOPICS
                                                                        • BACKGROUND NEBS
                                                                        • 20+ YEARS OF NEBS PROGRESShellip
                                                                        • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                        • KEY APPLICATIONS OF NEBS
                                                                        • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                        • Neb measurement
                                                                        • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                        • MEASURING PARTICIPANTNEBs
                                                                        • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                        • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                        • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                        • PROGRESS TRANSFERABILITY OF NEBS
                                                                        • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                        • PROGRESS AND GAPS
                                                                        • PROGRESS AND GAPS
                                                                        • USING NEB NEI VALUES
                                                                        • Slide Number 19
                                                                        • ARE NEBS HIGH VALUE
                                                                        • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                        • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                        • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                        • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                        • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                        • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                        • AUDIT FINANCIAL PROGRAM
                                                                        • CampI NEW CONSTRUCTION
                                                                        • CampI TECHNICAL ASSISTANCE
                                                                        • EXPRESSING NEBS VALUEndashCxe
                                                                        • Slide Number 31
                                                                        • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                        • EMISSIONS (SOCIETAL NEBS)
                                                                        • UTILITY NEBS
                                                                        • ADJUSTED PAYBACKS
                                                                        • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                        • NEBS IN BC
                                                                        • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                        • APPLICATION TO BC
                                                                        • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                        • STATE TREATMENT OF NEBS NEBs
                                                                        • Slide Number 42
                                                                        • SELECTED STATE PROGRESS
                                                                        • SELECTED STATE PROGRESS
                                                                        • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                        • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                        • DISCOUNT RATE MATTERS
                                                                        • EUL VARIATIONS- ADOPTED VALUES
                                                                        • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                        • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                        • TAKEAWAYSON NEBSNEIS
                                                                        • THANK YOUQuestions
                                                                        • Slide Number 53
                                                                        • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                        • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                          Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                          m Product losses20
                                                                          l Ease of selling leasing building33
                                                                          n Labor requirements45
                                                                          h Aesthetics appearance59
                                                                          j Noise50
                                                                          b Equipment maintenance660
                                                                          f Tenant satisfaction60
                                                                          d Equipment lifetimes718
                                                                          e Productivity724
                                                                          k Building safety[2]73
                                                                          g Comfort933
                                                                          i Lighting Quality of Light916
                                                                          c Equipment performance[1]1019
                                                                          a Operating costs1165
                                                                          o Doing good for the Environment[3]1132
                                                                          Other03
                                                                          [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                          [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                          [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                          m Product losses
                                                                          l Ease of selling leasing building
                                                                          n Labor requirements
                                                                          h Aesthetics appearance
                                                                          j Noise
                                                                          b Equipment maintenance
                                                                          f Tenant satisfaction
                                                                          d Equipment lifetimes
                                                                          e Productivity
                                                                          k Building safety[2]
                                                                          g Comfort
                                                                          i Lighting Quality of Light
                                                                          c Equipment performance[1]
                                                                          a Operating costs
                                                                          o Doing good for the Environment[3]
                                                                          Share of ZALEH NEBS from various measures
                                                                          Each presented as multiple of the (overall) energy savings
                                                                          Super Insulation037
                                                                          Double Glazing026
                                                                          Solar Design025
                                                                          Solar Water Heating005
                                                                          EnergySmart (insul Retrofit)Dunedin (insul)
                                                                          Other124
                                                                          Noise24
                                                                          Budgetfinances25
                                                                          Environmental65
                                                                          Health78
                                                                          Reduced moves86
                                                                          Ease of sellingrenting130
                                                                          DampnessMold1316
                                                                          Comfort-related4933
                                                                          101101
                                                                          Operational NEBs44
                                                                          DesignConstruction NEBs31
                                                                          Occupant NEBs25
                                                                          1
                                                                          OtherOther
                                                                          NoiseNoise
                                                                          BudgetfinancesBudgetfinances
                                                                          EnvironmentalEnvironmental
                                                                          HealthHealth
                                                                          Reduced movesReduced moves
                                                                          Ease of sellingrentingEase of sellingrenting
                                                                          DampnessMoldDampnessMold
                                                                          Comfort-relatedComfort-related

                                                                          Chart5

                                                                          EnergySmart (insul Retrofit)
                                                                          Dunedin (insul)
                                                                          001
                                                                          024
                                                                          002
                                                                          004
                                                                          002
                                                                          005
                                                                          006
                                                                          005
                                                                          007
                                                                          008
                                                                          008
                                                                          006
                                                                          013
                                                                          0
                                                                          013
                                                                          016
                                                                          049
                                                                          033

                                                                          cx

                                                                          cx

                                                                          nz

                                                                          Sheet3

                                                                          Sheet3

                                                                          Super Insulation
                                                                          Double Glazing
                                                                          Solar Design
                                                                          Solar Water Heating
                                                                          Super Insulation
                                                                          Double Glazing
                                                                          Solar Design
                                                                          Solar Water Heating
                                                                          EnergySmart (insul Retrofit)
                                                                          Dunedin (insul)

                                                                          SERA

                                                                          ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                                          0

                                                                          002

                                                                          004

                                                                          006

                                                                          008

                                                                          01

                                                                          012

                                                                          Maint

                                                                          Perf LifeSatis

                                                                          f

                                                                          ComfortAesth

                                                                          NoiseSafe

                                                                          ty

                                                                          LightingSelli

                                                                          ng

                                                                          Stay in home

                                                                          Enviro

                                                                          Sick Days

                                                                          Calls

                                                                          ES Homes HP wES Energy savings multiplier=1

                                                                          Source SERA Research

                                                                          SERA

                                                                          CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                          DG CHP HVAC Equipment rebate Other

                                                                          Building codes incentives by cities

                                                                          Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                          Variety of end uses

                                                                          SERA

                                                                          AUDIT FINANCIAL PROGRAM

                                                                          0

                                                                          01

                                                                          02

                                                                          03

                                                                          04

                                                                          05

                                                                          06

                                                                          07

                                                                          Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                          Source SERA Research

                                                                          SERA

                                                                          CampI NEW CONSTRUCTION

                                                                          -005

                                                                          0

                                                                          005

                                                                          01

                                                                          015

                                                                          02

                                                                          AampE Owner(Pct of Total Partic NEBs)

                                                                          Source SERA Research

                                                                          SERA

                                                                          CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                          0 2 4 6 8 10 12

                                                                          m Product losses

                                                                          l Ease of selling leasing building

                                                                          n Labor requirements

                                                                          h Aesthetics appearance

                                                                          j Noise

                                                                          b Equipment maintenance

                                                                          f Tenant satisfaction

                                                                          d Equipment lifetimes

                                                                          e Productivity

                                                                          k Building safety[2]

                                                                          g Comfort

                                                                          i Lighting Quality of Light

                                                                          c Equipment performance[1]

                                                                          a Operating costs

                                                                          o Doing good for the Environment[3]

                                                                          Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                          vcv

                                                                          Source SERA Research

                                                                          Chart1

                                                                          CampI Technical Assistance NEBs
                                                                          002
                                                                          003
                                                                          004
                                                                          005
                                                                          005
                                                                          006
                                                                          006
                                                                          007
                                                                          007
                                                                          007
                                                                          009
                                                                          009
                                                                          01
                                                                          011
                                                                          011

                                                                          Sheet1

                                                                          Sheet1

                                                                          CampI Technical Assistance Program NEBs (Participant)

                                                                          Sheet2

                                                                          Sheet3

                                                                          SERA

                                                                          EXPRESSING NEBS VALUEndashCxe

                                                                          Strong value from RetroCxSource SERA Research

                                                                          SERA

                                                                          SOCIETAL NEB EXAMPLES

                                                                          31

                                                                          SERA

                                                                          JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                          32

                                                                          0

                                                                          02

                                                                          04

                                                                          06

                                                                          08

                                                                          1

                                                                          12

                                                                          HPWxRetrof Appliance

                                                                          CA

                                                                          WI

                                                                          Natl

                                                                          (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                          Jobs Economic

                                                                          SERA

                                                                          EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                          33

                                                                          SERA

                                                                          UTILITY NEBSUtility NEBs for Template Program

                                                                          Debt WriteOff (util)13

                                                                          Rate Subsidy(util)61

                                                                          HealthSafety(util)0

                                                                          Colln Costs (util)0

                                                                          Gas Calls (util)0

                                                                          Calls to CSRs(util)2

                                                                          TampD (util)16

                                                                          Arrears (util)0

                                                                          Reconnects (util)0

                                                                          Notices (util)7

                                                                          Shutoffs (util)1

                                                                          Rate subsidy TampD

                                                                          Payment-related

                                                                          Source Skumatz Economic Research Associates research

                                                                          Example Low Income Weatherization

                                                                          SERA

                                                                          ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                          be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                          Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                          Value of 100 HALVES Payback doubles ROI

                                                                          Source Skumatz Economics (SERA)

                                                                          35

                                                                          ( for NEBs Multiplier of 125)

                                                                          SERA

                                                                          NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                          36

                                                                          hellip2001 ON

                                                                          SERA

                                                                          37

                                                                          NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                          For true representation of B amp C elements of NEBs address missing factor bias

                                                                          Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                          BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                          SERA

                                                                          CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                          38

                                                                          Test Benefits Costs States Using Traditionally

                                                                          Improved treat-ment with NEBs

                                                                          Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                          bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                          bull Avoided gas and water supply costs

                                                                          bull Program administration

                                                                          bull Participant incentives

                                                                          bull Increased supply cost

                                                                          CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                          Use cost only paid by the utility

                                                                          Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                          Same as above plus bull increased revenue

                                                                          Same as above plus bull Decreased revenue

                                                                          AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                          Participant cost bull Utility bill reductions bull Participant incentives

                                                                          bull Participant direct costs

                                                                          AR CA FL HI IA IN MI MN NY VA

                                                                          Participant NEBs

                                                                          Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                          Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                          SERA

                                                                          APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                          Expansion Concern with risk calculation accuracy

                                                                          39

                                                                          Source Skumatz SERA research

                                                                          SERA

                                                                          40

                                                                          METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                          Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                          Minimize Regulatory amp Implementer Risk

                                                                          Minimize Evaluation Cost

                                                                          Adder

                                                                          Readily MeasurableHybrid

                                                                          All NEBsSource SERA Research

                                                                          SERA

                                                                          41

                                                                          STATE TREATMENT OF NEBS NEBsRegulatory

                                                                          Screening Application Utilities regions

                                                                          Program Marketing Fairly widespread use in utilities states across the country

                                                                          MO

                                                                          RE AGGRESSIVE ==gt

                                                                          Test Pgm Screen - adder

                                                                          IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                          Test Pgm Screen - readily measured

                                                                          MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                          Test Hybrid (potential adder amp measured)

                                                                          CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                          Test Pgm screen - Broad

                                                                          With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                          Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                          SERA

                                                                          42

                                                                          Key

                                                                          States with NEBs in CE tests for At least one type of program

                                                                          copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                          STATES WITH NEBS IN CE TESTS

                                                                          NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                          groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                          SERA

                                                                          SELECTED STATE PROGRESS CA

                                                                          2001 ndash LIPPT model NY

                                                                          Mid-2000s measurement amp scenarios not included CO

                                                                          Adders (10 electric 25 LI 5 gas) VT

                                                                          15 adder helped by previous research DC

                                                                          10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                          43

                                                                          Source Skumatz SERA research

                                                                          SERA

                                                                          SELECTED STATE PROGRESS Dominos ongoing

                                                                          Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                          Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                          Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                          Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                          44

                                                                          Source Skumatz SERA research

                                                                          SERA

                                                                          APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                          Expansion Concern with risk calculation accuracy

                                                                          45

                                                                          Source Skumatz SERA research

                                                                          SERA

                                                                          46

                                                                          NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                          [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                          SourceSkumatz SERA Independent research

                                                                          NTG ndash accuracy measurement incomplete

                                                                          Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                          NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                          EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                          Cost Complicated expensive local changes

                                                                          RELATIVE Risk from NEBs

                                                                          Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                          Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                          NEBs as accurate as many inputs (all could improve)

                                                                          SERA

                                                                          DISCOUNT RATE MATTERS

                                                                          47

                                                                          Source Skumatz SERA Research

                                                                          0

                                                                          20

                                                                          40

                                                                          60

                                                                          80

                                                                          100

                                                                          120

                                                                          1 2 3 4 5 6 7 8 9 10

                                                                          PV Ratios vs 1 Discount Rate

                                                                          10 yr PV 20 yr PV

                                                                          Discount rate - percent

                                                                          WACCSocialLTT

                                                                          Source Skumatz Electricity Journal IEPEC paper independent research

                                                                          SERA

                                                                          EUL VARIATIONS-ADOPTED VALUES

                                                                          Source Skumatz Economic Research Associates SERA independent research

                                                                          0

                                                                          50

                                                                          100

                                                                          150

                                                                          200

                                                                          250

                                                                          300

                                                                          350

                                                                          400

                                                                          450

                                                                          0 10 20 30 40 50 60 70

                                                                          EUL Variation by Measure

                                                                          67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                          Poorly supportedOldTechnology change

                                                                          SERA

                                                                          KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                          Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                          measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                          change the BC conclusion

                                                                          On the cusphellip

                                                                          If NO Yoursquore doneAnd bias addressedsufficiently

                                                                          IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                          Source SERA all rights reserved

                                                                          SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                          SERA

                                                                          IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                          Utility Soc Part Conserv Recrsquom

                                                                          Rationale

                                                                          Base Percent X X X Program-invariant

                                                                          Low Income X X X X Multiple sources

                                                                          Weatherization X X X Substantial Participantimpacts

                                                                          Measure Program-specific

                                                                          X Varies by measure sector

                                                                          Other Recomrsquos Local Research

                                                                          50

                                                                          Source Skumatz SERA

                                                                          Developing values for multiple states amp utilities

                                                                          SERA

                                                                          TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                          Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                          Other sources of bias to research (EUL NTG etc) bring high variation

                                                                          5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                          6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                          THANK YOU

                                                                          Questions

                                                                          Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                          SERA

                                                                          POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                          53

                                                                          SERA

                                                                          NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                          Negative NEB values cost of barrier

                                                                          Solar Water Heat NZ$ Euros

                                                                          Solar Design NZ$ Euros

                                                                          Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                          -23 -12 (079)

                                                                          -11 -6 (06)

                                                                          Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                          Source Skumatz Economic Research Associates research

                                                                          SERA

                                                                          CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                          NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                          -5-4-3-2-1012345

                                                                          Op Co

                                                                          st

                                                                          Init C

                                                                          ost

                                                                          Maint

                                                                          Perfo

                                                                          rm

                                                                          Prod

                                                                          uctiv

                                                                          Tena

                                                                          nt Sa

                                                                          t

                                                                          Comf

                                                                          ort

                                                                          Appe

                                                                          ar

                                                                          Light

                                                                          Qual

                                                                          SellL

                                                                          ease

                                                                          Devp Ownr AampE

                                                                          Note AgtE (~4 vs ~2) for these NEBs

                                                                          Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                          Source Skumatz Economic Research Associates Inc 2002

                                                                          Owners more positive ndashUnderinvestment train

                                                                          • Soup to nuts on non-energy benefits NEIs
                                                                          • TOPICS
                                                                          • BACKGROUND NEBS
                                                                          • 20+ YEARS OF NEBS PROGRESShellip
                                                                          • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                          • KEY APPLICATIONS OF NEBS
                                                                          • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                          • Neb measurement
                                                                          • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                          • MEASURING PARTICIPANTNEBs
                                                                          • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                          • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                          • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                          • PROGRESS TRANSFERABILITY OF NEBS
                                                                          • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                          • PROGRESS AND GAPS
                                                                          • PROGRESS AND GAPS
                                                                          • USING NEB NEI VALUES
                                                                          • Slide Number 19
                                                                          • ARE NEBS HIGH VALUE
                                                                          • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                          • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                          • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                          • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                          • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                          • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                          • AUDIT FINANCIAL PROGRAM
                                                                          • CampI NEW CONSTRUCTION
                                                                          • CampI TECHNICAL ASSISTANCE
                                                                          • EXPRESSING NEBS VALUEndashCxe
                                                                          • Slide Number 31
                                                                          • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                          • EMISSIONS (SOCIETAL NEBS)
                                                                          • UTILITY NEBS
                                                                          • ADJUSTED PAYBACKS
                                                                          • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                          • NEBS IN BC
                                                                          • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                          • APPLICATION TO BC
                                                                          • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                          • STATE TREATMENT OF NEBS NEBs
                                                                          • Slide Number 42
                                                                          • SELECTED STATE PROGRESS
                                                                          • SELECTED STATE PROGRESS
                                                                          • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                          • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                          • DISCOUNT RATE MATTERS
                                                                          • EUL VARIATIONS- ADOPTED VALUES
                                                                          • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                          • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                          • TAKEAWAYSON NEBSNEIS
                                                                          • THANK YOUQuestions
                                                                          • Slide Number 53
                                                                          • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                          • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                            Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                            m Product losses20
                                                                            l Ease of selling leasing building33
                                                                            n Labor requirements45
                                                                            h Aesthetics appearance59
                                                                            j Noise50
                                                                            b Equipment maintenance660
                                                                            f Tenant satisfaction60
                                                                            d Equipment lifetimes718
                                                                            e Productivity724
                                                                            k Building safety[2]73
                                                                            g Comfort933
                                                                            i Lighting Quality of Light916
                                                                            c Equipment performance[1]1019
                                                                            a Operating costs1165
                                                                            o Doing good for the Environment[3]1132
                                                                            Other03
                                                                            [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                            [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                            [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                            m Product losses
                                                                            l Ease of selling leasing building
                                                                            n Labor requirements
                                                                            h Aesthetics appearance
                                                                            j Noise
                                                                            b Equipment maintenance
                                                                            f Tenant satisfaction
                                                                            d Equipment lifetimes
                                                                            e Productivity
                                                                            k Building safety[2]
                                                                            g Comfort
                                                                            i Lighting Quality of Light
                                                                            c Equipment performance[1]
                                                                            a Operating costs
                                                                            o Doing good for the Environment[3]
                                                                            Share of ZALEH NEBS from various measures
                                                                            Each presented as multiple of the (overall) energy savings
                                                                            Super Insulation037
                                                                            Double Glazing026
                                                                            Solar Design025
                                                                            Solar Water Heating005
                                                                            EnergySmart (insul Retrofit)Dunedin (insul)
                                                                            Other124
                                                                            Noise24
                                                                            Budgetfinances25
                                                                            Environmental65
                                                                            Health78
                                                                            Reduced moves86
                                                                            Ease of sellingrenting130
                                                                            DampnessMold1316
                                                                            Comfort-related4933
                                                                            101101
                                                                            Operational NEBs44
                                                                            DesignConstruction NEBs31
                                                                            Occupant NEBs25
                                                                            1
                                                                            OtherOther
                                                                            NoiseNoise
                                                                            BudgetfinancesBudgetfinances
                                                                            EnvironmentalEnvironmental
                                                                            HealthHealth
                                                                            Reduced movesReduced moves
                                                                            Ease of sellingrentingEase of sellingrenting
                                                                            DampnessMoldDampnessMold
                                                                            Comfort-relatedComfort-related

                                                                            cx

                                                                            cx

                                                                            nz

                                                                            Sheet3

                                                                            Sheet3

                                                                            Super Insulation
                                                                            Double Glazing
                                                                            Solar Design
                                                                            Solar Water Heating
                                                                            Super Insulation
                                                                            Double Glazing
                                                                            Solar Design
                                                                            Solar Water Heating
                                                                            EnergySmart (insul Retrofit)
                                                                            Dunedin (insul)

                                                                            SERA

                                                                            ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                                            0

                                                                            002

                                                                            004

                                                                            006

                                                                            008

                                                                            01

                                                                            012

                                                                            Maint

                                                                            Perf LifeSatis

                                                                            f

                                                                            ComfortAesth

                                                                            NoiseSafe

                                                                            ty

                                                                            LightingSelli

                                                                            ng

                                                                            Stay in home

                                                                            Enviro

                                                                            Sick Days

                                                                            Calls

                                                                            ES Homes HP wES Energy savings multiplier=1

                                                                            Source SERA Research

                                                                            SERA

                                                                            CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                            DG CHP HVAC Equipment rebate Other

                                                                            Building codes incentives by cities

                                                                            Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                            Variety of end uses

                                                                            SERA

                                                                            AUDIT FINANCIAL PROGRAM

                                                                            0

                                                                            01

                                                                            02

                                                                            03

                                                                            04

                                                                            05

                                                                            06

                                                                            07

                                                                            Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                            Source SERA Research

                                                                            SERA

                                                                            CampI NEW CONSTRUCTION

                                                                            -005

                                                                            0

                                                                            005

                                                                            01

                                                                            015

                                                                            02

                                                                            AampE Owner(Pct of Total Partic NEBs)

                                                                            Source SERA Research

                                                                            SERA

                                                                            CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                            0 2 4 6 8 10 12

                                                                            m Product losses

                                                                            l Ease of selling leasing building

                                                                            n Labor requirements

                                                                            h Aesthetics appearance

                                                                            j Noise

                                                                            b Equipment maintenance

                                                                            f Tenant satisfaction

                                                                            d Equipment lifetimes

                                                                            e Productivity

                                                                            k Building safety[2]

                                                                            g Comfort

                                                                            i Lighting Quality of Light

                                                                            c Equipment performance[1]

                                                                            a Operating costs

                                                                            o Doing good for the Environment[3]

                                                                            Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                            vcv

                                                                            Source SERA Research

                                                                            Chart1

                                                                            CampI Technical Assistance NEBs
                                                                            002
                                                                            003
                                                                            004
                                                                            005
                                                                            005
                                                                            006
                                                                            006
                                                                            007
                                                                            007
                                                                            007
                                                                            009
                                                                            009
                                                                            01
                                                                            011
                                                                            011

                                                                            Sheet1

                                                                            Sheet1

                                                                            CampI Technical Assistance Program NEBs (Participant)

                                                                            Sheet2

                                                                            Sheet3

                                                                            SERA

                                                                            EXPRESSING NEBS VALUEndashCxe

                                                                            Strong value from RetroCxSource SERA Research

                                                                            SERA

                                                                            SOCIETAL NEB EXAMPLES

                                                                            31

                                                                            SERA

                                                                            JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                            32

                                                                            0

                                                                            02

                                                                            04

                                                                            06

                                                                            08

                                                                            1

                                                                            12

                                                                            HPWxRetrof Appliance

                                                                            CA

                                                                            WI

                                                                            Natl

                                                                            (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                            Jobs Economic

                                                                            SERA

                                                                            EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                            33

                                                                            SERA

                                                                            UTILITY NEBSUtility NEBs for Template Program

                                                                            Debt WriteOff (util)13

                                                                            Rate Subsidy(util)61

                                                                            HealthSafety(util)0

                                                                            Colln Costs (util)0

                                                                            Gas Calls (util)0

                                                                            Calls to CSRs(util)2

                                                                            TampD (util)16

                                                                            Arrears (util)0

                                                                            Reconnects (util)0

                                                                            Notices (util)7

                                                                            Shutoffs (util)1

                                                                            Rate subsidy TampD

                                                                            Payment-related

                                                                            Source Skumatz Economic Research Associates research

                                                                            Example Low Income Weatherization

                                                                            SERA

                                                                            ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                            be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                            Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                            Value of 100 HALVES Payback doubles ROI

                                                                            Source Skumatz Economics (SERA)

                                                                            35

                                                                            ( for NEBs Multiplier of 125)

                                                                            SERA

                                                                            NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                            36

                                                                            hellip2001 ON

                                                                            SERA

                                                                            37

                                                                            NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                            For true representation of B amp C elements of NEBs address missing factor bias

                                                                            Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                            BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                            SERA

                                                                            CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                            38

                                                                            Test Benefits Costs States Using Traditionally

                                                                            Improved treat-ment with NEBs

                                                                            Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                            bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                            bull Avoided gas and water supply costs

                                                                            bull Program administration

                                                                            bull Participant incentives

                                                                            bull Increased supply cost

                                                                            CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                            Use cost only paid by the utility

                                                                            Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                            Same as above plus bull increased revenue

                                                                            Same as above plus bull Decreased revenue

                                                                            AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                            Participant cost bull Utility bill reductions bull Participant incentives

                                                                            bull Participant direct costs

                                                                            AR CA FL HI IA IN MI MN NY VA

                                                                            Participant NEBs

                                                                            Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                            Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                            SERA

                                                                            APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                            Expansion Concern with risk calculation accuracy

                                                                            39

                                                                            Source Skumatz SERA research

                                                                            SERA

                                                                            40

                                                                            METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                            Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                            Minimize Regulatory amp Implementer Risk

                                                                            Minimize Evaluation Cost

                                                                            Adder

                                                                            Readily MeasurableHybrid

                                                                            All NEBsSource SERA Research

                                                                            SERA

                                                                            41

                                                                            STATE TREATMENT OF NEBS NEBsRegulatory

                                                                            Screening Application Utilities regions

                                                                            Program Marketing Fairly widespread use in utilities states across the country

                                                                            MO

                                                                            RE AGGRESSIVE ==gt

                                                                            Test Pgm Screen - adder

                                                                            IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                            Test Pgm Screen - readily measured

                                                                            MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                            Test Hybrid (potential adder amp measured)

                                                                            CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                            Test Pgm screen - Broad

                                                                            With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                            Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                            SERA

                                                                            42

                                                                            Key

                                                                            States with NEBs in CE tests for At least one type of program

                                                                            copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                            STATES WITH NEBS IN CE TESTS

                                                                            NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                            groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                            SERA

                                                                            SELECTED STATE PROGRESS CA

                                                                            2001 ndash LIPPT model NY

                                                                            Mid-2000s measurement amp scenarios not included CO

                                                                            Adders (10 electric 25 LI 5 gas) VT

                                                                            15 adder helped by previous research DC

                                                                            10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                            43

                                                                            Source Skumatz SERA research

                                                                            SERA

                                                                            SELECTED STATE PROGRESS Dominos ongoing

                                                                            Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                            Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                            Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                            Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                            44

                                                                            Source Skumatz SERA research

                                                                            SERA

                                                                            APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                            Expansion Concern with risk calculation accuracy

                                                                            45

                                                                            Source Skumatz SERA research

                                                                            SERA

                                                                            46

                                                                            NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                            [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                            SourceSkumatz SERA Independent research

                                                                            NTG ndash accuracy measurement incomplete

                                                                            Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                            NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                            EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                            Cost Complicated expensive local changes

                                                                            RELATIVE Risk from NEBs

                                                                            Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                            Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                            NEBs as accurate as many inputs (all could improve)

                                                                            SERA

                                                                            DISCOUNT RATE MATTERS

                                                                            47

                                                                            Source Skumatz SERA Research

                                                                            0

                                                                            20

                                                                            40

                                                                            60

                                                                            80

                                                                            100

                                                                            120

                                                                            1 2 3 4 5 6 7 8 9 10

                                                                            PV Ratios vs 1 Discount Rate

                                                                            10 yr PV 20 yr PV

                                                                            Discount rate - percent

                                                                            WACCSocialLTT

                                                                            Source Skumatz Electricity Journal IEPEC paper independent research

                                                                            SERA

                                                                            EUL VARIATIONS-ADOPTED VALUES

                                                                            Source Skumatz Economic Research Associates SERA independent research

                                                                            0

                                                                            50

                                                                            100

                                                                            150

                                                                            200

                                                                            250

                                                                            300

                                                                            350

                                                                            400

                                                                            450

                                                                            0 10 20 30 40 50 60 70

                                                                            EUL Variation by Measure

                                                                            67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                            Poorly supportedOldTechnology change

                                                                            SERA

                                                                            KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                            Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                            measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                            change the BC conclusion

                                                                            On the cusphellip

                                                                            If NO Yoursquore doneAnd bias addressedsufficiently

                                                                            IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                            Source SERA all rights reserved

                                                                            SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                            SERA

                                                                            IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                            Utility Soc Part Conserv Recrsquom

                                                                            Rationale

                                                                            Base Percent X X X Program-invariant

                                                                            Low Income X X X X Multiple sources

                                                                            Weatherization X X X Substantial Participantimpacts

                                                                            Measure Program-specific

                                                                            X Varies by measure sector

                                                                            Other Recomrsquos Local Research

                                                                            50

                                                                            Source Skumatz SERA

                                                                            Developing values for multiple states amp utilities

                                                                            SERA

                                                                            TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                            Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                            Other sources of bias to research (EUL NTG etc) bring high variation

                                                                            5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                            6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                            THANK YOU

                                                                            Questions

                                                                            Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                            SERA

                                                                            POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                            53

                                                                            SERA

                                                                            NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                            Negative NEB values cost of barrier

                                                                            Solar Water Heat NZ$ Euros

                                                                            Solar Design NZ$ Euros

                                                                            Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                            -23 -12 (079)

                                                                            -11 -6 (06)

                                                                            Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                            Source Skumatz Economic Research Associates research

                                                                            SERA

                                                                            CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                            NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                            -5-4-3-2-1012345

                                                                            Op Co

                                                                            st

                                                                            Init C

                                                                            ost

                                                                            Maint

                                                                            Perfo

                                                                            rm

                                                                            Prod

                                                                            uctiv

                                                                            Tena

                                                                            nt Sa

                                                                            t

                                                                            Comf

                                                                            ort

                                                                            Appe

                                                                            ar

                                                                            Light

                                                                            Qual

                                                                            SellL

                                                                            ease

                                                                            Devp Ownr AampE

                                                                            Note AgtE (~4 vs ~2) for these NEBs

                                                                            Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                            Source Skumatz Economic Research Associates Inc 2002

                                                                            Owners more positive ndashUnderinvestment train

                                                                            • Soup to nuts on non-energy benefits NEIs
                                                                            • TOPICS
                                                                            • BACKGROUND NEBS
                                                                            • 20+ YEARS OF NEBS PROGRESShellip
                                                                            • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                            • KEY APPLICATIONS OF NEBS
                                                                            • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                            • Neb measurement
                                                                            • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                            • MEASURING PARTICIPANTNEBs
                                                                            • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                            • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                            • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                            • PROGRESS TRANSFERABILITY OF NEBS
                                                                            • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                            • PROGRESS AND GAPS
                                                                            • PROGRESS AND GAPS
                                                                            • USING NEB NEI VALUES
                                                                            • Slide Number 19
                                                                            • ARE NEBS HIGH VALUE
                                                                            • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                            • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                            • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                            • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                            • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                            • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                            • AUDIT FINANCIAL PROGRAM
                                                                            • CampI NEW CONSTRUCTION
                                                                            • CampI TECHNICAL ASSISTANCE
                                                                            • EXPRESSING NEBS VALUEndashCxe
                                                                            • Slide Number 31
                                                                            • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                            • EMISSIONS (SOCIETAL NEBS)
                                                                            • UTILITY NEBS
                                                                            • ADJUSTED PAYBACKS
                                                                            • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                            • NEBS IN BC
                                                                            • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                            • APPLICATION TO BC
                                                                            • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                            • STATE TREATMENT OF NEBS NEBs
                                                                            • Slide Number 42
                                                                            • SELECTED STATE PROGRESS
                                                                            • SELECTED STATE PROGRESS
                                                                            • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                            • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                            • DISCOUNT RATE MATTERS
                                                                            • EUL VARIATIONS- ADOPTED VALUES
                                                                            • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                            • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                            • TAKEAWAYSON NEBSNEIS
                                                                            • THANK YOUQuestions
                                                                            • Slide Number 53
                                                                            • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                            • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                              Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                              m Product losses20
                                                                              l Ease of selling leasing building33
                                                                              n Labor requirements45
                                                                              h Aesthetics appearance59
                                                                              j Noise50
                                                                              b Equipment maintenance660
                                                                              f Tenant satisfaction60
                                                                              d Equipment lifetimes718
                                                                              e Productivity724
                                                                              k Building safety[2]73
                                                                              g Comfort933
                                                                              i Lighting Quality of Light916
                                                                              c Equipment performance[1]1019
                                                                              a Operating costs1165
                                                                              o Doing good for the Environment[3]1132
                                                                              Other03
                                                                              [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                              [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                              [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                              m Product losses
                                                                              l Ease of selling leasing building
                                                                              n Labor requirements
                                                                              h Aesthetics appearance
                                                                              j Noise
                                                                              b Equipment maintenance
                                                                              f Tenant satisfaction
                                                                              d Equipment lifetimes
                                                                              e Productivity
                                                                              k Building safety[2]
                                                                              g Comfort
                                                                              i Lighting Quality of Light
                                                                              c Equipment performance[1]
                                                                              a Operating costs
                                                                              o Doing good for the Environment[3]
                                                                              Share of ZALEH NEBS from various measures
                                                                              Each presented as multiple of the (overall) energy savings
                                                                              Super Insulation037
                                                                              Double Glazing026
                                                                              Solar Design025
                                                                              Solar Water Heating005
                                                                              EnergySmart (insul Retrofit)Dunedin (insul)
                                                                              Other124
                                                                              Noise24
                                                                              Budgetfinances25
                                                                              Environmental65
                                                                              Health78
                                                                              Reduced moves86
                                                                              Ease of sellingrenting130
                                                                              DampnessMold1316
                                                                              Comfort-related4933
                                                                              101101
                                                                              Operational NEBs44
                                                                              DesignConstruction NEBs31
                                                                              Occupant NEBs25
                                                                              1

                                                                              cx

                                                                              nz

                                                                              Sheet3

                                                                              Sheet3

                                                                              Super Insulation
                                                                              Double Glazing
                                                                              Solar Design
                                                                              Solar Water Heating
                                                                              Super Insulation
                                                                              Double Glazing
                                                                              Solar Design
                                                                              Solar Water Heating
                                                                              EnergySmart (insul Retrofit)
                                                                              Dunedin (insul)

                                                                              SERA

                                                                              ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                                              0

                                                                              002

                                                                              004

                                                                              006

                                                                              008

                                                                              01

                                                                              012

                                                                              Maint

                                                                              Perf LifeSatis

                                                                              f

                                                                              ComfortAesth

                                                                              NoiseSafe

                                                                              ty

                                                                              LightingSelli

                                                                              ng

                                                                              Stay in home

                                                                              Enviro

                                                                              Sick Days

                                                                              Calls

                                                                              ES Homes HP wES Energy savings multiplier=1

                                                                              Source SERA Research

                                                                              SERA

                                                                              CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                              DG CHP HVAC Equipment rebate Other

                                                                              Building codes incentives by cities

                                                                              Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                              Variety of end uses

                                                                              SERA

                                                                              AUDIT FINANCIAL PROGRAM

                                                                              0

                                                                              01

                                                                              02

                                                                              03

                                                                              04

                                                                              05

                                                                              06

                                                                              07

                                                                              Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                              Source SERA Research

                                                                              SERA

                                                                              CampI NEW CONSTRUCTION

                                                                              -005

                                                                              0

                                                                              005

                                                                              01

                                                                              015

                                                                              02

                                                                              AampE Owner(Pct of Total Partic NEBs)

                                                                              Source SERA Research

                                                                              SERA

                                                                              CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                              0 2 4 6 8 10 12

                                                                              m Product losses

                                                                              l Ease of selling leasing building

                                                                              n Labor requirements

                                                                              h Aesthetics appearance

                                                                              j Noise

                                                                              b Equipment maintenance

                                                                              f Tenant satisfaction

                                                                              d Equipment lifetimes

                                                                              e Productivity

                                                                              k Building safety[2]

                                                                              g Comfort

                                                                              i Lighting Quality of Light

                                                                              c Equipment performance[1]

                                                                              a Operating costs

                                                                              o Doing good for the Environment[3]

                                                                              Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                              vcv

                                                                              Source SERA Research

                                                                              Chart1

                                                                              CampI Technical Assistance NEBs
                                                                              002
                                                                              003
                                                                              004
                                                                              005
                                                                              005
                                                                              006
                                                                              006
                                                                              007
                                                                              007
                                                                              007
                                                                              009
                                                                              009
                                                                              01
                                                                              011
                                                                              011

                                                                              Sheet1

                                                                              Sheet1

                                                                              CampI Technical Assistance Program NEBs (Participant)

                                                                              Sheet2

                                                                              Sheet3

                                                                              SERA

                                                                              EXPRESSING NEBS VALUEndashCxe

                                                                              Strong value from RetroCxSource SERA Research

                                                                              SERA

                                                                              SOCIETAL NEB EXAMPLES

                                                                              31

                                                                              SERA

                                                                              JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                              32

                                                                              0

                                                                              02

                                                                              04

                                                                              06

                                                                              08

                                                                              1

                                                                              12

                                                                              HPWxRetrof Appliance

                                                                              CA

                                                                              WI

                                                                              Natl

                                                                              (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                              Jobs Economic

                                                                              SERA

                                                                              EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                              33

                                                                              SERA

                                                                              UTILITY NEBSUtility NEBs for Template Program

                                                                              Debt WriteOff (util)13

                                                                              Rate Subsidy(util)61

                                                                              HealthSafety(util)0

                                                                              Colln Costs (util)0

                                                                              Gas Calls (util)0

                                                                              Calls to CSRs(util)2

                                                                              TampD (util)16

                                                                              Arrears (util)0

                                                                              Reconnects (util)0

                                                                              Notices (util)7

                                                                              Shutoffs (util)1

                                                                              Rate subsidy TampD

                                                                              Payment-related

                                                                              Source Skumatz Economic Research Associates research

                                                                              Example Low Income Weatherization

                                                                              SERA

                                                                              ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                              be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                              Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                              Value of 100 HALVES Payback doubles ROI

                                                                              Source Skumatz Economics (SERA)

                                                                              35

                                                                              ( for NEBs Multiplier of 125)

                                                                              SERA

                                                                              NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                              36

                                                                              hellip2001 ON

                                                                              SERA

                                                                              37

                                                                              NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                              For true representation of B amp C elements of NEBs address missing factor bias

                                                                              Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                              BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                              SERA

                                                                              CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                              38

                                                                              Test Benefits Costs States Using Traditionally

                                                                              Improved treat-ment with NEBs

                                                                              Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                              bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                              bull Avoided gas and water supply costs

                                                                              bull Program administration

                                                                              bull Participant incentives

                                                                              bull Increased supply cost

                                                                              CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                              Use cost only paid by the utility

                                                                              Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                              Same as above plus bull increased revenue

                                                                              Same as above plus bull Decreased revenue

                                                                              AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                              Participant cost bull Utility bill reductions bull Participant incentives

                                                                              bull Participant direct costs

                                                                              AR CA FL HI IA IN MI MN NY VA

                                                                              Participant NEBs

                                                                              Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                              Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                              SERA

                                                                              APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                              Expansion Concern with risk calculation accuracy

                                                                              39

                                                                              Source Skumatz SERA research

                                                                              SERA

                                                                              40

                                                                              METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                              Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                              Minimize Regulatory amp Implementer Risk

                                                                              Minimize Evaluation Cost

                                                                              Adder

                                                                              Readily MeasurableHybrid

                                                                              All NEBsSource SERA Research

                                                                              SERA

                                                                              41

                                                                              STATE TREATMENT OF NEBS NEBsRegulatory

                                                                              Screening Application Utilities regions

                                                                              Program Marketing Fairly widespread use in utilities states across the country

                                                                              MO

                                                                              RE AGGRESSIVE ==gt

                                                                              Test Pgm Screen - adder

                                                                              IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                              Test Pgm Screen - readily measured

                                                                              MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                              Test Hybrid (potential adder amp measured)

                                                                              CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                              Test Pgm screen - Broad

                                                                              With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                              Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                              SERA

                                                                              42

                                                                              Key

                                                                              States with NEBs in CE tests for At least one type of program

                                                                              copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                              STATES WITH NEBS IN CE TESTS

                                                                              NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                              groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                              SERA

                                                                              SELECTED STATE PROGRESS CA

                                                                              2001 ndash LIPPT model NY

                                                                              Mid-2000s measurement amp scenarios not included CO

                                                                              Adders (10 electric 25 LI 5 gas) VT

                                                                              15 adder helped by previous research DC

                                                                              10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                              43

                                                                              Source Skumatz SERA research

                                                                              SERA

                                                                              SELECTED STATE PROGRESS Dominos ongoing

                                                                              Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                              Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                              Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                              Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                              44

                                                                              Source Skumatz SERA research

                                                                              SERA

                                                                              APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                              Expansion Concern with risk calculation accuracy

                                                                              45

                                                                              Source Skumatz SERA research

                                                                              SERA

                                                                              46

                                                                              NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                              [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                              SourceSkumatz SERA Independent research

                                                                              NTG ndash accuracy measurement incomplete

                                                                              Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                              NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                              EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                              Cost Complicated expensive local changes

                                                                              RELATIVE Risk from NEBs

                                                                              Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                              Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                              NEBs as accurate as many inputs (all could improve)

                                                                              SERA

                                                                              DISCOUNT RATE MATTERS

                                                                              47

                                                                              Source Skumatz SERA Research

                                                                              0

                                                                              20

                                                                              40

                                                                              60

                                                                              80

                                                                              100

                                                                              120

                                                                              1 2 3 4 5 6 7 8 9 10

                                                                              PV Ratios vs 1 Discount Rate

                                                                              10 yr PV 20 yr PV

                                                                              Discount rate - percent

                                                                              WACCSocialLTT

                                                                              Source Skumatz Electricity Journal IEPEC paper independent research

                                                                              SERA

                                                                              EUL VARIATIONS-ADOPTED VALUES

                                                                              Source Skumatz Economic Research Associates SERA independent research

                                                                              0

                                                                              50

                                                                              100

                                                                              150

                                                                              200

                                                                              250

                                                                              300

                                                                              350

                                                                              400

                                                                              450

                                                                              0 10 20 30 40 50 60 70

                                                                              EUL Variation by Measure

                                                                              67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                              Poorly supportedOldTechnology change

                                                                              SERA

                                                                              KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                              Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                              measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                              change the BC conclusion

                                                                              On the cusphellip

                                                                              If NO Yoursquore doneAnd bias addressedsufficiently

                                                                              IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                              Source SERA all rights reserved

                                                                              SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                              SERA

                                                                              IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                              Utility Soc Part Conserv Recrsquom

                                                                              Rationale

                                                                              Base Percent X X X Program-invariant

                                                                              Low Income X X X X Multiple sources

                                                                              Weatherization X X X Substantial Participantimpacts

                                                                              Measure Program-specific

                                                                              X Varies by measure sector

                                                                              Other Recomrsquos Local Research

                                                                              50

                                                                              Source Skumatz SERA

                                                                              Developing values for multiple states amp utilities

                                                                              SERA

                                                                              TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                              Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                              Other sources of bias to research (EUL NTG etc) bring high variation

                                                                              5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                              6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                              THANK YOU

                                                                              Questions

                                                                              Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                              SERA

                                                                              POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                              53

                                                                              SERA

                                                                              NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                              Negative NEB values cost of barrier

                                                                              Solar Water Heat NZ$ Euros

                                                                              Solar Design NZ$ Euros

                                                                              Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                              -23 -12 (079)

                                                                              -11 -6 (06)

                                                                              Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                              Source Skumatz Economic Research Associates research

                                                                              SERA

                                                                              CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                              NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                              -5-4-3-2-1012345

                                                                              Op Co

                                                                              st

                                                                              Init C

                                                                              ost

                                                                              Maint

                                                                              Perfo

                                                                              rm

                                                                              Prod

                                                                              uctiv

                                                                              Tena

                                                                              nt Sa

                                                                              t

                                                                              Comf

                                                                              ort

                                                                              Appe

                                                                              ar

                                                                              Light

                                                                              Qual

                                                                              SellL

                                                                              ease

                                                                              Devp Ownr AampE

                                                                              Note AgtE (~4 vs ~2) for these NEBs

                                                                              Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                              Source Skumatz Economic Research Associates Inc 2002

                                                                              Owners more positive ndashUnderinvestment train

                                                                              • Soup to nuts on non-energy benefits NEIs
                                                                              • TOPICS
                                                                              • BACKGROUND NEBS
                                                                              • 20+ YEARS OF NEBS PROGRESShellip
                                                                              • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                              • KEY APPLICATIONS OF NEBS
                                                                              • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                              • Neb measurement
                                                                              • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                              • MEASURING PARTICIPANTNEBs
                                                                              • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                              • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                              • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                              • PROGRESS TRANSFERABILITY OF NEBS
                                                                              • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                              • PROGRESS AND GAPS
                                                                              • PROGRESS AND GAPS
                                                                              • USING NEB NEI VALUES
                                                                              • Slide Number 19
                                                                              • ARE NEBS HIGH VALUE
                                                                              • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                              • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                              • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                              • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                              • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                              • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                              • AUDIT FINANCIAL PROGRAM
                                                                              • CampI NEW CONSTRUCTION
                                                                              • CampI TECHNICAL ASSISTANCE
                                                                              • EXPRESSING NEBS VALUEndashCxe
                                                                              • Slide Number 31
                                                                              • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                              • EMISSIONS (SOCIETAL NEBS)
                                                                              • UTILITY NEBS
                                                                              • ADJUSTED PAYBACKS
                                                                              • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                              • NEBS IN BC
                                                                              • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                              • APPLICATION TO BC
                                                                              • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                              • STATE TREATMENT OF NEBS NEBs
                                                                              • Slide Number 42
                                                                              • SELECTED STATE PROGRESS
                                                                              • SELECTED STATE PROGRESS
                                                                              • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                              • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                              • DISCOUNT RATE MATTERS
                                                                              • EUL VARIATIONS- ADOPTED VALUES
                                                                              • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                              • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                              • TAKEAWAYSON NEBSNEIS
                                                                              • THANK YOUQuestions
                                                                              • Slide Number 53
                                                                              • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                              • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                                Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                                m Product losses20
                                                                                l Ease of selling leasing building33
                                                                                n Labor requirements45
                                                                                h Aesthetics appearance59
                                                                                j Noise50
                                                                                b Equipment maintenance660
                                                                                f Tenant satisfaction60
                                                                                d Equipment lifetimes718
                                                                                e Productivity724
                                                                                k Building safety[2]73
                                                                                g Comfort933
                                                                                i Lighting Quality of Light916
                                                                                c Equipment performance[1]1019
                                                                                a Operating costs1165
                                                                                o Doing good for the Environment[3]1132
                                                                                Other03
                                                                                [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                                [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                                [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                                m Product losses
                                                                                l Ease of selling leasing building
                                                                                n Labor requirements
                                                                                h Aesthetics appearance
                                                                                j Noise
                                                                                b Equipment maintenance
                                                                                f Tenant satisfaction
                                                                                d Equipment lifetimes
                                                                                e Productivity
                                                                                k Building safety[2]
                                                                                g Comfort
                                                                                i Lighting Quality of Light
                                                                                c Equipment performance[1]
                                                                                a Operating costs
                                                                                o Doing good for the Environment[3]
                                                                                Share of ZALEH NEBS from various measures
                                                                                Each presented as multiple of the (overall) energy savings
                                                                                Super Insulation037
                                                                                Double Glazing026
                                                                                Solar Design025
                                                                                Solar Water Heating005
                                                                                EnergySmart (insul Retrofit)Dunedin (insul)
                                                                                Other124
                                                                                Noise24
                                                                                Budgetfinances25
                                                                                Environmental65
                                                                                Health78
                                                                                Reduced moves86
                                                                                Ease of sellingrenting130
                                                                                DampnessMold1316
                                                                                Comfort-related4933
                                                                                101101

                                                                                nz

                                                                                Sheet3

                                                                                Sheet3

                                                                                Super Insulation
                                                                                Double Glazing
                                                                                Solar Design
                                                                                Solar Water Heating
                                                                                Super Insulation
                                                                                Double Glazing
                                                                                Solar Design
                                                                                Solar Water Heating
                                                                                EnergySmart (insul Retrofit)
                                                                                Dunedin (insul)

                                                                                SERA

                                                                                ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                                                0

                                                                                002

                                                                                004

                                                                                006

                                                                                008

                                                                                01

                                                                                012

                                                                                Maint

                                                                                Perf LifeSatis

                                                                                f

                                                                                ComfortAesth

                                                                                NoiseSafe

                                                                                ty

                                                                                LightingSelli

                                                                                ng

                                                                                Stay in home

                                                                                Enviro

                                                                                Sick Days

                                                                                Calls

                                                                                ES Homes HP wES Energy savings multiplier=1

                                                                                Source SERA Research

                                                                                SERA

                                                                                CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                                DG CHP HVAC Equipment rebate Other

                                                                                Building codes incentives by cities

                                                                                Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                                Variety of end uses

                                                                                SERA

                                                                                AUDIT FINANCIAL PROGRAM

                                                                                0

                                                                                01

                                                                                02

                                                                                03

                                                                                04

                                                                                05

                                                                                06

                                                                                07

                                                                                Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                                Source SERA Research

                                                                                SERA

                                                                                CampI NEW CONSTRUCTION

                                                                                -005

                                                                                0

                                                                                005

                                                                                01

                                                                                015

                                                                                02

                                                                                AampE Owner(Pct of Total Partic NEBs)

                                                                                Source SERA Research

                                                                                SERA

                                                                                CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                                0 2 4 6 8 10 12

                                                                                m Product losses

                                                                                l Ease of selling leasing building

                                                                                n Labor requirements

                                                                                h Aesthetics appearance

                                                                                j Noise

                                                                                b Equipment maintenance

                                                                                f Tenant satisfaction

                                                                                d Equipment lifetimes

                                                                                e Productivity

                                                                                k Building safety[2]

                                                                                g Comfort

                                                                                i Lighting Quality of Light

                                                                                c Equipment performance[1]

                                                                                a Operating costs

                                                                                o Doing good for the Environment[3]

                                                                                Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                                vcv

                                                                                Source SERA Research

                                                                                Chart1

                                                                                CampI Technical Assistance NEBs
                                                                                002
                                                                                003
                                                                                004
                                                                                005
                                                                                005
                                                                                006
                                                                                006
                                                                                007
                                                                                007
                                                                                007
                                                                                009
                                                                                009
                                                                                01
                                                                                011
                                                                                011

                                                                                Sheet1

                                                                                Sheet1

                                                                                CampI Technical Assistance Program NEBs (Participant)

                                                                                Sheet2

                                                                                Sheet3

                                                                                SERA

                                                                                EXPRESSING NEBS VALUEndashCxe

                                                                                Strong value from RetroCxSource SERA Research

                                                                                SERA

                                                                                SOCIETAL NEB EXAMPLES

                                                                                31

                                                                                SERA

                                                                                JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                32

                                                                                0

                                                                                02

                                                                                04

                                                                                06

                                                                                08

                                                                                1

                                                                                12

                                                                                HPWxRetrof Appliance

                                                                                CA

                                                                                WI

                                                                                Natl

                                                                                (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                Jobs Economic

                                                                                SERA

                                                                                EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                33

                                                                                SERA

                                                                                UTILITY NEBSUtility NEBs for Template Program

                                                                                Debt WriteOff (util)13

                                                                                Rate Subsidy(util)61

                                                                                HealthSafety(util)0

                                                                                Colln Costs (util)0

                                                                                Gas Calls (util)0

                                                                                Calls to CSRs(util)2

                                                                                TampD (util)16

                                                                                Arrears (util)0

                                                                                Reconnects (util)0

                                                                                Notices (util)7

                                                                                Shutoffs (util)1

                                                                                Rate subsidy TampD

                                                                                Payment-related

                                                                                Source Skumatz Economic Research Associates research

                                                                                Example Low Income Weatherization

                                                                                SERA

                                                                                ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                Value of 100 HALVES Payback doubles ROI

                                                                                Source Skumatz Economics (SERA)

                                                                                35

                                                                                ( for NEBs Multiplier of 125)

                                                                                SERA

                                                                                NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                36

                                                                                hellip2001 ON

                                                                                SERA

                                                                                37

                                                                                NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                For true representation of B amp C elements of NEBs address missing factor bias

                                                                                Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                SERA

                                                                                CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                38

                                                                                Test Benefits Costs States Using Traditionally

                                                                                Improved treat-ment with NEBs

                                                                                Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                bull Avoided gas and water supply costs

                                                                                bull Program administration

                                                                                bull Participant incentives

                                                                                bull Increased supply cost

                                                                                CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                Use cost only paid by the utility

                                                                                Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                Same as above plus bull increased revenue

                                                                                Same as above plus bull Decreased revenue

                                                                                AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                Participant cost bull Utility bill reductions bull Participant incentives

                                                                                bull Participant direct costs

                                                                                AR CA FL HI IA IN MI MN NY VA

                                                                                Participant NEBs

                                                                                Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                SERA

                                                                                APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                Expansion Concern with risk calculation accuracy

                                                                                39

                                                                                Source Skumatz SERA research

                                                                                SERA

                                                                                40

                                                                                METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                Minimize Regulatory amp Implementer Risk

                                                                                Minimize Evaluation Cost

                                                                                Adder

                                                                                Readily MeasurableHybrid

                                                                                All NEBsSource SERA Research

                                                                                SERA

                                                                                41

                                                                                STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                Screening Application Utilities regions

                                                                                Program Marketing Fairly widespread use in utilities states across the country

                                                                                MO

                                                                                RE AGGRESSIVE ==gt

                                                                                Test Pgm Screen - adder

                                                                                IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                Test Pgm Screen - readily measured

                                                                                MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                Test Hybrid (potential adder amp measured)

                                                                                CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                Test Pgm screen - Broad

                                                                                With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                SERA

                                                                                42

                                                                                Key

                                                                                States with NEBs in CE tests for At least one type of program

                                                                                copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                STATES WITH NEBS IN CE TESTS

                                                                                NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                SERA

                                                                                SELECTED STATE PROGRESS CA

                                                                                2001 ndash LIPPT model NY

                                                                                Mid-2000s measurement amp scenarios not included CO

                                                                                Adders (10 electric 25 LI 5 gas) VT

                                                                                15 adder helped by previous research DC

                                                                                10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                43

                                                                                Source Skumatz SERA research

                                                                                SERA

                                                                                SELECTED STATE PROGRESS Dominos ongoing

                                                                                Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                44

                                                                                Source Skumatz SERA research

                                                                                SERA

                                                                                APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                Expansion Concern with risk calculation accuracy

                                                                                45

                                                                                Source Skumatz SERA research

                                                                                SERA

                                                                                46

                                                                                NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                SourceSkumatz SERA Independent research

                                                                                NTG ndash accuracy measurement incomplete

                                                                                Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                Cost Complicated expensive local changes

                                                                                RELATIVE Risk from NEBs

                                                                                Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                NEBs as accurate as many inputs (all could improve)

                                                                                SERA

                                                                                DISCOUNT RATE MATTERS

                                                                                47

                                                                                Source Skumatz SERA Research

                                                                                0

                                                                                20

                                                                                40

                                                                                60

                                                                                80

                                                                                100

                                                                                120

                                                                                1 2 3 4 5 6 7 8 9 10

                                                                                PV Ratios vs 1 Discount Rate

                                                                                10 yr PV 20 yr PV

                                                                                Discount rate - percent

                                                                                WACCSocialLTT

                                                                                Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                SERA

                                                                                EUL VARIATIONS-ADOPTED VALUES

                                                                                Source Skumatz Economic Research Associates SERA independent research

                                                                                0

                                                                                50

                                                                                100

                                                                                150

                                                                                200

                                                                                250

                                                                                300

                                                                                350

                                                                                400

                                                                                450

                                                                                0 10 20 30 40 50 60 70

                                                                                EUL Variation by Measure

                                                                                67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                Poorly supportedOldTechnology change

                                                                                SERA

                                                                                KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                change the BC conclusion

                                                                                On the cusphellip

                                                                                If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                Source SERA all rights reserved

                                                                                SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                SERA

                                                                                IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                Utility Soc Part Conserv Recrsquom

                                                                                Rationale

                                                                                Base Percent X X X Program-invariant

                                                                                Low Income X X X X Multiple sources

                                                                                Weatherization X X X Substantial Participantimpacts

                                                                                Measure Program-specific

                                                                                X Varies by measure sector

                                                                                Other Recomrsquos Local Research

                                                                                50

                                                                                Source Skumatz SERA

                                                                                Developing values for multiple states amp utilities

                                                                                SERA

                                                                                TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                THANK YOU

                                                                                Questions

                                                                                Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                SERA

                                                                                POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                53

                                                                                SERA

                                                                                NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                Negative NEB values cost of barrier

                                                                                Solar Water Heat NZ$ Euros

                                                                                Solar Design NZ$ Euros

                                                                                Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                -23 -12 (079)

                                                                                -11 -6 (06)

                                                                                Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                Source Skumatz Economic Research Associates research

                                                                                SERA

                                                                                CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                -5-4-3-2-1012345

                                                                                Op Co

                                                                                st

                                                                                Init C

                                                                                ost

                                                                                Maint

                                                                                Perfo

                                                                                rm

                                                                                Prod

                                                                                uctiv

                                                                                Tena

                                                                                nt Sa

                                                                                t

                                                                                Comf

                                                                                ort

                                                                                Appe

                                                                                ar

                                                                                Light

                                                                                Qual

                                                                                SellL

                                                                                ease

                                                                                Devp Ownr AampE

                                                                                Note AgtE (~4 vs ~2) for these NEBs

                                                                                Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                Source Skumatz Economic Research Associates Inc 2002

                                                                                Owners more positive ndashUnderinvestment train

                                                                                • Soup to nuts on non-energy benefits NEIs
                                                                                • TOPICS
                                                                                • BACKGROUND NEBS
                                                                                • 20+ YEARS OF NEBS PROGRESShellip
                                                                                • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                • KEY APPLICATIONS OF NEBS
                                                                                • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                • Neb measurement
                                                                                • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                • MEASURING PARTICIPANTNEBs
                                                                                • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                • PROGRESS TRANSFERABILITY OF NEBS
                                                                                • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                • PROGRESS AND GAPS
                                                                                • PROGRESS AND GAPS
                                                                                • USING NEB NEI VALUES
                                                                                • Slide Number 19
                                                                                • ARE NEBS HIGH VALUE
                                                                                • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                • AUDIT FINANCIAL PROGRAM
                                                                                • CampI NEW CONSTRUCTION
                                                                                • CampI TECHNICAL ASSISTANCE
                                                                                • EXPRESSING NEBS VALUEndashCxe
                                                                                • Slide Number 31
                                                                                • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                • EMISSIONS (SOCIETAL NEBS)
                                                                                • UTILITY NEBS
                                                                                • ADJUSTED PAYBACKS
                                                                                • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                • NEBS IN BC
                                                                                • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                • APPLICATION TO BC
                                                                                • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                • STATE TREATMENT OF NEBS NEBs
                                                                                • Slide Number 42
                                                                                • SELECTED STATE PROGRESS
                                                                                • SELECTED STATE PROGRESS
                                                                                • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                • DISCOUNT RATE MATTERS
                                                                                • EUL VARIATIONS- ADOPTED VALUES
                                                                                • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                • TAKEAWAYSON NEBSNEIS
                                                                                • THANK YOUQuestions
                                                                                • Slide Number 53
                                                                                • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                                  Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                                  m Product losses20
                                                                                  l Ease of selling leasing building33
                                                                                  n Labor requirements45
                                                                                  h Aesthetics appearance59
                                                                                  j Noise50
                                                                                  b Equipment maintenance660
                                                                                  f Tenant satisfaction60
                                                                                  d Equipment lifetimes718
                                                                                  e Productivity724
                                                                                  k Building safety[2]73
                                                                                  g Comfort933
                                                                                  i Lighting Quality of Light916
                                                                                  c Equipment performance[1]1019
                                                                                  a Operating costs1165
                                                                                  o Doing good for the Environment[3]1132
                                                                                  Other03
                                                                                  [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                                  [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                                  [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                                  m Product losses
                                                                                  l Ease of selling leasing building
                                                                                  n Labor requirements
                                                                                  h Aesthetics appearance
                                                                                  j Noise
                                                                                  b Equipment maintenance
                                                                                  f Tenant satisfaction
                                                                                  d Equipment lifetimes
                                                                                  e Productivity
                                                                                  k Building safety[2]
                                                                                  g Comfort
                                                                                  i Lighting Quality of Light
                                                                                  c Equipment performance[1]
                                                                                  a Operating costs
                                                                                  o Doing good for the Environment[3]
                                                                                  Share of ZALEH NEBS from various measures
                                                                                  Each presented as multiple of the (overall) energy savings
                                                                                  Super Insulation037
                                                                                  Double Glazing026
                                                                                  Solar Design025
                                                                                  Solar Water Heating005
                                                                                  EnergySmart (insul Retrofit)Dunedin (insul)
                                                                                  Other124
                                                                                  Noise24
                                                                                  Budgetfinances25
                                                                                  Environmental65
                                                                                  Health78
                                                                                  Reduced moves86
                                                                                  Ease of sellingrenting130
                                                                                  DampnessMold1316
                                                                                  Comfort-related4933
                                                                                  101101

                                                                                  Sheet3

                                                                                  Sheet3

                                                                                  Super Insulation
                                                                                  Double Glazing
                                                                                  Solar Design
                                                                                  Solar Water Heating
                                                                                  Super Insulation
                                                                                  Double Glazing
                                                                                  Solar Design
                                                                                  Solar Water Heating
                                                                                  EnergySmart (insul Retrofit)
                                                                                  Dunedin (insul)

                                                                                  SERA

                                                                                  ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                                                  0

                                                                                  002

                                                                                  004

                                                                                  006

                                                                                  008

                                                                                  01

                                                                                  012

                                                                                  Maint

                                                                                  Perf LifeSatis

                                                                                  f

                                                                                  ComfortAesth

                                                                                  NoiseSafe

                                                                                  ty

                                                                                  LightingSelli

                                                                                  ng

                                                                                  Stay in home

                                                                                  Enviro

                                                                                  Sick Days

                                                                                  Calls

                                                                                  ES Homes HP wES Energy savings multiplier=1

                                                                                  Source SERA Research

                                                                                  SERA

                                                                                  CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                                  DG CHP HVAC Equipment rebate Other

                                                                                  Building codes incentives by cities

                                                                                  Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                                  Variety of end uses

                                                                                  SERA

                                                                                  AUDIT FINANCIAL PROGRAM

                                                                                  0

                                                                                  01

                                                                                  02

                                                                                  03

                                                                                  04

                                                                                  05

                                                                                  06

                                                                                  07

                                                                                  Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                                  Source SERA Research

                                                                                  SERA

                                                                                  CampI NEW CONSTRUCTION

                                                                                  -005

                                                                                  0

                                                                                  005

                                                                                  01

                                                                                  015

                                                                                  02

                                                                                  AampE Owner(Pct of Total Partic NEBs)

                                                                                  Source SERA Research

                                                                                  SERA

                                                                                  CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                                  0 2 4 6 8 10 12

                                                                                  m Product losses

                                                                                  l Ease of selling leasing building

                                                                                  n Labor requirements

                                                                                  h Aesthetics appearance

                                                                                  j Noise

                                                                                  b Equipment maintenance

                                                                                  f Tenant satisfaction

                                                                                  d Equipment lifetimes

                                                                                  e Productivity

                                                                                  k Building safety[2]

                                                                                  g Comfort

                                                                                  i Lighting Quality of Light

                                                                                  c Equipment performance[1]

                                                                                  a Operating costs

                                                                                  o Doing good for the Environment[3]

                                                                                  Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                                  vcv

                                                                                  Source SERA Research

                                                                                  Chart1

                                                                                  CampI Technical Assistance NEBs
                                                                                  002
                                                                                  003
                                                                                  004
                                                                                  005
                                                                                  005
                                                                                  006
                                                                                  006
                                                                                  007
                                                                                  007
                                                                                  007
                                                                                  009
                                                                                  009
                                                                                  01
                                                                                  011
                                                                                  011

                                                                                  Sheet1

                                                                                  Sheet1

                                                                                  CampI Technical Assistance Program NEBs (Participant)

                                                                                  Sheet2

                                                                                  Sheet3

                                                                                  SERA

                                                                                  EXPRESSING NEBS VALUEndashCxe

                                                                                  Strong value from RetroCxSource SERA Research

                                                                                  SERA

                                                                                  SOCIETAL NEB EXAMPLES

                                                                                  31

                                                                                  SERA

                                                                                  JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                  32

                                                                                  0

                                                                                  02

                                                                                  04

                                                                                  06

                                                                                  08

                                                                                  1

                                                                                  12

                                                                                  HPWxRetrof Appliance

                                                                                  CA

                                                                                  WI

                                                                                  Natl

                                                                                  (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                  Jobs Economic

                                                                                  SERA

                                                                                  EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                  33

                                                                                  SERA

                                                                                  UTILITY NEBSUtility NEBs for Template Program

                                                                                  Debt WriteOff (util)13

                                                                                  Rate Subsidy(util)61

                                                                                  HealthSafety(util)0

                                                                                  Colln Costs (util)0

                                                                                  Gas Calls (util)0

                                                                                  Calls to CSRs(util)2

                                                                                  TampD (util)16

                                                                                  Arrears (util)0

                                                                                  Reconnects (util)0

                                                                                  Notices (util)7

                                                                                  Shutoffs (util)1

                                                                                  Rate subsidy TampD

                                                                                  Payment-related

                                                                                  Source Skumatz Economic Research Associates research

                                                                                  Example Low Income Weatherization

                                                                                  SERA

                                                                                  ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                  be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                  Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                  Value of 100 HALVES Payback doubles ROI

                                                                                  Source Skumatz Economics (SERA)

                                                                                  35

                                                                                  ( for NEBs Multiplier of 125)

                                                                                  SERA

                                                                                  NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                  36

                                                                                  hellip2001 ON

                                                                                  SERA

                                                                                  37

                                                                                  NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                  For true representation of B amp C elements of NEBs address missing factor bias

                                                                                  Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                  BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                  SERA

                                                                                  CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                  38

                                                                                  Test Benefits Costs States Using Traditionally

                                                                                  Improved treat-ment with NEBs

                                                                                  Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                  bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                  bull Avoided gas and water supply costs

                                                                                  bull Program administration

                                                                                  bull Participant incentives

                                                                                  bull Increased supply cost

                                                                                  CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                  Use cost only paid by the utility

                                                                                  Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                  Same as above plus bull increased revenue

                                                                                  Same as above plus bull Decreased revenue

                                                                                  AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                  Participant cost bull Utility bill reductions bull Participant incentives

                                                                                  bull Participant direct costs

                                                                                  AR CA FL HI IA IN MI MN NY VA

                                                                                  Participant NEBs

                                                                                  Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                  Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                  SERA

                                                                                  APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                  Expansion Concern with risk calculation accuracy

                                                                                  39

                                                                                  Source Skumatz SERA research

                                                                                  SERA

                                                                                  40

                                                                                  METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                  Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                  Minimize Regulatory amp Implementer Risk

                                                                                  Minimize Evaluation Cost

                                                                                  Adder

                                                                                  Readily MeasurableHybrid

                                                                                  All NEBsSource SERA Research

                                                                                  SERA

                                                                                  41

                                                                                  STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                  Screening Application Utilities regions

                                                                                  Program Marketing Fairly widespread use in utilities states across the country

                                                                                  MO

                                                                                  RE AGGRESSIVE ==gt

                                                                                  Test Pgm Screen - adder

                                                                                  IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                  Test Pgm Screen - readily measured

                                                                                  MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                  Test Hybrid (potential adder amp measured)

                                                                                  CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                  Test Pgm screen - Broad

                                                                                  With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                  Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                  SERA

                                                                                  42

                                                                                  Key

                                                                                  States with NEBs in CE tests for At least one type of program

                                                                                  copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                  STATES WITH NEBS IN CE TESTS

                                                                                  NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                  groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                  SERA

                                                                                  SELECTED STATE PROGRESS CA

                                                                                  2001 ndash LIPPT model NY

                                                                                  Mid-2000s measurement amp scenarios not included CO

                                                                                  Adders (10 electric 25 LI 5 gas) VT

                                                                                  15 adder helped by previous research DC

                                                                                  10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                  43

                                                                                  Source Skumatz SERA research

                                                                                  SERA

                                                                                  SELECTED STATE PROGRESS Dominos ongoing

                                                                                  Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                  Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                  Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                  Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                  44

                                                                                  Source Skumatz SERA research

                                                                                  SERA

                                                                                  APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                  Expansion Concern with risk calculation accuracy

                                                                                  45

                                                                                  Source Skumatz SERA research

                                                                                  SERA

                                                                                  46

                                                                                  NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                  [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                  SourceSkumatz SERA Independent research

                                                                                  NTG ndash accuracy measurement incomplete

                                                                                  Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                  NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                  EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                  Cost Complicated expensive local changes

                                                                                  RELATIVE Risk from NEBs

                                                                                  Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                  Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                  NEBs as accurate as many inputs (all could improve)

                                                                                  SERA

                                                                                  DISCOUNT RATE MATTERS

                                                                                  47

                                                                                  Source Skumatz SERA Research

                                                                                  0

                                                                                  20

                                                                                  40

                                                                                  60

                                                                                  80

                                                                                  100

                                                                                  120

                                                                                  1 2 3 4 5 6 7 8 9 10

                                                                                  PV Ratios vs 1 Discount Rate

                                                                                  10 yr PV 20 yr PV

                                                                                  Discount rate - percent

                                                                                  WACCSocialLTT

                                                                                  Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                  SERA

                                                                                  EUL VARIATIONS-ADOPTED VALUES

                                                                                  Source Skumatz Economic Research Associates SERA independent research

                                                                                  0

                                                                                  50

                                                                                  100

                                                                                  150

                                                                                  200

                                                                                  250

                                                                                  300

                                                                                  350

                                                                                  400

                                                                                  450

                                                                                  0 10 20 30 40 50 60 70

                                                                                  EUL Variation by Measure

                                                                                  67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                  Poorly supportedOldTechnology change

                                                                                  SERA

                                                                                  KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                  Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                  measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                  change the BC conclusion

                                                                                  On the cusphellip

                                                                                  If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                  IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                  Source SERA all rights reserved

                                                                                  SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                  SERA

                                                                                  IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                  Utility Soc Part Conserv Recrsquom

                                                                                  Rationale

                                                                                  Base Percent X X X Program-invariant

                                                                                  Low Income X X X X Multiple sources

                                                                                  Weatherization X X X Substantial Participantimpacts

                                                                                  Measure Program-specific

                                                                                  X Varies by measure sector

                                                                                  Other Recomrsquos Local Research

                                                                                  50

                                                                                  Source Skumatz SERA

                                                                                  Developing values for multiple states amp utilities

                                                                                  SERA

                                                                                  TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                  Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                  Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                  5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                  6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                  THANK YOU

                                                                                  Questions

                                                                                  Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                  SERA

                                                                                  POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                  53

                                                                                  SERA

                                                                                  NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                  Negative NEB values cost of barrier

                                                                                  Solar Water Heat NZ$ Euros

                                                                                  Solar Design NZ$ Euros

                                                                                  Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                  -23 -12 (079)

                                                                                  -11 -6 (06)

                                                                                  Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                  Source Skumatz Economic Research Associates research

                                                                                  SERA

                                                                                  CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                  NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                  -5-4-3-2-1012345

                                                                                  Op Co

                                                                                  st

                                                                                  Init C

                                                                                  ost

                                                                                  Maint

                                                                                  Perfo

                                                                                  rm

                                                                                  Prod

                                                                                  uctiv

                                                                                  Tena

                                                                                  nt Sa

                                                                                  t

                                                                                  Comf

                                                                                  ort

                                                                                  Appe

                                                                                  ar

                                                                                  Light

                                                                                  Qual

                                                                                  SellL

                                                                                  ease

                                                                                  Devp Ownr AampE

                                                                                  Note AgtE (~4 vs ~2) for these NEBs

                                                                                  Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                  Source Skumatz Economic Research Associates Inc 2002

                                                                                  Owners more positive ndashUnderinvestment train

                                                                                  • Soup to nuts on non-energy benefits NEIs
                                                                                  • TOPICS
                                                                                  • BACKGROUND NEBS
                                                                                  • 20+ YEARS OF NEBS PROGRESShellip
                                                                                  • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                  • KEY APPLICATIONS OF NEBS
                                                                                  • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                  • Neb measurement
                                                                                  • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                  • MEASURING PARTICIPANTNEBs
                                                                                  • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                  • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                  • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                  • PROGRESS TRANSFERABILITY OF NEBS
                                                                                  • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                  • PROGRESS AND GAPS
                                                                                  • PROGRESS AND GAPS
                                                                                  • USING NEB NEI VALUES
                                                                                  • Slide Number 19
                                                                                  • ARE NEBS HIGH VALUE
                                                                                  • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                  • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                  • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                  • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                  • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                  • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                  • AUDIT FINANCIAL PROGRAM
                                                                                  • CampI NEW CONSTRUCTION
                                                                                  • CampI TECHNICAL ASSISTANCE
                                                                                  • EXPRESSING NEBS VALUEndashCxe
                                                                                  • Slide Number 31
                                                                                  • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                  • EMISSIONS (SOCIETAL NEBS)
                                                                                  • UTILITY NEBS
                                                                                  • ADJUSTED PAYBACKS
                                                                                  • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                  • NEBS IN BC
                                                                                  • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                  • APPLICATION TO BC
                                                                                  • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                  • STATE TREATMENT OF NEBS NEBs
                                                                                  • Slide Number 42
                                                                                  • SELECTED STATE PROGRESS
                                                                                  • SELECTED STATE PROGRESS
                                                                                  • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                  • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                  • DISCOUNT RATE MATTERS
                                                                                  • EUL VARIATIONS- ADOPTED VALUES
                                                                                  • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                  • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                  • TAKEAWAYSON NEBSNEIS
                                                                                  • THANK YOUQuestions
                                                                                  • Slide Number 53
                                                                                  • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                  • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                                    Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                                    m Product losses20
                                                                                    l Ease of selling leasing building33
                                                                                    n Labor requirements45
                                                                                    h Aesthetics appearance59
                                                                                    j Noise50
                                                                                    b Equipment maintenance660
                                                                                    f Tenant satisfaction60
                                                                                    d Equipment lifetimes718
                                                                                    e Productivity724
                                                                                    k Building safety[2]73
                                                                                    g Comfort933
                                                                                    i Lighting Quality of Light916
                                                                                    c Equipment performance[1]1019
                                                                                    a Operating costs1165
                                                                                    o Doing good for the Environment[3]1132
                                                                                    Other03
                                                                                    [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                                    [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                                    [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                                    m Product losses
                                                                                    l Ease of selling leasing building
                                                                                    n Labor requirements
                                                                                    h Aesthetics appearance
                                                                                    j Noise
                                                                                    b Equipment maintenance
                                                                                    f Tenant satisfaction
                                                                                    d Equipment lifetimes
                                                                                    e Productivity
                                                                                    k Building safety[2]
                                                                                    g Comfort
                                                                                    i Lighting Quality of Light
                                                                                    c Equipment performance[1]
                                                                                    a Operating costs
                                                                                    o Doing good for the Environment[3]
                                                                                    Share of ZALEH NEBS from various measures
                                                                                    Each presented as multiple of the (overall) energy savings
                                                                                    Super Insulation037
                                                                                    Double Glazing026
                                                                                    Solar Design025
                                                                                    Solar Water Heating005
                                                                                    EnergySmart (insul Retrofit)Dunedin (insul)
                                                                                    Other124
                                                                                    Noise24
                                                                                    Budgetfinances25
                                                                                    Environmental65
                                                                                    Health78
                                                                                    Reduced moves86
                                                                                    Ease of sellingrenting130
                                                                                    DampnessMold1316
                                                                                    Comfort-related4933
                                                                                    101101

                                                                                    Sheet3

                                                                                    Super Insulation
                                                                                    Double Glazing
                                                                                    Solar Design
                                                                                    Solar Water Heating
                                                                                    Super Insulation
                                                                                    Double Glazing
                                                                                    Solar Design
                                                                                    Solar Water Heating
                                                                                    EnergySmart (insul Retrofit)
                                                                                    Dunedin (insul)

                                                                                    SERA

                                                                                    ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                                                    0

                                                                                    002

                                                                                    004

                                                                                    006

                                                                                    008

                                                                                    01

                                                                                    012

                                                                                    Maint

                                                                                    Perf LifeSatis

                                                                                    f

                                                                                    ComfortAesth

                                                                                    NoiseSafe

                                                                                    ty

                                                                                    LightingSelli

                                                                                    ng

                                                                                    Stay in home

                                                                                    Enviro

                                                                                    Sick Days

                                                                                    Calls

                                                                                    ES Homes HP wES Energy savings multiplier=1

                                                                                    Source SERA Research

                                                                                    SERA

                                                                                    CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                                    DG CHP HVAC Equipment rebate Other

                                                                                    Building codes incentives by cities

                                                                                    Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                                    Variety of end uses

                                                                                    SERA

                                                                                    AUDIT FINANCIAL PROGRAM

                                                                                    0

                                                                                    01

                                                                                    02

                                                                                    03

                                                                                    04

                                                                                    05

                                                                                    06

                                                                                    07

                                                                                    Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                                    Source SERA Research

                                                                                    SERA

                                                                                    CampI NEW CONSTRUCTION

                                                                                    -005

                                                                                    0

                                                                                    005

                                                                                    01

                                                                                    015

                                                                                    02

                                                                                    AampE Owner(Pct of Total Partic NEBs)

                                                                                    Source SERA Research

                                                                                    SERA

                                                                                    CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                                    0 2 4 6 8 10 12

                                                                                    m Product losses

                                                                                    l Ease of selling leasing building

                                                                                    n Labor requirements

                                                                                    h Aesthetics appearance

                                                                                    j Noise

                                                                                    b Equipment maintenance

                                                                                    f Tenant satisfaction

                                                                                    d Equipment lifetimes

                                                                                    e Productivity

                                                                                    k Building safety[2]

                                                                                    g Comfort

                                                                                    i Lighting Quality of Light

                                                                                    c Equipment performance[1]

                                                                                    a Operating costs

                                                                                    o Doing good for the Environment[3]

                                                                                    Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                                    vcv

                                                                                    Source SERA Research

                                                                                    Chart1

                                                                                    CampI Technical Assistance NEBs
                                                                                    002
                                                                                    003
                                                                                    004
                                                                                    005
                                                                                    005
                                                                                    006
                                                                                    006
                                                                                    007
                                                                                    007
                                                                                    007
                                                                                    009
                                                                                    009
                                                                                    01
                                                                                    011
                                                                                    011

                                                                                    Sheet1

                                                                                    Sheet1

                                                                                    CampI Technical Assistance Program NEBs (Participant)

                                                                                    Sheet2

                                                                                    Sheet3

                                                                                    SERA

                                                                                    EXPRESSING NEBS VALUEndashCxe

                                                                                    Strong value from RetroCxSource SERA Research

                                                                                    SERA

                                                                                    SOCIETAL NEB EXAMPLES

                                                                                    31

                                                                                    SERA

                                                                                    JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                    32

                                                                                    0

                                                                                    02

                                                                                    04

                                                                                    06

                                                                                    08

                                                                                    1

                                                                                    12

                                                                                    HPWxRetrof Appliance

                                                                                    CA

                                                                                    WI

                                                                                    Natl

                                                                                    (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                    Jobs Economic

                                                                                    SERA

                                                                                    EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                    33

                                                                                    SERA

                                                                                    UTILITY NEBSUtility NEBs for Template Program

                                                                                    Debt WriteOff (util)13

                                                                                    Rate Subsidy(util)61

                                                                                    HealthSafety(util)0

                                                                                    Colln Costs (util)0

                                                                                    Gas Calls (util)0

                                                                                    Calls to CSRs(util)2

                                                                                    TampD (util)16

                                                                                    Arrears (util)0

                                                                                    Reconnects (util)0

                                                                                    Notices (util)7

                                                                                    Shutoffs (util)1

                                                                                    Rate subsidy TampD

                                                                                    Payment-related

                                                                                    Source Skumatz Economic Research Associates research

                                                                                    Example Low Income Weatherization

                                                                                    SERA

                                                                                    ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                    be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                    Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                    Value of 100 HALVES Payback doubles ROI

                                                                                    Source Skumatz Economics (SERA)

                                                                                    35

                                                                                    ( for NEBs Multiplier of 125)

                                                                                    SERA

                                                                                    NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                    36

                                                                                    hellip2001 ON

                                                                                    SERA

                                                                                    37

                                                                                    NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                    For true representation of B amp C elements of NEBs address missing factor bias

                                                                                    Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                    BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                    SERA

                                                                                    CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                    38

                                                                                    Test Benefits Costs States Using Traditionally

                                                                                    Improved treat-ment with NEBs

                                                                                    Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                    bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                    bull Avoided gas and water supply costs

                                                                                    bull Program administration

                                                                                    bull Participant incentives

                                                                                    bull Increased supply cost

                                                                                    CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                    Use cost only paid by the utility

                                                                                    Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                    Same as above plus bull increased revenue

                                                                                    Same as above plus bull Decreased revenue

                                                                                    AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                    Participant cost bull Utility bill reductions bull Participant incentives

                                                                                    bull Participant direct costs

                                                                                    AR CA FL HI IA IN MI MN NY VA

                                                                                    Participant NEBs

                                                                                    Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                    Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                    SERA

                                                                                    APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                    Expansion Concern with risk calculation accuracy

                                                                                    39

                                                                                    Source Skumatz SERA research

                                                                                    SERA

                                                                                    40

                                                                                    METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                    Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                    Minimize Regulatory amp Implementer Risk

                                                                                    Minimize Evaluation Cost

                                                                                    Adder

                                                                                    Readily MeasurableHybrid

                                                                                    All NEBsSource SERA Research

                                                                                    SERA

                                                                                    41

                                                                                    STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                    Screening Application Utilities regions

                                                                                    Program Marketing Fairly widespread use in utilities states across the country

                                                                                    MO

                                                                                    RE AGGRESSIVE ==gt

                                                                                    Test Pgm Screen - adder

                                                                                    IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                    Test Pgm Screen - readily measured

                                                                                    MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                    Test Hybrid (potential adder amp measured)

                                                                                    CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                    Test Pgm screen - Broad

                                                                                    With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                    Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                    SERA

                                                                                    42

                                                                                    Key

                                                                                    States with NEBs in CE tests for At least one type of program

                                                                                    copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                    STATES WITH NEBS IN CE TESTS

                                                                                    NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                    groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                    SERA

                                                                                    SELECTED STATE PROGRESS CA

                                                                                    2001 ndash LIPPT model NY

                                                                                    Mid-2000s measurement amp scenarios not included CO

                                                                                    Adders (10 electric 25 LI 5 gas) VT

                                                                                    15 adder helped by previous research DC

                                                                                    10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                    43

                                                                                    Source Skumatz SERA research

                                                                                    SERA

                                                                                    SELECTED STATE PROGRESS Dominos ongoing

                                                                                    Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                    Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                    Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                    Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                    44

                                                                                    Source Skumatz SERA research

                                                                                    SERA

                                                                                    APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                    Expansion Concern with risk calculation accuracy

                                                                                    45

                                                                                    Source Skumatz SERA research

                                                                                    SERA

                                                                                    46

                                                                                    NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                    [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                    SourceSkumatz SERA Independent research

                                                                                    NTG ndash accuracy measurement incomplete

                                                                                    Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                    NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                    EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                    Cost Complicated expensive local changes

                                                                                    RELATIVE Risk from NEBs

                                                                                    Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                    Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                    NEBs as accurate as many inputs (all could improve)

                                                                                    SERA

                                                                                    DISCOUNT RATE MATTERS

                                                                                    47

                                                                                    Source Skumatz SERA Research

                                                                                    0

                                                                                    20

                                                                                    40

                                                                                    60

                                                                                    80

                                                                                    100

                                                                                    120

                                                                                    1 2 3 4 5 6 7 8 9 10

                                                                                    PV Ratios vs 1 Discount Rate

                                                                                    10 yr PV 20 yr PV

                                                                                    Discount rate - percent

                                                                                    WACCSocialLTT

                                                                                    Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                    SERA

                                                                                    EUL VARIATIONS-ADOPTED VALUES

                                                                                    Source Skumatz Economic Research Associates SERA independent research

                                                                                    0

                                                                                    50

                                                                                    100

                                                                                    150

                                                                                    200

                                                                                    250

                                                                                    300

                                                                                    350

                                                                                    400

                                                                                    450

                                                                                    0 10 20 30 40 50 60 70

                                                                                    EUL Variation by Measure

                                                                                    67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                    Poorly supportedOldTechnology change

                                                                                    SERA

                                                                                    KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                    Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                    measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                    change the BC conclusion

                                                                                    On the cusphellip

                                                                                    If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                    IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                    Source SERA all rights reserved

                                                                                    SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                    SERA

                                                                                    IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                    Utility Soc Part Conserv Recrsquom

                                                                                    Rationale

                                                                                    Base Percent X X X Program-invariant

                                                                                    Low Income X X X X Multiple sources

                                                                                    Weatherization X X X Substantial Participantimpacts

                                                                                    Measure Program-specific

                                                                                    X Varies by measure sector

                                                                                    Other Recomrsquos Local Research

                                                                                    50

                                                                                    Source Skumatz SERA

                                                                                    Developing values for multiple states amp utilities

                                                                                    SERA

                                                                                    TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                    Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                    Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                    5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                    6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                    THANK YOU

                                                                                    Questions

                                                                                    Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                    SERA

                                                                                    POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                    53

                                                                                    SERA

                                                                                    NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                    Negative NEB values cost of barrier

                                                                                    Solar Water Heat NZ$ Euros

                                                                                    Solar Design NZ$ Euros

                                                                                    Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                    -23 -12 (079)

                                                                                    -11 -6 (06)

                                                                                    Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                    Source Skumatz Economic Research Associates research

                                                                                    SERA

                                                                                    CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                    NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                    -5-4-3-2-1012345

                                                                                    Op Co

                                                                                    st

                                                                                    Init C

                                                                                    ost

                                                                                    Maint

                                                                                    Perfo

                                                                                    rm

                                                                                    Prod

                                                                                    uctiv

                                                                                    Tena

                                                                                    nt Sa

                                                                                    t

                                                                                    Comf

                                                                                    ort

                                                                                    Appe

                                                                                    ar

                                                                                    Light

                                                                                    Qual

                                                                                    SellL

                                                                                    ease

                                                                                    Devp Ownr AampE

                                                                                    Note AgtE (~4 vs ~2) for these NEBs

                                                                                    Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                    Source Skumatz Economic Research Associates Inc 2002

                                                                                    Owners more positive ndashUnderinvestment train

                                                                                    • Soup to nuts on non-energy benefits NEIs
                                                                                    • TOPICS
                                                                                    • BACKGROUND NEBS
                                                                                    • 20+ YEARS OF NEBS PROGRESShellip
                                                                                    • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                    • KEY APPLICATIONS OF NEBS
                                                                                    • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                    • Neb measurement
                                                                                    • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                    • MEASURING PARTICIPANTNEBs
                                                                                    • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                    • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                    • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                    • PROGRESS TRANSFERABILITY OF NEBS
                                                                                    • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                    • PROGRESS AND GAPS
                                                                                    • PROGRESS AND GAPS
                                                                                    • USING NEB NEI VALUES
                                                                                    • Slide Number 19
                                                                                    • ARE NEBS HIGH VALUE
                                                                                    • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                    • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                    • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                    • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                    • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                    • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                    • AUDIT FINANCIAL PROGRAM
                                                                                    • CampI NEW CONSTRUCTION
                                                                                    • CampI TECHNICAL ASSISTANCE
                                                                                    • EXPRESSING NEBS VALUEndashCxe
                                                                                    • Slide Number 31
                                                                                    • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                    • EMISSIONS (SOCIETAL NEBS)
                                                                                    • UTILITY NEBS
                                                                                    • ADJUSTED PAYBACKS
                                                                                    • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                    • NEBS IN BC
                                                                                    • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                    • APPLICATION TO BC
                                                                                    • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                    • STATE TREATMENT OF NEBS NEBs
                                                                                    • Slide Number 42
                                                                                    • SELECTED STATE PROGRESS
                                                                                    • SELECTED STATE PROGRESS
                                                                                    • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                    • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                    • DISCOUNT RATE MATTERS
                                                                                    • EUL VARIATIONS- ADOPTED VALUES
                                                                                    • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                    • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                    • TAKEAWAYSON NEBSNEIS
                                                                                    • THANK YOUQuestions
                                                                                    • Slide Number 53
                                                                                    • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                    • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                                      Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                                      m Product losses20
                                                                                      l Ease of selling leasing building33
                                                                                      n Labor requirements45
                                                                                      h Aesthetics appearance59
                                                                                      j Noise50
                                                                                      b Equipment maintenance660
                                                                                      f Tenant satisfaction60
                                                                                      d Equipment lifetimes718
                                                                                      e Productivity724
                                                                                      k Building safety[2]73
                                                                                      g Comfort933
                                                                                      i Lighting Quality of Light916
                                                                                      c Equipment performance[1]1019
                                                                                      a Operating costs1165
                                                                                      o Doing good for the Environment[3]1132
                                                                                      Other03
                                                                                      [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                                      [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                                      [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                                      m Product losses
                                                                                      l Ease of selling leasing building
                                                                                      n Labor requirements
                                                                                      h Aesthetics appearance
                                                                                      j Noise
                                                                                      b Equipment maintenance
                                                                                      f Tenant satisfaction
                                                                                      d Equipment lifetimes
                                                                                      e Productivity
                                                                                      k Building safety[2]
                                                                                      g Comfort
                                                                                      i Lighting Quality of Light
                                                                                      c Equipment performance[1]
                                                                                      a Operating costs
                                                                                      o Doing good for the Environment[3]
                                                                                      Super Insulation
                                                                                      Double Glazing
                                                                                      Solar Design
                                                                                      Solar Water Heating
                                                                                      EnergySmart (insul Retrofit)
                                                                                      Dunedin (insul)

                                                                                      SERA

                                                                                      ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                                                      0

                                                                                      002

                                                                                      004

                                                                                      006

                                                                                      008

                                                                                      01

                                                                                      012

                                                                                      Maint

                                                                                      Perf LifeSatis

                                                                                      f

                                                                                      ComfortAesth

                                                                                      NoiseSafe

                                                                                      ty

                                                                                      LightingSelli

                                                                                      ng

                                                                                      Stay in home

                                                                                      Enviro

                                                                                      Sick Days

                                                                                      Calls

                                                                                      ES Homes HP wES Energy savings multiplier=1

                                                                                      Source SERA Research

                                                                                      SERA

                                                                                      CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                                      DG CHP HVAC Equipment rebate Other

                                                                                      Building codes incentives by cities

                                                                                      Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                                      Variety of end uses

                                                                                      SERA

                                                                                      AUDIT FINANCIAL PROGRAM

                                                                                      0

                                                                                      01

                                                                                      02

                                                                                      03

                                                                                      04

                                                                                      05

                                                                                      06

                                                                                      07

                                                                                      Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                                      Source SERA Research

                                                                                      SERA

                                                                                      CampI NEW CONSTRUCTION

                                                                                      -005

                                                                                      0

                                                                                      005

                                                                                      01

                                                                                      015

                                                                                      02

                                                                                      AampE Owner(Pct of Total Partic NEBs)

                                                                                      Source SERA Research

                                                                                      SERA

                                                                                      CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                                      0 2 4 6 8 10 12

                                                                                      m Product losses

                                                                                      l Ease of selling leasing building

                                                                                      n Labor requirements

                                                                                      h Aesthetics appearance

                                                                                      j Noise

                                                                                      b Equipment maintenance

                                                                                      f Tenant satisfaction

                                                                                      d Equipment lifetimes

                                                                                      e Productivity

                                                                                      k Building safety[2]

                                                                                      g Comfort

                                                                                      i Lighting Quality of Light

                                                                                      c Equipment performance[1]

                                                                                      a Operating costs

                                                                                      o Doing good for the Environment[3]

                                                                                      Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                                      vcv

                                                                                      Source SERA Research

                                                                                      Chart1

                                                                                      CampI Technical Assistance NEBs
                                                                                      002
                                                                                      003
                                                                                      004
                                                                                      005
                                                                                      005
                                                                                      006
                                                                                      006
                                                                                      007
                                                                                      007
                                                                                      007
                                                                                      009
                                                                                      009
                                                                                      01
                                                                                      011
                                                                                      011

                                                                                      Sheet1

                                                                                      Sheet1

                                                                                      CampI Technical Assistance Program NEBs (Participant)

                                                                                      Sheet2

                                                                                      Sheet3

                                                                                      SERA

                                                                                      EXPRESSING NEBS VALUEndashCxe

                                                                                      Strong value from RetroCxSource SERA Research

                                                                                      SERA

                                                                                      SOCIETAL NEB EXAMPLES

                                                                                      31

                                                                                      SERA

                                                                                      JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                      32

                                                                                      0

                                                                                      02

                                                                                      04

                                                                                      06

                                                                                      08

                                                                                      1

                                                                                      12

                                                                                      HPWxRetrof Appliance

                                                                                      CA

                                                                                      WI

                                                                                      Natl

                                                                                      (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                      Jobs Economic

                                                                                      SERA

                                                                                      EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                      33

                                                                                      SERA

                                                                                      UTILITY NEBSUtility NEBs for Template Program

                                                                                      Debt WriteOff (util)13

                                                                                      Rate Subsidy(util)61

                                                                                      HealthSafety(util)0

                                                                                      Colln Costs (util)0

                                                                                      Gas Calls (util)0

                                                                                      Calls to CSRs(util)2

                                                                                      TampD (util)16

                                                                                      Arrears (util)0

                                                                                      Reconnects (util)0

                                                                                      Notices (util)7

                                                                                      Shutoffs (util)1

                                                                                      Rate subsidy TampD

                                                                                      Payment-related

                                                                                      Source Skumatz Economic Research Associates research

                                                                                      Example Low Income Weatherization

                                                                                      SERA

                                                                                      ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                      be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                      Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                      Value of 100 HALVES Payback doubles ROI

                                                                                      Source Skumatz Economics (SERA)

                                                                                      35

                                                                                      ( for NEBs Multiplier of 125)

                                                                                      SERA

                                                                                      NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                      36

                                                                                      hellip2001 ON

                                                                                      SERA

                                                                                      37

                                                                                      NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                      For true representation of B amp C elements of NEBs address missing factor bias

                                                                                      Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                      BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                      SERA

                                                                                      CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                      38

                                                                                      Test Benefits Costs States Using Traditionally

                                                                                      Improved treat-ment with NEBs

                                                                                      Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                      bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                      bull Avoided gas and water supply costs

                                                                                      bull Program administration

                                                                                      bull Participant incentives

                                                                                      bull Increased supply cost

                                                                                      CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                      Use cost only paid by the utility

                                                                                      Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                      Same as above plus bull increased revenue

                                                                                      Same as above plus bull Decreased revenue

                                                                                      AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                      Participant cost bull Utility bill reductions bull Participant incentives

                                                                                      bull Participant direct costs

                                                                                      AR CA FL HI IA IN MI MN NY VA

                                                                                      Participant NEBs

                                                                                      Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                      Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                      SERA

                                                                                      APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                      Expansion Concern with risk calculation accuracy

                                                                                      39

                                                                                      Source Skumatz SERA research

                                                                                      SERA

                                                                                      40

                                                                                      METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                      Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                      Minimize Regulatory amp Implementer Risk

                                                                                      Minimize Evaluation Cost

                                                                                      Adder

                                                                                      Readily MeasurableHybrid

                                                                                      All NEBsSource SERA Research

                                                                                      SERA

                                                                                      41

                                                                                      STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                      Screening Application Utilities regions

                                                                                      Program Marketing Fairly widespread use in utilities states across the country

                                                                                      MO

                                                                                      RE AGGRESSIVE ==gt

                                                                                      Test Pgm Screen - adder

                                                                                      IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                      Test Pgm Screen - readily measured

                                                                                      MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                      Test Hybrid (potential adder amp measured)

                                                                                      CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                      Test Pgm screen - Broad

                                                                                      With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                      Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                      SERA

                                                                                      42

                                                                                      Key

                                                                                      States with NEBs in CE tests for At least one type of program

                                                                                      copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                      STATES WITH NEBS IN CE TESTS

                                                                                      NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                      groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                      SERA

                                                                                      SELECTED STATE PROGRESS CA

                                                                                      2001 ndash LIPPT model NY

                                                                                      Mid-2000s measurement amp scenarios not included CO

                                                                                      Adders (10 electric 25 LI 5 gas) VT

                                                                                      15 adder helped by previous research DC

                                                                                      10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                      43

                                                                                      Source Skumatz SERA research

                                                                                      SERA

                                                                                      SELECTED STATE PROGRESS Dominos ongoing

                                                                                      Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                      Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                      Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                      Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                      44

                                                                                      Source Skumatz SERA research

                                                                                      SERA

                                                                                      APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                      Expansion Concern with risk calculation accuracy

                                                                                      45

                                                                                      Source Skumatz SERA research

                                                                                      SERA

                                                                                      46

                                                                                      NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                      [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                      SourceSkumatz SERA Independent research

                                                                                      NTG ndash accuracy measurement incomplete

                                                                                      Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                      NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                      EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                      Cost Complicated expensive local changes

                                                                                      RELATIVE Risk from NEBs

                                                                                      Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                      Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                      NEBs as accurate as many inputs (all could improve)

                                                                                      SERA

                                                                                      DISCOUNT RATE MATTERS

                                                                                      47

                                                                                      Source Skumatz SERA Research

                                                                                      0

                                                                                      20

                                                                                      40

                                                                                      60

                                                                                      80

                                                                                      100

                                                                                      120

                                                                                      1 2 3 4 5 6 7 8 9 10

                                                                                      PV Ratios vs 1 Discount Rate

                                                                                      10 yr PV 20 yr PV

                                                                                      Discount rate - percent

                                                                                      WACCSocialLTT

                                                                                      Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                      SERA

                                                                                      EUL VARIATIONS-ADOPTED VALUES

                                                                                      Source Skumatz Economic Research Associates SERA independent research

                                                                                      0

                                                                                      50

                                                                                      100

                                                                                      150

                                                                                      200

                                                                                      250

                                                                                      300

                                                                                      350

                                                                                      400

                                                                                      450

                                                                                      0 10 20 30 40 50 60 70

                                                                                      EUL Variation by Measure

                                                                                      67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                      Poorly supportedOldTechnology change

                                                                                      SERA

                                                                                      KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                      Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                      measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                      change the BC conclusion

                                                                                      On the cusphellip

                                                                                      If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                      IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                      Source SERA all rights reserved

                                                                                      SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                      SERA

                                                                                      IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                      Utility Soc Part Conserv Recrsquom

                                                                                      Rationale

                                                                                      Base Percent X X X Program-invariant

                                                                                      Low Income X X X X Multiple sources

                                                                                      Weatherization X X X Substantial Participantimpacts

                                                                                      Measure Program-specific

                                                                                      X Varies by measure sector

                                                                                      Other Recomrsquos Local Research

                                                                                      50

                                                                                      Source Skumatz SERA

                                                                                      Developing values for multiple states amp utilities

                                                                                      SERA

                                                                                      TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                      Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                      Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                      5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                      6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                      THANK YOU

                                                                                      Questions

                                                                                      Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                      SERA

                                                                                      POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                      53

                                                                                      SERA

                                                                                      NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                      Negative NEB values cost of barrier

                                                                                      Solar Water Heat NZ$ Euros

                                                                                      Solar Design NZ$ Euros

                                                                                      Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                      -23 -12 (079)

                                                                                      -11 -6 (06)

                                                                                      Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                      Source Skumatz Economic Research Associates research

                                                                                      SERA

                                                                                      CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                      NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                      -5-4-3-2-1012345

                                                                                      Op Co

                                                                                      st

                                                                                      Init C

                                                                                      ost

                                                                                      Maint

                                                                                      Perfo

                                                                                      rm

                                                                                      Prod

                                                                                      uctiv

                                                                                      Tena

                                                                                      nt Sa

                                                                                      t

                                                                                      Comf

                                                                                      ort

                                                                                      Appe

                                                                                      ar

                                                                                      Light

                                                                                      Qual

                                                                                      SellL

                                                                                      ease

                                                                                      Devp Ownr AampE

                                                                                      Note AgtE (~4 vs ~2) for these NEBs

                                                                                      Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                      Source Skumatz Economic Research Associates Inc 2002

                                                                                      Owners more positive ndashUnderinvestment train

                                                                                      • Soup to nuts on non-energy benefits NEIs
                                                                                      • TOPICS
                                                                                      • BACKGROUND NEBS
                                                                                      • 20+ YEARS OF NEBS PROGRESShellip
                                                                                      • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                      • KEY APPLICATIONS OF NEBS
                                                                                      • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                      • Neb measurement
                                                                                      • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                      • MEASURING PARTICIPANTNEBs
                                                                                      • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                      • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                      • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                      • PROGRESS TRANSFERABILITY OF NEBS
                                                                                      • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                      • PROGRESS AND GAPS
                                                                                      • PROGRESS AND GAPS
                                                                                      • USING NEB NEI VALUES
                                                                                      • Slide Number 19
                                                                                      • ARE NEBS HIGH VALUE
                                                                                      • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                      • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                      • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                      • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                      • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                      • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                      • AUDIT FINANCIAL PROGRAM
                                                                                      • CampI NEW CONSTRUCTION
                                                                                      • CampI TECHNICAL ASSISTANCE
                                                                                      • EXPRESSING NEBS VALUEndashCxe
                                                                                      • Slide Number 31
                                                                                      • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                      • EMISSIONS (SOCIETAL NEBS)
                                                                                      • UTILITY NEBS
                                                                                      • ADJUSTED PAYBACKS
                                                                                      • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                      • NEBS IN BC
                                                                                      • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                      • APPLICATION TO BC
                                                                                      • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                      • STATE TREATMENT OF NEBS NEBs
                                                                                      • Slide Number 42
                                                                                      • SELECTED STATE PROGRESS
                                                                                      • SELECTED STATE PROGRESS
                                                                                      • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                      • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                      • DISCOUNT RATE MATTERS
                                                                                      • EUL VARIATIONS- ADOPTED VALUES
                                                                                      • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                      • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                      • TAKEAWAYSON NEBSNEIS
                                                                                      • THANK YOUQuestions
                                                                                      • Slide Number 53
                                                                                      • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                      • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                                        Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                                        m Product losses20
                                                                                        l Ease of selling leasing building33
                                                                                        n Labor requirements45
                                                                                        h Aesthetics appearance59
                                                                                        j Noise50
                                                                                        b Equipment maintenance660
                                                                                        f Tenant satisfaction60
                                                                                        d Equipment lifetimes718
                                                                                        e Productivity724
                                                                                        k Building safety[2]73
                                                                                        g Comfort933
                                                                                        i Lighting Quality of Light916
                                                                                        c Equipment performance[1]1019
                                                                                        a Operating costs1165
                                                                                        o Doing good for the Environment[3]1132
                                                                                        Other03
                                                                                        [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                                        [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                                        [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                                        m Product losses
                                                                                        l Ease of selling leasing building
                                                                                        n Labor requirements
                                                                                        h Aesthetics appearance
                                                                                        j Noise
                                                                                        b Equipment maintenance
                                                                                        f Tenant satisfaction
                                                                                        d Equipment lifetimes
                                                                                        e Productivity
                                                                                        k Building safety[2]
                                                                                        g Comfort
                                                                                        i Lighting Quality of Light
                                                                                        c Equipment performance[1]
                                                                                        a Operating costs
                                                                                        o Doing good for the Environment[3]
                                                                                        EnergySmart (insul Retrofit)
                                                                                        Dunedin (insul)

                                                                                        SERA

                                                                                        ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                                                        0

                                                                                        002

                                                                                        004

                                                                                        006

                                                                                        008

                                                                                        01

                                                                                        012

                                                                                        Maint

                                                                                        Perf LifeSatis

                                                                                        f

                                                                                        ComfortAesth

                                                                                        NoiseSafe

                                                                                        ty

                                                                                        LightingSelli

                                                                                        ng

                                                                                        Stay in home

                                                                                        Enviro

                                                                                        Sick Days

                                                                                        Calls

                                                                                        ES Homes HP wES Energy savings multiplier=1

                                                                                        Source SERA Research

                                                                                        SERA

                                                                                        CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                                        DG CHP HVAC Equipment rebate Other

                                                                                        Building codes incentives by cities

                                                                                        Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                                        Variety of end uses

                                                                                        SERA

                                                                                        AUDIT FINANCIAL PROGRAM

                                                                                        0

                                                                                        01

                                                                                        02

                                                                                        03

                                                                                        04

                                                                                        05

                                                                                        06

                                                                                        07

                                                                                        Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                                        Source SERA Research

                                                                                        SERA

                                                                                        CampI NEW CONSTRUCTION

                                                                                        -005

                                                                                        0

                                                                                        005

                                                                                        01

                                                                                        015

                                                                                        02

                                                                                        AampE Owner(Pct of Total Partic NEBs)

                                                                                        Source SERA Research

                                                                                        SERA

                                                                                        CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                                        0 2 4 6 8 10 12

                                                                                        m Product losses

                                                                                        l Ease of selling leasing building

                                                                                        n Labor requirements

                                                                                        h Aesthetics appearance

                                                                                        j Noise

                                                                                        b Equipment maintenance

                                                                                        f Tenant satisfaction

                                                                                        d Equipment lifetimes

                                                                                        e Productivity

                                                                                        k Building safety[2]

                                                                                        g Comfort

                                                                                        i Lighting Quality of Light

                                                                                        c Equipment performance[1]

                                                                                        a Operating costs

                                                                                        o Doing good for the Environment[3]

                                                                                        Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                                        vcv

                                                                                        Source SERA Research

                                                                                        Chart1

                                                                                        CampI Technical Assistance NEBs
                                                                                        002
                                                                                        003
                                                                                        004
                                                                                        005
                                                                                        005
                                                                                        006
                                                                                        006
                                                                                        007
                                                                                        007
                                                                                        007
                                                                                        009
                                                                                        009
                                                                                        01
                                                                                        011
                                                                                        011

                                                                                        Sheet1

                                                                                        Sheet1

                                                                                        CampI Technical Assistance Program NEBs (Participant)

                                                                                        Sheet2

                                                                                        Sheet3

                                                                                        SERA

                                                                                        EXPRESSING NEBS VALUEndashCxe

                                                                                        Strong value from RetroCxSource SERA Research

                                                                                        SERA

                                                                                        SOCIETAL NEB EXAMPLES

                                                                                        31

                                                                                        SERA

                                                                                        JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                        32

                                                                                        0

                                                                                        02

                                                                                        04

                                                                                        06

                                                                                        08

                                                                                        1

                                                                                        12

                                                                                        HPWxRetrof Appliance

                                                                                        CA

                                                                                        WI

                                                                                        Natl

                                                                                        (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                        Jobs Economic

                                                                                        SERA

                                                                                        EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                        33

                                                                                        SERA

                                                                                        UTILITY NEBSUtility NEBs for Template Program

                                                                                        Debt WriteOff (util)13

                                                                                        Rate Subsidy(util)61

                                                                                        HealthSafety(util)0

                                                                                        Colln Costs (util)0

                                                                                        Gas Calls (util)0

                                                                                        Calls to CSRs(util)2

                                                                                        TampD (util)16

                                                                                        Arrears (util)0

                                                                                        Reconnects (util)0

                                                                                        Notices (util)7

                                                                                        Shutoffs (util)1

                                                                                        Rate subsidy TampD

                                                                                        Payment-related

                                                                                        Source Skumatz Economic Research Associates research

                                                                                        Example Low Income Weatherization

                                                                                        SERA

                                                                                        ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                        be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                        Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                        Value of 100 HALVES Payback doubles ROI

                                                                                        Source Skumatz Economics (SERA)

                                                                                        35

                                                                                        ( for NEBs Multiplier of 125)

                                                                                        SERA

                                                                                        NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                        36

                                                                                        hellip2001 ON

                                                                                        SERA

                                                                                        37

                                                                                        NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                        For true representation of B amp C elements of NEBs address missing factor bias

                                                                                        Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                        BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                        SERA

                                                                                        CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                        38

                                                                                        Test Benefits Costs States Using Traditionally

                                                                                        Improved treat-ment with NEBs

                                                                                        Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                        bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                        bull Avoided gas and water supply costs

                                                                                        bull Program administration

                                                                                        bull Participant incentives

                                                                                        bull Increased supply cost

                                                                                        CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                        Use cost only paid by the utility

                                                                                        Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                        Same as above plus bull increased revenue

                                                                                        Same as above plus bull Decreased revenue

                                                                                        AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                        Participant cost bull Utility bill reductions bull Participant incentives

                                                                                        bull Participant direct costs

                                                                                        AR CA FL HI IA IN MI MN NY VA

                                                                                        Participant NEBs

                                                                                        Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                        Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                        SERA

                                                                                        APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                        Expansion Concern with risk calculation accuracy

                                                                                        39

                                                                                        Source Skumatz SERA research

                                                                                        SERA

                                                                                        40

                                                                                        METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                        Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                        Minimize Regulatory amp Implementer Risk

                                                                                        Minimize Evaluation Cost

                                                                                        Adder

                                                                                        Readily MeasurableHybrid

                                                                                        All NEBsSource SERA Research

                                                                                        SERA

                                                                                        41

                                                                                        STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                        Screening Application Utilities regions

                                                                                        Program Marketing Fairly widespread use in utilities states across the country

                                                                                        MO

                                                                                        RE AGGRESSIVE ==gt

                                                                                        Test Pgm Screen - adder

                                                                                        IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                        Test Pgm Screen - readily measured

                                                                                        MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                        Test Hybrid (potential adder amp measured)

                                                                                        CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                        Test Pgm screen - Broad

                                                                                        With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                        Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                        SERA

                                                                                        42

                                                                                        Key

                                                                                        States with NEBs in CE tests for At least one type of program

                                                                                        copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                        STATES WITH NEBS IN CE TESTS

                                                                                        NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                        groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                        SERA

                                                                                        SELECTED STATE PROGRESS CA

                                                                                        2001 ndash LIPPT model NY

                                                                                        Mid-2000s measurement amp scenarios not included CO

                                                                                        Adders (10 electric 25 LI 5 gas) VT

                                                                                        15 adder helped by previous research DC

                                                                                        10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                        43

                                                                                        Source Skumatz SERA research

                                                                                        SERA

                                                                                        SELECTED STATE PROGRESS Dominos ongoing

                                                                                        Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                        Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                        Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                        Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                        44

                                                                                        Source Skumatz SERA research

                                                                                        SERA

                                                                                        APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                        Expansion Concern with risk calculation accuracy

                                                                                        45

                                                                                        Source Skumatz SERA research

                                                                                        SERA

                                                                                        46

                                                                                        NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                        [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                        SourceSkumatz SERA Independent research

                                                                                        NTG ndash accuracy measurement incomplete

                                                                                        Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                        NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                        EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                        Cost Complicated expensive local changes

                                                                                        RELATIVE Risk from NEBs

                                                                                        Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                        Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                        NEBs as accurate as many inputs (all could improve)

                                                                                        SERA

                                                                                        DISCOUNT RATE MATTERS

                                                                                        47

                                                                                        Source Skumatz SERA Research

                                                                                        0

                                                                                        20

                                                                                        40

                                                                                        60

                                                                                        80

                                                                                        100

                                                                                        120

                                                                                        1 2 3 4 5 6 7 8 9 10

                                                                                        PV Ratios vs 1 Discount Rate

                                                                                        10 yr PV 20 yr PV

                                                                                        Discount rate - percent

                                                                                        WACCSocialLTT

                                                                                        Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                        SERA

                                                                                        EUL VARIATIONS-ADOPTED VALUES

                                                                                        Source Skumatz Economic Research Associates SERA independent research

                                                                                        0

                                                                                        50

                                                                                        100

                                                                                        150

                                                                                        200

                                                                                        250

                                                                                        300

                                                                                        350

                                                                                        400

                                                                                        450

                                                                                        0 10 20 30 40 50 60 70

                                                                                        EUL Variation by Measure

                                                                                        67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                        Poorly supportedOldTechnology change

                                                                                        SERA

                                                                                        KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                        Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                        measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                        change the BC conclusion

                                                                                        On the cusphellip

                                                                                        If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                        IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                        Source SERA all rights reserved

                                                                                        SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                        SERA

                                                                                        IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                        Utility Soc Part Conserv Recrsquom

                                                                                        Rationale

                                                                                        Base Percent X X X Program-invariant

                                                                                        Low Income X X X X Multiple sources

                                                                                        Weatherization X X X Substantial Participantimpacts

                                                                                        Measure Program-specific

                                                                                        X Varies by measure sector

                                                                                        Other Recomrsquos Local Research

                                                                                        50

                                                                                        Source Skumatz SERA

                                                                                        Developing values for multiple states amp utilities

                                                                                        SERA

                                                                                        TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                        Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                        Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                        5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                        6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                        THANK YOU

                                                                                        Questions

                                                                                        Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                        SERA

                                                                                        POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                        53

                                                                                        SERA

                                                                                        NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                        Negative NEB values cost of barrier

                                                                                        Solar Water Heat NZ$ Euros

                                                                                        Solar Design NZ$ Euros

                                                                                        Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                        -23 -12 (079)

                                                                                        -11 -6 (06)

                                                                                        Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                        Source Skumatz Economic Research Associates research

                                                                                        SERA

                                                                                        CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                        NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                        -5-4-3-2-1012345

                                                                                        Op Co

                                                                                        st

                                                                                        Init C

                                                                                        ost

                                                                                        Maint

                                                                                        Perfo

                                                                                        rm

                                                                                        Prod

                                                                                        uctiv

                                                                                        Tena

                                                                                        nt Sa

                                                                                        t

                                                                                        Comf

                                                                                        ort

                                                                                        Appe

                                                                                        ar

                                                                                        Light

                                                                                        Qual

                                                                                        SellL

                                                                                        ease

                                                                                        Devp Ownr AampE

                                                                                        Note AgtE (~4 vs ~2) for these NEBs

                                                                                        Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                        Source Skumatz Economic Research Associates Inc 2002

                                                                                        Owners more positive ndashUnderinvestment train

                                                                                        • Soup to nuts on non-energy benefits NEIs
                                                                                        • TOPICS
                                                                                        • BACKGROUND NEBS
                                                                                        • 20+ YEARS OF NEBS PROGRESShellip
                                                                                        • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                        • KEY APPLICATIONS OF NEBS
                                                                                        • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                        • Neb measurement
                                                                                        • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                        • MEASURING PARTICIPANTNEBs
                                                                                        • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                        • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                        • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                        • PROGRESS TRANSFERABILITY OF NEBS
                                                                                        • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                        • PROGRESS AND GAPS
                                                                                        • PROGRESS AND GAPS
                                                                                        • USING NEB NEI VALUES
                                                                                        • Slide Number 19
                                                                                        • ARE NEBS HIGH VALUE
                                                                                        • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                        • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                        • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                        • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                        • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                        • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                        • AUDIT FINANCIAL PROGRAM
                                                                                        • CampI NEW CONSTRUCTION
                                                                                        • CampI TECHNICAL ASSISTANCE
                                                                                        • EXPRESSING NEBS VALUEndashCxe
                                                                                        • Slide Number 31
                                                                                        • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                        • EMISSIONS (SOCIETAL NEBS)
                                                                                        • UTILITY NEBS
                                                                                        • ADJUSTED PAYBACKS
                                                                                        • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                        • NEBS IN BC
                                                                                        • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                        • APPLICATION TO BC
                                                                                        • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                        • STATE TREATMENT OF NEBS NEBs
                                                                                        • Slide Number 42
                                                                                        • SELECTED STATE PROGRESS
                                                                                        • SELECTED STATE PROGRESS
                                                                                        • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                        • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                        • DISCOUNT RATE MATTERS
                                                                                        • EUL VARIATIONS- ADOPTED VALUES
                                                                                        • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                        • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                        • TAKEAWAYSON NEBSNEIS
                                                                                        • THANK YOUQuestions
                                                                                        • Slide Number 53
                                                                                        • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                        • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                                          Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                                          m Product losses20
                                                                                          l Ease of selling leasing building33
                                                                                          n Labor requirements45
                                                                                          h Aesthetics appearance59
                                                                                          j Noise50
                                                                                          b Equipment maintenance660
                                                                                          f Tenant satisfaction60
                                                                                          d Equipment lifetimes718
                                                                                          e Productivity724
                                                                                          k Building safety[2]73
                                                                                          g Comfort933
                                                                                          i Lighting Quality of Light916
                                                                                          c Equipment performance[1]1019
                                                                                          a Operating costs1165
                                                                                          o Doing good for the Environment[3]1132
                                                                                          Other03
                                                                                          [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                                          [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                                          [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                                          m Product losses
                                                                                          l Ease of selling leasing building
                                                                                          n Labor requirements
                                                                                          h Aesthetics appearance
                                                                                          j Noise
                                                                                          b Equipment maintenance
                                                                                          f Tenant satisfaction
                                                                                          d Equipment lifetimes
                                                                                          e Productivity
                                                                                          k Building safety[2]
                                                                                          g Comfort
                                                                                          i Lighting Quality of Light
                                                                                          c Equipment performance[1]
                                                                                          a Operating costs
                                                                                          o Doing good for the Environment[3]

                                                                                          SERA

                                                                                          ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                                                          0

                                                                                          002

                                                                                          004

                                                                                          006

                                                                                          008

                                                                                          01

                                                                                          012

                                                                                          Maint

                                                                                          Perf LifeSatis

                                                                                          f

                                                                                          ComfortAesth

                                                                                          NoiseSafe

                                                                                          ty

                                                                                          LightingSelli

                                                                                          ng

                                                                                          Stay in home

                                                                                          Enviro

                                                                                          Sick Days

                                                                                          Calls

                                                                                          ES Homes HP wES Energy savings multiplier=1

                                                                                          Source SERA Research

                                                                                          SERA

                                                                                          CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                                          DG CHP HVAC Equipment rebate Other

                                                                                          Building codes incentives by cities

                                                                                          Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                                          Variety of end uses

                                                                                          SERA

                                                                                          AUDIT FINANCIAL PROGRAM

                                                                                          0

                                                                                          01

                                                                                          02

                                                                                          03

                                                                                          04

                                                                                          05

                                                                                          06

                                                                                          07

                                                                                          Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                                          Source SERA Research

                                                                                          SERA

                                                                                          CampI NEW CONSTRUCTION

                                                                                          -005

                                                                                          0

                                                                                          005

                                                                                          01

                                                                                          015

                                                                                          02

                                                                                          AampE Owner(Pct of Total Partic NEBs)

                                                                                          Source SERA Research

                                                                                          SERA

                                                                                          CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                                          0 2 4 6 8 10 12

                                                                                          m Product losses

                                                                                          l Ease of selling leasing building

                                                                                          n Labor requirements

                                                                                          h Aesthetics appearance

                                                                                          j Noise

                                                                                          b Equipment maintenance

                                                                                          f Tenant satisfaction

                                                                                          d Equipment lifetimes

                                                                                          e Productivity

                                                                                          k Building safety[2]

                                                                                          g Comfort

                                                                                          i Lighting Quality of Light

                                                                                          c Equipment performance[1]

                                                                                          a Operating costs

                                                                                          o Doing good for the Environment[3]

                                                                                          Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                                          vcv

                                                                                          Source SERA Research

                                                                                          Chart1

                                                                                          CampI Technical Assistance NEBs
                                                                                          002
                                                                                          003
                                                                                          004
                                                                                          005
                                                                                          005
                                                                                          006
                                                                                          006
                                                                                          007
                                                                                          007
                                                                                          007
                                                                                          009
                                                                                          009
                                                                                          01
                                                                                          011
                                                                                          011

                                                                                          Sheet1

                                                                                          Sheet1

                                                                                          CampI Technical Assistance Program NEBs (Participant)

                                                                                          Sheet2

                                                                                          Sheet3

                                                                                          SERA

                                                                                          EXPRESSING NEBS VALUEndashCxe

                                                                                          Strong value from RetroCxSource SERA Research

                                                                                          SERA

                                                                                          SOCIETAL NEB EXAMPLES

                                                                                          31

                                                                                          SERA

                                                                                          JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                          32

                                                                                          0

                                                                                          02

                                                                                          04

                                                                                          06

                                                                                          08

                                                                                          1

                                                                                          12

                                                                                          HPWxRetrof Appliance

                                                                                          CA

                                                                                          WI

                                                                                          Natl

                                                                                          (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                          Jobs Economic

                                                                                          SERA

                                                                                          EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                          33

                                                                                          SERA

                                                                                          UTILITY NEBSUtility NEBs for Template Program

                                                                                          Debt WriteOff (util)13

                                                                                          Rate Subsidy(util)61

                                                                                          HealthSafety(util)0

                                                                                          Colln Costs (util)0

                                                                                          Gas Calls (util)0

                                                                                          Calls to CSRs(util)2

                                                                                          TampD (util)16

                                                                                          Arrears (util)0

                                                                                          Reconnects (util)0

                                                                                          Notices (util)7

                                                                                          Shutoffs (util)1

                                                                                          Rate subsidy TampD

                                                                                          Payment-related

                                                                                          Source Skumatz Economic Research Associates research

                                                                                          Example Low Income Weatherization

                                                                                          SERA

                                                                                          ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                          be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                          Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                          Value of 100 HALVES Payback doubles ROI

                                                                                          Source Skumatz Economics (SERA)

                                                                                          35

                                                                                          ( for NEBs Multiplier of 125)

                                                                                          SERA

                                                                                          NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                          36

                                                                                          hellip2001 ON

                                                                                          SERA

                                                                                          37

                                                                                          NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                          For true representation of B amp C elements of NEBs address missing factor bias

                                                                                          Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                          BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                          SERA

                                                                                          CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                          38

                                                                                          Test Benefits Costs States Using Traditionally

                                                                                          Improved treat-ment with NEBs

                                                                                          Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                          bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                          bull Avoided gas and water supply costs

                                                                                          bull Program administration

                                                                                          bull Participant incentives

                                                                                          bull Increased supply cost

                                                                                          CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                          Use cost only paid by the utility

                                                                                          Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                          Same as above plus bull increased revenue

                                                                                          Same as above plus bull Decreased revenue

                                                                                          AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                          Participant cost bull Utility bill reductions bull Participant incentives

                                                                                          bull Participant direct costs

                                                                                          AR CA FL HI IA IN MI MN NY VA

                                                                                          Participant NEBs

                                                                                          Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                          Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                          SERA

                                                                                          APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                          Expansion Concern with risk calculation accuracy

                                                                                          39

                                                                                          Source Skumatz SERA research

                                                                                          SERA

                                                                                          40

                                                                                          METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                          Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                          Minimize Regulatory amp Implementer Risk

                                                                                          Minimize Evaluation Cost

                                                                                          Adder

                                                                                          Readily MeasurableHybrid

                                                                                          All NEBsSource SERA Research

                                                                                          SERA

                                                                                          41

                                                                                          STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                          Screening Application Utilities regions

                                                                                          Program Marketing Fairly widespread use in utilities states across the country

                                                                                          MO

                                                                                          RE AGGRESSIVE ==gt

                                                                                          Test Pgm Screen - adder

                                                                                          IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                          Test Pgm Screen - readily measured

                                                                                          MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                          Test Hybrid (potential adder amp measured)

                                                                                          CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                          Test Pgm screen - Broad

                                                                                          With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                          Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                          SERA

                                                                                          42

                                                                                          Key

                                                                                          States with NEBs in CE tests for At least one type of program

                                                                                          copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                          STATES WITH NEBS IN CE TESTS

                                                                                          NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                          groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                          SERA

                                                                                          SELECTED STATE PROGRESS CA

                                                                                          2001 ndash LIPPT model NY

                                                                                          Mid-2000s measurement amp scenarios not included CO

                                                                                          Adders (10 electric 25 LI 5 gas) VT

                                                                                          15 adder helped by previous research DC

                                                                                          10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                          43

                                                                                          Source Skumatz SERA research

                                                                                          SERA

                                                                                          SELECTED STATE PROGRESS Dominos ongoing

                                                                                          Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                          Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                          Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                          Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                          44

                                                                                          Source Skumatz SERA research

                                                                                          SERA

                                                                                          APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                          Expansion Concern with risk calculation accuracy

                                                                                          45

                                                                                          Source Skumatz SERA research

                                                                                          SERA

                                                                                          46

                                                                                          NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                          [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                          SourceSkumatz SERA Independent research

                                                                                          NTG ndash accuracy measurement incomplete

                                                                                          Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                          NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                          EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                          Cost Complicated expensive local changes

                                                                                          RELATIVE Risk from NEBs

                                                                                          Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                          Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                          NEBs as accurate as many inputs (all could improve)

                                                                                          SERA

                                                                                          DISCOUNT RATE MATTERS

                                                                                          47

                                                                                          Source Skumatz SERA Research

                                                                                          0

                                                                                          20

                                                                                          40

                                                                                          60

                                                                                          80

                                                                                          100

                                                                                          120

                                                                                          1 2 3 4 5 6 7 8 9 10

                                                                                          PV Ratios vs 1 Discount Rate

                                                                                          10 yr PV 20 yr PV

                                                                                          Discount rate - percent

                                                                                          WACCSocialLTT

                                                                                          Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                          SERA

                                                                                          EUL VARIATIONS-ADOPTED VALUES

                                                                                          Source Skumatz Economic Research Associates SERA independent research

                                                                                          0

                                                                                          50

                                                                                          100

                                                                                          150

                                                                                          200

                                                                                          250

                                                                                          300

                                                                                          350

                                                                                          400

                                                                                          450

                                                                                          0 10 20 30 40 50 60 70

                                                                                          EUL Variation by Measure

                                                                                          67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                          Poorly supportedOldTechnology change

                                                                                          SERA

                                                                                          KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                          Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                          measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                          change the BC conclusion

                                                                                          On the cusphellip

                                                                                          If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                          IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                          Source SERA all rights reserved

                                                                                          SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                          SERA

                                                                                          IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                          Utility Soc Part Conserv Recrsquom

                                                                                          Rationale

                                                                                          Base Percent X X X Program-invariant

                                                                                          Low Income X X X X Multiple sources

                                                                                          Weatherization X X X Substantial Participantimpacts

                                                                                          Measure Program-specific

                                                                                          X Varies by measure sector

                                                                                          Other Recomrsquos Local Research

                                                                                          50

                                                                                          Source Skumatz SERA

                                                                                          Developing values for multiple states amp utilities

                                                                                          SERA

                                                                                          TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                          Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                          Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                          5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                          6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                          THANK YOU

                                                                                          Questions

                                                                                          Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                          SERA

                                                                                          POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                          53

                                                                                          SERA

                                                                                          NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                          Negative NEB values cost of barrier

                                                                                          Solar Water Heat NZ$ Euros

                                                                                          Solar Design NZ$ Euros

                                                                                          Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                          -23 -12 (079)

                                                                                          -11 -6 (06)

                                                                                          Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                          Source Skumatz Economic Research Associates research

                                                                                          SERA

                                                                                          CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                          NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                          -5-4-3-2-1012345

                                                                                          Op Co

                                                                                          st

                                                                                          Init C

                                                                                          ost

                                                                                          Maint

                                                                                          Perfo

                                                                                          rm

                                                                                          Prod

                                                                                          uctiv

                                                                                          Tena

                                                                                          nt Sa

                                                                                          t

                                                                                          Comf

                                                                                          ort

                                                                                          Appe

                                                                                          ar

                                                                                          Light

                                                                                          Qual

                                                                                          SellL

                                                                                          ease

                                                                                          Devp Ownr AampE

                                                                                          Note AgtE (~4 vs ~2) for these NEBs

                                                                                          Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                          Source Skumatz Economic Research Associates Inc 2002

                                                                                          Owners more positive ndashUnderinvestment train

                                                                                          • Soup to nuts on non-energy benefits NEIs
                                                                                          • TOPICS
                                                                                          • BACKGROUND NEBS
                                                                                          • 20+ YEARS OF NEBS PROGRESShellip
                                                                                          • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                          • KEY APPLICATIONS OF NEBS
                                                                                          • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                          • Neb measurement
                                                                                          • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                          • MEASURING PARTICIPANTNEBs
                                                                                          • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                          • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                          • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                          • PROGRESS TRANSFERABILITY OF NEBS
                                                                                          • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                          • PROGRESS AND GAPS
                                                                                          • PROGRESS AND GAPS
                                                                                          • USING NEB NEI VALUES
                                                                                          • Slide Number 19
                                                                                          • ARE NEBS HIGH VALUE
                                                                                          • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                          • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                          • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                          • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                          • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                          • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                          • AUDIT FINANCIAL PROGRAM
                                                                                          • CampI NEW CONSTRUCTION
                                                                                          • CampI TECHNICAL ASSISTANCE
                                                                                          • EXPRESSING NEBS VALUEndashCxe
                                                                                          • Slide Number 31
                                                                                          • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                          • EMISSIONS (SOCIETAL NEBS)
                                                                                          • UTILITY NEBS
                                                                                          • ADJUSTED PAYBACKS
                                                                                          • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                          • NEBS IN BC
                                                                                          • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                          • APPLICATION TO BC
                                                                                          • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                          • STATE TREATMENT OF NEBS NEBs
                                                                                          • Slide Number 42
                                                                                          • SELECTED STATE PROGRESS
                                                                                          • SELECTED STATE PROGRESS
                                                                                          • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                          • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                          • DISCOUNT RATE MATTERS
                                                                                          • EUL VARIATIONS- ADOPTED VALUES
                                                                                          • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                          • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                          • TAKEAWAYSON NEBSNEIS
                                                                                          • THANK YOUQuestions
                                                                                          • Slide Number 53
                                                                                          • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                          • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                                            Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                                            m Product losses20
                                                                                            l Ease of selling leasing building33
                                                                                            n Labor requirements45
                                                                                            h Aesthetics appearance59
                                                                                            j Noise50
                                                                                            b Equipment maintenance660
                                                                                            f Tenant satisfaction60
                                                                                            d Equipment lifetimes718
                                                                                            e Productivity724
                                                                                            k Building safety[2]73
                                                                                            g Comfort933
                                                                                            i Lighting Quality of Light916
                                                                                            c Equipment performance[1]1019
                                                                                            a Operating costs1165
                                                                                            o Doing good for the Environment[3]1132
                                                                                            Other03
                                                                                            [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                                            [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                                            [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                                            m Product losses
                                                                                            l Ease of selling leasing building
                                                                                            n Labor requirements
                                                                                            h Aesthetics appearance
                                                                                            j Noise
                                                                                            b Equipment maintenance
                                                                                            f Tenant satisfaction
                                                                                            d Equipment lifetimes
                                                                                            e Productivity
                                                                                            k Building safety[2]
                                                                                            g Comfort
                                                                                            i Lighting Quality of Light
                                                                                            c Equipment performance[1]
                                                                                            a Operating costs
                                                                                            o Doing good for the Environment[3]

                                                                                            SERA

                                                                                            ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS

                                                                                            0

                                                                                            002

                                                                                            004

                                                                                            006

                                                                                            008

                                                                                            01

                                                                                            012

                                                                                            Maint

                                                                                            Perf LifeSatis

                                                                                            f

                                                                                            ComfortAesth

                                                                                            NoiseSafe

                                                                                            ty

                                                                                            LightingSelli

                                                                                            ng

                                                                                            Stay in home

                                                                                            Enviro

                                                                                            Sick Days

                                                                                            Calls

                                                                                            ES Homes HP wES Energy savings multiplier=1

                                                                                            Source SERA Research

                                                                                            SERA

                                                                                            CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                                            DG CHP HVAC Equipment rebate Other

                                                                                            Building codes incentives by cities

                                                                                            Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                                            Variety of end uses

                                                                                            SERA

                                                                                            AUDIT FINANCIAL PROGRAM

                                                                                            0

                                                                                            01

                                                                                            02

                                                                                            03

                                                                                            04

                                                                                            05

                                                                                            06

                                                                                            07

                                                                                            Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                                            Source SERA Research

                                                                                            SERA

                                                                                            CampI NEW CONSTRUCTION

                                                                                            -005

                                                                                            0

                                                                                            005

                                                                                            01

                                                                                            015

                                                                                            02

                                                                                            AampE Owner(Pct of Total Partic NEBs)

                                                                                            Source SERA Research

                                                                                            SERA

                                                                                            CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                                            0 2 4 6 8 10 12

                                                                                            m Product losses

                                                                                            l Ease of selling leasing building

                                                                                            n Labor requirements

                                                                                            h Aesthetics appearance

                                                                                            j Noise

                                                                                            b Equipment maintenance

                                                                                            f Tenant satisfaction

                                                                                            d Equipment lifetimes

                                                                                            e Productivity

                                                                                            k Building safety[2]

                                                                                            g Comfort

                                                                                            i Lighting Quality of Light

                                                                                            c Equipment performance[1]

                                                                                            a Operating costs

                                                                                            o Doing good for the Environment[3]

                                                                                            Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                                            vcv

                                                                                            Source SERA Research

                                                                                            Chart1

                                                                                            CampI Technical Assistance NEBs
                                                                                            002
                                                                                            003
                                                                                            004
                                                                                            005
                                                                                            005
                                                                                            006
                                                                                            006
                                                                                            007
                                                                                            007
                                                                                            007
                                                                                            009
                                                                                            009
                                                                                            01
                                                                                            011
                                                                                            011

                                                                                            Sheet1

                                                                                            Sheet1

                                                                                            CampI Technical Assistance Program NEBs (Participant)

                                                                                            Sheet2

                                                                                            Sheet3

                                                                                            SERA

                                                                                            EXPRESSING NEBS VALUEndashCxe

                                                                                            Strong value from RetroCxSource SERA Research

                                                                                            SERA

                                                                                            SOCIETAL NEB EXAMPLES

                                                                                            31

                                                                                            SERA

                                                                                            JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                            32

                                                                                            0

                                                                                            02

                                                                                            04

                                                                                            06

                                                                                            08

                                                                                            1

                                                                                            12

                                                                                            HPWxRetrof Appliance

                                                                                            CA

                                                                                            WI

                                                                                            Natl

                                                                                            (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                            Jobs Economic

                                                                                            SERA

                                                                                            EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                            33

                                                                                            SERA

                                                                                            UTILITY NEBSUtility NEBs for Template Program

                                                                                            Debt WriteOff (util)13

                                                                                            Rate Subsidy(util)61

                                                                                            HealthSafety(util)0

                                                                                            Colln Costs (util)0

                                                                                            Gas Calls (util)0

                                                                                            Calls to CSRs(util)2

                                                                                            TampD (util)16

                                                                                            Arrears (util)0

                                                                                            Reconnects (util)0

                                                                                            Notices (util)7

                                                                                            Shutoffs (util)1

                                                                                            Rate subsidy TampD

                                                                                            Payment-related

                                                                                            Source Skumatz Economic Research Associates research

                                                                                            Example Low Income Weatherization

                                                                                            SERA

                                                                                            ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                            be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                            Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                            Value of 100 HALVES Payback doubles ROI

                                                                                            Source Skumatz Economics (SERA)

                                                                                            35

                                                                                            ( for NEBs Multiplier of 125)

                                                                                            SERA

                                                                                            NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                            36

                                                                                            hellip2001 ON

                                                                                            SERA

                                                                                            37

                                                                                            NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                            For true representation of B amp C elements of NEBs address missing factor bias

                                                                                            Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                            BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                            SERA

                                                                                            CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                            38

                                                                                            Test Benefits Costs States Using Traditionally

                                                                                            Improved treat-ment with NEBs

                                                                                            Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                            bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                            bull Avoided gas and water supply costs

                                                                                            bull Program administration

                                                                                            bull Participant incentives

                                                                                            bull Increased supply cost

                                                                                            CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                            Use cost only paid by the utility

                                                                                            Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                            Same as above plus bull increased revenue

                                                                                            Same as above plus bull Decreased revenue

                                                                                            AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                            Participant cost bull Utility bill reductions bull Participant incentives

                                                                                            bull Participant direct costs

                                                                                            AR CA FL HI IA IN MI MN NY VA

                                                                                            Participant NEBs

                                                                                            Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                            Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                            SERA

                                                                                            APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                            Expansion Concern with risk calculation accuracy

                                                                                            39

                                                                                            Source Skumatz SERA research

                                                                                            SERA

                                                                                            40

                                                                                            METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                            Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                            Minimize Regulatory amp Implementer Risk

                                                                                            Minimize Evaluation Cost

                                                                                            Adder

                                                                                            Readily MeasurableHybrid

                                                                                            All NEBsSource SERA Research

                                                                                            SERA

                                                                                            41

                                                                                            STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                            Screening Application Utilities regions

                                                                                            Program Marketing Fairly widespread use in utilities states across the country

                                                                                            MO

                                                                                            RE AGGRESSIVE ==gt

                                                                                            Test Pgm Screen - adder

                                                                                            IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                            Test Pgm Screen - readily measured

                                                                                            MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                            Test Hybrid (potential adder amp measured)

                                                                                            CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                            Test Pgm screen - Broad

                                                                                            With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                            Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                            SERA

                                                                                            42

                                                                                            Key

                                                                                            States with NEBs in CE tests for At least one type of program

                                                                                            copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                            STATES WITH NEBS IN CE TESTS

                                                                                            NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                            groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                            SERA

                                                                                            SELECTED STATE PROGRESS CA

                                                                                            2001 ndash LIPPT model NY

                                                                                            Mid-2000s measurement amp scenarios not included CO

                                                                                            Adders (10 electric 25 LI 5 gas) VT

                                                                                            15 adder helped by previous research DC

                                                                                            10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                            43

                                                                                            Source Skumatz SERA research

                                                                                            SERA

                                                                                            SELECTED STATE PROGRESS Dominos ongoing

                                                                                            Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                            Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                            Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                            Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                            44

                                                                                            Source Skumatz SERA research

                                                                                            SERA

                                                                                            APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                            Expansion Concern with risk calculation accuracy

                                                                                            45

                                                                                            Source Skumatz SERA research

                                                                                            SERA

                                                                                            46

                                                                                            NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                            [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                            SourceSkumatz SERA Independent research

                                                                                            NTG ndash accuracy measurement incomplete

                                                                                            Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                            NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                            EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                            Cost Complicated expensive local changes

                                                                                            RELATIVE Risk from NEBs

                                                                                            Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                            Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                            NEBs as accurate as many inputs (all could improve)

                                                                                            SERA

                                                                                            DISCOUNT RATE MATTERS

                                                                                            47

                                                                                            Source Skumatz SERA Research

                                                                                            0

                                                                                            20

                                                                                            40

                                                                                            60

                                                                                            80

                                                                                            100

                                                                                            120

                                                                                            1 2 3 4 5 6 7 8 9 10

                                                                                            PV Ratios vs 1 Discount Rate

                                                                                            10 yr PV 20 yr PV

                                                                                            Discount rate - percent

                                                                                            WACCSocialLTT

                                                                                            Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                            SERA

                                                                                            EUL VARIATIONS-ADOPTED VALUES

                                                                                            Source Skumatz Economic Research Associates SERA independent research

                                                                                            0

                                                                                            50

                                                                                            100

                                                                                            150

                                                                                            200

                                                                                            250

                                                                                            300

                                                                                            350

                                                                                            400

                                                                                            450

                                                                                            0 10 20 30 40 50 60 70

                                                                                            EUL Variation by Measure

                                                                                            67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                            Poorly supportedOldTechnology change

                                                                                            SERA

                                                                                            KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                            Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                            measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                            change the BC conclusion

                                                                                            On the cusphellip

                                                                                            If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                            IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                            Source SERA all rights reserved

                                                                                            SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                            SERA

                                                                                            IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                            Utility Soc Part Conserv Recrsquom

                                                                                            Rationale

                                                                                            Base Percent X X X Program-invariant

                                                                                            Low Income X X X X Multiple sources

                                                                                            Weatherization X X X Substantial Participantimpacts

                                                                                            Measure Program-specific

                                                                                            X Varies by measure sector

                                                                                            Other Recomrsquos Local Research

                                                                                            50

                                                                                            Source Skumatz SERA

                                                                                            Developing values for multiple states amp utilities

                                                                                            SERA

                                                                                            TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                            Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                            Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                            5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                            6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                            THANK YOU

                                                                                            Questions

                                                                                            Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                            SERA

                                                                                            POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                            53

                                                                                            SERA

                                                                                            NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                            Negative NEB values cost of barrier

                                                                                            Solar Water Heat NZ$ Euros

                                                                                            Solar Design NZ$ Euros

                                                                                            Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                            -23 -12 (079)

                                                                                            -11 -6 (06)

                                                                                            Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                            Source Skumatz Economic Research Associates research

                                                                                            SERA

                                                                                            CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                            NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                            -5-4-3-2-1012345

                                                                                            Op Co

                                                                                            st

                                                                                            Init C

                                                                                            ost

                                                                                            Maint

                                                                                            Perfo

                                                                                            rm

                                                                                            Prod

                                                                                            uctiv

                                                                                            Tena

                                                                                            nt Sa

                                                                                            t

                                                                                            Comf

                                                                                            ort

                                                                                            Appe

                                                                                            ar

                                                                                            Light

                                                                                            Qual

                                                                                            SellL

                                                                                            ease

                                                                                            Devp Ownr AampE

                                                                                            Note AgtE (~4 vs ~2) for these NEBs

                                                                                            Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                            Source Skumatz Economic Research Associates Inc 2002

                                                                                            Owners more positive ndashUnderinvestment train

                                                                                            • Soup to nuts on non-energy benefits NEIs
                                                                                            • TOPICS
                                                                                            • BACKGROUND NEBS
                                                                                            • 20+ YEARS OF NEBS PROGRESShellip
                                                                                            • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                            • KEY APPLICATIONS OF NEBS
                                                                                            • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                            • Neb measurement
                                                                                            • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                            • MEASURING PARTICIPANTNEBs
                                                                                            • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                            • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                            • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                            • PROGRESS TRANSFERABILITY OF NEBS
                                                                                            • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                            • PROGRESS AND GAPS
                                                                                            • PROGRESS AND GAPS
                                                                                            • USING NEB NEI VALUES
                                                                                            • Slide Number 19
                                                                                            • ARE NEBS HIGH VALUE
                                                                                            • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                            • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                            • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                            • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                            • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                            • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                            • AUDIT FINANCIAL PROGRAM
                                                                                            • CampI NEW CONSTRUCTION
                                                                                            • CampI TECHNICAL ASSISTANCE
                                                                                            • EXPRESSING NEBS VALUEndashCxe
                                                                                            • Slide Number 31
                                                                                            • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                            • EMISSIONS (SOCIETAL NEBS)
                                                                                            • UTILITY NEBS
                                                                                            • ADJUSTED PAYBACKS
                                                                                            • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                            • NEBS IN BC
                                                                                            • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                            • APPLICATION TO BC
                                                                                            • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                            • STATE TREATMENT OF NEBS NEBs
                                                                                            • Slide Number 42
                                                                                            • SELECTED STATE PROGRESS
                                                                                            • SELECTED STATE PROGRESS
                                                                                            • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                            • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                            • DISCOUNT RATE MATTERS
                                                                                            • EUL VARIATIONS- ADOPTED VALUES
                                                                                            • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                            • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                            • TAKEAWAYSON NEBSNEIS
                                                                                            • THANK YOUQuestions
                                                                                            • Slide Number 53
                                                                                            • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                            • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                                              Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                                              m Product losses20
                                                                                              l Ease of selling leasing building33
                                                                                              n Labor requirements45
                                                                                              h Aesthetics appearance59
                                                                                              j Noise50
                                                                                              b Equipment maintenance660
                                                                                              f Tenant satisfaction60
                                                                                              d Equipment lifetimes718
                                                                                              e Productivity724
                                                                                              k Building safety[2]73
                                                                                              g Comfort933
                                                                                              i Lighting Quality of Light916
                                                                                              c Equipment performance[1]1019
                                                                                              a Operating costs1165
                                                                                              o Doing good for the Environment[3]1132
                                                                                              Other03
                                                                                              [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                                              [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                                              [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                                              m Product losses
                                                                                              l Ease of selling leasing building
                                                                                              n Labor requirements
                                                                                              h Aesthetics appearance
                                                                                              j Noise
                                                                                              b Equipment maintenance
                                                                                              f Tenant satisfaction
                                                                                              d Equipment lifetimes
                                                                                              e Productivity
                                                                                              k Building safety[2]
                                                                                              g Comfort
                                                                                              i Lighting Quality of Light
                                                                                              c Equipment performance[1]
                                                                                              a Operating costs
                                                                                              o Doing good for the Environment[3]

                                                                                              SERA

                                                                                              CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC

                                                                                              DG CHP HVAC Equipment rebate Other

                                                                                              Building codes incentives by cities

                                                                                              Thousands of surveys results By measures By program types By many sectors By stakeholders By geography

                                                                                              Variety of end uses

                                                                                              SERA

                                                                                              AUDIT FINANCIAL PROGRAM

                                                                                              0

                                                                                              01

                                                                                              02

                                                                                              03

                                                                                              04

                                                                                              05

                                                                                              06

                                                                                              07

                                                                                              Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                                              Source SERA Research

                                                                                              SERA

                                                                                              CampI NEW CONSTRUCTION

                                                                                              -005

                                                                                              0

                                                                                              005

                                                                                              01

                                                                                              015

                                                                                              02

                                                                                              AampE Owner(Pct of Total Partic NEBs)

                                                                                              Source SERA Research

                                                                                              SERA

                                                                                              CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                                              0 2 4 6 8 10 12

                                                                                              m Product losses

                                                                                              l Ease of selling leasing building

                                                                                              n Labor requirements

                                                                                              h Aesthetics appearance

                                                                                              j Noise

                                                                                              b Equipment maintenance

                                                                                              f Tenant satisfaction

                                                                                              d Equipment lifetimes

                                                                                              e Productivity

                                                                                              k Building safety[2]

                                                                                              g Comfort

                                                                                              i Lighting Quality of Light

                                                                                              c Equipment performance[1]

                                                                                              a Operating costs

                                                                                              o Doing good for the Environment[3]

                                                                                              Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                                              vcv

                                                                                              Source SERA Research

                                                                                              Chart1

                                                                                              CampI Technical Assistance NEBs
                                                                                              002
                                                                                              003
                                                                                              004
                                                                                              005
                                                                                              005
                                                                                              006
                                                                                              006
                                                                                              007
                                                                                              007
                                                                                              007
                                                                                              009
                                                                                              009
                                                                                              01
                                                                                              011
                                                                                              011

                                                                                              Sheet1

                                                                                              Sheet1

                                                                                              CampI Technical Assistance Program NEBs (Participant)

                                                                                              Sheet2

                                                                                              Sheet3

                                                                                              SERA

                                                                                              EXPRESSING NEBS VALUEndashCxe

                                                                                              Strong value from RetroCxSource SERA Research

                                                                                              SERA

                                                                                              SOCIETAL NEB EXAMPLES

                                                                                              31

                                                                                              SERA

                                                                                              JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                              32

                                                                                              0

                                                                                              02

                                                                                              04

                                                                                              06

                                                                                              08

                                                                                              1

                                                                                              12

                                                                                              HPWxRetrof Appliance

                                                                                              CA

                                                                                              WI

                                                                                              Natl

                                                                                              (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                              Jobs Economic

                                                                                              SERA

                                                                                              EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                              33

                                                                                              SERA

                                                                                              UTILITY NEBSUtility NEBs for Template Program

                                                                                              Debt WriteOff (util)13

                                                                                              Rate Subsidy(util)61

                                                                                              HealthSafety(util)0

                                                                                              Colln Costs (util)0

                                                                                              Gas Calls (util)0

                                                                                              Calls to CSRs(util)2

                                                                                              TampD (util)16

                                                                                              Arrears (util)0

                                                                                              Reconnects (util)0

                                                                                              Notices (util)7

                                                                                              Shutoffs (util)1

                                                                                              Rate subsidy TampD

                                                                                              Payment-related

                                                                                              Source Skumatz Economic Research Associates research

                                                                                              Example Low Income Weatherization

                                                                                              SERA

                                                                                              ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                              be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                              Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                              Value of 100 HALVES Payback doubles ROI

                                                                                              Source Skumatz Economics (SERA)

                                                                                              35

                                                                                              ( for NEBs Multiplier of 125)

                                                                                              SERA

                                                                                              NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                              36

                                                                                              hellip2001 ON

                                                                                              SERA

                                                                                              37

                                                                                              NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                              For true representation of B amp C elements of NEBs address missing factor bias

                                                                                              Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                              BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                              SERA

                                                                                              CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                              38

                                                                                              Test Benefits Costs States Using Traditionally

                                                                                              Improved treat-ment with NEBs

                                                                                              Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                              bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                              bull Avoided gas and water supply costs

                                                                                              bull Program administration

                                                                                              bull Participant incentives

                                                                                              bull Increased supply cost

                                                                                              CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                              Use cost only paid by the utility

                                                                                              Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                              Same as above plus bull increased revenue

                                                                                              Same as above plus bull Decreased revenue

                                                                                              AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                              Participant cost bull Utility bill reductions bull Participant incentives

                                                                                              bull Participant direct costs

                                                                                              AR CA FL HI IA IN MI MN NY VA

                                                                                              Participant NEBs

                                                                                              Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                              Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                              SERA

                                                                                              APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                              Expansion Concern with risk calculation accuracy

                                                                                              39

                                                                                              Source Skumatz SERA research

                                                                                              SERA

                                                                                              40

                                                                                              METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                              Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                              Minimize Regulatory amp Implementer Risk

                                                                                              Minimize Evaluation Cost

                                                                                              Adder

                                                                                              Readily MeasurableHybrid

                                                                                              All NEBsSource SERA Research

                                                                                              SERA

                                                                                              41

                                                                                              STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                              Screening Application Utilities regions

                                                                                              Program Marketing Fairly widespread use in utilities states across the country

                                                                                              MO

                                                                                              RE AGGRESSIVE ==gt

                                                                                              Test Pgm Screen - adder

                                                                                              IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                              Test Pgm Screen - readily measured

                                                                                              MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                              Test Hybrid (potential adder amp measured)

                                                                                              CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                              Test Pgm screen - Broad

                                                                                              With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                              Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                              SERA

                                                                                              42

                                                                                              Key

                                                                                              States with NEBs in CE tests for At least one type of program

                                                                                              copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                              STATES WITH NEBS IN CE TESTS

                                                                                              NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                              groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                              SERA

                                                                                              SELECTED STATE PROGRESS CA

                                                                                              2001 ndash LIPPT model NY

                                                                                              Mid-2000s measurement amp scenarios not included CO

                                                                                              Adders (10 electric 25 LI 5 gas) VT

                                                                                              15 adder helped by previous research DC

                                                                                              10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                              43

                                                                                              Source Skumatz SERA research

                                                                                              SERA

                                                                                              SELECTED STATE PROGRESS Dominos ongoing

                                                                                              Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                              Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                              Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                              Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                              44

                                                                                              Source Skumatz SERA research

                                                                                              SERA

                                                                                              APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                              Expansion Concern with risk calculation accuracy

                                                                                              45

                                                                                              Source Skumatz SERA research

                                                                                              SERA

                                                                                              46

                                                                                              NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                              [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                              SourceSkumatz SERA Independent research

                                                                                              NTG ndash accuracy measurement incomplete

                                                                                              Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                              NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                              EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                              Cost Complicated expensive local changes

                                                                                              RELATIVE Risk from NEBs

                                                                                              Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                              Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                              NEBs as accurate as many inputs (all could improve)

                                                                                              SERA

                                                                                              DISCOUNT RATE MATTERS

                                                                                              47

                                                                                              Source Skumatz SERA Research

                                                                                              0

                                                                                              20

                                                                                              40

                                                                                              60

                                                                                              80

                                                                                              100

                                                                                              120

                                                                                              1 2 3 4 5 6 7 8 9 10

                                                                                              PV Ratios vs 1 Discount Rate

                                                                                              10 yr PV 20 yr PV

                                                                                              Discount rate - percent

                                                                                              WACCSocialLTT

                                                                                              Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                              SERA

                                                                                              EUL VARIATIONS-ADOPTED VALUES

                                                                                              Source Skumatz Economic Research Associates SERA independent research

                                                                                              0

                                                                                              50

                                                                                              100

                                                                                              150

                                                                                              200

                                                                                              250

                                                                                              300

                                                                                              350

                                                                                              400

                                                                                              450

                                                                                              0 10 20 30 40 50 60 70

                                                                                              EUL Variation by Measure

                                                                                              67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                              Poorly supportedOldTechnology change

                                                                                              SERA

                                                                                              KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                              Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                              measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                              change the BC conclusion

                                                                                              On the cusphellip

                                                                                              If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                              IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                              Source SERA all rights reserved

                                                                                              SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                              SERA

                                                                                              IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                              Utility Soc Part Conserv Recrsquom

                                                                                              Rationale

                                                                                              Base Percent X X X Program-invariant

                                                                                              Low Income X X X X Multiple sources

                                                                                              Weatherization X X X Substantial Participantimpacts

                                                                                              Measure Program-specific

                                                                                              X Varies by measure sector

                                                                                              Other Recomrsquos Local Research

                                                                                              50

                                                                                              Source Skumatz SERA

                                                                                              Developing values for multiple states amp utilities

                                                                                              SERA

                                                                                              TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                              Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                              Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                              5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                              6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                              THANK YOU

                                                                                              Questions

                                                                                              Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                              SERA

                                                                                              POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                              53

                                                                                              SERA

                                                                                              NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                              Negative NEB values cost of barrier

                                                                                              Solar Water Heat NZ$ Euros

                                                                                              Solar Design NZ$ Euros

                                                                                              Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                              -23 -12 (079)

                                                                                              -11 -6 (06)

                                                                                              Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                              Source Skumatz Economic Research Associates research

                                                                                              SERA

                                                                                              CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                              NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                              -5-4-3-2-1012345

                                                                                              Op Co

                                                                                              st

                                                                                              Init C

                                                                                              ost

                                                                                              Maint

                                                                                              Perfo

                                                                                              rm

                                                                                              Prod

                                                                                              uctiv

                                                                                              Tena

                                                                                              nt Sa

                                                                                              t

                                                                                              Comf

                                                                                              ort

                                                                                              Appe

                                                                                              ar

                                                                                              Light

                                                                                              Qual

                                                                                              SellL

                                                                                              ease

                                                                                              Devp Ownr AampE

                                                                                              Note AgtE (~4 vs ~2) for these NEBs

                                                                                              Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                              Source Skumatz Economic Research Associates Inc 2002

                                                                                              Owners more positive ndashUnderinvestment train

                                                                                              • Soup to nuts on non-energy benefits NEIs
                                                                                              • TOPICS
                                                                                              • BACKGROUND NEBS
                                                                                              • 20+ YEARS OF NEBS PROGRESShellip
                                                                                              • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                              • KEY APPLICATIONS OF NEBS
                                                                                              • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                              • Neb measurement
                                                                                              • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                              • MEASURING PARTICIPANTNEBs
                                                                                              • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                              • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                              • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                              • PROGRESS TRANSFERABILITY OF NEBS
                                                                                              • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                              • PROGRESS AND GAPS
                                                                                              • PROGRESS AND GAPS
                                                                                              • USING NEB NEI VALUES
                                                                                              • Slide Number 19
                                                                                              • ARE NEBS HIGH VALUE
                                                                                              • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                              • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                              • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                              • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                              • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                              • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                              • AUDIT FINANCIAL PROGRAM
                                                                                              • CampI NEW CONSTRUCTION
                                                                                              • CampI TECHNICAL ASSISTANCE
                                                                                              • EXPRESSING NEBS VALUEndashCxe
                                                                                              • Slide Number 31
                                                                                              • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                              • EMISSIONS (SOCIETAL NEBS)
                                                                                              • UTILITY NEBS
                                                                                              • ADJUSTED PAYBACKS
                                                                                              • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                              • NEBS IN BC
                                                                                              • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                              • APPLICATION TO BC
                                                                                              • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                              • STATE TREATMENT OF NEBS NEBs
                                                                                              • Slide Number 42
                                                                                              • SELECTED STATE PROGRESS
                                                                                              • SELECTED STATE PROGRESS
                                                                                              • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                              • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                              • DISCOUNT RATE MATTERS
                                                                                              • EUL VARIATIONS- ADOPTED VALUES
                                                                                              • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                              • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                              • TAKEAWAYSON NEBSNEIS
                                                                                              • THANK YOUQuestions
                                                                                              • Slide Number 53
                                                                                              • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                              • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                                                Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                                                m Product losses20
                                                                                                l Ease of selling leasing building33
                                                                                                n Labor requirements45
                                                                                                h Aesthetics appearance59
                                                                                                j Noise50
                                                                                                b Equipment maintenance660
                                                                                                f Tenant satisfaction60
                                                                                                d Equipment lifetimes718
                                                                                                e Productivity724
                                                                                                k Building safety[2]73
                                                                                                g Comfort933
                                                                                                i Lighting Quality of Light916
                                                                                                c Equipment performance[1]1019
                                                                                                a Operating costs1165
                                                                                                o Doing good for the Environment[3]1132
                                                                                                Other03
                                                                                                [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                                                [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                                                [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                                                m Product losses
                                                                                                l Ease of selling leasing building
                                                                                                n Labor requirements
                                                                                                h Aesthetics appearance
                                                                                                j Noise
                                                                                                b Equipment maintenance
                                                                                                f Tenant satisfaction
                                                                                                d Equipment lifetimes
                                                                                                e Productivity
                                                                                                k Building safety[2]
                                                                                                g Comfort
                                                                                                i Lighting Quality of Light
                                                                                                c Equipment performance[1]
                                                                                                a Operating costs
                                                                                                o Doing good for the Environment[3]

                                                                                                SERA

                                                                                                AUDIT FINANCIAL PROGRAM

                                                                                                0

                                                                                                01

                                                                                                02

                                                                                                03

                                                                                                04

                                                                                                05

                                                                                                06

                                                                                                07

                                                                                                Variety of measureshellip disaggregatedResults by measure type businessType etchellip

                                                                                                Source SERA Research

                                                                                                SERA

                                                                                                CampI NEW CONSTRUCTION

                                                                                                -005

                                                                                                0

                                                                                                005

                                                                                                01

                                                                                                015

                                                                                                02

                                                                                                AampE Owner(Pct of Total Partic NEBs)

                                                                                                Source SERA Research

                                                                                                SERA

                                                                                                CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                                                0 2 4 6 8 10 12

                                                                                                m Product losses

                                                                                                l Ease of selling leasing building

                                                                                                n Labor requirements

                                                                                                h Aesthetics appearance

                                                                                                j Noise

                                                                                                b Equipment maintenance

                                                                                                f Tenant satisfaction

                                                                                                d Equipment lifetimes

                                                                                                e Productivity

                                                                                                k Building safety[2]

                                                                                                g Comfort

                                                                                                i Lighting Quality of Light

                                                                                                c Equipment performance[1]

                                                                                                a Operating costs

                                                                                                o Doing good for the Environment[3]

                                                                                                Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                                                vcv

                                                                                                Source SERA Research

                                                                                                Chart1

                                                                                                CampI Technical Assistance NEBs
                                                                                                002
                                                                                                003
                                                                                                004
                                                                                                005
                                                                                                005
                                                                                                006
                                                                                                006
                                                                                                007
                                                                                                007
                                                                                                007
                                                                                                009
                                                                                                009
                                                                                                01
                                                                                                011
                                                                                                011

                                                                                                Sheet1

                                                                                                Sheet1

                                                                                                CampI Technical Assistance Program NEBs (Participant)

                                                                                                Sheet2

                                                                                                Sheet3

                                                                                                SERA

                                                                                                EXPRESSING NEBS VALUEndashCxe

                                                                                                Strong value from RetroCxSource SERA Research

                                                                                                SERA

                                                                                                SOCIETAL NEB EXAMPLES

                                                                                                31

                                                                                                SERA

                                                                                                JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                                32

                                                                                                0

                                                                                                02

                                                                                                04

                                                                                                06

                                                                                                08

                                                                                                1

                                                                                                12

                                                                                                HPWxRetrof Appliance

                                                                                                CA

                                                                                                WI

                                                                                                Natl

                                                                                                (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                                Jobs Economic

                                                                                                SERA

                                                                                                EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                                33

                                                                                                SERA

                                                                                                UTILITY NEBSUtility NEBs for Template Program

                                                                                                Debt WriteOff (util)13

                                                                                                Rate Subsidy(util)61

                                                                                                HealthSafety(util)0

                                                                                                Colln Costs (util)0

                                                                                                Gas Calls (util)0

                                                                                                Calls to CSRs(util)2

                                                                                                TampD (util)16

                                                                                                Arrears (util)0

                                                                                                Reconnects (util)0

                                                                                                Notices (util)7

                                                                                                Shutoffs (util)1

                                                                                                Rate subsidy TampD

                                                                                                Payment-related

                                                                                                Source Skumatz Economic Research Associates research

                                                                                                Example Low Income Weatherization

                                                                                                SERA

                                                                                                ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                                be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                                Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                                Value of 100 HALVES Payback doubles ROI

                                                                                                Source Skumatz Economics (SERA)

                                                                                                35

                                                                                                ( for NEBs Multiplier of 125)

                                                                                                SERA

                                                                                                NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                                36

                                                                                                hellip2001 ON

                                                                                                SERA

                                                                                                37

                                                                                                NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                SERA

                                                                                                CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                38

                                                                                                Test Benefits Costs States Using Traditionally

                                                                                                Improved treat-ment with NEBs

                                                                                                Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                bull Avoided gas and water supply costs

                                                                                                bull Program administration

                                                                                                bull Participant incentives

                                                                                                bull Increased supply cost

                                                                                                CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                Use cost only paid by the utility

                                                                                                Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                Same as above plus bull increased revenue

                                                                                                Same as above plus bull Decreased revenue

                                                                                                AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                bull Participant direct costs

                                                                                                AR CA FL HI IA IN MI MN NY VA

                                                                                                Participant NEBs

                                                                                                Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                SERA

                                                                                                APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                Expansion Concern with risk calculation accuracy

                                                                                                39

                                                                                                Source Skumatz SERA research

                                                                                                SERA

                                                                                                40

                                                                                                METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                Minimize Regulatory amp Implementer Risk

                                                                                                Minimize Evaluation Cost

                                                                                                Adder

                                                                                                Readily MeasurableHybrid

                                                                                                All NEBsSource SERA Research

                                                                                                SERA

                                                                                                41

                                                                                                STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                Screening Application Utilities regions

                                                                                                Program Marketing Fairly widespread use in utilities states across the country

                                                                                                MO

                                                                                                RE AGGRESSIVE ==gt

                                                                                                Test Pgm Screen - adder

                                                                                                IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                Test Pgm Screen - readily measured

                                                                                                MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                Test Hybrid (potential adder amp measured)

                                                                                                CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                Test Pgm screen - Broad

                                                                                                With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                SERA

                                                                                                42

                                                                                                Key

                                                                                                States with NEBs in CE tests for At least one type of program

                                                                                                copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                STATES WITH NEBS IN CE TESTS

                                                                                                NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                SERA

                                                                                                SELECTED STATE PROGRESS CA

                                                                                                2001 ndash LIPPT model NY

                                                                                                Mid-2000s measurement amp scenarios not included CO

                                                                                                Adders (10 electric 25 LI 5 gas) VT

                                                                                                15 adder helped by previous research DC

                                                                                                10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                43

                                                                                                Source Skumatz SERA research

                                                                                                SERA

                                                                                                SELECTED STATE PROGRESS Dominos ongoing

                                                                                                Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                44

                                                                                                Source Skumatz SERA research

                                                                                                SERA

                                                                                                APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                Expansion Concern with risk calculation accuracy

                                                                                                45

                                                                                                Source Skumatz SERA research

                                                                                                SERA

                                                                                                46

                                                                                                NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                SourceSkumatz SERA Independent research

                                                                                                NTG ndash accuracy measurement incomplete

                                                                                                Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                Cost Complicated expensive local changes

                                                                                                RELATIVE Risk from NEBs

                                                                                                Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                NEBs as accurate as many inputs (all could improve)

                                                                                                SERA

                                                                                                DISCOUNT RATE MATTERS

                                                                                                47

                                                                                                Source Skumatz SERA Research

                                                                                                0

                                                                                                20

                                                                                                40

                                                                                                60

                                                                                                80

                                                                                                100

                                                                                                120

                                                                                                1 2 3 4 5 6 7 8 9 10

                                                                                                PV Ratios vs 1 Discount Rate

                                                                                                10 yr PV 20 yr PV

                                                                                                Discount rate - percent

                                                                                                WACCSocialLTT

                                                                                                Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                SERA

                                                                                                EUL VARIATIONS-ADOPTED VALUES

                                                                                                Source Skumatz Economic Research Associates SERA independent research

                                                                                                0

                                                                                                50

                                                                                                100

                                                                                                150

                                                                                                200

                                                                                                250

                                                                                                300

                                                                                                350

                                                                                                400

                                                                                                450

                                                                                                0 10 20 30 40 50 60 70

                                                                                                EUL Variation by Measure

                                                                                                67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                Poorly supportedOldTechnology change

                                                                                                SERA

                                                                                                KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                change the BC conclusion

                                                                                                On the cusphellip

                                                                                                If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                Source SERA all rights reserved

                                                                                                SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                SERA

                                                                                                IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                Utility Soc Part Conserv Recrsquom

                                                                                                Rationale

                                                                                                Base Percent X X X Program-invariant

                                                                                                Low Income X X X X Multiple sources

                                                                                                Weatherization X X X Substantial Participantimpacts

                                                                                                Measure Program-specific

                                                                                                X Varies by measure sector

                                                                                                Other Recomrsquos Local Research

                                                                                                50

                                                                                                Source Skumatz SERA

                                                                                                Developing values for multiple states amp utilities

                                                                                                SERA

                                                                                                TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                THANK YOU

                                                                                                Questions

                                                                                                Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                SERA

                                                                                                POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                53

                                                                                                SERA

                                                                                                NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                Negative NEB values cost of barrier

                                                                                                Solar Water Heat NZ$ Euros

                                                                                                Solar Design NZ$ Euros

                                                                                                Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                -23 -12 (079)

                                                                                                -11 -6 (06)

                                                                                                Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                Source Skumatz Economic Research Associates research

                                                                                                SERA

                                                                                                CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                -5-4-3-2-1012345

                                                                                                Op Co

                                                                                                st

                                                                                                Init C

                                                                                                ost

                                                                                                Maint

                                                                                                Perfo

                                                                                                rm

                                                                                                Prod

                                                                                                uctiv

                                                                                                Tena

                                                                                                nt Sa

                                                                                                t

                                                                                                Comf

                                                                                                ort

                                                                                                Appe

                                                                                                ar

                                                                                                Light

                                                                                                Qual

                                                                                                SellL

                                                                                                ease

                                                                                                Devp Ownr AampE

                                                                                                Note AgtE (~4 vs ~2) for these NEBs

                                                                                                Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                Source Skumatz Economic Research Associates Inc 2002

                                                                                                Owners more positive ndashUnderinvestment train

                                                                                                • Soup to nuts on non-energy benefits NEIs
                                                                                                • TOPICS
                                                                                                • BACKGROUND NEBS
                                                                                                • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                • KEY APPLICATIONS OF NEBS
                                                                                                • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                • Neb measurement
                                                                                                • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                • MEASURING PARTICIPANTNEBs
                                                                                                • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                • PROGRESS AND GAPS
                                                                                                • PROGRESS AND GAPS
                                                                                                • USING NEB NEI VALUES
                                                                                                • Slide Number 19
                                                                                                • ARE NEBS HIGH VALUE
                                                                                                • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                • AUDIT FINANCIAL PROGRAM
                                                                                                • CampI NEW CONSTRUCTION
                                                                                                • CampI TECHNICAL ASSISTANCE
                                                                                                • EXPRESSING NEBS VALUEndashCxe
                                                                                                • Slide Number 31
                                                                                                • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                • EMISSIONS (SOCIETAL NEBS)
                                                                                                • UTILITY NEBS
                                                                                                • ADJUSTED PAYBACKS
                                                                                                • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                • NEBS IN BC
                                                                                                • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                • APPLICATION TO BC
                                                                                                • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                • STATE TREATMENT OF NEBS NEBs
                                                                                                • Slide Number 42
                                                                                                • SELECTED STATE PROGRESS
                                                                                                • SELECTED STATE PROGRESS
                                                                                                • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                • DISCOUNT RATE MATTERS
                                                                                                • EUL VARIATIONS- ADOPTED VALUES
                                                                                                • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                • TAKEAWAYSON NEBSNEIS
                                                                                                • THANK YOUQuestions
                                                                                                • Slide Number 53
                                                                                                • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                                                  Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                                                  m Product losses20
                                                                                                  l Ease of selling leasing building33
                                                                                                  n Labor requirements45
                                                                                                  h Aesthetics appearance59
                                                                                                  j Noise50
                                                                                                  b Equipment maintenance660
                                                                                                  f Tenant satisfaction60
                                                                                                  d Equipment lifetimes718
                                                                                                  e Productivity724
                                                                                                  k Building safety[2]73
                                                                                                  g Comfort933
                                                                                                  i Lighting Quality of Light916
                                                                                                  c Equipment performance[1]1019
                                                                                                  a Operating costs1165
                                                                                                  o Doing good for the Environment[3]1132
                                                                                                  Other03
                                                                                                  [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                                                  [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                                                  [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                                                  m Product losses
                                                                                                  l Ease of selling leasing building
                                                                                                  n Labor requirements
                                                                                                  h Aesthetics appearance
                                                                                                  j Noise
                                                                                                  b Equipment maintenance
                                                                                                  f Tenant satisfaction
                                                                                                  d Equipment lifetimes
                                                                                                  e Productivity
                                                                                                  k Building safety[2]
                                                                                                  g Comfort
                                                                                                  i Lighting Quality of Light
                                                                                                  c Equipment performance[1]
                                                                                                  a Operating costs
                                                                                                  o Doing good for the Environment[3]

                                                                                                  SERA

                                                                                                  CampI NEW CONSTRUCTION

                                                                                                  -005

                                                                                                  0

                                                                                                  005

                                                                                                  01

                                                                                                  015

                                                                                                  02

                                                                                                  AampE Owner(Pct of Total Partic NEBs)

                                                                                                  Source SERA Research

                                                                                                  SERA

                                                                                                  CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                                                  0 2 4 6 8 10 12

                                                                                                  m Product losses

                                                                                                  l Ease of selling leasing building

                                                                                                  n Labor requirements

                                                                                                  h Aesthetics appearance

                                                                                                  j Noise

                                                                                                  b Equipment maintenance

                                                                                                  f Tenant satisfaction

                                                                                                  d Equipment lifetimes

                                                                                                  e Productivity

                                                                                                  k Building safety[2]

                                                                                                  g Comfort

                                                                                                  i Lighting Quality of Light

                                                                                                  c Equipment performance[1]

                                                                                                  a Operating costs

                                                                                                  o Doing good for the Environment[3]

                                                                                                  Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                                                  vcv

                                                                                                  Source SERA Research

                                                                                                  Chart1

                                                                                                  CampI Technical Assistance NEBs
                                                                                                  002
                                                                                                  003
                                                                                                  004
                                                                                                  005
                                                                                                  005
                                                                                                  006
                                                                                                  006
                                                                                                  007
                                                                                                  007
                                                                                                  007
                                                                                                  009
                                                                                                  009
                                                                                                  01
                                                                                                  011
                                                                                                  011

                                                                                                  Sheet1

                                                                                                  Sheet1

                                                                                                  CampI Technical Assistance Program NEBs (Participant)

                                                                                                  Sheet2

                                                                                                  Sheet3

                                                                                                  SERA

                                                                                                  EXPRESSING NEBS VALUEndashCxe

                                                                                                  Strong value from RetroCxSource SERA Research

                                                                                                  SERA

                                                                                                  SOCIETAL NEB EXAMPLES

                                                                                                  31

                                                                                                  SERA

                                                                                                  JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                                  32

                                                                                                  0

                                                                                                  02

                                                                                                  04

                                                                                                  06

                                                                                                  08

                                                                                                  1

                                                                                                  12

                                                                                                  HPWxRetrof Appliance

                                                                                                  CA

                                                                                                  WI

                                                                                                  Natl

                                                                                                  (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                                  Jobs Economic

                                                                                                  SERA

                                                                                                  EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                                  33

                                                                                                  SERA

                                                                                                  UTILITY NEBSUtility NEBs for Template Program

                                                                                                  Debt WriteOff (util)13

                                                                                                  Rate Subsidy(util)61

                                                                                                  HealthSafety(util)0

                                                                                                  Colln Costs (util)0

                                                                                                  Gas Calls (util)0

                                                                                                  Calls to CSRs(util)2

                                                                                                  TampD (util)16

                                                                                                  Arrears (util)0

                                                                                                  Reconnects (util)0

                                                                                                  Notices (util)7

                                                                                                  Shutoffs (util)1

                                                                                                  Rate subsidy TampD

                                                                                                  Payment-related

                                                                                                  Source Skumatz Economic Research Associates research

                                                                                                  Example Low Income Weatherization

                                                                                                  SERA

                                                                                                  ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                                  be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                                  Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                                  Value of 100 HALVES Payback doubles ROI

                                                                                                  Source Skumatz Economics (SERA)

                                                                                                  35

                                                                                                  ( for NEBs Multiplier of 125)

                                                                                                  SERA

                                                                                                  NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                                  36

                                                                                                  hellip2001 ON

                                                                                                  SERA

                                                                                                  37

                                                                                                  NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                  For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                  Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                  BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                  SERA

                                                                                                  CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                  38

                                                                                                  Test Benefits Costs States Using Traditionally

                                                                                                  Improved treat-ment with NEBs

                                                                                                  Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                  bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                  bull Avoided gas and water supply costs

                                                                                                  bull Program administration

                                                                                                  bull Participant incentives

                                                                                                  bull Increased supply cost

                                                                                                  CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                  Use cost only paid by the utility

                                                                                                  Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                  Same as above plus bull increased revenue

                                                                                                  Same as above plus bull Decreased revenue

                                                                                                  AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                  Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                  bull Participant direct costs

                                                                                                  AR CA FL HI IA IN MI MN NY VA

                                                                                                  Participant NEBs

                                                                                                  Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                  Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                  SERA

                                                                                                  APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                  Expansion Concern with risk calculation accuracy

                                                                                                  39

                                                                                                  Source Skumatz SERA research

                                                                                                  SERA

                                                                                                  40

                                                                                                  METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                  Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                  Minimize Regulatory amp Implementer Risk

                                                                                                  Minimize Evaluation Cost

                                                                                                  Adder

                                                                                                  Readily MeasurableHybrid

                                                                                                  All NEBsSource SERA Research

                                                                                                  SERA

                                                                                                  41

                                                                                                  STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                  Screening Application Utilities regions

                                                                                                  Program Marketing Fairly widespread use in utilities states across the country

                                                                                                  MO

                                                                                                  RE AGGRESSIVE ==gt

                                                                                                  Test Pgm Screen - adder

                                                                                                  IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                  Test Pgm Screen - readily measured

                                                                                                  MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                  Test Hybrid (potential adder amp measured)

                                                                                                  CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                  Test Pgm screen - Broad

                                                                                                  With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                  Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                  SERA

                                                                                                  42

                                                                                                  Key

                                                                                                  States with NEBs in CE tests for At least one type of program

                                                                                                  copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                  STATES WITH NEBS IN CE TESTS

                                                                                                  NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                  groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                  SERA

                                                                                                  SELECTED STATE PROGRESS CA

                                                                                                  2001 ndash LIPPT model NY

                                                                                                  Mid-2000s measurement amp scenarios not included CO

                                                                                                  Adders (10 electric 25 LI 5 gas) VT

                                                                                                  15 adder helped by previous research DC

                                                                                                  10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                  43

                                                                                                  Source Skumatz SERA research

                                                                                                  SERA

                                                                                                  SELECTED STATE PROGRESS Dominos ongoing

                                                                                                  Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                  Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                  Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                  Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                  44

                                                                                                  Source Skumatz SERA research

                                                                                                  SERA

                                                                                                  APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                  Expansion Concern with risk calculation accuracy

                                                                                                  45

                                                                                                  Source Skumatz SERA research

                                                                                                  SERA

                                                                                                  46

                                                                                                  NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                  [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                  SourceSkumatz SERA Independent research

                                                                                                  NTG ndash accuracy measurement incomplete

                                                                                                  Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                  NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                  EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                  Cost Complicated expensive local changes

                                                                                                  RELATIVE Risk from NEBs

                                                                                                  Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                  Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                  NEBs as accurate as many inputs (all could improve)

                                                                                                  SERA

                                                                                                  DISCOUNT RATE MATTERS

                                                                                                  47

                                                                                                  Source Skumatz SERA Research

                                                                                                  0

                                                                                                  20

                                                                                                  40

                                                                                                  60

                                                                                                  80

                                                                                                  100

                                                                                                  120

                                                                                                  1 2 3 4 5 6 7 8 9 10

                                                                                                  PV Ratios vs 1 Discount Rate

                                                                                                  10 yr PV 20 yr PV

                                                                                                  Discount rate - percent

                                                                                                  WACCSocialLTT

                                                                                                  Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                  SERA

                                                                                                  EUL VARIATIONS-ADOPTED VALUES

                                                                                                  Source Skumatz Economic Research Associates SERA independent research

                                                                                                  0

                                                                                                  50

                                                                                                  100

                                                                                                  150

                                                                                                  200

                                                                                                  250

                                                                                                  300

                                                                                                  350

                                                                                                  400

                                                                                                  450

                                                                                                  0 10 20 30 40 50 60 70

                                                                                                  EUL Variation by Measure

                                                                                                  67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                  Poorly supportedOldTechnology change

                                                                                                  SERA

                                                                                                  KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                  Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                  measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                  change the BC conclusion

                                                                                                  On the cusphellip

                                                                                                  If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                  IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                  Source SERA all rights reserved

                                                                                                  SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                  SERA

                                                                                                  IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                  Utility Soc Part Conserv Recrsquom

                                                                                                  Rationale

                                                                                                  Base Percent X X X Program-invariant

                                                                                                  Low Income X X X X Multiple sources

                                                                                                  Weatherization X X X Substantial Participantimpacts

                                                                                                  Measure Program-specific

                                                                                                  X Varies by measure sector

                                                                                                  Other Recomrsquos Local Research

                                                                                                  50

                                                                                                  Source Skumatz SERA

                                                                                                  Developing values for multiple states amp utilities

                                                                                                  SERA

                                                                                                  TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                  Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                  Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                  5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                  6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                  THANK YOU

                                                                                                  Questions

                                                                                                  Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                  SERA

                                                                                                  POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                  53

                                                                                                  SERA

                                                                                                  NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                  Negative NEB values cost of barrier

                                                                                                  Solar Water Heat NZ$ Euros

                                                                                                  Solar Design NZ$ Euros

                                                                                                  Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                  -23 -12 (079)

                                                                                                  -11 -6 (06)

                                                                                                  Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                  Source Skumatz Economic Research Associates research

                                                                                                  SERA

                                                                                                  CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                  NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                  -5-4-3-2-1012345

                                                                                                  Op Co

                                                                                                  st

                                                                                                  Init C

                                                                                                  ost

                                                                                                  Maint

                                                                                                  Perfo

                                                                                                  rm

                                                                                                  Prod

                                                                                                  uctiv

                                                                                                  Tena

                                                                                                  nt Sa

                                                                                                  t

                                                                                                  Comf

                                                                                                  ort

                                                                                                  Appe

                                                                                                  ar

                                                                                                  Light

                                                                                                  Qual

                                                                                                  SellL

                                                                                                  ease

                                                                                                  Devp Ownr AampE

                                                                                                  Note AgtE (~4 vs ~2) for these NEBs

                                                                                                  Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                  Source Skumatz Economic Research Associates Inc 2002

                                                                                                  Owners more positive ndashUnderinvestment train

                                                                                                  • Soup to nuts on non-energy benefits NEIs
                                                                                                  • TOPICS
                                                                                                  • BACKGROUND NEBS
                                                                                                  • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                  • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                  • KEY APPLICATIONS OF NEBS
                                                                                                  • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                  • Neb measurement
                                                                                                  • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                  • MEASURING PARTICIPANTNEBs
                                                                                                  • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                  • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                  • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                  • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                  • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                  • PROGRESS AND GAPS
                                                                                                  • PROGRESS AND GAPS
                                                                                                  • USING NEB NEI VALUES
                                                                                                  • Slide Number 19
                                                                                                  • ARE NEBS HIGH VALUE
                                                                                                  • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                  • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                  • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                  • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                  • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                  • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                  • AUDIT FINANCIAL PROGRAM
                                                                                                  • CampI NEW CONSTRUCTION
                                                                                                  • CampI TECHNICAL ASSISTANCE
                                                                                                  • EXPRESSING NEBS VALUEndashCxe
                                                                                                  • Slide Number 31
                                                                                                  • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                  • EMISSIONS (SOCIETAL NEBS)
                                                                                                  • UTILITY NEBS
                                                                                                  • ADJUSTED PAYBACKS
                                                                                                  • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                  • NEBS IN BC
                                                                                                  • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                  • APPLICATION TO BC
                                                                                                  • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                  • STATE TREATMENT OF NEBS NEBs
                                                                                                  • Slide Number 42
                                                                                                  • SELECTED STATE PROGRESS
                                                                                                  • SELECTED STATE PROGRESS
                                                                                                  • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                  • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                  • DISCOUNT RATE MATTERS
                                                                                                  • EUL VARIATIONS- ADOPTED VALUES
                                                                                                  • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                  • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                  • TAKEAWAYSON NEBSNEIS
                                                                                                  • THANK YOUQuestions
                                                                                                  • Slide Number 53
                                                                                                  • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                  • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                                                    Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                                                    m Product losses20
                                                                                                    l Ease of selling leasing building33
                                                                                                    n Labor requirements45
                                                                                                    h Aesthetics appearance59
                                                                                                    j Noise50
                                                                                                    b Equipment maintenance660
                                                                                                    f Tenant satisfaction60
                                                                                                    d Equipment lifetimes718
                                                                                                    e Productivity724
                                                                                                    k Building safety[2]73
                                                                                                    g Comfort933
                                                                                                    i Lighting Quality of Light916
                                                                                                    c Equipment performance[1]1019
                                                                                                    a Operating costs1165
                                                                                                    o Doing good for the Environment[3]1132
                                                                                                    Other03
                                                                                                    [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                                                    [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                                                    [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                                                    m Product losses
                                                                                                    l Ease of selling leasing building
                                                                                                    n Labor requirements
                                                                                                    h Aesthetics appearance
                                                                                                    j Noise
                                                                                                    b Equipment maintenance
                                                                                                    f Tenant satisfaction
                                                                                                    d Equipment lifetimes
                                                                                                    e Productivity
                                                                                                    k Building safety[2]
                                                                                                    g Comfort
                                                                                                    i Lighting Quality of Light
                                                                                                    c Equipment performance[1]
                                                                                                    a Operating costs
                                                                                                    o Doing good for the Environment[3]

                                                                                                    SERA

                                                                                                    CampI TECHNICAL ASSISTANCE CampI Technical Assistance NEBs

                                                                                                    0 2 4 6 8 10 12

                                                                                                    m Product losses

                                                                                                    l Ease of selling leasing building

                                                                                                    n Labor requirements

                                                                                                    h Aesthetics appearance

                                                                                                    j Noise

                                                                                                    b Equipment maintenance

                                                                                                    f Tenant satisfaction

                                                                                                    d Equipment lifetimes

                                                                                                    e Productivity

                                                                                                    k Building safety[2]

                                                                                                    g Comfort

                                                                                                    i Lighting Quality of Light

                                                                                                    c Equipment performance[1]

                                                                                                    a Operating costs

                                                                                                    o Doing good for the Environment[3]

                                                                                                    Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip

                                                                                                    vcv

                                                                                                    Source SERA Research

                                                                                                    Chart1

                                                                                                    CampI Technical Assistance NEBs
                                                                                                    002
                                                                                                    003
                                                                                                    004
                                                                                                    005
                                                                                                    005
                                                                                                    006
                                                                                                    006
                                                                                                    007
                                                                                                    007
                                                                                                    007
                                                                                                    009
                                                                                                    009
                                                                                                    01
                                                                                                    011
                                                                                                    011

                                                                                                    Sheet1

                                                                                                    Sheet1

                                                                                                    CampI Technical Assistance Program NEBs (Participant)

                                                                                                    Sheet2

                                                                                                    Sheet3

                                                                                                    SERA

                                                                                                    EXPRESSING NEBS VALUEndashCxe

                                                                                                    Strong value from RetroCxSource SERA Research

                                                                                                    SERA

                                                                                                    SOCIETAL NEB EXAMPLES

                                                                                                    31

                                                                                                    SERA

                                                                                                    JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                                    32

                                                                                                    0

                                                                                                    02

                                                                                                    04

                                                                                                    06

                                                                                                    08

                                                                                                    1

                                                                                                    12

                                                                                                    HPWxRetrof Appliance

                                                                                                    CA

                                                                                                    WI

                                                                                                    Natl

                                                                                                    (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                                    Jobs Economic

                                                                                                    SERA

                                                                                                    EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                                    33

                                                                                                    SERA

                                                                                                    UTILITY NEBSUtility NEBs for Template Program

                                                                                                    Debt WriteOff (util)13

                                                                                                    Rate Subsidy(util)61

                                                                                                    HealthSafety(util)0

                                                                                                    Colln Costs (util)0

                                                                                                    Gas Calls (util)0

                                                                                                    Calls to CSRs(util)2

                                                                                                    TampD (util)16

                                                                                                    Arrears (util)0

                                                                                                    Reconnects (util)0

                                                                                                    Notices (util)7

                                                                                                    Shutoffs (util)1

                                                                                                    Rate subsidy TampD

                                                                                                    Payment-related

                                                                                                    Source Skumatz Economic Research Associates research

                                                                                                    Example Low Income Weatherization

                                                                                                    SERA

                                                                                                    ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                                    be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                                    Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                                    Value of 100 HALVES Payback doubles ROI

                                                                                                    Source Skumatz Economics (SERA)

                                                                                                    35

                                                                                                    ( for NEBs Multiplier of 125)

                                                                                                    SERA

                                                                                                    NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                                    36

                                                                                                    hellip2001 ON

                                                                                                    SERA

                                                                                                    37

                                                                                                    NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                    For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                    Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                    BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                    SERA

                                                                                                    CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                    38

                                                                                                    Test Benefits Costs States Using Traditionally

                                                                                                    Improved treat-ment with NEBs

                                                                                                    Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                    bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                    bull Avoided gas and water supply costs

                                                                                                    bull Program administration

                                                                                                    bull Participant incentives

                                                                                                    bull Increased supply cost

                                                                                                    CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                    Use cost only paid by the utility

                                                                                                    Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                    Same as above plus bull increased revenue

                                                                                                    Same as above plus bull Decreased revenue

                                                                                                    AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                    Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                    bull Participant direct costs

                                                                                                    AR CA FL HI IA IN MI MN NY VA

                                                                                                    Participant NEBs

                                                                                                    Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                    Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                    SERA

                                                                                                    APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                    Expansion Concern with risk calculation accuracy

                                                                                                    39

                                                                                                    Source Skumatz SERA research

                                                                                                    SERA

                                                                                                    40

                                                                                                    METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                    Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                    Minimize Regulatory amp Implementer Risk

                                                                                                    Minimize Evaluation Cost

                                                                                                    Adder

                                                                                                    Readily MeasurableHybrid

                                                                                                    All NEBsSource SERA Research

                                                                                                    SERA

                                                                                                    41

                                                                                                    STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                    Screening Application Utilities regions

                                                                                                    Program Marketing Fairly widespread use in utilities states across the country

                                                                                                    MO

                                                                                                    RE AGGRESSIVE ==gt

                                                                                                    Test Pgm Screen - adder

                                                                                                    IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                    Test Pgm Screen - readily measured

                                                                                                    MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                    Test Hybrid (potential adder amp measured)

                                                                                                    CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                    Test Pgm screen - Broad

                                                                                                    With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                    Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                    SERA

                                                                                                    42

                                                                                                    Key

                                                                                                    States with NEBs in CE tests for At least one type of program

                                                                                                    copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                    STATES WITH NEBS IN CE TESTS

                                                                                                    NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                    groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                    SERA

                                                                                                    SELECTED STATE PROGRESS CA

                                                                                                    2001 ndash LIPPT model NY

                                                                                                    Mid-2000s measurement amp scenarios not included CO

                                                                                                    Adders (10 electric 25 LI 5 gas) VT

                                                                                                    15 adder helped by previous research DC

                                                                                                    10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                    43

                                                                                                    Source Skumatz SERA research

                                                                                                    SERA

                                                                                                    SELECTED STATE PROGRESS Dominos ongoing

                                                                                                    Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                    Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                    Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                    Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                    44

                                                                                                    Source Skumatz SERA research

                                                                                                    SERA

                                                                                                    APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                    Expansion Concern with risk calculation accuracy

                                                                                                    45

                                                                                                    Source Skumatz SERA research

                                                                                                    SERA

                                                                                                    46

                                                                                                    NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                    [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                    SourceSkumatz SERA Independent research

                                                                                                    NTG ndash accuracy measurement incomplete

                                                                                                    Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                    NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                    EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                    Cost Complicated expensive local changes

                                                                                                    RELATIVE Risk from NEBs

                                                                                                    Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                    Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                    NEBs as accurate as many inputs (all could improve)

                                                                                                    SERA

                                                                                                    DISCOUNT RATE MATTERS

                                                                                                    47

                                                                                                    Source Skumatz SERA Research

                                                                                                    0

                                                                                                    20

                                                                                                    40

                                                                                                    60

                                                                                                    80

                                                                                                    100

                                                                                                    120

                                                                                                    1 2 3 4 5 6 7 8 9 10

                                                                                                    PV Ratios vs 1 Discount Rate

                                                                                                    10 yr PV 20 yr PV

                                                                                                    Discount rate - percent

                                                                                                    WACCSocialLTT

                                                                                                    Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                    SERA

                                                                                                    EUL VARIATIONS-ADOPTED VALUES

                                                                                                    Source Skumatz Economic Research Associates SERA independent research

                                                                                                    0

                                                                                                    50

                                                                                                    100

                                                                                                    150

                                                                                                    200

                                                                                                    250

                                                                                                    300

                                                                                                    350

                                                                                                    400

                                                                                                    450

                                                                                                    0 10 20 30 40 50 60 70

                                                                                                    EUL Variation by Measure

                                                                                                    67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                    Poorly supportedOldTechnology change

                                                                                                    SERA

                                                                                                    KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                    Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                    measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                    change the BC conclusion

                                                                                                    On the cusphellip

                                                                                                    If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                    IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                    Source SERA all rights reserved

                                                                                                    SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                    SERA

                                                                                                    IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                    Utility Soc Part Conserv Recrsquom

                                                                                                    Rationale

                                                                                                    Base Percent X X X Program-invariant

                                                                                                    Low Income X X X X Multiple sources

                                                                                                    Weatherization X X X Substantial Participantimpacts

                                                                                                    Measure Program-specific

                                                                                                    X Varies by measure sector

                                                                                                    Other Recomrsquos Local Research

                                                                                                    50

                                                                                                    Source Skumatz SERA

                                                                                                    Developing values for multiple states amp utilities

                                                                                                    SERA

                                                                                                    TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                    Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                    Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                    5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                    6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                    THANK YOU

                                                                                                    Questions

                                                                                                    Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                    SERA

                                                                                                    POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                    53

                                                                                                    SERA

                                                                                                    NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                    Negative NEB values cost of barrier

                                                                                                    Solar Water Heat NZ$ Euros

                                                                                                    Solar Design NZ$ Euros

                                                                                                    Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                    -23 -12 (079)

                                                                                                    -11 -6 (06)

                                                                                                    Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                    Source Skumatz Economic Research Associates research

                                                                                                    SERA

                                                                                                    CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                    NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                    -5-4-3-2-1012345

                                                                                                    Op Co

                                                                                                    st

                                                                                                    Init C

                                                                                                    ost

                                                                                                    Maint

                                                                                                    Perfo

                                                                                                    rm

                                                                                                    Prod

                                                                                                    uctiv

                                                                                                    Tena

                                                                                                    nt Sa

                                                                                                    t

                                                                                                    Comf

                                                                                                    ort

                                                                                                    Appe

                                                                                                    ar

                                                                                                    Light

                                                                                                    Qual

                                                                                                    SellL

                                                                                                    ease

                                                                                                    Devp Ownr AampE

                                                                                                    Note AgtE (~4 vs ~2) for these NEBs

                                                                                                    Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                    Source Skumatz Economic Research Associates Inc 2002

                                                                                                    Owners more positive ndashUnderinvestment train

                                                                                                    • Soup to nuts on non-energy benefits NEIs
                                                                                                    • TOPICS
                                                                                                    • BACKGROUND NEBS
                                                                                                    • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                    • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                    • KEY APPLICATIONS OF NEBS
                                                                                                    • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                    • Neb measurement
                                                                                                    • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                    • MEASURING PARTICIPANTNEBs
                                                                                                    • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                    • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                    • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                    • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                    • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                    • PROGRESS AND GAPS
                                                                                                    • PROGRESS AND GAPS
                                                                                                    • USING NEB NEI VALUES
                                                                                                    • Slide Number 19
                                                                                                    • ARE NEBS HIGH VALUE
                                                                                                    • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                    • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                    • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                    • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                    • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                    • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                    • AUDIT FINANCIAL PROGRAM
                                                                                                    • CampI NEW CONSTRUCTION
                                                                                                    • CampI TECHNICAL ASSISTANCE
                                                                                                    • EXPRESSING NEBS VALUEndashCxe
                                                                                                    • Slide Number 31
                                                                                                    • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                    • EMISSIONS (SOCIETAL NEBS)
                                                                                                    • UTILITY NEBS
                                                                                                    • ADJUSTED PAYBACKS
                                                                                                    • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                    • NEBS IN BC
                                                                                                    • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                    • APPLICATION TO BC
                                                                                                    • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                    • STATE TREATMENT OF NEBS NEBs
                                                                                                    • Slide Number 42
                                                                                                    • SELECTED STATE PROGRESS
                                                                                                    • SELECTED STATE PROGRESS
                                                                                                    • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                    • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                    • DISCOUNT RATE MATTERS
                                                                                                    • EUL VARIATIONS- ADOPTED VALUES
                                                                                                    • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                    • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                    • TAKEAWAYSON NEBSNEIS
                                                                                                    • THANK YOUQuestions
                                                                                                    • Slide Number 53
                                                                                                    • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                    • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                                                      Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                                                      m Product losses20
                                                                                                      l Ease of selling leasing building33
                                                                                                      n Labor requirements45
                                                                                                      h Aesthetics appearance59
                                                                                                      j Noise50
                                                                                                      b Equipment maintenance660
                                                                                                      f Tenant satisfaction60
                                                                                                      d Equipment lifetimes718
                                                                                                      e Productivity724
                                                                                                      k Building safety[2]73
                                                                                                      g Comfort933
                                                                                                      i Lighting Quality of Light916
                                                                                                      c Equipment performance[1]1019
                                                                                                      a Operating costs1165
                                                                                                      o Doing good for the Environment[3]1132
                                                                                                      Other03
                                                                                                      [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                                                      [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                                                      [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                                                      m Product losses
                                                                                                      l Ease of selling leasing building
                                                                                                      n Labor requirements
                                                                                                      h Aesthetics appearance
                                                                                                      j Noise
                                                                                                      b Equipment maintenance
                                                                                                      f Tenant satisfaction
                                                                                                      d Equipment lifetimes
                                                                                                      e Productivity
                                                                                                      k Building safety[2]
                                                                                                      g Comfort
                                                                                                      i Lighting Quality of Light
                                                                                                      c Equipment performance[1]
                                                                                                      a Operating costs
                                                                                                      o Doing good for the Environment[3]

                                                                                                      Chart1

                                                                                                      CampI Technical Assistance NEBs
                                                                                                      002
                                                                                                      003
                                                                                                      004
                                                                                                      005
                                                                                                      005
                                                                                                      006
                                                                                                      006
                                                                                                      007
                                                                                                      007
                                                                                                      007
                                                                                                      009
                                                                                                      009
                                                                                                      01
                                                                                                      011
                                                                                                      011

                                                                                                      Sheet1

                                                                                                      Sheet1

                                                                                                      CampI Technical Assistance Program NEBs (Participant)

                                                                                                      Sheet2

                                                                                                      Sheet3

                                                                                                      SERA

                                                                                                      EXPRESSING NEBS VALUEndashCxe

                                                                                                      Strong value from RetroCxSource SERA Research

                                                                                                      SERA

                                                                                                      SOCIETAL NEB EXAMPLES

                                                                                                      31

                                                                                                      SERA

                                                                                                      JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                                      32

                                                                                                      0

                                                                                                      02

                                                                                                      04

                                                                                                      06

                                                                                                      08

                                                                                                      1

                                                                                                      12

                                                                                                      HPWxRetrof Appliance

                                                                                                      CA

                                                                                                      WI

                                                                                                      Natl

                                                                                                      (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                                      Jobs Economic

                                                                                                      SERA

                                                                                                      EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                                      33

                                                                                                      SERA

                                                                                                      UTILITY NEBSUtility NEBs for Template Program

                                                                                                      Debt WriteOff (util)13

                                                                                                      Rate Subsidy(util)61

                                                                                                      HealthSafety(util)0

                                                                                                      Colln Costs (util)0

                                                                                                      Gas Calls (util)0

                                                                                                      Calls to CSRs(util)2

                                                                                                      TampD (util)16

                                                                                                      Arrears (util)0

                                                                                                      Reconnects (util)0

                                                                                                      Notices (util)7

                                                                                                      Shutoffs (util)1

                                                                                                      Rate subsidy TampD

                                                                                                      Payment-related

                                                                                                      Source Skumatz Economic Research Associates research

                                                                                                      Example Low Income Weatherization

                                                                                                      SERA

                                                                                                      ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                                      be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                                      Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                                      Value of 100 HALVES Payback doubles ROI

                                                                                                      Source Skumatz Economics (SERA)

                                                                                                      35

                                                                                                      ( for NEBs Multiplier of 125)

                                                                                                      SERA

                                                                                                      NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                                      36

                                                                                                      hellip2001 ON

                                                                                                      SERA

                                                                                                      37

                                                                                                      NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                      For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                      Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                      BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                      SERA

                                                                                                      CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                      38

                                                                                                      Test Benefits Costs States Using Traditionally

                                                                                                      Improved treat-ment with NEBs

                                                                                                      Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                      bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                      bull Avoided gas and water supply costs

                                                                                                      bull Program administration

                                                                                                      bull Participant incentives

                                                                                                      bull Increased supply cost

                                                                                                      CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                      Use cost only paid by the utility

                                                                                                      Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                      Same as above plus bull increased revenue

                                                                                                      Same as above plus bull Decreased revenue

                                                                                                      AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                      Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                      bull Participant direct costs

                                                                                                      AR CA FL HI IA IN MI MN NY VA

                                                                                                      Participant NEBs

                                                                                                      Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                      Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                      SERA

                                                                                                      APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                      Expansion Concern with risk calculation accuracy

                                                                                                      39

                                                                                                      Source Skumatz SERA research

                                                                                                      SERA

                                                                                                      40

                                                                                                      METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                      Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                      Minimize Regulatory amp Implementer Risk

                                                                                                      Minimize Evaluation Cost

                                                                                                      Adder

                                                                                                      Readily MeasurableHybrid

                                                                                                      All NEBsSource SERA Research

                                                                                                      SERA

                                                                                                      41

                                                                                                      STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                      Screening Application Utilities regions

                                                                                                      Program Marketing Fairly widespread use in utilities states across the country

                                                                                                      MO

                                                                                                      RE AGGRESSIVE ==gt

                                                                                                      Test Pgm Screen - adder

                                                                                                      IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                      Test Pgm Screen - readily measured

                                                                                                      MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                      Test Hybrid (potential adder amp measured)

                                                                                                      CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                      Test Pgm screen - Broad

                                                                                                      With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                      Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                      SERA

                                                                                                      42

                                                                                                      Key

                                                                                                      States with NEBs in CE tests for At least one type of program

                                                                                                      copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                      STATES WITH NEBS IN CE TESTS

                                                                                                      NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                      groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                      SERA

                                                                                                      SELECTED STATE PROGRESS CA

                                                                                                      2001 ndash LIPPT model NY

                                                                                                      Mid-2000s measurement amp scenarios not included CO

                                                                                                      Adders (10 electric 25 LI 5 gas) VT

                                                                                                      15 adder helped by previous research DC

                                                                                                      10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                      43

                                                                                                      Source Skumatz SERA research

                                                                                                      SERA

                                                                                                      SELECTED STATE PROGRESS Dominos ongoing

                                                                                                      Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                      Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                      Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                      Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                      44

                                                                                                      Source Skumatz SERA research

                                                                                                      SERA

                                                                                                      APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                      Expansion Concern with risk calculation accuracy

                                                                                                      45

                                                                                                      Source Skumatz SERA research

                                                                                                      SERA

                                                                                                      46

                                                                                                      NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                      [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                      SourceSkumatz SERA Independent research

                                                                                                      NTG ndash accuracy measurement incomplete

                                                                                                      Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                      NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                      EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                      Cost Complicated expensive local changes

                                                                                                      RELATIVE Risk from NEBs

                                                                                                      Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                      Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                      NEBs as accurate as many inputs (all could improve)

                                                                                                      SERA

                                                                                                      DISCOUNT RATE MATTERS

                                                                                                      47

                                                                                                      Source Skumatz SERA Research

                                                                                                      0

                                                                                                      20

                                                                                                      40

                                                                                                      60

                                                                                                      80

                                                                                                      100

                                                                                                      120

                                                                                                      1 2 3 4 5 6 7 8 9 10

                                                                                                      PV Ratios vs 1 Discount Rate

                                                                                                      10 yr PV 20 yr PV

                                                                                                      Discount rate - percent

                                                                                                      WACCSocialLTT

                                                                                                      Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                      SERA

                                                                                                      EUL VARIATIONS-ADOPTED VALUES

                                                                                                      Source Skumatz Economic Research Associates SERA independent research

                                                                                                      0

                                                                                                      50

                                                                                                      100

                                                                                                      150

                                                                                                      200

                                                                                                      250

                                                                                                      300

                                                                                                      350

                                                                                                      400

                                                                                                      450

                                                                                                      0 10 20 30 40 50 60 70

                                                                                                      EUL Variation by Measure

                                                                                                      67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                      Poorly supportedOldTechnology change

                                                                                                      SERA

                                                                                                      KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                      Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                      measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                      change the BC conclusion

                                                                                                      On the cusphellip

                                                                                                      If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                      IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                      Source SERA all rights reserved

                                                                                                      SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                      SERA

                                                                                                      IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                      Utility Soc Part Conserv Recrsquom

                                                                                                      Rationale

                                                                                                      Base Percent X X X Program-invariant

                                                                                                      Low Income X X X X Multiple sources

                                                                                                      Weatherization X X X Substantial Participantimpacts

                                                                                                      Measure Program-specific

                                                                                                      X Varies by measure sector

                                                                                                      Other Recomrsquos Local Research

                                                                                                      50

                                                                                                      Source Skumatz SERA

                                                                                                      Developing values for multiple states amp utilities

                                                                                                      SERA

                                                                                                      TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                      Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                      Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                      5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                      6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                      THANK YOU

                                                                                                      Questions

                                                                                                      Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                      SERA

                                                                                                      POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                      53

                                                                                                      SERA

                                                                                                      NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                      Negative NEB values cost of barrier

                                                                                                      Solar Water Heat NZ$ Euros

                                                                                                      Solar Design NZ$ Euros

                                                                                                      Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                      -23 -12 (079)

                                                                                                      -11 -6 (06)

                                                                                                      Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                      Source Skumatz Economic Research Associates research

                                                                                                      SERA

                                                                                                      CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                      NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                      -5-4-3-2-1012345

                                                                                                      Op Co

                                                                                                      st

                                                                                                      Init C

                                                                                                      ost

                                                                                                      Maint

                                                                                                      Perfo

                                                                                                      rm

                                                                                                      Prod

                                                                                                      uctiv

                                                                                                      Tena

                                                                                                      nt Sa

                                                                                                      t

                                                                                                      Comf

                                                                                                      ort

                                                                                                      Appe

                                                                                                      ar

                                                                                                      Light

                                                                                                      Qual

                                                                                                      SellL

                                                                                                      ease

                                                                                                      Devp Ownr AampE

                                                                                                      Note AgtE (~4 vs ~2) for these NEBs

                                                                                                      Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                      Source Skumatz Economic Research Associates Inc 2002

                                                                                                      Owners more positive ndashUnderinvestment train

                                                                                                      • Soup to nuts on non-energy benefits NEIs
                                                                                                      • TOPICS
                                                                                                      • BACKGROUND NEBS
                                                                                                      • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                      • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                      • KEY APPLICATIONS OF NEBS
                                                                                                      • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                      • Neb measurement
                                                                                                      • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                      • MEASURING PARTICIPANTNEBs
                                                                                                      • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                      • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                      • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                      • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                      • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                      • PROGRESS AND GAPS
                                                                                                      • PROGRESS AND GAPS
                                                                                                      • USING NEB NEI VALUES
                                                                                                      • Slide Number 19
                                                                                                      • ARE NEBS HIGH VALUE
                                                                                                      • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                      • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                      • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                      • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                      • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                      • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                      • AUDIT FINANCIAL PROGRAM
                                                                                                      • CampI NEW CONSTRUCTION
                                                                                                      • CampI TECHNICAL ASSISTANCE
                                                                                                      • EXPRESSING NEBS VALUEndashCxe
                                                                                                      • Slide Number 31
                                                                                                      • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                      • EMISSIONS (SOCIETAL NEBS)
                                                                                                      • UTILITY NEBS
                                                                                                      • ADJUSTED PAYBACKS
                                                                                                      • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                      • NEBS IN BC
                                                                                                      • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                      • APPLICATION TO BC
                                                                                                      • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                      • STATE TREATMENT OF NEBS NEBs
                                                                                                      • Slide Number 42
                                                                                                      • SELECTED STATE PROGRESS
                                                                                                      • SELECTED STATE PROGRESS
                                                                                                      • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                      • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                      • DISCOUNT RATE MATTERS
                                                                                                      • EUL VARIATIONS- ADOPTED VALUES
                                                                                                      • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                      • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                      • TAKEAWAYSON NEBSNEIS
                                                                                                      • THANK YOUQuestions
                                                                                                      • Slide Number 53
                                                                                                      • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                      • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                                                        Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                                                        m Product losses20
                                                                                                        l Ease of selling leasing building33
                                                                                                        n Labor requirements45
                                                                                                        h Aesthetics appearance59
                                                                                                        j Noise50
                                                                                                        b Equipment maintenance660
                                                                                                        f Tenant satisfaction60
                                                                                                        d Equipment lifetimes718
                                                                                                        e Productivity724
                                                                                                        k Building safety[2]73
                                                                                                        g Comfort933
                                                                                                        i Lighting Quality of Light916
                                                                                                        c Equipment performance[1]1019
                                                                                                        a Operating costs1165
                                                                                                        o Doing good for the Environment[3]1132
                                                                                                        Other03
                                                                                                        [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                                                        [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                                                        [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
                                                                                                        m Product losses
                                                                                                        l Ease of selling leasing building
                                                                                                        n Labor requirements
                                                                                                        h Aesthetics appearance
                                                                                                        j Noise
                                                                                                        b Equipment maintenance
                                                                                                        f Tenant satisfaction
                                                                                                        d Equipment lifetimes
                                                                                                        e Productivity
                                                                                                        k Building safety[2]
                                                                                                        g Comfort
                                                                                                        i Lighting Quality of Light
                                                                                                        c Equipment performance[1]
                                                                                                        a Operating costs
                                                                                                        o Doing good for the Environment[3]

                                                                                                        Sheet1

                                                                                                        Sheet1

                                                                                                        CampI Technical Assistance Program NEBs (Participant)

                                                                                                        Sheet2

                                                                                                        Sheet3

                                                                                                        SERA

                                                                                                        EXPRESSING NEBS VALUEndashCxe

                                                                                                        Strong value from RetroCxSource SERA Research

                                                                                                        SERA

                                                                                                        SOCIETAL NEB EXAMPLES

                                                                                                        31

                                                                                                        SERA

                                                                                                        JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                                        32

                                                                                                        0

                                                                                                        02

                                                                                                        04

                                                                                                        06

                                                                                                        08

                                                                                                        1

                                                                                                        12

                                                                                                        HPWxRetrof Appliance

                                                                                                        CA

                                                                                                        WI

                                                                                                        Natl

                                                                                                        (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                                        Jobs Economic

                                                                                                        SERA

                                                                                                        EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                                        33

                                                                                                        SERA

                                                                                                        UTILITY NEBSUtility NEBs for Template Program

                                                                                                        Debt WriteOff (util)13

                                                                                                        Rate Subsidy(util)61

                                                                                                        HealthSafety(util)0

                                                                                                        Colln Costs (util)0

                                                                                                        Gas Calls (util)0

                                                                                                        Calls to CSRs(util)2

                                                                                                        TampD (util)16

                                                                                                        Arrears (util)0

                                                                                                        Reconnects (util)0

                                                                                                        Notices (util)7

                                                                                                        Shutoffs (util)1

                                                                                                        Rate subsidy TampD

                                                                                                        Payment-related

                                                                                                        Source Skumatz Economic Research Associates research

                                                                                                        Example Low Income Weatherization

                                                                                                        SERA

                                                                                                        ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                                        be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                                        Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                                        Value of 100 HALVES Payback doubles ROI

                                                                                                        Source Skumatz Economics (SERA)

                                                                                                        35

                                                                                                        ( for NEBs Multiplier of 125)

                                                                                                        SERA

                                                                                                        NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                                        36

                                                                                                        hellip2001 ON

                                                                                                        SERA

                                                                                                        37

                                                                                                        NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                        For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                        Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                        BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                        SERA

                                                                                                        CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                        38

                                                                                                        Test Benefits Costs States Using Traditionally

                                                                                                        Improved treat-ment with NEBs

                                                                                                        Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                        bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                        bull Avoided gas and water supply costs

                                                                                                        bull Program administration

                                                                                                        bull Participant incentives

                                                                                                        bull Increased supply cost

                                                                                                        CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                        Use cost only paid by the utility

                                                                                                        Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                        Same as above plus bull increased revenue

                                                                                                        Same as above plus bull Decreased revenue

                                                                                                        AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                        Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                        bull Participant direct costs

                                                                                                        AR CA FL HI IA IN MI MN NY VA

                                                                                                        Participant NEBs

                                                                                                        Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                        Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                        SERA

                                                                                                        APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                        Expansion Concern with risk calculation accuracy

                                                                                                        39

                                                                                                        Source Skumatz SERA research

                                                                                                        SERA

                                                                                                        40

                                                                                                        METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                        Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                        Minimize Regulatory amp Implementer Risk

                                                                                                        Minimize Evaluation Cost

                                                                                                        Adder

                                                                                                        Readily MeasurableHybrid

                                                                                                        All NEBsSource SERA Research

                                                                                                        SERA

                                                                                                        41

                                                                                                        STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                        Screening Application Utilities regions

                                                                                                        Program Marketing Fairly widespread use in utilities states across the country

                                                                                                        MO

                                                                                                        RE AGGRESSIVE ==gt

                                                                                                        Test Pgm Screen - adder

                                                                                                        IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                        Test Pgm Screen - readily measured

                                                                                                        MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                        Test Hybrid (potential adder amp measured)

                                                                                                        CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                        Test Pgm screen - Broad

                                                                                                        With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                        Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                        SERA

                                                                                                        42

                                                                                                        Key

                                                                                                        States with NEBs in CE tests for At least one type of program

                                                                                                        copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                        STATES WITH NEBS IN CE TESTS

                                                                                                        NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                        groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                        SERA

                                                                                                        SELECTED STATE PROGRESS CA

                                                                                                        2001 ndash LIPPT model NY

                                                                                                        Mid-2000s measurement amp scenarios not included CO

                                                                                                        Adders (10 electric 25 LI 5 gas) VT

                                                                                                        15 adder helped by previous research DC

                                                                                                        10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                        43

                                                                                                        Source Skumatz SERA research

                                                                                                        SERA

                                                                                                        SELECTED STATE PROGRESS Dominos ongoing

                                                                                                        Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                        Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                        Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                        Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                        44

                                                                                                        Source Skumatz SERA research

                                                                                                        SERA

                                                                                                        APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                        Expansion Concern with risk calculation accuracy

                                                                                                        45

                                                                                                        Source Skumatz SERA research

                                                                                                        SERA

                                                                                                        46

                                                                                                        NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                        [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                        SourceSkumatz SERA Independent research

                                                                                                        NTG ndash accuracy measurement incomplete

                                                                                                        Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                        NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                        EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                        Cost Complicated expensive local changes

                                                                                                        RELATIVE Risk from NEBs

                                                                                                        Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                        Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                        NEBs as accurate as many inputs (all could improve)

                                                                                                        SERA

                                                                                                        DISCOUNT RATE MATTERS

                                                                                                        47

                                                                                                        Source Skumatz SERA Research

                                                                                                        0

                                                                                                        20

                                                                                                        40

                                                                                                        60

                                                                                                        80

                                                                                                        100

                                                                                                        120

                                                                                                        1 2 3 4 5 6 7 8 9 10

                                                                                                        PV Ratios vs 1 Discount Rate

                                                                                                        10 yr PV 20 yr PV

                                                                                                        Discount rate - percent

                                                                                                        WACCSocialLTT

                                                                                                        Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                        SERA

                                                                                                        EUL VARIATIONS-ADOPTED VALUES

                                                                                                        Source Skumatz Economic Research Associates SERA independent research

                                                                                                        0

                                                                                                        50

                                                                                                        100

                                                                                                        150

                                                                                                        200

                                                                                                        250

                                                                                                        300

                                                                                                        350

                                                                                                        400

                                                                                                        450

                                                                                                        0 10 20 30 40 50 60 70

                                                                                                        EUL Variation by Measure

                                                                                                        67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                        Poorly supportedOldTechnology change

                                                                                                        SERA

                                                                                                        KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                        Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                        measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                        change the BC conclusion

                                                                                                        On the cusphellip

                                                                                                        If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                        IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                        Source SERA all rights reserved

                                                                                                        SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                        SERA

                                                                                                        IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                        Utility Soc Part Conserv Recrsquom

                                                                                                        Rationale

                                                                                                        Base Percent X X X Program-invariant

                                                                                                        Low Income X X X X Multiple sources

                                                                                                        Weatherization X X X Substantial Participantimpacts

                                                                                                        Measure Program-specific

                                                                                                        X Varies by measure sector

                                                                                                        Other Recomrsquos Local Research

                                                                                                        50

                                                                                                        Source Skumatz SERA

                                                                                                        Developing values for multiple states amp utilities

                                                                                                        SERA

                                                                                                        TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                        Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                        Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                        5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                        6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                        THANK YOU

                                                                                                        Questions

                                                                                                        Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                        SERA

                                                                                                        POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                        53

                                                                                                        SERA

                                                                                                        NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                        Negative NEB values cost of barrier

                                                                                                        Solar Water Heat NZ$ Euros

                                                                                                        Solar Design NZ$ Euros

                                                                                                        Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                        -23 -12 (079)

                                                                                                        -11 -6 (06)

                                                                                                        Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                        Source Skumatz Economic Research Associates research

                                                                                                        SERA

                                                                                                        CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                        NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                        -5-4-3-2-1012345

                                                                                                        Op Co

                                                                                                        st

                                                                                                        Init C

                                                                                                        ost

                                                                                                        Maint

                                                                                                        Perfo

                                                                                                        rm

                                                                                                        Prod

                                                                                                        uctiv

                                                                                                        Tena

                                                                                                        nt Sa

                                                                                                        t

                                                                                                        Comf

                                                                                                        ort

                                                                                                        Appe

                                                                                                        ar

                                                                                                        Light

                                                                                                        Qual

                                                                                                        SellL

                                                                                                        ease

                                                                                                        Devp Ownr AampE

                                                                                                        Note AgtE (~4 vs ~2) for these NEBs

                                                                                                        Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                        Source Skumatz Economic Research Associates Inc 2002

                                                                                                        Owners more positive ndashUnderinvestment train

                                                                                                        • Soup to nuts on non-energy benefits NEIs
                                                                                                        • TOPICS
                                                                                                        • BACKGROUND NEBS
                                                                                                        • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                        • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                        • KEY APPLICATIONS OF NEBS
                                                                                                        • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                        • Neb measurement
                                                                                                        • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                        • MEASURING PARTICIPANTNEBs
                                                                                                        • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                        • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                        • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                        • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                        • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                        • PROGRESS AND GAPS
                                                                                                        • PROGRESS AND GAPS
                                                                                                        • USING NEB NEI VALUES
                                                                                                        • Slide Number 19
                                                                                                        • ARE NEBS HIGH VALUE
                                                                                                        • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                        • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                        • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                        • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                        • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                        • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                        • AUDIT FINANCIAL PROGRAM
                                                                                                        • CampI NEW CONSTRUCTION
                                                                                                        • CampI TECHNICAL ASSISTANCE
                                                                                                        • EXPRESSING NEBS VALUEndashCxe
                                                                                                        • Slide Number 31
                                                                                                        • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                        • EMISSIONS (SOCIETAL NEBS)
                                                                                                        • UTILITY NEBS
                                                                                                        • ADJUSTED PAYBACKS
                                                                                                        • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                        • NEBS IN BC
                                                                                                        • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                        • APPLICATION TO BC
                                                                                                        • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                        • STATE TREATMENT OF NEBS NEBs
                                                                                                        • Slide Number 42
                                                                                                        • SELECTED STATE PROGRESS
                                                                                                        • SELECTED STATE PROGRESS
                                                                                                        • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                        • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                        • DISCOUNT RATE MATTERS
                                                                                                        • EUL VARIATIONS- ADOPTED VALUES
                                                                                                        • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                        • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                        • TAKEAWAYSON NEBSNEIS
                                                                                                        • THANK YOUQuestions
                                                                                                        • Slide Number 53
                                                                                                        • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                        • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
                                                                                                          Percent of NEBs by category per participating buildingPercent Ranking this NEB in the Top Three
                                                                                                          m Product losses20
                                                                                                          l Ease of selling leasing building33
                                                                                                          n Labor requirements45
                                                                                                          h Aesthetics appearance59
                                                                                                          j Noise50
                                                                                                          b Equipment maintenance660
                                                                                                          f Tenant satisfaction60
                                                                                                          d Equipment lifetimes718
                                                                                                          e Productivity724
                                                                                                          k Building safety[2]73
                                                                                                          g Comfort933
                                                                                                          i Lighting Quality of Light916
                                                                                                          c Equipment performance[1]1019
                                                                                                          a Operating costs1165
                                                                                                          o Doing good for the Environment[3]1132
                                                                                                          Other03
                                                                                                          [1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
                                                                                                          [2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
                                                                                                          [3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment

                                                                                                          Sheet1

                                                                                                          CampI Technical Assistance Program NEBs (Participant)

                                                                                                          Sheet2

                                                                                                          Sheet3

                                                                                                          SERA

                                                                                                          EXPRESSING NEBS VALUEndashCxe

                                                                                                          Strong value from RetroCxSource SERA Research

                                                                                                          SERA

                                                                                                          SOCIETAL NEB EXAMPLES

                                                                                                          31

                                                                                                          SERA

                                                                                                          JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                                          32

                                                                                                          0

                                                                                                          02

                                                                                                          04

                                                                                                          06

                                                                                                          08

                                                                                                          1

                                                                                                          12

                                                                                                          HPWxRetrof Appliance

                                                                                                          CA

                                                                                                          WI

                                                                                                          Natl

                                                                                                          (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                                          Jobs Economic

                                                                                                          SERA

                                                                                                          EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                                          33

                                                                                                          SERA

                                                                                                          UTILITY NEBSUtility NEBs for Template Program

                                                                                                          Debt WriteOff (util)13

                                                                                                          Rate Subsidy(util)61

                                                                                                          HealthSafety(util)0

                                                                                                          Colln Costs (util)0

                                                                                                          Gas Calls (util)0

                                                                                                          Calls to CSRs(util)2

                                                                                                          TampD (util)16

                                                                                                          Arrears (util)0

                                                                                                          Reconnects (util)0

                                                                                                          Notices (util)7

                                                                                                          Shutoffs (util)1

                                                                                                          Rate subsidy TampD

                                                                                                          Payment-related

                                                                                                          Source Skumatz Economic Research Associates research

                                                                                                          Example Low Income Weatherization

                                                                                                          SERA

                                                                                                          ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                                          be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                                          Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                                          Value of 100 HALVES Payback doubles ROI

                                                                                                          Source Skumatz Economics (SERA)

                                                                                                          35

                                                                                                          ( for NEBs Multiplier of 125)

                                                                                                          SERA

                                                                                                          NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                                          36

                                                                                                          hellip2001 ON

                                                                                                          SERA

                                                                                                          37

                                                                                                          NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                          For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                          Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                          BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                          SERA

                                                                                                          CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                          38

                                                                                                          Test Benefits Costs States Using Traditionally

                                                                                                          Improved treat-ment with NEBs

                                                                                                          Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                          bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                          bull Avoided gas and water supply costs

                                                                                                          bull Program administration

                                                                                                          bull Participant incentives

                                                                                                          bull Increased supply cost

                                                                                                          CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                          Use cost only paid by the utility

                                                                                                          Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                          Same as above plus bull increased revenue

                                                                                                          Same as above plus bull Decreased revenue

                                                                                                          AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                          Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                          bull Participant direct costs

                                                                                                          AR CA FL HI IA IN MI MN NY VA

                                                                                                          Participant NEBs

                                                                                                          Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                          Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                          SERA

                                                                                                          APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                          Expansion Concern with risk calculation accuracy

                                                                                                          39

                                                                                                          Source Skumatz SERA research

                                                                                                          SERA

                                                                                                          40

                                                                                                          METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                          Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                          Minimize Regulatory amp Implementer Risk

                                                                                                          Minimize Evaluation Cost

                                                                                                          Adder

                                                                                                          Readily MeasurableHybrid

                                                                                                          All NEBsSource SERA Research

                                                                                                          SERA

                                                                                                          41

                                                                                                          STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                          Screening Application Utilities regions

                                                                                                          Program Marketing Fairly widespread use in utilities states across the country

                                                                                                          MO

                                                                                                          RE AGGRESSIVE ==gt

                                                                                                          Test Pgm Screen - adder

                                                                                                          IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                          Test Pgm Screen - readily measured

                                                                                                          MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                          Test Hybrid (potential adder amp measured)

                                                                                                          CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                          Test Pgm screen - Broad

                                                                                                          With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                          Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                          SERA

                                                                                                          42

                                                                                                          Key

                                                                                                          States with NEBs in CE tests for At least one type of program

                                                                                                          copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                          STATES WITH NEBS IN CE TESTS

                                                                                                          NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                          groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                          SERA

                                                                                                          SELECTED STATE PROGRESS CA

                                                                                                          2001 ndash LIPPT model NY

                                                                                                          Mid-2000s measurement amp scenarios not included CO

                                                                                                          Adders (10 electric 25 LI 5 gas) VT

                                                                                                          15 adder helped by previous research DC

                                                                                                          10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                          43

                                                                                                          Source Skumatz SERA research

                                                                                                          SERA

                                                                                                          SELECTED STATE PROGRESS Dominos ongoing

                                                                                                          Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                          Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                          Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                          Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                          44

                                                                                                          Source Skumatz SERA research

                                                                                                          SERA

                                                                                                          APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                          Expansion Concern with risk calculation accuracy

                                                                                                          45

                                                                                                          Source Skumatz SERA research

                                                                                                          SERA

                                                                                                          46

                                                                                                          NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                          [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                          SourceSkumatz SERA Independent research

                                                                                                          NTG ndash accuracy measurement incomplete

                                                                                                          Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                          NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                          EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                          Cost Complicated expensive local changes

                                                                                                          RELATIVE Risk from NEBs

                                                                                                          Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                          Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                          NEBs as accurate as many inputs (all could improve)

                                                                                                          SERA

                                                                                                          DISCOUNT RATE MATTERS

                                                                                                          47

                                                                                                          Source Skumatz SERA Research

                                                                                                          0

                                                                                                          20

                                                                                                          40

                                                                                                          60

                                                                                                          80

                                                                                                          100

                                                                                                          120

                                                                                                          1 2 3 4 5 6 7 8 9 10

                                                                                                          PV Ratios vs 1 Discount Rate

                                                                                                          10 yr PV 20 yr PV

                                                                                                          Discount rate - percent

                                                                                                          WACCSocialLTT

                                                                                                          Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                          SERA

                                                                                                          EUL VARIATIONS-ADOPTED VALUES

                                                                                                          Source Skumatz Economic Research Associates SERA independent research

                                                                                                          0

                                                                                                          50

                                                                                                          100

                                                                                                          150

                                                                                                          200

                                                                                                          250

                                                                                                          300

                                                                                                          350

                                                                                                          400

                                                                                                          450

                                                                                                          0 10 20 30 40 50 60 70

                                                                                                          EUL Variation by Measure

                                                                                                          67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                          Poorly supportedOldTechnology change

                                                                                                          SERA

                                                                                                          KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                          Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                          measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                          change the BC conclusion

                                                                                                          On the cusphellip

                                                                                                          If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                          IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                          Source SERA all rights reserved

                                                                                                          SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                          SERA

                                                                                                          IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                          Utility Soc Part Conserv Recrsquom

                                                                                                          Rationale

                                                                                                          Base Percent X X X Program-invariant

                                                                                                          Low Income X X X X Multiple sources

                                                                                                          Weatherization X X X Substantial Participantimpacts

                                                                                                          Measure Program-specific

                                                                                                          X Varies by measure sector

                                                                                                          Other Recomrsquos Local Research

                                                                                                          50

                                                                                                          Source Skumatz SERA

                                                                                                          Developing values for multiple states amp utilities

                                                                                                          SERA

                                                                                                          TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                          Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                          Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                          5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                          6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                          THANK YOU

                                                                                                          Questions

                                                                                                          Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                          SERA

                                                                                                          POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                          53

                                                                                                          SERA

                                                                                                          NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                          Negative NEB values cost of barrier

                                                                                                          Solar Water Heat NZ$ Euros

                                                                                                          Solar Design NZ$ Euros

                                                                                                          Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                          -23 -12 (079)

                                                                                                          -11 -6 (06)

                                                                                                          Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                          Source Skumatz Economic Research Associates research

                                                                                                          SERA

                                                                                                          CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                          NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                          -5-4-3-2-1012345

                                                                                                          Op Co

                                                                                                          st

                                                                                                          Init C

                                                                                                          ost

                                                                                                          Maint

                                                                                                          Perfo

                                                                                                          rm

                                                                                                          Prod

                                                                                                          uctiv

                                                                                                          Tena

                                                                                                          nt Sa

                                                                                                          t

                                                                                                          Comf

                                                                                                          ort

                                                                                                          Appe

                                                                                                          ar

                                                                                                          Light

                                                                                                          Qual

                                                                                                          SellL

                                                                                                          ease

                                                                                                          Devp Ownr AampE

                                                                                                          Note AgtE (~4 vs ~2) for these NEBs

                                                                                                          Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                          Source Skumatz Economic Research Associates Inc 2002

                                                                                                          Owners more positive ndashUnderinvestment train

                                                                                                          • Soup to nuts on non-energy benefits NEIs
                                                                                                          • TOPICS
                                                                                                          • BACKGROUND NEBS
                                                                                                          • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                          • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                          • KEY APPLICATIONS OF NEBS
                                                                                                          • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                          • Neb measurement
                                                                                                          • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                          • MEASURING PARTICIPANTNEBs
                                                                                                          • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                          • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                          • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                          • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                          • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                          • PROGRESS AND GAPS
                                                                                                          • PROGRESS AND GAPS
                                                                                                          • USING NEB NEI VALUES
                                                                                                          • Slide Number 19
                                                                                                          • ARE NEBS HIGH VALUE
                                                                                                          • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                          • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                          • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                          • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                          • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                          • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                          • AUDIT FINANCIAL PROGRAM
                                                                                                          • CampI NEW CONSTRUCTION
                                                                                                          • CampI TECHNICAL ASSISTANCE
                                                                                                          • EXPRESSING NEBS VALUEndashCxe
                                                                                                          • Slide Number 31
                                                                                                          • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                          • EMISSIONS (SOCIETAL NEBS)
                                                                                                          • UTILITY NEBS
                                                                                                          • ADJUSTED PAYBACKS
                                                                                                          • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                          • NEBS IN BC
                                                                                                          • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                          • APPLICATION TO BC
                                                                                                          • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                          • STATE TREATMENT OF NEBS NEBs
                                                                                                          • Slide Number 42
                                                                                                          • SELECTED STATE PROGRESS
                                                                                                          • SELECTED STATE PROGRESS
                                                                                                          • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                          • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                          • DISCOUNT RATE MATTERS
                                                                                                          • EUL VARIATIONS- ADOPTED VALUES
                                                                                                          • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                          • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                          • TAKEAWAYSON NEBSNEIS
                                                                                                          • THANK YOUQuestions
                                                                                                          • Slide Number 53
                                                                                                          • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                          • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                            Sheet2

                                                                                                            Sheet3

                                                                                                            SERA

                                                                                                            EXPRESSING NEBS VALUEndashCxe

                                                                                                            Strong value from RetroCxSource SERA Research

                                                                                                            SERA

                                                                                                            SOCIETAL NEB EXAMPLES

                                                                                                            31

                                                                                                            SERA

                                                                                                            JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                                            32

                                                                                                            0

                                                                                                            02

                                                                                                            04

                                                                                                            06

                                                                                                            08

                                                                                                            1

                                                                                                            12

                                                                                                            HPWxRetrof Appliance

                                                                                                            CA

                                                                                                            WI

                                                                                                            Natl

                                                                                                            (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                                            Jobs Economic

                                                                                                            SERA

                                                                                                            EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                                            33

                                                                                                            SERA

                                                                                                            UTILITY NEBSUtility NEBs for Template Program

                                                                                                            Debt WriteOff (util)13

                                                                                                            Rate Subsidy(util)61

                                                                                                            HealthSafety(util)0

                                                                                                            Colln Costs (util)0

                                                                                                            Gas Calls (util)0

                                                                                                            Calls to CSRs(util)2

                                                                                                            TampD (util)16

                                                                                                            Arrears (util)0

                                                                                                            Reconnects (util)0

                                                                                                            Notices (util)7

                                                                                                            Shutoffs (util)1

                                                                                                            Rate subsidy TampD

                                                                                                            Payment-related

                                                                                                            Source Skumatz Economic Research Associates research

                                                                                                            Example Low Income Weatherization

                                                                                                            SERA

                                                                                                            ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                                            be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                                            Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                                            Value of 100 HALVES Payback doubles ROI

                                                                                                            Source Skumatz Economics (SERA)

                                                                                                            35

                                                                                                            ( for NEBs Multiplier of 125)

                                                                                                            SERA

                                                                                                            NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                                            36

                                                                                                            hellip2001 ON

                                                                                                            SERA

                                                                                                            37

                                                                                                            NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                            For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                            Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                            BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                            SERA

                                                                                                            CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                            38

                                                                                                            Test Benefits Costs States Using Traditionally

                                                                                                            Improved treat-ment with NEBs

                                                                                                            Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                            bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                            bull Avoided gas and water supply costs

                                                                                                            bull Program administration

                                                                                                            bull Participant incentives

                                                                                                            bull Increased supply cost

                                                                                                            CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                            Use cost only paid by the utility

                                                                                                            Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                            Same as above plus bull increased revenue

                                                                                                            Same as above plus bull Decreased revenue

                                                                                                            AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                            Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                            bull Participant direct costs

                                                                                                            AR CA FL HI IA IN MI MN NY VA

                                                                                                            Participant NEBs

                                                                                                            Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                            Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                            SERA

                                                                                                            APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                            Expansion Concern with risk calculation accuracy

                                                                                                            39

                                                                                                            Source Skumatz SERA research

                                                                                                            SERA

                                                                                                            40

                                                                                                            METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                            Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                            Minimize Regulatory amp Implementer Risk

                                                                                                            Minimize Evaluation Cost

                                                                                                            Adder

                                                                                                            Readily MeasurableHybrid

                                                                                                            All NEBsSource SERA Research

                                                                                                            SERA

                                                                                                            41

                                                                                                            STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                            Screening Application Utilities regions

                                                                                                            Program Marketing Fairly widespread use in utilities states across the country

                                                                                                            MO

                                                                                                            RE AGGRESSIVE ==gt

                                                                                                            Test Pgm Screen - adder

                                                                                                            IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                            Test Pgm Screen - readily measured

                                                                                                            MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                            Test Hybrid (potential adder amp measured)

                                                                                                            CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                            Test Pgm screen - Broad

                                                                                                            With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                            Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                            SERA

                                                                                                            42

                                                                                                            Key

                                                                                                            States with NEBs in CE tests for At least one type of program

                                                                                                            copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                            STATES WITH NEBS IN CE TESTS

                                                                                                            NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                            groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                            SERA

                                                                                                            SELECTED STATE PROGRESS CA

                                                                                                            2001 ndash LIPPT model NY

                                                                                                            Mid-2000s measurement amp scenarios not included CO

                                                                                                            Adders (10 electric 25 LI 5 gas) VT

                                                                                                            15 adder helped by previous research DC

                                                                                                            10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                            43

                                                                                                            Source Skumatz SERA research

                                                                                                            SERA

                                                                                                            SELECTED STATE PROGRESS Dominos ongoing

                                                                                                            Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                            Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                            Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                            Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                            44

                                                                                                            Source Skumatz SERA research

                                                                                                            SERA

                                                                                                            APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                            Expansion Concern with risk calculation accuracy

                                                                                                            45

                                                                                                            Source Skumatz SERA research

                                                                                                            SERA

                                                                                                            46

                                                                                                            NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                            [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                            SourceSkumatz SERA Independent research

                                                                                                            NTG ndash accuracy measurement incomplete

                                                                                                            Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                            NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                            EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                            Cost Complicated expensive local changes

                                                                                                            RELATIVE Risk from NEBs

                                                                                                            Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                            Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                            NEBs as accurate as many inputs (all could improve)

                                                                                                            SERA

                                                                                                            DISCOUNT RATE MATTERS

                                                                                                            47

                                                                                                            Source Skumatz SERA Research

                                                                                                            0

                                                                                                            20

                                                                                                            40

                                                                                                            60

                                                                                                            80

                                                                                                            100

                                                                                                            120

                                                                                                            1 2 3 4 5 6 7 8 9 10

                                                                                                            PV Ratios vs 1 Discount Rate

                                                                                                            10 yr PV 20 yr PV

                                                                                                            Discount rate - percent

                                                                                                            WACCSocialLTT

                                                                                                            Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                            SERA

                                                                                                            EUL VARIATIONS-ADOPTED VALUES

                                                                                                            Source Skumatz Economic Research Associates SERA independent research

                                                                                                            0

                                                                                                            50

                                                                                                            100

                                                                                                            150

                                                                                                            200

                                                                                                            250

                                                                                                            300

                                                                                                            350

                                                                                                            400

                                                                                                            450

                                                                                                            0 10 20 30 40 50 60 70

                                                                                                            EUL Variation by Measure

                                                                                                            67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                            Poorly supportedOldTechnology change

                                                                                                            SERA

                                                                                                            KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                            Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                            measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                            change the BC conclusion

                                                                                                            On the cusphellip

                                                                                                            If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                            IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                            Source SERA all rights reserved

                                                                                                            SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                            SERA

                                                                                                            IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                            Utility Soc Part Conserv Recrsquom

                                                                                                            Rationale

                                                                                                            Base Percent X X X Program-invariant

                                                                                                            Low Income X X X X Multiple sources

                                                                                                            Weatherization X X X Substantial Participantimpacts

                                                                                                            Measure Program-specific

                                                                                                            X Varies by measure sector

                                                                                                            Other Recomrsquos Local Research

                                                                                                            50

                                                                                                            Source Skumatz SERA

                                                                                                            Developing values for multiple states amp utilities

                                                                                                            SERA

                                                                                                            TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                            Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                            Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                            5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                            6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                            THANK YOU

                                                                                                            Questions

                                                                                                            Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                            SERA

                                                                                                            POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                            53

                                                                                                            SERA

                                                                                                            NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                            Negative NEB values cost of barrier

                                                                                                            Solar Water Heat NZ$ Euros

                                                                                                            Solar Design NZ$ Euros

                                                                                                            Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                            -23 -12 (079)

                                                                                                            -11 -6 (06)

                                                                                                            Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                            Source Skumatz Economic Research Associates research

                                                                                                            SERA

                                                                                                            CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                            NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                            -5-4-3-2-1012345

                                                                                                            Op Co

                                                                                                            st

                                                                                                            Init C

                                                                                                            ost

                                                                                                            Maint

                                                                                                            Perfo

                                                                                                            rm

                                                                                                            Prod

                                                                                                            uctiv

                                                                                                            Tena

                                                                                                            nt Sa

                                                                                                            t

                                                                                                            Comf

                                                                                                            ort

                                                                                                            Appe

                                                                                                            ar

                                                                                                            Light

                                                                                                            Qual

                                                                                                            SellL

                                                                                                            ease

                                                                                                            Devp Ownr AampE

                                                                                                            Note AgtE (~4 vs ~2) for these NEBs

                                                                                                            Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                            Source Skumatz Economic Research Associates Inc 2002

                                                                                                            Owners more positive ndashUnderinvestment train

                                                                                                            • Soup to nuts on non-energy benefits NEIs
                                                                                                            • TOPICS
                                                                                                            • BACKGROUND NEBS
                                                                                                            • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                            • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                            • KEY APPLICATIONS OF NEBS
                                                                                                            • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                            • Neb measurement
                                                                                                            • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                            • MEASURING PARTICIPANTNEBs
                                                                                                            • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                            • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                            • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                            • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                            • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                            • PROGRESS AND GAPS
                                                                                                            • PROGRESS AND GAPS
                                                                                                            • USING NEB NEI VALUES
                                                                                                            • Slide Number 19
                                                                                                            • ARE NEBS HIGH VALUE
                                                                                                            • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                            • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                            • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                            • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                            • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                            • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                            • AUDIT FINANCIAL PROGRAM
                                                                                                            • CampI NEW CONSTRUCTION
                                                                                                            • CampI TECHNICAL ASSISTANCE
                                                                                                            • EXPRESSING NEBS VALUEndashCxe
                                                                                                            • Slide Number 31
                                                                                                            • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                            • EMISSIONS (SOCIETAL NEBS)
                                                                                                            • UTILITY NEBS
                                                                                                            • ADJUSTED PAYBACKS
                                                                                                            • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                            • NEBS IN BC
                                                                                                            • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                            • APPLICATION TO BC
                                                                                                            • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                            • STATE TREATMENT OF NEBS NEBs
                                                                                                            • Slide Number 42
                                                                                                            • SELECTED STATE PROGRESS
                                                                                                            • SELECTED STATE PROGRESS
                                                                                                            • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                            • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                            • DISCOUNT RATE MATTERS
                                                                                                            • EUL VARIATIONS- ADOPTED VALUES
                                                                                                            • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                            • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                            • TAKEAWAYSON NEBSNEIS
                                                                                                            • THANK YOUQuestions
                                                                                                            • Slide Number 53
                                                                                                            • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                            • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                              Sheet3

                                                                                                              SERA

                                                                                                              EXPRESSING NEBS VALUEndashCxe

                                                                                                              Strong value from RetroCxSource SERA Research

                                                                                                              SERA

                                                                                                              SOCIETAL NEB EXAMPLES

                                                                                                              31

                                                                                                              SERA

                                                                                                              JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                                              32

                                                                                                              0

                                                                                                              02

                                                                                                              04

                                                                                                              06

                                                                                                              08

                                                                                                              1

                                                                                                              12

                                                                                                              HPWxRetrof Appliance

                                                                                                              CA

                                                                                                              WI

                                                                                                              Natl

                                                                                                              (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                                              Jobs Economic

                                                                                                              SERA

                                                                                                              EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                                              33

                                                                                                              SERA

                                                                                                              UTILITY NEBSUtility NEBs for Template Program

                                                                                                              Debt WriteOff (util)13

                                                                                                              Rate Subsidy(util)61

                                                                                                              HealthSafety(util)0

                                                                                                              Colln Costs (util)0

                                                                                                              Gas Calls (util)0

                                                                                                              Calls to CSRs(util)2

                                                                                                              TampD (util)16

                                                                                                              Arrears (util)0

                                                                                                              Reconnects (util)0

                                                                                                              Notices (util)7

                                                                                                              Shutoffs (util)1

                                                                                                              Rate subsidy TampD

                                                                                                              Payment-related

                                                                                                              Source Skumatz Economic Research Associates research

                                                                                                              Example Low Income Weatherization

                                                                                                              SERA

                                                                                                              ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                                              be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                                              Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                                              Value of 100 HALVES Payback doubles ROI

                                                                                                              Source Skumatz Economics (SERA)

                                                                                                              35

                                                                                                              ( for NEBs Multiplier of 125)

                                                                                                              SERA

                                                                                                              NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                                              36

                                                                                                              hellip2001 ON

                                                                                                              SERA

                                                                                                              37

                                                                                                              NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                              For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                              Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                              BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                              SERA

                                                                                                              CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                              38

                                                                                                              Test Benefits Costs States Using Traditionally

                                                                                                              Improved treat-ment with NEBs

                                                                                                              Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                              bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                              bull Avoided gas and water supply costs

                                                                                                              bull Program administration

                                                                                                              bull Participant incentives

                                                                                                              bull Increased supply cost

                                                                                                              CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                              Use cost only paid by the utility

                                                                                                              Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                              Same as above plus bull increased revenue

                                                                                                              Same as above plus bull Decreased revenue

                                                                                                              AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                              Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                              bull Participant direct costs

                                                                                                              AR CA FL HI IA IN MI MN NY VA

                                                                                                              Participant NEBs

                                                                                                              Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                              Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                              SERA

                                                                                                              APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                              Expansion Concern with risk calculation accuracy

                                                                                                              39

                                                                                                              Source Skumatz SERA research

                                                                                                              SERA

                                                                                                              40

                                                                                                              METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                              Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                              Minimize Regulatory amp Implementer Risk

                                                                                                              Minimize Evaluation Cost

                                                                                                              Adder

                                                                                                              Readily MeasurableHybrid

                                                                                                              All NEBsSource SERA Research

                                                                                                              SERA

                                                                                                              41

                                                                                                              STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                              Screening Application Utilities regions

                                                                                                              Program Marketing Fairly widespread use in utilities states across the country

                                                                                                              MO

                                                                                                              RE AGGRESSIVE ==gt

                                                                                                              Test Pgm Screen - adder

                                                                                                              IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                              Test Pgm Screen - readily measured

                                                                                                              MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                              Test Hybrid (potential adder amp measured)

                                                                                                              CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                              Test Pgm screen - Broad

                                                                                                              With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                              Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                              SERA

                                                                                                              42

                                                                                                              Key

                                                                                                              States with NEBs in CE tests for At least one type of program

                                                                                                              copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                              STATES WITH NEBS IN CE TESTS

                                                                                                              NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                              groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                              SERA

                                                                                                              SELECTED STATE PROGRESS CA

                                                                                                              2001 ndash LIPPT model NY

                                                                                                              Mid-2000s measurement amp scenarios not included CO

                                                                                                              Adders (10 electric 25 LI 5 gas) VT

                                                                                                              15 adder helped by previous research DC

                                                                                                              10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                              43

                                                                                                              Source Skumatz SERA research

                                                                                                              SERA

                                                                                                              SELECTED STATE PROGRESS Dominos ongoing

                                                                                                              Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                              Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                              Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                              Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                              44

                                                                                                              Source Skumatz SERA research

                                                                                                              SERA

                                                                                                              APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                              Expansion Concern with risk calculation accuracy

                                                                                                              45

                                                                                                              Source Skumatz SERA research

                                                                                                              SERA

                                                                                                              46

                                                                                                              NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                              [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                              SourceSkumatz SERA Independent research

                                                                                                              NTG ndash accuracy measurement incomplete

                                                                                                              Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                              NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                              EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                              Cost Complicated expensive local changes

                                                                                                              RELATIVE Risk from NEBs

                                                                                                              Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                              Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                              NEBs as accurate as many inputs (all could improve)

                                                                                                              SERA

                                                                                                              DISCOUNT RATE MATTERS

                                                                                                              47

                                                                                                              Source Skumatz SERA Research

                                                                                                              0

                                                                                                              20

                                                                                                              40

                                                                                                              60

                                                                                                              80

                                                                                                              100

                                                                                                              120

                                                                                                              1 2 3 4 5 6 7 8 9 10

                                                                                                              PV Ratios vs 1 Discount Rate

                                                                                                              10 yr PV 20 yr PV

                                                                                                              Discount rate - percent

                                                                                                              WACCSocialLTT

                                                                                                              Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                              SERA

                                                                                                              EUL VARIATIONS-ADOPTED VALUES

                                                                                                              Source Skumatz Economic Research Associates SERA independent research

                                                                                                              0

                                                                                                              50

                                                                                                              100

                                                                                                              150

                                                                                                              200

                                                                                                              250

                                                                                                              300

                                                                                                              350

                                                                                                              400

                                                                                                              450

                                                                                                              0 10 20 30 40 50 60 70

                                                                                                              EUL Variation by Measure

                                                                                                              67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                              Poorly supportedOldTechnology change

                                                                                                              SERA

                                                                                                              KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                              Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                              measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                              change the BC conclusion

                                                                                                              On the cusphellip

                                                                                                              If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                              IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                              Source SERA all rights reserved

                                                                                                              SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                              SERA

                                                                                                              IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                              Utility Soc Part Conserv Recrsquom

                                                                                                              Rationale

                                                                                                              Base Percent X X X Program-invariant

                                                                                                              Low Income X X X X Multiple sources

                                                                                                              Weatherization X X X Substantial Participantimpacts

                                                                                                              Measure Program-specific

                                                                                                              X Varies by measure sector

                                                                                                              Other Recomrsquos Local Research

                                                                                                              50

                                                                                                              Source Skumatz SERA

                                                                                                              Developing values for multiple states amp utilities

                                                                                                              SERA

                                                                                                              TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                              Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                              Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                              5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                              6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                              THANK YOU

                                                                                                              Questions

                                                                                                              Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                              SERA

                                                                                                              POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                              53

                                                                                                              SERA

                                                                                                              NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                              Negative NEB values cost of barrier

                                                                                                              Solar Water Heat NZ$ Euros

                                                                                                              Solar Design NZ$ Euros

                                                                                                              Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                              -23 -12 (079)

                                                                                                              -11 -6 (06)

                                                                                                              Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                              Source Skumatz Economic Research Associates research

                                                                                                              SERA

                                                                                                              CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                              NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                              -5-4-3-2-1012345

                                                                                                              Op Co

                                                                                                              st

                                                                                                              Init C

                                                                                                              ost

                                                                                                              Maint

                                                                                                              Perfo

                                                                                                              rm

                                                                                                              Prod

                                                                                                              uctiv

                                                                                                              Tena

                                                                                                              nt Sa

                                                                                                              t

                                                                                                              Comf

                                                                                                              ort

                                                                                                              Appe

                                                                                                              ar

                                                                                                              Light

                                                                                                              Qual

                                                                                                              SellL

                                                                                                              ease

                                                                                                              Devp Ownr AampE

                                                                                                              Note AgtE (~4 vs ~2) for these NEBs

                                                                                                              Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                              Source Skumatz Economic Research Associates Inc 2002

                                                                                                              Owners more positive ndashUnderinvestment train

                                                                                                              • Soup to nuts on non-energy benefits NEIs
                                                                                                              • TOPICS
                                                                                                              • BACKGROUND NEBS
                                                                                                              • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                              • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                              • KEY APPLICATIONS OF NEBS
                                                                                                              • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                              • Neb measurement
                                                                                                              • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                              • MEASURING PARTICIPANTNEBs
                                                                                                              • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                              • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                              • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                              • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                              • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                              • PROGRESS AND GAPS
                                                                                                              • PROGRESS AND GAPS
                                                                                                              • USING NEB NEI VALUES
                                                                                                              • Slide Number 19
                                                                                                              • ARE NEBS HIGH VALUE
                                                                                                              • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                              • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                              • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                              • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                              • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                              • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                              • AUDIT FINANCIAL PROGRAM
                                                                                                              • CampI NEW CONSTRUCTION
                                                                                                              • CampI TECHNICAL ASSISTANCE
                                                                                                              • EXPRESSING NEBS VALUEndashCxe
                                                                                                              • Slide Number 31
                                                                                                              • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                              • EMISSIONS (SOCIETAL NEBS)
                                                                                                              • UTILITY NEBS
                                                                                                              • ADJUSTED PAYBACKS
                                                                                                              • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                              • NEBS IN BC
                                                                                                              • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                              • APPLICATION TO BC
                                                                                                              • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                              • STATE TREATMENT OF NEBS NEBs
                                                                                                              • Slide Number 42
                                                                                                              • SELECTED STATE PROGRESS
                                                                                                              • SELECTED STATE PROGRESS
                                                                                                              • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                              • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                              • DISCOUNT RATE MATTERS
                                                                                                              • EUL VARIATIONS- ADOPTED VALUES
                                                                                                              • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                              • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                              • TAKEAWAYSON NEBSNEIS
                                                                                                              • THANK YOUQuestions
                                                                                                              • Slide Number 53
                                                                                                              • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                              • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                SERA

                                                                                                                EXPRESSING NEBS VALUEndashCxe

                                                                                                                Strong value from RetroCxSource SERA Research

                                                                                                                SERA

                                                                                                                SOCIETAL NEB EXAMPLES

                                                                                                                31

                                                                                                                SERA

                                                                                                                JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                                                32

                                                                                                                0

                                                                                                                02

                                                                                                                04

                                                                                                                06

                                                                                                                08

                                                                                                                1

                                                                                                                12

                                                                                                                HPWxRetrof Appliance

                                                                                                                CA

                                                                                                                WI

                                                                                                                Natl

                                                                                                                (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                                                Jobs Economic

                                                                                                                SERA

                                                                                                                EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                                                33

                                                                                                                SERA

                                                                                                                UTILITY NEBSUtility NEBs for Template Program

                                                                                                                Debt WriteOff (util)13

                                                                                                                Rate Subsidy(util)61

                                                                                                                HealthSafety(util)0

                                                                                                                Colln Costs (util)0

                                                                                                                Gas Calls (util)0

                                                                                                                Calls to CSRs(util)2

                                                                                                                TampD (util)16

                                                                                                                Arrears (util)0

                                                                                                                Reconnects (util)0

                                                                                                                Notices (util)7

                                                                                                                Shutoffs (util)1

                                                                                                                Rate subsidy TampD

                                                                                                                Payment-related

                                                                                                                Source Skumatz Economic Research Associates research

                                                                                                                Example Low Income Weatherization

                                                                                                                SERA

                                                                                                                ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                                                be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                                                Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                                                Value of 100 HALVES Payback doubles ROI

                                                                                                                Source Skumatz Economics (SERA)

                                                                                                                35

                                                                                                                ( for NEBs Multiplier of 125)

                                                                                                                SERA

                                                                                                                NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                                                36

                                                                                                                hellip2001 ON

                                                                                                                SERA

                                                                                                                37

                                                                                                                NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                                For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                                Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                                BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                                SERA

                                                                                                                CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                                38

                                                                                                                Test Benefits Costs States Using Traditionally

                                                                                                                Improved treat-ment with NEBs

                                                                                                                Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                                bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                                bull Avoided gas and water supply costs

                                                                                                                bull Program administration

                                                                                                                bull Participant incentives

                                                                                                                bull Increased supply cost

                                                                                                                CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                                Use cost only paid by the utility

                                                                                                                Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                                Same as above plus bull increased revenue

                                                                                                                Same as above plus bull Decreased revenue

                                                                                                                AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                                Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                                bull Participant direct costs

                                                                                                                AR CA FL HI IA IN MI MN NY VA

                                                                                                                Participant NEBs

                                                                                                                Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                                Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                                SERA

                                                                                                                APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                Expansion Concern with risk calculation accuracy

                                                                                                                39

                                                                                                                Source Skumatz SERA research

                                                                                                                SERA

                                                                                                                40

                                                                                                                METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                                Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                                Minimize Regulatory amp Implementer Risk

                                                                                                                Minimize Evaluation Cost

                                                                                                                Adder

                                                                                                                Readily MeasurableHybrid

                                                                                                                All NEBsSource SERA Research

                                                                                                                SERA

                                                                                                                41

                                                                                                                STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                                Screening Application Utilities regions

                                                                                                                Program Marketing Fairly widespread use in utilities states across the country

                                                                                                                MO

                                                                                                                RE AGGRESSIVE ==gt

                                                                                                                Test Pgm Screen - adder

                                                                                                                IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                                Test Pgm Screen - readily measured

                                                                                                                MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                                Test Hybrid (potential adder amp measured)

                                                                                                                CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                                Test Pgm screen - Broad

                                                                                                                With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                                Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                                SERA

                                                                                                                42

                                                                                                                Key

                                                                                                                States with NEBs in CE tests for At least one type of program

                                                                                                                copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                                STATES WITH NEBS IN CE TESTS

                                                                                                                NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                                groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                                SERA

                                                                                                                SELECTED STATE PROGRESS CA

                                                                                                                2001 ndash LIPPT model NY

                                                                                                                Mid-2000s measurement amp scenarios not included CO

                                                                                                                Adders (10 electric 25 LI 5 gas) VT

                                                                                                                15 adder helped by previous research DC

                                                                                                                10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                                43

                                                                                                                Source Skumatz SERA research

                                                                                                                SERA

                                                                                                                SELECTED STATE PROGRESS Dominos ongoing

                                                                                                                Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                                Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                                Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                                Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                                44

                                                                                                                Source Skumatz SERA research

                                                                                                                SERA

                                                                                                                APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                Expansion Concern with risk calculation accuracy

                                                                                                                45

                                                                                                                Source Skumatz SERA research

                                                                                                                SERA

                                                                                                                46

                                                                                                                NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                SourceSkumatz SERA Independent research

                                                                                                                NTG ndash accuracy measurement incomplete

                                                                                                                Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                Cost Complicated expensive local changes

                                                                                                                RELATIVE Risk from NEBs

                                                                                                                Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                NEBs as accurate as many inputs (all could improve)

                                                                                                                SERA

                                                                                                                DISCOUNT RATE MATTERS

                                                                                                                47

                                                                                                                Source Skumatz SERA Research

                                                                                                                0

                                                                                                                20

                                                                                                                40

                                                                                                                60

                                                                                                                80

                                                                                                                100

                                                                                                                120

                                                                                                                1 2 3 4 5 6 7 8 9 10

                                                                                                                PV Ratios vs 1 Discount Rate

                                                                                                                10 yr PV 20 yr PV

                                                                                                                Discount rate - percent

                                                                                                                WACCSocialLTT

                                                                                                                Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                SERA

                                                                                                                EUL VARIATIONS-ADOPTED VALUES

                                                                                                                Source Skumatz Economic Research Associates SERA independent research

                                                                                                                0

                                                                                                                50

                                                                                                                100

                                                                                                                150

                                                                                                                200

                                                                                                                250

                                                                                                                300

                                                                                                                350

                                                                                                                400

                                                                                                                450

                                                                                                                0 10 20 30 40 50 60 70

                                                                                                                EUL Variation by Measure

                                                                                                                67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                Poorly supportedOldTechnology change

                                                                                                                SERA

                                                                                                                KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                change the BC conclusion

                                                                                                                On the cusphellip

                                                                                                                If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                Source SERA all rights reserved

                                                                                                                SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                SERA

                                                                                                                IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                Utility Soc Part Conserv Recrsquom

                                                                                                                Rationale

                                                                                                                Base Percent X X X Program-invariant

                                                                                                                Low Income X X X X Multiple sources

                                                                                                                Weatherization X X X Substantial Participantimpacts

                                                                                                                Measure Program-specific

                                                                                                                X Varies by measure sector

                                                                                                                Other Recomrsquos Local Research

                                                                                                                50

                                                                                                                Source Skumatz SERA

                                                                                                                Developing values for multiple states amp utilities

                                                                                                                SERA

                                                                                                                TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                THANK YOU

                                                                                                                Questions

                                                                                                                Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                SERA

                                                                                                                POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                53

                                                                                                                SERA

                                                                                                                NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                Negative NEB values cost of barrier

                                                                                                                Solar Water Heat NZ$ Euros

                                                                                                                Solar Design NZ$ Euros

                                                                                                                Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                -23 -12 (079)

                                                                                                                -11 -6 (06)

                                                                                                                Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                Source Skumatz Economic Research Associates research

                                                                                                                SERA

                                                                                                                CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                -5-4-3-2-1012345

                                                                                                                Op Co

                                                                                                                st

                                                                                                                Init C

                                                                                                                ost

                                                                                                                Maint

                                                                                                                Perfo

                                                                                                                rm

                                                                                                                Prod

                                                                                                                uctiv

                                                                                                                Tena

                                                                                                                nt Sa

                                                                                                                t

                                                                                                                Comf

                                                                                                                ort

                                                                                                                Appe

                                                                                                                ar

                                                                                                                Light

                                                                                                                Qual

                                                                                                                SellL

                                                                                                                ease

                                                                                                                Devp Ownr AampE

                                                                                                                Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                Source Skumatz Economic Research Associates Inc 2002

                                                                                                                Owners more positive ndashUnderinvestment train

                                                                                                                • Soup to nuts on non-energy benefits NEIs
                                                                                                                • TOPICS
                                                                                                                • BACKGROUND NEBS
                                                                                                                • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                • KEY APPLICATIONS OF NEBS
                                                                                                                • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                • Neb measurement
                                                                                                                • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                • MEASURING PARTICIPANTNEBs
                                                                                                                • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                • PROGRESS AND GAPS
                                                                                                                • PROGRESS AND GAPS
                                                                                                                • USING NEB NEI VALUES
                                                                                                                • Slide Number 19
                                                                                                                • ARE NEBS HIGH VALUE
                                                                                                                • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                • AUDIT FINANCIAL PROGRAM
                                                                                                                • CampI NEW CONSTRUCTION
                                                                                                                • CampI TECHNICAL ASSISTANCE
                                                                                                                • EXPRESSING NEBS VALUEndashCxe
                                                                                                                • Slide Number 31
                                                                                                                • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                • EMISSIONS (SOCIETAL NEBS)
                                                                                                                • UTILITY NEBS
                                                                                                                • ADJUSTED PAYBACKS
                                                                                                                • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                • NEBS IN BC
                                                                                                                • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                • APPLICATION TO BC
                                                                                                                • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                • STATE TREATMENT OF NEBS NEBs
                                                                                                                • Slide Number 42
                                                                                                                • SELECTED STATE PROGRESS
                                                                                                                • SELECTED STATE PROGRESS
                                                                                                                • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                • DISCOUNT RATE MATTERS
                                                                                                                • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                • TAKEAWAYSON NEBSNEIS
                                                                                                                • THANK YOUQuestions
                                                                                                                • Slide Number 53
                                                                                                                • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                  SERA

                                                                                                                  SOCIETAL NEB EXAMPLES

                                                                                                                  31

                                                                                                                  SERA

                                                                                                                  JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                                                  32

                                                                                                                  0

                                                                                                                  02

                                                                                                                  04

                                                                                                                  06

                                                                                                                  08

                                                                                                                  1

                                                                                                                  12

                                                                                                                  HPWxRetrof Appliance

                                                                                                                  CA

                                                                                                                  WI

                                                                                                                  Natl

                                                                                                                  (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                                                  Jobs Economic

                                                                                                                  SERA

                                                                                                                  EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                                                  33

                                                                                                                  SERA

                                                                                                                  UTILITY NEBSUtility NEBs for Template Program

                                                                                                                  Debt WriteOff (util)13

                                                                                                                  Rate Subsidy(util)61

                                                                                                                  HealthSafety(util)0

                                                                                                                  Colln Costs (util)0

                                                                                                                  Gas Calls (util)0

                                                                                                                  Calls to CSRs(util)2

                                                                                                                  TampD (util)16

                                                                                                                  Arrears (util)0

                                                                                                                  Reconnects (util)0

                                                                                                                  Notices (util)7

                                                                                                                  Shutoffs (util)1

                                                                                                                  Rate subsidy TampD

                                                                                                                  Payment-related

                                                                                                                  Source Skumatz Economic Research Associates research

                                                                                                                  Example Low Income Weatherization

                                                                                                                  SERA

                                                                                                                  ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                                                  be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                                                  Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                                                  Value of 100 HALVES Payback doubles ROI

                                                                                                                  Source Skumatz Economics (SERA)

                                                                                                                  35

                                                                                                                  ( for NEBs Multiplier of 125)

                                                                                                                  SERA

                                                                                                                  NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                                                  36

                                                                                                                  hellip2001 ON

                                                                                                                  SERA

                                                                                                                  37

                                                                                                                  NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                                  For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                                  Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                                  BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                                  SERA

                                                                                                                  CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                                  38

                                                                                                                  Test Benefits Costs States Using Traditionally

                                                                                                                  Improved treat-ment with NEBs

                                                                                                                  Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                                  bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                                  bull Avoided gas and water supply costs

                                                                                                                  bull Program administration

                                                                                                                  bull Participant incentives

                                                                                                                  bull Increased supply cost

                                                                                                                  CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                                  Use cost only paid by the utility

                                                                                                                  Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                                  Same as above plus bull increased revenue

                                                                                                                  Same as above plus bull Decreased revenue

                                                                                                                  AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                                  Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                                  bull Participant direct costs

                                                                                                                  AR CA FL HI IA IN MI MN NY VA

                                                                                                                  Participant NEBs

                                                                                                                  Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                                  Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                                  SERA

                                                                                                                  APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                  Expansion Concern with risk calculation accuracy

                                                                                                                  39

                                                                                                                  Source Skumatz SERA research

                                                                                                                  SERA

                                                                                                                  40

                                                                                                                  METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                                  Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                                  Minimize Regulatory amp Implementer Risk

                                                                                                                  Minimize Evaluation Cost

                                                                                                                  Adder

                                                                                                                  Readily MeasurableHybrid

                                                                                                                  All NEBsSource SERA Research

                                                                                                                  SERA

                                                                                                                  41

                                                                                                                  STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                                  Screening Application Utilities regions

                                                                                                                  Program Marketing Fairly widespread use in utilities states across the country

                                                                                                                  MO

                                                                                                                  RE AGGRESSIVE ==gt

                                                                                                                  Test Pgm Screen - adder

                                                                                                                  IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                                  Test Pgm Screen - readily measured

                                                                                                                  MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                                  Test Hybrid (potential adder amp measured)

                                                                                                                  CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                                  Test Pgm screen - Broad

                                                                                                                  With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                                  Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                                  SERA

                                                                                                                  42

                                                                                                                  Key

                                                                                                                  States with NEBs in CE tests for At least one type of program

                                                                                                                  copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                                  STATES WITH NEBS IN CE TESTS

                                                                                                                  NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                                  groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                                  SERA

                                                                                                                  SELECTED STATE PROGRESS CA

                                                                                                                  2001 ndash LIPPT model NY

                                                                                                                  Mid-2000s measurement amp scenarios not included CO

                                                                                                                  Adders (10 electric 25 LI 5 gas) VT

                                                                                                                  15 adder helped by previous research DC

                                                                                                                  10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                                  43

                                                                                                                  Source Skumatz SERA research

                                                                                                                  SERA

                                                                                                                  SELECTED STATE PROGRESS Dominos ongoing

                                                                                                                  Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                                  Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                                  Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                                  Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                                  44

                                                                                                                  Source Skumatz SERA research

                                                                                                                  SERA

                                                                                                                  APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                  Expansion Concern with risk calculation accuracy

                                                                                                                  45

                                                                                                                  Source Skumatz SERA research

                                                                                                                  SERA

                                                                                                                  46

                                                                                                                  NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                  [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                  SourceSkumatz SERA Independent research

                                                                                                                  NTG ndash accuracy measurement incomplete

                                                                                                                  Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                  NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                  EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                  Cost Complicated expensive local changes

                                                                                                                  RELATIVE Risk from NEBs

                                                                                                                  Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                  Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                  NEBs as accurate as many inputs (all could improve)

                                                                                                                  SERA

                                                                                                                  DISCOUNT RATE MATTERS

                                                                                                                  47

                                                                                                                  Source Skumatz SERA Research

                                                                                                                  0

                                                                                                                  20

                                                                                                                  40

                                                                                                                  60

                                                                                                                  80

                                                                                                                  100

                                                                                                                  120

                                                                                                                  1 2 3 4 5 6 7 8 9 10

                                                                                                                  PV Ratios vs 1 Discount Rate

                                                                                                                  10 yr PV 20 yr PV

                                                                                                                  Discount rate - percent

                                                                                                                  WACCSocialLTT

                                                                                                                  Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                  SERA

                                                                                                                  EUL VARIATIONS-ADOPTED VALUES

                                                                                                                  Source Skumatz Economic Research Associates SERA independent research

                                                                                                                  0

                                                                                                                  50

                                                                                                                  100

                                                                                                                  150

                                                                                                                  200

                                                                                                                  250

                                                                                                                  300

                                                                                                                  350

                                                                                                                  400

                                                                                                                  450

                                                                                                                  0 10 20 30 40 50 60 70

                                                                                                                  EUL Variation by Measure

                                                                                                                  67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                  Poorly supportedOldTechnology change

                                                                                                                  SERA

                                                                                                                  KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                  Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                  measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                  change the BC conclusion

                                                                                                                  On the cusphellip

                                                                                                                  If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                  IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                  Source SERA all rights reserved

                                                                                                                  SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                  SERA

                                                                                                                  IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                  Utility Soc Part Conserv Recrsquom

                                                                                                                  Rationale

                                                                                                                  Base Percent X X X Program-invariant

                                                                                                                  Low Income X X X X Multiple sources

                                                                                                                  Weatherization X X X Substantial Participantimpacts

                                                                                                                  Measure Program-specific

                                                                                                                  X Varies by measure sector

                                                                                                                  Other Recomrsquos Local Research

                                                                                                                  50

                                                                                                                  Source Skumatz SERA

                                                                                                                  Developing values for multiple states amp utilities

                                                                                                                  SERA

                                                                                                                  TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                  Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                  Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                  5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                  6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                  THANK YOU

                                                                                                                  Questions

                                                                                                                  Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                  SERA

                                                                                                                  POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                  53

                                                                                                                  SERA

                                                                                                                  NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                  Negative NEB values cost of barrier

                                                                                                                  Solar Water Heat NZ$ Euros

                                                                                                                  Solar Design NZ$ Euros

                                                                                                                  Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                  -23 -12 (079)

                                                                                                                  -11 -6 (06)

                                                                                                                  Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                  Source Skumatz Economic Research Associates research

                                                                                                                  SERA

                                                                                                                  CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                  NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                  -5-4-3-2-1012345

                                                                                                                  Op Co

                                                                                                                  st

                                                                                                                  Init C

                                                                                                                  ost

                                                                                                                  Maint

                                                                                                                  Perfo

                                                                                                                  rm

                                                                                                                  Prod

                                                                                                                  uctiv

                                                                                                                  Tena

                                                                                                                  nt Sa

                                                                                                                  t

                                                                                                                  Comf

                                                                                                                  ort

                                                                                                                  Appe

                                                                                                                  ar

                                                                                                                  Light

                                                                                                                  Qual

                                                                                                                  SellL

                                                                                                                  ease

                                                                                                                  Devp Ownr AampE

                                                                                                                  Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                  Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                  Source Skumatz Economic Research Associates Inc 2002

                                                                                                                  Owners more positive ndashUnderinvestment train

                                                                                                                  • Soup to nuts on non-energy benefits NEIs
                                                                                                                  • TOPICS
                                                                                                                  • BACKGROUND NEBS
                                                                                                                  • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                  • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                  • KEY APPLICATIONS OF NEBS
                                                                                                                  • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                  • Neb measurement
                                                                                                                  • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                  • MEASURING PARTICIPANTNEBs
                                                                                                                  • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                  • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                  • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                  • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                  • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                  • PROGRESS AND GAPS
                                                                                                                  • PROGRESS AND GAPS
                                                                                                                  • USING NEB NEI VALUES
                                                                                                                  • Slide Number 19
                                                                                                                  • ARE NEBS HIGH VALUE
                                                                                                                  • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                  • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                  • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                  • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                  • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                  • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                  • AUDIT FINANCIAL PROGRAM
                                                                                                                  • CampI NEW CONSTRUCTION
                                                                                                                  • CampI TECHNICAL ASSISTANCE
                                                                                                                  • EXPRESSING NEBS VALUEndashCxe
                                                                                                                  • Slide Number 31
                                                                                                                  • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                  • EMISSIONS (SOCIETAL NEBS)
                                                                                                                  • UTILITY NEBS
                                                                                                                  • ADJUSTED PAYBACKS
                                                                                                                  • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                  • NEBS IN BC
                                                                                                                  • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                  • APPLICATION TO BC
                                                                                                                  • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                  • STATE TREATMENT OF NEBS NEBs
                                                                                                                  • Slide Number 42
                                                                                                                  • SELECTED STATE PROGRESS
                                                                                                                  • SELECTED STATE PROGRESS
                                                                                                                  • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                  • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                  • DISCOUNT RATE MATTERS
                                                                                                                  • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                  • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                  • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                  • TAKEAWAYSON NEBSNEIS
                                                                                                                  • THANK YOUQuestions
                                                                                                                  • Slide Number 53
                                                                                                                  • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                  • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                    SERA

                                                                                                                    JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship

                                                                                                                    32

                                                                                                                    0

                                                                                                                    02

                                                                                                                    04

                                                                                                                    06

                                                                                                                    08

                                                                                                                    1

                                                                                                                    12

                                                                                                                    HPWxRetrof Appliance

                                                                                                                    CA

                                                                                                                    WI

                                                                                                                    Natl

                                                                                                                    (Source Skumatz SERAECEEE 2007 ACEEE 2006)

                                                                                                                    Jobs Economic

                                                                                                                    SERA

                                                                                                                    EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                                                    33

                                                                                                                    SERA

                                                                                                                    UTILITY NEBSUtility NEBs for Template Program

                                                                                                                    Debt WriteOff (util)13

                                                                                                                    Rate Subsidy(util)61

                                                                                                                    HealthSafety(util)0

                                                                                                                    Colln Costs (util)0

                                                                                                                    Gas Calls (util)0

                                                                                                                    Calls to CSRs(util)2

                                                                                                                    TampD (util)16

                                                                                                                    Arrears (util)0

                                                                                                                    Reconnects (util)0

                                                                                                                    Notices (util)7

                                                                                                                    Shutoffs (util)1

                                                                                                                    Rate subsidy TampD

                                                                                                                    Payment-related

                                                                                                                    Source Skumatz Economic Research Associates research

                                                                                                                    Example Low Income Weatherization

                                                                                                                    SERA

                                                                                                                    ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                                                    be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                                                    Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                                                    Value of 100 HALVES Payback doubles ROI

                                                                                                                    Source Skumatz Economics (SERA)

                                                                                                                    35

                                                                                                                    ( for NEBs Multiplier of 125)

                                                                                                                    SERA

                                                                                                                    NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                                                    36

                                                                                                                    hellip2001 ON

                                                                                                                    SERA

                                                                                                                    37

                                                                                                                    NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                                    For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                                    Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                                    BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                                    SERA

                                                                                                                    CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                                    38

                                                                                                                    Test Benefits Costs States Using Traditionally

                                                                                                                    Improved treat-ment with NEBs

                                                                                                                    Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                                    bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                                    bull Avoided gas and water supply costs

                                                                                                                    bull Program administration

                                                                                                                    bull Participant incentives

                                                                                                                    bull Increased supply cost

                                                                                                                    CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                                    Use cost only paid by the utility

                                                                                                                    Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                                    Same as above plus bull increased revenue

                                                                                                                    Same as above plus bull Decreased revenue

                                                                                                                    AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                                    Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                                    bull Participant direct costs

                                                                                                                    AR CA FL HI IA IN MI MN NY VA

                                                                                                                    Participant NEBs

                                                                                                                    Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                                    Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                                    SERA

                                                                                                                    APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                    Expansion Concern with risk calculation accuracy

                                                                                                                    39

                                                                                                                    Source Skumatz SERA research

                                                                                                                    SERA

                                                                                                                    40

                                                                                                                    METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                                    Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                                    Minimize Regulatory amp Implementer Risk

                                                                                                                    Minimize Evaluation Cost

                                                                                                                    Adder

                                                                                                                    Readily MeasurableHybrid

                                                                                                                    All NEBsSource SERA Research

                                                                                                                    SERA

                                                                                                                    41

                                                                                                                    STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                                    Screening Application Utilities regions

                                                                                                                    Program Marketing Fairly widespread use in utilities states across the country

                                                                                                                    MO

                                                                                                                    RE AGGRESSIVE ==gt

                                                                                                                    Test Pgm Screen - adder

                                                                                                                    IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                                    Test Pgm Screen - readily measured

                                                                                                                    MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                                    Test Hybrid (potential adder amp measured)

                                                                                                                    CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                                    Test Pgm screen - Broad

                                                                                                                    With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                                    Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                                    SERA

                                                                                                                    42

                                                                                                                    Key

                                                                                                                    States with NEBs in CE tests for At least one type of program

                                                                                                                    copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                                    STATES WITH NEBS IN CE TESTS

                                                                                                                    NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                                    groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                                    SERA

                                                                                                                    SELECTED STATE PROGRESS CA

                                                                                                                    2001 ndash LIPPT model NY

                                                                                                                    Mid-2000s measurement amp scenarios not included CO

                                                                                                                    Adders (10 electric 25 LI 5 gas) VT

                                                                                                                    15 adder helped by previous research DC

                                                                                                                    10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                                    43

                                                                                                                    Source Skumatz SERA research

                                                                                                                    SERA

                                                                                                                    SELECTED STATE PROGRESS Dominos ongoing

                                                                                                                    Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                                    Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                                    Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                                    Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                                    44

                                                                                                                    Source Skumatz SERA research

                                                                                                                    SERA

                                                                                                                    APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                    Expansion Concern with risk calculation accuracy

                                                                                                                    45

                                                                                                                    Source Skumatz SERA research

                                                                                                                    SERA

                                                                                                                    46

                                                                                                                    NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                    [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                    SourceSkumatz SERA Independent research

                                                                                                                    NTG ndash accuracy measurement incomplete

                                                                                                                    Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                    NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                    EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                    Cost Complicated expensive local changes

                                                                                                                    RELATIVE Risk from NEBs

                                                                                                                    Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                    Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                    NEBs as accurate as many inputs (all could improve)

                                                                                                                    SERA

                                                                                                                    DISCOUNT RATE MATTERS

                                                                                                                    47

                                                                                                                    Source Skumatz SERA Research

                                                                                                                    0

                                                                                                                    20

                                                                                                                    40

                                                                                                                    60

                                                                                                                    80

                                                                                                                    100

                                                                                                                    120

                                                                                                                    1 2 3 4 5 6 7 8 9 10

                                                                                                                    PV Ratios vs 1 Discount Rate

                                                                                                                    10 yr PV 20 yr PV

                                                                                                                    Discount rate - percent

                                                                                                                    WACCSocialLTT

                                                                                                                    Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                    SERA

                                                                                                                    EUL VARIATIONS-ADOPTED VALUES

                                                                                                                    Source Skumatz Economic Research Associates SERA independent research

                                                                                                                    0

                                                                                                                    50

                                                                                                                    100

                                                                                                                    150

                                                                                                                    200

                                                                                                                    250

                                                                                                                    300

                                                                                                                    350

                                                                                                                    400

                                                                                                                    450

                                                                                                                    0 10 20 30 40 50 60 70

                                                                                                                    EUL Variation by Measure

                                                                                                                    67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                    Poorly supportedOldTechnology change

                                                                                                                    SERA

                                                                                                                    KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                    Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                    measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                    change the BC conclusion

                                                                                                                    On the cusphellip

                                                                                                                    If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                    IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                    Source SERA all rights reserved

                                                                                                                    SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                    SERA

                                                                                                                    IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                    Utility Soc Part Conserv Recrsquom

                                                                                                                    Rationale

                                                                                                                    Base Percent X X X Program-invariant

                                                                                                                    Low Income X X X X Multiple sources

                                                                                                                    Weatherization X X X Substantial Participantimpacts

                                                                                                                    Measure Program-specific

                                                                                                                    X Varies by measure sector

                                                                                                                    Other Recomrsquos Local Research

                                                                                                                    50

                                                                                                                    Source Skumatz SERA

                                                                                                                    Developing values for multiple states amp utilities

                                                                                                                    SERA

                                                                                                                    TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                    Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                    Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                    5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                    6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                    THANK YOU

                                                                                                                    Questions

                                                                                                                    Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                    SERA

                                                                                                                    POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                    53

                                                                                                                    SERA

                                                                                                                    NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                    Negative NEB values cost of barrier

                                                                                                                    Solar Water Heat NZ$ Euros

                                                                                                                    Solar Design NZ$ Euros

                                                                                                                    Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                    -23 -12 (079)

                                                                                                                    -11 -6 (06)

                                                                                                                    Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                    Source Skumatz Economic Research Associates research

                                                                                                                    SERA

                                                                                                                    CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                    NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                    -5-4-3-2-1012345

                                                                                                                    Op Co

                                                                                                                    st

                                                                                                                    Init C

                                                                                                                    ost

                                                                                                                    Maint

                                                                                                                    Perfo

                                                                                                                    rm

                                                                                                                    Prod

                                                                                                                    uctiv

                                                                                                                    Tena

                                                                                                                    nt Sa

                                                                                                                    t

                                                                                                                    Comf

                                                                                                                    ort

                                                                                                                    Appe

                                                                                                                    ar

                                                                                                                    Light

                                                                                                                    Qual

                                                                                                                    SellL

                                                                                                                    ease

                                                                                                                    Devp Ownr AampE

                                                                                                                    Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                    Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                    Source Skumatz Economic Research Associates Inc 2002

                                                                                                                    Owners more positive ndashUnderinvestment train

                                                                                                                    • Soup to nuts on non-energy benefits NEIs
                                                                                                                    • TOPICS
                                                                                                                    • BACKGROUND NEBS
                                                                                                                    • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                    • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                    • KEY APPLICATIONS OF NEBS
                                                                                                                    • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                    • Neb measurement
                                                                                                                    • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                    • MEASURING PARTICIPANTNEBs
                                                                                                                    • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                    • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                    • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                    • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                    • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                    • PROGRESS AND GAPS
                                                                                                                    • PROGRESS AND GAPS
                                                                                                                    • USING NEB NEI VALUES
                                                                                                                    • Slide Number 19
                                                                                                                    • ARE NEBS HIGH VALUE
                                                                                                                    • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                    • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                    • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                    • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                    • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                    • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                    • AUDIT FINANCIAL PROGRAM
                                                                                                                    • CampI NEW CONSTRUCTION
                                                                                                                    • CampI TECHNICAL ASSISTANCE
                                                                                                                    • EXPRESSING NEBS VALUEndashCxe
                                                                                                                    • Slide Number 31
                                                                                                                    • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                    • EMISSIONS (SOCIETAL NEBS)
                                                                                                                    • UTILITY NEBS
                                                                                                                    • ADJUSTED PAYBACKS
                                                                                                                    • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                    • NEBS IN BC
                                                                                                                    • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                    • APPLICATION TO BC
                                                                                                                    • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                    • STATE TREATMENT OF NEBS NEBs
                                                                                                                    • Slide Number 42
                                                                                                                    • SELECTED STATE PROGRESS
                                                                                                                    • SELECTED STATE PROGRESS
                                                                                                                    • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                    • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                    • DISCOUNT RATE MATTERS
                                                                                                                    • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                    • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                    • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                    • TAKEAWAYSON NEBSNEIS
                                                                                                                    • THANK YOUQuestions
                                                                                                                    • Slide Number 53
                                                                                                                    • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                    • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                      SERA

                                                                                                                      EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder

                                                                                                                      33

                                                                                                                      SERA

                                                                                                                      UTILITY NEBSUtility NEBs for Template Program

                                                                                                                      Debt WriteOff (util)13

                                                                                                                      Rate Subsidy(util)61

                                                                                                                      HealthSafety(util)0

                                                                                                                      Colln Costs (util)0

                                                                                                                      Gas Calls (util)0

                                                                                                                      Calls to CSRs(util)2

                                                                                                                      TampD (util)16

                                                                                                                      Arrears (util)0

                                                                                                                      Reconnects (util)0

                                                                                                                      Notices (util)7

                                                                                                                      Shutoffs (util)1

                                                                                                                      Rate subsidy TampD

                                                                                                                      Payment-related

                                                                                                                      Source Skumatz Economic Research Associates research

                                                                                                                      Example Low Income Weatherization

                                                                                                                      SERA

                                                                                                                      ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                                                      be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                                                      Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                                                      Value of 100 HALVES Payback doubles ROI

                                                                                                                      Source Skumatz Economics (SERA)

                                                                                                                      35

                                                                                                                      ( for NEBs Multiplier of 125)

                                                                                                                      SERA

                                                                                                                      NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                                                      36

                                                                                                                      hellip2001 ON

                                                                                                                      SERA

                                                                                                                      37

                                                                                                                      NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                                      For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                                      Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                                      BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                                      SERA

                                                                                                                      CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                                      38

                                                                                                                      Test Benefits Costs States Using Traditionally

                                                                                                                      Improved treat-ment with NEBs

                                                                                                                      Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                                      bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                                      bull Avoided gas and water supply costs

                                                                                                                      bull Program administration

                                                                                                                      bull Participant incentives

                                                                                                                      bull Increased supply cost

                                                                                                                      CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                                      Use cost only paid by the utility

                                                                                                                      Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                                      Same as above plus bull increased revenue

                                                                                                                      Same as above plus bull Decreased revenue

                                                                                                                      AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                                      Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                                      bull Participant direct costs

                                                                                                                      AR CA FL HI IA IN MI MN NY VA

                                                                                                                      Participant NEBs

                                                                                                                      Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                                      Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                                      SERA

                                                                                                                      APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                      Expansion Concern with risk calculation accuracy

                                                                                                                      39

                                                                                                                      Source Skumatz SERA research

                                                                                                                      SERA

                                                                                                                      40

                                                                                                                      METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                                      Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                                      Minimize Regulatory amp Implementer Risk

                                                                                                                      Minimize Evaluation Cost

                                                                                                                      Adder

                                                                                                                      Readily MeasurableHybrid

                                                                                                                      All NEBsSource SERA Research

                                                                                                                      SERA

                                                                                                                      41

                                                                                                                      STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                                      Screening Application Utilities regions

                                                                                                                      Program Marketing Fairly widespread use in utilities states across the country

                                                                                                                      MO

                                                                                                                      RE AGGRESSIVE ==gt

                                                                                                                      Test Pgm Screen - adder

                                                                                                                      IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                                      Test Pgm Screen - readily measured

                                                                                                                      MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                                      Test Hybrid (potential adder amp measured)

                                                                                                                      CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                                      Test Pgm screen - Broad

                                                                                                                      With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                                      Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                                      SERA

                                                                                                                      42

                                                                                                                      Key

                                                                                                                      States with NEBs in CE tests for At least one type of program

                                                                                                                      copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                                      STATES WITH NEBS IN CE TESTS

                                                                                                                      NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                                      groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                                      SERA

                                                                                                                      SELECTED STATE PROGRESS CA

                                                                                                                      2001 ndash LIPPT model NY

                                                                                                                      Mid-2000s measurement amp scenarios not included CO

                                                                                                                      Adders (10 electric 25 LI 5 gas) VT

                                                                                                                      15 adder helped by previous research DC

                                                                                                                      10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                                      43

                                                                                                                      Source Skumatz SERA research

                                                                                                                      SERA

                                                                                                                      SELECTED STATE PROGRESS Dominos ongoing

                                                                                                                      Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                                      Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                                      Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                                      Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                                      44

                                                                                                                      Source Skumatz SERA research

                                                                                                                      SERA

                                                                                                                      APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                      Expansion Concern with risk calculation accuracy

                                                                                                                      45

                                                                                                                      Source Skumatz SERA research

                                                                                                                      SERA

                                                                                                                      46

                                                                                                                      NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                      [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                      SourceSkumatz SERA Independent research

                                                                                                                      NTG ndash accuracy measurement incomplete

                                                                                                                      Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                      NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                      EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                      Cost Complicated expensive local changes

                                                                                                                      RELATIVE Risk from NEBs

                                                                                                                      Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                      Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                      NEBs as accurate as many inputs (all could improve)

                                                                                                                      SERA

                                                                                                                      DISCOUNT RATE MATTERS

                                                                                                                      47

                                                                                                                      Source Skumatz SERA Research

                                                                                                                      0

                                                                                                                      20

                                                                                                                      40

                                                                                                                      60

                                                                                                                      80

                                                                                                                      100

                                                                                                                      120

                                                                                                                      1 2 3 4 5 6 7 8 9 10

                                                                                                                      PV Ratios vs 1 Discount Rate

                                                                                                                      10 yr PV 20 yr PV

                                                                                                                      Discount rate - percent

                                                                                                                      WACCSocialLTT

                                                                                                                      Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                      SERA

                                                                                                                      EUL VARIATIONS-ADOPTED VALUES

                                                                                                                      Source Skumatz Economic Research Associates SERA independent research

                                                                                                                      0

                                                                                                                      50

                                                                                                                      100

                                                                                                                      150

                                                                                                                      200

                                                                                                                      250

                                                                                                                      300

                                                                                                                      350

                                                                                                                      400

                                                                                                                      450

                                                                                                                      0 10 20 30 40 50 60 70

                                                                                                                      EUL Variation by Measure

                                                                                                                      67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                      Poorly supportedOldTechnology change

                                                                                                                      SERA

                                                                                                                      KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                      Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                      measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                      change the BC conclusion

                                                                                                                      On the cusphellip

                                                                                                                      If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                      IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                      Source SERA all rights reserved

                                                                                                                      SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                      SERA

                                                                                                                      IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                      Utility Soc Part Conserv Recrsquom

                                                                                                                      Rationale

                                                                                                                      Base Percent X X X Program-invariant

                                                                                                                      Low Income X X X X Multiple sources

                                                                                                                      Weatherization X X X Substantial Participantimpacts

                                                                                                                      Measure Program-specific

                                                                                                                      X Varies by measure sector

                                                                                                                      Other Recomrsquos Local Research

                                                                                                                      50

                                                                                                                      Source Skumatz SERA

                                                                                                                      Developing values for multiple states amp utilities

                                                                                                                      SERA

                                                                                                                      TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                      Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                      Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                      5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                      6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                      THANK YOU

                                                                                                                      Questions

                                                                                                                      Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                      SERA

                                                                                                                      POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                      53

                                                                                                                      SERA

                                                                                                                      NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                      Negative NEB values cost of barrier

                                                                                                                      Solar Water Heat NZ$ Euros

                                                                                                                      Solar Design NZ$ Euros

                                                                                                                      Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                      -23 -12 (079)

                                                                                                                      -11 -6 (06)

                                                                                                                      Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                      Source Skumatz Economic Research Associates research

                                                                                                                      SERA

                                                                                                                      CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                      NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                      -5-4-3-2-1012345

                                                                                                                      Op Co

                                                                                                                      st

                                                                                                                      Init C

                                                                                                                      ost

                                                                                                                      Maint

                                                                                                                      Perfo

                                                                                                                      rm

                                                                                                                      Prod

                                                                                                                      uctiv

                                                                                                                      Tena

                                                                                                                      nt Sa

                                                                                                                      t

                                                                                                                      Comf

                                                                                                                      ort

                                                                                                                      Appe

                                                                                                                      ar

                                                                                                                      Light

                                                                                                                      Qual

                                                                                                                      SellL

                                                                                                                      ease

                                                                                                                      Devp Ownr AampE

                                                                                                                      Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                      Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                      Source Skumatz Economic Research Associates Inc 2002

                                                                                                                      Owners more positive ndashUnderinvestment train

                                                                                                                      • Soup to nuts on non-energy benefits NEIs
                                                                                                                      • TOPICS
                                                                                                                      • BACKGROUND NEBS
                                                                                                                      • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                      • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                      • KEY APPLICATIONS OF NEBS
                                                                                                                      • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                      • Neb measurement
                                                                                                                      • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                      • MEASURING PARTICIPANTNEBs
                                                                                                                      • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                      • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                      • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                      • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                      • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                      • PROGRESS AND GAPS
                                                                                                                      • PROGRESS AND GAPS
                                                                                                                      • USING NEB NEI VALUES
                                                                                                                      • Slide Number 19
                                                                                                                      • ARE NEBS HIGH VALUE
                                                                                                                      • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                      • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                      • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                      • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                      • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                      • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                      • AUDIT FINANCIAL PROGRAM
                                                                                                                      • CampI NEW CONSTRUCTION
                                                                                                                      • CampI TECHNICAL ASSISTANCE
                                                                                                                      • EXPRESSING NEBS VALUEndashCxe
                                                                                                                      • Slide Number 31
                                                                                                                      • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                      • EMISSIONS (SOCIETAL NEBS)
                                                                                                                      • UTILITY NEBS
                                                                                                                      • ADJUSTED PAYBACKS
                                                                                                                      • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                      • NEBS IN BC
                                                                                                                      • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                      • APPLICATION TO BC
                                                                                                                      • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                      • STATE TREATMENT OF NEBS NEBs
                                                                                                                      • Slide Number 42
                                                                                                                      • SELECTED STATE PROGRESS
                                                                                                                      • SELECTED STATE PROGRESS
                                                                                                                      • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                      • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                      • DISCOUNT RATE MATTERS
                                                                                                                      • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                      • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                      • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                      • TAKEAWAYSON NEBSNEIS
                                                                                                                      • THANK YOUQuestions
                                                                                                                      • Slide Number 53
                                                                                                                      • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                      • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                        SERA

                                                                                                                        UTILITY NEBSUtility NEBs for Template Program

                                                                                                                        Debt WriteOff (util)13

                                                                                                                        Rate Subsidy(util)61

                                                                                                                        HealthSafety(util)0

                                                                                                                        Colln Costs (util)0

                                                                                                                        Gas Calls (util)0

                                                                                                                        Calls to CSRs(util)2

                                                                                                                        TampD (util)16

                                                                                                                        Arrears (util)0

                                                                                                                        Reconnects (util)0

                                                                                                                        Notices (util)7

                                                                                                                        Shutoffs (util)1

                                                                                                                        Rate subsidy TampD

                                                                                                                        Payment-related

                                                                                                                        Source Skumatz Economic Research Associates research

                                                                                                                        Example Low Income Weatherization

                                                                                                                        SERA

                                                                                                                        ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                                                        be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                                                        Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                                                        Value of 100 HALVES Payback doubles ROI

                                                                                                                        Source Skumatz Economics (SERA)

                                                                                                                        35

                                                                                                                        ( for NEBs Multiplier of 125)

                                                                                                                        SERA

                                                                                                                        NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                                                        36

                                                                                                                        hellip2001 ON

                                                                                                                        SERA

                                                                                                                        37

                                                                                                                        NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                                        For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                                        Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                                        BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                                        SERA

                                                                                                                        CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                                        38

                                                                                                                        Test Benefits Costs States Using Traditionally

                                                                                                                        Improved treat-ment with NEBs

                                                                                                                        Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                                        bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                                        bull Avoided gas and water supply costs

                                                                                                                        bull Program administration

                                                                                                                        bull Participant incentives

                                                                                                                        bull Increased supply cost

                                                                                                                        CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                                        Use cost only paid by the utility

                                                                                                                        Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                                        Same as above plus bull increased revenue

                                                                                                                        Same as above plus bull Decreased revenue

                                                                                                                        AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                                        Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                                        bull Participant direct costs

                                                                                                                        AR CA FL HI IA IN MI MN NY VA

                                                                                                                        Participant NEBs

                                                                                                                        Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                                        Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                                        SERA

                                                                                                                        APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                        Expansion Concern with risk calculation accuracy

                                                                                                                        39

                                                                                                                        Source Skumatz SERA research

                                                                                                                        SERA

                                                                                                                        40

                                                                                                                        METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                                        Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                                        Minimize Regulatory amp Implementer Risk

                                                                                                                        Minimize Evaluation Cost

                                                                                                                        Adder

                                                                                                                        Readily MeasurableHybrid

                                                                                                                        All NEBsSource SERA Research

                                                                                                                        SERA

                                                                                                                        41

                                                                                                                        STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                                        Screening Application Utilities regions

                                                                                                                        Program Marketing Fairly widespread use in utilities states across the country

                                                                                                                        MO

                                                                                                                        RE AGGRESSIVE ==gt

                                                                                                                        Test Pgm Screen - adder

                                                                                                                        IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                                        Test Pgm Screen - readily measured

                                                                                                                        MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                                        Test Hybrid (potential adder amp measured)

                                                                                                                        CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                                        Test Pgm screen - Broad

                                                                                                                        With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                                        Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                                        SERA

                                                                                                                        42

                                                                                                                        Key

                                                                                                                        States with NEBs in CE tests for At least one type of program

                                                                                                                        copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                                        STATES WITH NEBS IN CE TESTS

                                                                                                                        NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                                        groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                                        SERA

                                                                                                                        SELECTED STATE PROGRESS CA

                                                                                                                        2001 ndash LIPPT model NY

                                                                                                                        Mid-2000s measurement amp scenarios not included CO

                                                                                                                        Adders (10 electric 25 LI 5 gas) VT

                                                                                                                        15 adder helped by previous research DC

                                                                                                                        10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                                        43

                                                                                                                        Source Skumatz SERA research

                                                                                                                        SERA

                                                                                                                        SELECTED STATE PROGRESS Dominos ongoing

                                                                                                                        Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                                        Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                                        Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                                        Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                                        44

                                                                                                                        Source Skumatz SERA research

                                                                                                                        SERA

                                                                                                                        APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                        Expansion Concern with risk calculation accuracy

                                                                                                                        45

                                                                                                                        Source Skumatz SERA research

                                                                                                                        SERA

                                                                                                                        46

                                                                                                                        NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                        [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                        SourceSkumatz SERA Independent research

                                                                                                                        NTG ndash accuracy measurement incomplete

                                                                                                                        Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                        NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                        EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                        Cost Complicated expensive local changes

                                                                                                                        RELATIVE Risk from NEBs

                                                                                                                        Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                        Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                        NEBs as accurate as many inputs (all could improve)

                                                                                                                        SERA

                                                                                                                        DISCOUNT RATE MATTERS

                                                                                                                        47

                                                                                                                        Source Skumatz SERA Research

                                                                                                                        0

                                                                                                                        20

                                                                                                                        40

                                                                                                                        60

                                                                                                                        80

                                                                                                                        100

                                                                                                                        120

                                                                                                                        1 2 3 4 5 6 7 8 9 10

                                                                                                                        PV Ratios vs 1 Discount Rate

                                                                                                                        10 yr PV 20 yr PV

                                                                                                                        Discount rate - percent

                                                                                                                        WACCSocialLTT

                                                                                                                        Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                        SERA

                                                                                                                        EUL VARIATIONS-ADOPTED VALUES

                                                                                                                        Source Skumatz Economic Research Associates SERA independent research

                                                                                                                        0

                                                                                                                        50

                                                                                                                        100

                                                                                                                        150

                                                                                                                        200

                                                                                                                        250

                                                                                                                        300

                                                                                                                        350

                                                                                                                        400

                                                                                                                        450

                                                                                                                        0 10 20 30 40 50 60 70

                                                                                                                        EUL Variation by Measure

                                                                                                                        67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                        Poorly supportedOldTechnology change

                                                                                                                        SERA

                                                                                                                        KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                        Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                        measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                        change the BC conclusion

                                                                                                                        On the cusphellip

                                                                                                                        If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                        IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                        Source SERA all rights reserved

                                                                                                                        SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                        SERA

                                                                                                                        IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                        Utility Soc Part Conserv Recrsquom

                                                                                                                        Rationale

                                                                                                                        Base Percent X X X Program-invariant

                                                                                                                        Low Income X X X X Multiple sources

                                                                                                                        Weatherization X X X Substantial Participantimpacts

                                                                                                                        Measure Program-specific

                                                                                                                        X Varies by measure sector

                                                                                                                        Other Recomrsquos Local Research

                                                                                                                        50

                                                                                                                        Source Skumatz SERA

                                                                                                                        Developing values for multiple states amp utilities

                                                                                                                        SERA

                                                                                                                        TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                        Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                        Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                        5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                        6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                        THANK YOU

                                                                                                                        Questions

                                                                                                                        Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                        SERA

                                                                                                                        POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                        53

                                                                                                                        SERA

                                                                                                                        NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                        Negative NEB values cost of barrier

                                                                                                                        Solar Water Heat NZ$ Euros

                                                                                                                        Solar Design NZ$ Euros

                                                                                                                        Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                        -23 -12 (079)

                                                                                                                        -11 -6 (06)

                                                                                                                        Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                        Source Skumatz Economic Research Associates research

                                                                                                                        SERA

                                                                                                                        CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                        NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                        -5-4-3-2-1012345

                                                                                                                        Op Co

                                                                                                                        st

                                                                                                                        Init C

                                                                                                                        ost

                                                                                                                        Maint

                                                                                                                        Perfo

                                                                                                                        rm

                                                                                                                        Prod

                                                                                                                        uctiv

                                                                                                                        Tena

                                                                                                                        nt Sa

                                                                                                                        t

                                                                                                                        Comf

                                                                                                                        ort

                                                                                                                        Appe

                                                                                                                        ar

                                                                                                                        Light

                                                                                                                        Qual

                                                                                                                        SellL

                                                                                                                        ease

                                                                                                                        Devp Ownr AampE

                                                                                                                        Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                        Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                        Source Skumatz Economic Research Associates Inc 2002

                                                                                                                        Owners more positive ndashUnderinvestment train

                                                                                                                        • Soup to nuts on non-energy benefits NEIs
                                                                                                                        • TOPICS
                                                                                                                        • BACKGROUND NEBS
                                                                                                                        • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                        • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                        • KEY APPLICATIONS OF NEBS
                                                                                                                        • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                        • Neb measurement
                                                                                                                        • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                        • MEASURING PARTICIPANTNEBs
                                                                                                                        • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                        • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                        • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                        • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                        • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                        • PROGRESS AND GAPS
                                                                                                                        • PROGRESS AND GAPS
                                                                                                                        • USING NEB NEI VALUES
                                                                                                                        • Slide Number 19
                                                                                                                        • ARE NEBS HIGH VALUE
                                                                                                                        • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                        • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                        • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                        • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                        • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                        • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                        • AUDIT FINANCIAL PROGRAM
                                                                                                                        • CampI NEW CONSTRUCTION
                                                                                                                        • CampI TECHNICAL ASSISTANCE
                                                                                                                        • EXPRESSING NEBS VALUEndashCxe
                                                                                                                        • Slide Number 31
                                                                                                                        • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                        • EMISSIONS (SOCIETAL NEBS)
                                                                                                                        • UTILITY NEBS
                                                                                                                        • ADJUSTED PAYBACKS
                                                                                                                        • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                        • NEBS IN BC
                                                                                                                        • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                        • APPLICATION TO BC
                                                                                                                        • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                        • STATE TREATMENT OF NEBS NEBs
                                                                                                                        • Slide Number 42
                                                                                                                        • SELECTED STATE PROGRESS
                                                                                                                        • SELECTED STATE PROGRESS
                                                                                                                        • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                        • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                        • DISCOUNT RATE MATTERS
                                                                                                                        • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                        • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                        • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                        • TAKEAWAYSON NEBSNEIS
                                                                                                                        • THANK YOUQuestions
                                                                                                                        • Slide Number 53
                                                                                                                        • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                        • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                          SERA

                                                                                                                          ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can

                                                                                                                          be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19

                                                                                                                          Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc

                                                                                                                          Value of 100 HALVES Payback doubles ROI

                                                                                                                          Source Skumatz Economics (SERA)

                                                                                                                          35

                                                                                                                          ( for NEBs Multiplier of 125)

                                                                                                                          SERA

                                                                                                                          NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                                                          36

                                                                                                                          hellip2001 ON

                                                                                                                          SERA

                                                                                                                          37

                                                                                                                          NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                                          For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                                          Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                                          BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                                          SERA

                                                                                                                          CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                                          38

                                                                                                                          Test Benefits Costs States Using Traditionally

                                                                                                                          Improved treat-ment with NEBs

                                                                                                                          Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                                          bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                                          bull Avoided gas and water supply costs

                                                                                                                          bull Program administration

                                                                                                                          bull Participant incentives

                                                                                                                          bull Increased supply cost

                                                                                                                          CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                                          Use cost only paid by the utility

                                                                                                                          Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                                          Same as above plus bull increased revenue

                                                                                                                          Same as above plus bull Decreased revenue

                                                                                                                          AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                                          Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                                          bull Participant direct costs

                                                                                                                          AR CA FL HI IA IN MI MN NY VA

                                                                                                                          Participant NEBs

                                                                                                                          Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                                          Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                                          SERA

                                                                                                                          APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                          Expansion Concern with risk calculation accuracy

                                                                                                                          39

                                                                                                                          Source Skumatz SERA research

                                                                                                                          SERA

                                                                                                                          40

                                                                                                                          METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                                          Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                                          Minimize Regulatory amp Implementer Risk

                                                                                                                          Minimize Evaluation Cost

                                                                                                                          Adder

                                                                                                                          Readily MeasurableHybrid

                                                                                                                          All NEBsSource SERA Research

                                                                                                                          SERA

                                                                                                                          41

                                                                                                                          STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                                          Screening Application Utilities regions

                                                                                                                          Program Marketing Fairly widespread use in utilities states across the country

                                                                                                                          MO

                                                                                                                          RE AGGRESSIVE ==gt

                                                                                                                          Test Pgm Screen - adder

                                                                                                                          IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                                          Test Pgm Screen - readily measured

                                                                                                                          MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                                          Test Hybrid (potential adder amp measured)

                                                                                                                          CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                                          Test Pgm screen - Broad

                                                                                                                          With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                                          Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                                          SERA

                                                                                                                          42

                                                                                                                          Key

                                                                                                                          States with NEBs in CE tests for At least one type of program

                                                                                                                          copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                                          STATES WITH NEBS IN CE TESTS

                                                                                                                          NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                                          groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                                          SERA

                                                                                                                          SELECTED STATE PROGRESS CA

                                                                                                                          2001 ndash LIPPT model NY

                                                                                                                          Mid-2000s measurement amp scenarios not included CO

                                                                                                                          Adders (10 electric 25 LI 5 gas) VT

                                                                                                                          15 adder helped by previous research DC

                                                                                                                          10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                                          43

                                                                                                                          Source Skumatz SERA research

                                                                                                                          SERA

                                                                                                                          SELECTED STATE PROGRESS Dominos ongoing

                                                                                                                          Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                                          Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                                          Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                                          Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                                          44

                                                                                                                          Source Skumatz SERA research

                                                                                                                          SERA

                                                                                                                          APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                          Expansion Concern with risk calculation accuracy

                                                                                                                          45

                                                                                                                          Source Skumatz SERA research

                                                                                                                          SERA

                                                                                                                          46

                                                                                                                          NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                          [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                          SourceSkumatz SERA Independent research

                                                                                                                          NTG ndash accuracy measurement incomplete

                                                                                                                          Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                          NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                          EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                          Cost Complicated expensive local changes

                                                                                                                          RELATIVE Risk from NEBs

                                                                                                                          Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                          Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                          NEBs as accurate as many inputs (all could improve)

                                                                                                                          SERA

                                                                                                                          DISCOUNT RATE MATTERS

                                                                                                                          47

                                                                                                                          Source Skumatz SERA Research

                                                                                                                          0

                                                                                                                          20

                                                                                                                          40

                                                                                                                          60

                                                                                                                          80

                                                                                                                          100

                                                                                                                          120

                                                                                                                          1 2 3 4 5 6 7 8 9 10

                                                                                                                          PV Ratios vs 1 Discount Rate

                                                                                                                          10 yr PV 20 yr PV

                                                                                                                          Discount rate - percent

                                                                                                                          WACCSocialLTT

                                                                                                                          Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                          SERA

                                                                                                                          EUL VARIATIONS-ADOPTED VALUES

                                                                                                                          Source Skumatz Economic Research Associates SERA independent research

                                                                                                                          0

                                                                                                                          50

                                                                                                                          100

                                                                                                                          150

                                                                                                                          200

                                                                                                                          250

                                                                                                                          300

                                                                                                                          350

                                                                                                                          400

                                                                                                                          450

                                                                                                                          0 10 20 30 40 50 60 70

                                                                                                                          EUL Variation by Measure

                                                                                                                          67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                          Poorly supportedOldTechnology change

                                                                                                                          SERA

                                                                                                                          KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                          Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                          measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                          change the BC conclusion

                                                                                                                          On the cusphellip

                                                                                                                          If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                          IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                          Source SERA all rights reserved

                                                                                                                          SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                          SERA

                                                                                                                          IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                          Utility Soc Part Conserv Recrsquom

                                                                                                                          Rationale

                                                                                                                          Base Percent X X X Program-invariant

                                                                                                                          Low Income X X X X Multiple sources

                                                                                                                          Weatherization X X X Substantial Participantimpacts

                                                                                                                          Measure Program-specific

                                                                                                                          X Varies by measure sector

                                                                                                                          Other Recomrsquos Local Research

                                                                                                                          50

                                                                                                                          Source Skumatz SERA

                                                                                                                          Developing values for multiple states amp utilities

                                                                                                                          SERA

                                                                                                                          TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                          Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                          Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                          5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                          6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                          THANK YOU

                                                                                                                          Questions

                                                                                                                          Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                          SERA

                                                                                                                          POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                          53

                                                                                                                          SERA

                                                                                                                          NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                          Negative NEB values cost of barrier

                                                                                                                          Solar Water Heat NZ$ Euros

                                                                                                                          Solar Design NZ$ Euros

                                                                                                                          Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                          -23 -12 (079)

                                                                                                                          -11 -6 (06)

                                                                                                                          Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                          Source Skumatz Economic Research Associates research

                                                                                                                          SERA

                                                                                                                          CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                          NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                          -5-4-3-2-1012345

                                                                                                                          Op Co

                                                                                                                          st

                                                                                                                          Init C

                                                                                                                          ost

                                                                                                                          Maint

                                                                                                                          Perfo

                                                                                                                          rm

                                                                                                                          Prod

                                                                                                                          uctiv

                                                                                                                          Tena

                                                                                                                          nt Sa

                                                                                                                          t

                                                                                                                          Comf

                                                                                                                          ort

                                                                                                                          Appe

                                                                                                                          ar

                                                                                                                          Light

                                                                                                                          Qual

                                                                                                                          SellL

                                                                                                                          ease

                                                                                                                          Devp Ownr AampE

                                                                                                                          Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                          Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                          Source Skumatz Economic Research Associates Inc 2002

                                                                                                                          Owners more positive ndashUnderinvestment train

                                                                                                                          • Soup to nuts on non-energy benefits NEIs
                                                                                                                          • TOPICS
                                                                                                                          • BACKGROUND NEBS
                                                                                                                          • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                          • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                          • KEY APPLICATIONS OF NEBS
                                                                                                                          • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                          • Neb measurement
                                                                                                                          • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                          • MEASURING PARTICIPANTNEBs
                                                                                                                          • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                          • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                          • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                          • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                          • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                          • PROGRESS AND GAPS
                                                                                                                          • PROGRESS AND GAPS
                                                                                                                          • USING NEB NEI VALUES
                                                                                                                          • Slide Number 19
                                                                                                                          • ARE NEBS HIGH VALUE
                                                                                                                          • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                          • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                          • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                          • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                          • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                          • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                          • AUDIT FINANCIAL PROGRAM
                                                                                                                          • CampI NEW CONSTRUCTION
                                                                                                                          • CampI TECHNICAL ASSISTANCE
                                                                                                                          • EXPRESSING NEBS VALUEndashCxe
                                                                                                                          • Slide Number 31
                                                                                                                          • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                          • EMISSIONS (SOCIETAL NEBS)
                                                                                                                          • UTILITY NEBS
                                                                                                                          • ADJUSTED PAYBACKS
                                                                                                                          • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                          • NEBS IN BC
                                                                                                                          • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                          • APPLICATION TO BC
                                                                                                                          • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                          • STATE TREATMENT OF NEBS NEBs
                                                                                                                          • Slide Number 42
                                                                                                                          • SELECTED STATE PROGRESS
                                                                                                                          • SELECTED STATE PROGRESS
                                                                                                                          • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                          • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                          • DISCOUNT RATE MATTERS
                                                                                                                          • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                          • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                          • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                          • TAKEAWAYSON NEBSNEIS
                                                                                                                          • THANK YOUQuestions
                                                                                                                          • Slide Number 53
                                                                                                                          • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                          • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                            SERA

                                                                                                                            NEBS IN COST-EFFECTIVENESS APPLICATIONS

                                                                                                                            36

                                                                                                                            hellip2001 ON

                                                                                                                            SERA

                                                                                                                            37

                                                                                                                            NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                                            For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                                            Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                                            BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                                            SERA

                                                                                                                            CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                                            38

                                                                                                                            Test Benefits Costs States Using Traditionally

                                                                                                                            Improved treat-ment with NEBs

                                                                                                                            Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                                            bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                                            bull Avoided gas and water supply costs

                                                                                                                            bull Program administration

                                                                                                                            bull Participant incentives

                                                                                                                            bull Increased supply cost

                                                                                                                            CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                                            Use cost only paid by the utility

                                                                                                                            Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                                            Same as above plus bull increased revenue

                                                                                                                            Same as above plus bull Decreased revenue

                                                                                                                            AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                                            Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                                            bull Participant direct costs

                                                                                                                            AR CA FL HI IA IN MI MN NY VA

                                                                                                                            Participant NEBs

                                                                                                                            Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                                            Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                                            SERA

                                                                                                                            APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                            Expansion Concern with risk calculation accuracy

                                                                                                                            39

                                                                                                                            Source Skumatz SERA research

                                                                                                                            SERA

                                                                                                                            40

                                                                                                                            METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                                            Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                                            Minimize Regulatory amp Implementer Risk

                                                                                                                            Minimize Evaluation Cost

                                                                                                                            Adder

                                                                                                                            Readily MeasurableHybrid

                                                                                                                            All NEBsSource SERA Research

                                                                                                                            SERA

                                                                                                                            41

                                                                                                                            STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                                            Screening Application Utilities regions

                                                                                                                            Program Marketing Fairly widespread use in utilities states across the country

                                                                                                                            MO

                                                                                                                            RE AGGRESSIVE ==gt

                                                                                                                            Test Pgm Screen - adder

                                                                                                                            IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                                            Test Pgm Screen - readily measured

                                                                                                                            MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                                            Test Hybrid (potential adder amp measured)

                                                                                                                            CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                                            Test Pgm screen - Broad

                                                                                                                            With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                                            Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                                            SERA

                                                                                                                            42

                                                                                                                            Key

                                                                                                                            States with NEBs in CE tests for At least one type of program

                                                                                                                            copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                                            STATES WITH NEBS IN CE TESTS

                                                                                                                            NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                                            groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                                            SERA

                                                                                                                            SELECTED STATE PROGRESS CA

                                                                                                                            2001 ndash LIPPT model NY

                                                                                                                            Mid-2000s measurement amp scenarios not included CO

                                                                                                                            Adders (10 electric 25 LI 5 gas) VT

                                                                                                                            15 adder helped by previous research DC

                                                                                                                            10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                                            43

                                                                                                                            Source Skumatz SERA research

                                                                                                                            SERA

                                                                                                                            SELECTED STATE PROGRESS Dominos ongoing

                                                                                                                            Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                                            Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                                            Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                                            Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                                            44

                                                                                                                            Source Skumatz SERA research

                                                                                                                            SERA

                                                                                                                            APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                            Expansion Concern with risk calculation accuracy

                                                                                                                            45

                                                                                                                            Source Skumatz SERA research

                                                                                                                            SERA

                                                                                                                            46

                                                                                                                            NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                            [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                            SourceSkumatz SERA Independent research

                                                                                                                            NTG ndash accuracy measurement incomplete

                                                                                                                            Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                            NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                            EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                            Cost Complicated expensive local changes

                                                                                                                            RELATIVE Risk from NEBs

                                                                                                                            Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                            Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                            NEBs as accurate as many inputs (all could improve)

                                                                                                                            SERA

                                                                                                                            DISCOUNT RATE MATTERS

                                                                                                                            47

                                                                                                                            Source Skumatz SERA Research

                                                                                                                            0

                                                                                                                            20

                                                                                                                            40

                                                                                                                            60

                                                                                                                            80

                                                                                                                            100

                                                                                                                            120

                                                                                                                            1 2 3 4 5 6 7 8 9 10

                                                                                                                            PV Ratios vs 1 Discount Rate

                                                                                                                            10 yr PV 20 yr PV

                                                                                                                            Discount rate - percent

                                                                                                                            WACCSocialLTT

                                                                                                                            Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                            SERA

                                                                                                                            EUL VARIATIONS-ADOPTED VALUES

                                                                                                                            Source Skumatz Economic Research Associates SERA independent research

                                                                                                                            0

                                                                                                                            50

                                                                                                                            100

                                                                                                                            150

                                                                                                                            200

                                                                                                                            250

                                                                                                                            300

                                                                                                                            350

                                                                                                                            400

                                                                                                                            450

                                                                                                                            0 10 20 30 40 50 60 70

                                                                                                                            EUL Variation by Measure

                                                                                                                            67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                            Poorly supportedOldTechnology change

                                                                                                                            SERA

                                                                                                                            KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                            Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                            measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                            change the BC conclusion

                                                                                                                            On the cusphellip

                                                                                                                            If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                            IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                            Source SERA all rights reserved

                                                                                                                            SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                            SERA

                                                                                                                            IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                            Utility Soc Part Conserv Recrsquom

                                                                                                                            Rationale

                                                                                                                            Base Percent X X X Program-invariant

                                                                                                                            Low Income X X X X Multiple sources

                                                                                                                            Weatherization X X X Substantial Participantimpacts

                                                                                                                            Measure Program-specific

                                                                                                                            X Varies by measure sector

                                                                                                                            Other Recomrsquos Local Research

                                                                                                                            50

                                                                                                                            Source Skumatz SERA

                                                                                                                            Developing values for multiple states amp utilities

                                                                                                                            SERA

                                                                                                                            TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                            Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                            Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                            5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                            6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                            THANK YOU

                                                                                                                            Questions

                                                                                                                            Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                            SERA

                                                                                                                            POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                            53

                                                                                                                            SERA

                                                                                                                            NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                            Negative NEB values cost of barrier

                                                                                                                            Solar Water Heat NZ$ Euros

                                                                                                                            Solar Design NZ$ Euros

                                                                                                                            Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                            -23 -12 (079)

                                                                                                                            -11 -6 (06)

                                                                                                                            Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                            Source Skumatz Economic Research Associates research

                                                                                                                            SERA

                                                                                                                            CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                            NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                            -5-4-3-2-1012345

                                                                                                                            Op Co

                                                                                                                            st

                                                                                                                            Init C

                                                                                                                            ost

                                                                                                                            Maint

                                                                                                                            Perfo

                                                                                                                            rm

                                                                                                                            Prod

                                                                                                                            uctiv

                                                                                                                            Tena

                                                                                                                            nt Sa

                                                                                                                            t

                                                                                                                            Comf

                                                                                                                            ort

                                                                                                                            Appe

                                                                                                                            ar

                                                                                                                            Light

                                                                                                                            Qual

                                                                                                                            SellL

                                                                                                                            ease

                                                                                                                            Devp Ownr AampE

                                                                                                                            Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                            Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                            Source Skumatz Economic Research Associates Inc 2002

                                                                                                                            Owners more positive ndashUnderinvestment train

                                                                                                                            • Soup to nuts on non-energy benefits NEIs
                                                                                                                            • TOPICS
                                                                                                                            • BACKGROUND NEBS
                                                                                                                            • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                            • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                            • KEY APPLICATIONS OF NEBS
                                                                                                                            • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                            • Neb measurement
                                                                                                                            • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                            • MEASURING PARTICIPANTNEBs
                                                                                                                            • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                            • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                            • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                            • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                            • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                            • PROGRESS AND GAPS
                                                                                                                            • PROGRESS AND GAPS
                                                                                                                            • USING NEB NEI VALUES
                                                                                                                            • Slide Number 19
                                                                                                                            • ARE NEBS HIGH VALUE
                                                                                                                            • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                            • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                            • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                            • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                            • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                            • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                            • AUDIT FINANCIAL PROGRAM
                                                                                                                            • CampI NEW CONSTRUCTION
                                                                                                                            • CampI TECHNICAL ASSISTANCE
                                                                                                                            • EXPRESSING NEBS VALUEndashCxe
                                                                                                                            • Slide Number 31
                                                                                                                            • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                            • EMISSIONS (SOCIETAL NEBS)
                                                                                                                            • UTILITY NEBS
                                                                                                                            • ADJUSTED PAYBACKS
                                                                                                                            • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                            • NEBS IN BC
                                                                                                                            • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                            • APPLICATION TO BC
                                                                                                                            • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                            • STATE TREATMENT OF NEBS NEBs
                                                                                                                            • Slide Number 42
                                                                                                                            • SELECTED STATE PROGRESS
                                                                                                                            • SELECTED STATE PROGRESS
                                                                                                                            • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                            • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                            • DISCOUNT RATE MATTERS
                                                                                                                            • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                            • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                            • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                            • TAKEAWAYSON NEBSNEIS
                                                                                                                            • THANK YOUQuestions
                                                                                                                            • Slide Number 53
                                                                                                                            • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                            • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                              SERA

                                                                                                                              37

                                                                                                                              NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs

                                                                                                                              For true representation of B amp C elements of NEBs address missing factor bias

                                                                                                                              Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT

                                                                                                                              BC applications were considered early on then pulled back awaiting more quantitative evidence

                                                                                                                              SERA

                                                                                                                              CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                                              38

                                                                                                                              Test Benefits Costs States Using Traditionally

                                                                                                                              Improved treat-ment with NEBs

                                                                                                                              Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                                              bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                                              bull Avoided gas and water supply costs

                                                                                                                              bull Program administration

                                                                                                                              bull Participant incentives

                                                                                                                              bull Increased supply cost

                                                                                                                              CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                                              Use cost only paid by the utility

                                                                                                                              Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                                              Same as above plus bull increased revenue

                                                                                                                              Same as above plus bull Decreased revenue

                                                                                                                              AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                                              Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                                              bull Participant direct costs

                                                                                                                              AR CA FL HI IA IN MI MN NY VA

                                                                                                                              Participant NEBs

                                                                                                                              Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                                              Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                                              SERA

                                                                                                                              APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                              Expansion Concern with risk calculation accuracy

                                                                                                                              39

                                                                                                                              Source Skumatz SERA research

                                                                                                                              SERA

                                                                                                                              40

                                                                                                                              METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                                              Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                                              Minimize Regulatory amp Implementer Risk

                                                                                                                              Minimize Evaluation Cost

                                                                                                                              Adder

                                                                                                                              Readily MeasurableHybrid

                                                                                                                              All NEBsSource SERA Research

                                                                                                                              SERA

                                                                                                                              41

                                                                                                                              STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                                              Screening Application Utilities regions

                                                                                                                              Program Marketing Fairly widespread use in utilities states across the country

                                                                                                                              MO

                                                                                                                              RE AGGRESSIVE ==gt

                                                                                                                              Test Pgm Screen - adder

                                                                                                                              IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                                              Test Pgm Screen - readily measured

                                                                                                                              MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                                              Test Hybrid (potential adder amp measured)

                                                                                                                              CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                                              Test Pgm screen - Broad

                                                                                                                              With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                                              Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                                              SERA

                                                                                                                              42

                                                                                                                              Key

                                                                                                                              States with NEBs in CE tests for At least one type of program

                                                                                                                              copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                                              STATES WITH NEBS IN CE TESTS

                                                                                                                              NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                                              groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                                              SERA

                                                                                                                              SELECTED STATE PROGRESS CA

                                                                                                                              2001 ndash LIPPT model NY

                                                                                                                              Mid-2000s measurement amp scenarios not included CO

                                                                                                                              Adders (10 electric 25 LI 5 gas) VT

                                                                                                                              15 adder helped by previous research DC

                                                                                                                              10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                                              43

                                                                                                                              Source Skumatz SERA research

                                                                                                                              SERA

                                                                                                                              SELECTED STATE PROGRESS Dominos ongoing

                                                                                                                              Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                                              Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                                              Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                                              Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                                              44

                                                                                                                              Source Skumatz SERA research

                                                                                                                              SERA

                                                                                                                              APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                              Expansion Concern with risk calculation accuracy

                                                                                                                              45

                                                                                                                              Source Skumatz SERA research

                                                                                                                              SERA

                                                                                                                              46

                                                                                                                              NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                              [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                              SourceSkumatz SERA Independent research

                                                                                                                              NTG ndash accuracy measurement incomplete

                                                                                                                              Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                              NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                              EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                              Cost Complicated expensive local changes

                                                                                                                              RELATIVE Risk from NEBs

                                                                                                                              Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                              Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                              NEBs as accurate as many inputs (all could improve)

                                                                                                                              SERA

                                                                                                                              DISCOUNT RATE MATTERS

                                                                                                                              47

                                                                                                                              Source Skumatz SERA Research

                                                                                                                              0

                                                                                                                              20

                                                                                                                              40

                                                                                                                              60

                                                                                                                              80

                                                                                                                              100

                                                                                                                              120

                                                                                                                              1 2 3 4 5 6 7 8 9 10

                                                                                                                              PV Ratios vs 1 Discount Rate

                                                                                                                              10 yr PV 20 yr PV

                                                                                                                              Discount rate - percent

                                                                                                                              WACCSocialLTT

                                                                                                                              Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                              SERA

                                                                                                                              EUL VARIATIONS-ADOPTED VALUES

                                                                                                                              Source Skumatz Economic Research Associates SERA independent research

                                                                                                                              0

                                                                                                                              50

                                                                                                                              100

                                                                                                                              150

                                                                                                                              200

                                                                                                                              250

                                                                                                                              300

                                                                                                                              350

                                                                                                                              400

                                                                                                                              450

                                                                                                                              0 10 20 30 40 50 60 70

                                                                                                                              EUL Variation by Measure

                                                                                                                              67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                              Poorly supportedOldTechnology change

                                                                                                                              SERA

                                                                                                                              KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                              Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                              measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                              change the BC conclusion

                                                                                                                              On the cusphellip

                                                                                                                              If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                              IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                              Source SERA all rights reserved

                                                                                                                              SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                              SERA

                                                                                                                              IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                              Utility Soc Part Conserv Recrsquom

                                                                                                                              Rationale

                                                                                                                              Base Percent X X X Program-invariant

                                                                                                                              Low Income X X X X Multiple sources

                                                                                                                              Weatherization X X X Substantial Participantimpacts

                                                                                                                              Measure Program-specific

                                                                                                                              X Varies by measure sector

                                                                                                                              Other Recomrsquos Local Research

                                                                                                                              50

                                                                                                                              Source Skumatz SERA

                                                                                                                              Developing values for multiple states amp utilities

                                                                                                                              SERA

                                                                                                                              TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                              Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                              Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                              5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                              6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                              THANK YOU

                                                                                                                              Questions

                                                                                                                              Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                              SERA

                                                                                                                              POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                              53

                                                                                                                              SERA

                                                                                                                              NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                              Negative NEB values cost of barrier

                                                                                                                              Solar Water Heat NZ$ Euros

                                                                                                                              Solar Design NZ$ Euros

                                                                                                                              Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                              -23 -12 (079)

                                                                                                                              -11 -6 (06)

                                                                                                                              Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                              Source Skumatz Economic Research Associates research

                                                                                                                              SERA

                                                                                                                              CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                              NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                              -5-4-3-2-1012345

                                                                                                                              Op Co

                                                                                                                              st

                                                                                                                              Init C

                                                                                                                              ost

                                                                                                                              Maint

                                                                                                                              Perfo

                                                                                                                              rm

                                                                                                                              Prod

                                                                                                                              uctiv

                                                                                                                              Tena

                                                                                                                              nt Sa

                                                                                                                              t

                                                                                                                              Comf

                                                                                                                              ort

                                                                                                                              Appe

                                                                                                                              ar

                                                                                                                              Light

                                                                                                                              Qual

                                                                                                                              SellL

                                                                                                                              ease

                                                                                                                              Devp Ownr AampE

                                                                                                                              Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                              Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                              Source Skumatz Economic Research Associates Inc 2002

                                                                                                                              Owners more positive ndashUnderinvestment train

                                                                                                                              • Soup to nuts on non-energy benefits NEIs
                                                                                                                              • TOPICS
                                                                                                                              • BACKGROUND NEBS
                                                                                                                              • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                              • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                              • KEY APPLICATIONS OF NEBS
                                                                                                                              • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                              • Neb measurement
                                                                                                                              • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                              • MEASURING PARTICIPANTNEBs
                                                                                                                              • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                              • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                              • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                              • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                              • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                              • PROGRESS AND GAPS
                                                                                                                              • PROGRESS AND GAPS
                                                                                                                              • USING NEB NEI VALUES
                                                                                                                              • Slide Number 19
                                                                                                                              • ARE NEBS HIGH VALUE
                                                                                                                              • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                              • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                              • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                              • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                              • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                              • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                              • AUDIT FINANCIAL PROGRAM
                                                                                                                              • CampI NEW CONSTRUCTION
                                                                                                                              • CampI TECHNICAL ASSISTANCE
                                                                                                                              • EXPRESSING NEBS VALUEndashCxe
                                                                                                                              • Slide Number 31
                                                                                                                              • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                              • EMISSIONS (SOCIETAL NEBS)
                                                                                                                              • UTILITY NEBS
                                                                                                                              • ADJUSTED PAYBACKS
                                                                                                                              • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                              • NEBS IN BC
                                                                                                                              • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                              • APPLICATION TO BC
                                                                                                                              • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                              • STATE TREATMENT OF NEBS NEBs
                                                                                                                              • Slide Number 42
                                                                                                                              • SELECTED STATE PROGRESS
                                                                                                                              • SELECTED STATE PROGRESS
                                                                                                                              • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                              • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                              • DISCOUNT RATE MATTERS
                                                                                                                              • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                              • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                              • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                              • TAKEAWAYSON NEBSNEIS
                                                                                                                              • THANK YOUQuestions
                                                                                                                              • Slide Number 53
                                                                                                                              • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                              • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                SERA

                                                                                                                                CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip

                                                                                                                                38

                                                                                                                                Test Benefits Costs States Using Traditionally

                                                                                                                                Improved treat-ment with NEBs

                                                                                                                                Utility Cost (or Program Administrator Test) (UCT or PAC)

                                                                                                                                bull Avoided supply costs for transmission distribution and generation (TDampG)

                                                                                                                                bull Avoided gas and water supply costs

                                                                                                                                bull Program administration

                                                                                                                                bull Participant incentives

                                                                                                                                bull Increased supply cost

                                                                                                                                CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA

                                                                                                                                Use cost only paid by the utility

                                                                                                                                Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)

                                                                                                                                Same as above plus bull increased revenue

                                                                                                                                Same as above plus bull Decreased revenue

                                                                                                                                AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI

                                                                                                                                Participant cost bull Utility bill reductions bull Participant incentives

                                                                                                                                bull Participant direct costs

                                                                                                                                AR CA FL HI IA IN MI MN NY VA

                                                                                                                                Participant NEBs

                                                                                                                                Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip

                                                                                                                                Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author

                                                                                                                                SERA

                                                                                                                                APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                                Expansion Concern with risk calculation accuracy

                                                                                                                                39

                                                                                                                                Source Skumatz SERA research

                                                                                                                                SERA

                                                                                                                                40

                                                                                                                                METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                                                Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                                                Minimize Regulatory amp Implementer Risk

                                                                                                                                Minimize Evaluation Cost

                                                                                                                                Adder

                                                                                                                                Readily MeasurableHybrid

                                                                                                                                All NEBsSource SERA Research

                                                                                                                                SERA

                                                                                                                                41

                                                                                                                                STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                                                Screening Application Utilities regions

                                                                                                                                Program Marketing Fairly widespread use in utilities states across the country

                                                                                                                                MO

                                                                                                                                RE AGGRESSIVE ==gt

                                                                                                                                Test Pgm Screen - adder

                                                                                                                                IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                                                Test Pgm Screen - readily measured

                                                                                                                                MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                                                Test Hybrid (potential adder amp measured)

                                                                                                                                CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                                                Test Pgm screen - Broad

                                                                                                                                With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                                                Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                                                SERA

                                                                                                                                42

                                                                                                                                Key

                                                                                                                                States with NEBs in CE tests for At least one type of program

                                                                                                                                copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                                                STATES WITH NEBS IN CE TESTS

                                                                                                                                NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                                                groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                                                SERA

                                                                                                                                SELECTED STATE PROGRESS CA

                                                                                                                                2001 ndash LIPPT model NY

                                                                                                                                Mid-2000s measurement amp scenarios not included CO

                                                                                                                                Adders (10 electric 25 LI 5 gas) VT

                                                                                                                                15 adder helped by previous research DC

                                                                                                                                10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                                                43

                                                                                                                                Source Skumatz SERA research

                                                                                                                                SERA

                                                                                                                                SELECTED STATE PROGRESS Dominos ongoing

                                                                                                                                Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                                                Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                                                Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                                                Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                                                44

                                                                                                                                Source Skumatz SERA research

                                                                                                                                SERA

                                                                                                                                APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                                Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                                Expansion Concern with risk calculation accuracy

                                                                                                                                45

                                                                                                                                Source Skumatz SERA research

                                                                                                                                SERA

                                                                                                                                46

                                                                                                                                NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                                [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                                SourceSkumatz SERA Independent research

                                                                                                                                NTG ndash accuracy measurement incomplete

                                                                                                                                Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                                NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                                EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                                Cost Complicated expensive local changes

                                                                                                                                RELATIVE Risk from NEBs

                                                                                                                                Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                                Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                                NEBs as accurate as many inputs (all could improve)

                                                                                                                                SERA

                                                                                                                                DISCOUNT RATE MATTERS

                                                                                                                                47

                                                                                                                                Source Skumatz SERA Research

                                                                                                                                0

                                                                                                                                20

                                                                                                                                40

                                                                                                                                60

                                                                                                                                80

                                                                                                                                100

                                                                                                                                120

                                                                                                                                1 2 3 4 5 6 7 8 9 10

                                                                                                                                PV Ratios vs 1 Discount Rate

                                                                                                                                10 yr PV 20 yr PV

                                                                                                                                Discount rate - percent

                                                                                                                                WACCSocialLTT

                                                                                                                                Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                                SERA

                                                                                                                                EUL VARIATIONS-ADOPTED VALUES

                                                                                                                                Source Skumatz Economic Research Associates SERA independent research

                                                                                                                                0

                                                                                                                                50

                                                                                                                                100

                                                                                                                                150

                                                                                                                                200

                                                                                                                                250

                                                                                                                                300

                                                                                                                                350

                                                                                                                                400

                                                                                                                                450

                                                                                                                                0 10 20 30 40 50 60 70

                                                                                                                                EUL Variation by Measure

                                                                                                                                67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                                Poorly supportedOldTechnology change

                                                                                                                                SERA

                                                                                                                                KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                                Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                                measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                                change the BC conclusion

                                                                                                                                On the cusphellip

                                                                                                                                If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                                IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                                Source SERA all rights reserved

                                                                                                                                SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                                SERA

                                                                                                                                IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                                Utility Soc Part Conserv Recrsquom

                                                                                                                                Rationale

                                                                                                                                Base Percent X X X Program-invariant

                                                                                                                                Low Income X X X X Multiple sources

                                                                                                                                Weatherization X X X Substantial Participantimpacts

                                                                                                                                Measure Program-specific

                                                                                                                                X Varies by measure sector

                                                                                                                                Other Recomrsquos Local Research

                                                                                                                                50

                                                                                                                                Source Skumatz SERA

                                                                                                                                Developing values for multiple states amp utilities

                                                                                                                                SERA

                                                                                                                                TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                                Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                                Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                                5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                                6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                                THANK YOU

                                                                                                                                Questions

                                                                                                                                Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                                SERA

                                                                                                                                POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                53

                                                                                                                                SERA

                                                                                                                                NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                Negative NEB values cost of barrier

                                                                                                                                Solar Water Heat NZ$ Euros

                                                                                                                                Solar Design NZ$ Euros

                                                                                                                                Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                -23 -12 (079)

                                                                                                                                -11 -6 (06)

                                                                                                                                Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                Source Skumatz Economic Research Associates research

                                                                                                                                SERA

                                                                                                                                CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                -5-4-3-2-1012345

                                                                                                                                Op Co

                                                                                                                                st

                                                                                                                                Init C

                                                                                                                                ost

                                                                                                                                Maint

                                                                                                                                Perfo

                                                                                                                                rm

                                                                                                                                Prod

                                                                                                                                uctiv

                                                                                                                                Tena

                                                                                                                                nt Sa

                                                                                                                                t

                                                                                                                                Comf

                                                                                                                                ort

                                                                                                                                Appe

                                                                                                                                ar

                                                                                                                                Light

                                                                                                                                Qual

                                                                                                                                SellL

                                                                                                                                ease

                                                                                                                                Devp Ownr AampE

                                                                                                                                Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                Owners more positive ndashUnderinvestment train

                                                                                                                                • Soup to nuts on non-energy benefits NEIs
                                                                                                                                • TOPICS
                                                                                                                                • BACKGROUND NEBS
                                                                                                                                • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                • KEY APPLICATIONS OF NEBS
                                                                                                                                • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                • Neb measurement
                                                                                                                                • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                • MEASURING PARTICIPANTNEBs
                                                                                                                                • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                • PROGRESS AND GAPS
                                                                                                                                • PROGRESS AND GAPS
                                                                                                                                • USING NEB NEI VALUES
                                                                                                                                • Slide Number 19
                                                                                                                                • ARE NEBS HIGH VALUE
                                                                                                                                • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                • AUDIT FINANCIAL PROGRAM
                                                                                                                                • CampI NEW CONSTRUCTION
                                                                                                                                • CampI TECHNICAL ASSISTANCE
                                                                                                                                • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                • Slide Number 31
                                                                                                                                • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                • UTILITY NEBS
                                                                                                                                • ADJUSTED PAYBACKS
                                                                                                                                • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                • NEBS IN BC
                                                                                                                                • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                • APPLICATION TO BC
                                                                                                                                • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                • STATE TREATMENT OF NEBS NEBs
                                                                                                                                • Slide Number 42
                                                                                                                                • SELECTED STATE PROGRESS
                                                                                                                                • SELECTED STATE PROGRESS
                                                                                                                                • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                • DISCOUNT RATE MATTERS
                                                                                                                                • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                • TAKEAWAYSON NEBSNEIS
                                                                                                                                • THANK YOUQuestions
                                                                                                                                • Slide Number 53
                                                                                                                                • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                  SERA

                                                                                                                                  APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                                  Expansion Concern with risk calculation accuracy

                                                                                                                                  39

                                                                                                                                  Source Skumatz SERA research

                                                                                                                                  SERA

                                                                                                                                  40

                                                                                                                                  METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                                                  Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                                                  Minimize Regulatory amp Implementer Risk

                                                                                                                                  Minimize Evaluation Cost

                                                                                                                                  Adder

                                                                                                                                  Readily MeasurableHybrid

                                                                                                                                  All NEBsSource SERA Research

                                                                                                                                  SERA

                                                                                                                                  41

                                                                                                                                  STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                                                  Screening Application Utilities regions

                                                                                                                                  Program Marketing Fairly widespread use in utilities states across the country

                                                                                                                                  MO

                                                                                                                                  RE AGGRESSIVE ==gt

                                                                                                                                  Test Pgm Screen - adder

                                                                                                                                  IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                                                  Test Pgm Screen - readily measured

                                                                                                                                  MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                                                  Test Hybrid (potential adder amp measured)

                                                                                                                                  CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                                                  Test Pgm screen - Broad

                                                                                                                                  With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                                                  Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                                                  SERA

                                                                                                                                  42

                                                                                                                                  Key

                                                                                                                                  States with NEBs in CE tests for At least one type of program

                                                                                                                                  copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                                                  STATES WITH NEBS IN CE TESTS

                                                                                                                                  NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                                                  groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                                                  SERA

                                                                                                                                  SELECTED STATE PROGRESS CA

                                                                                                                                  2001 ndash LIPPT model NY

                                                                                                                                  Mid-2000s measurement amp scenarios not included CO

                                                                                                                                  Adders (10 electric 25 LI 5 gas) VT

                                                                                                                                  15 adder helped by previous research DC

                                                                                                                                  10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                                                  43

                                                                                                                                  Source Skumatz SERA research

                                                                                                                                  SERA

                                                                                                                                  SELECTED STATE PROGRESS Dominos ongoing

                                                                                                                                  Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                                                  Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                                                  Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                                                  Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                                                  44

                                                                                                                                  Source Skumatz SERA research

                                                                                                                                  SERA

                                                                                                                                  APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                                  Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                                  Expansion Concern with risk calculation accuracy

                                                                                                                                  45

                                                                                                                                  Source Skumatz SERA research

                                                                                                                                  SERA

                                                                                                                                  46

                                                                                                                                  NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                                  [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                                  SourceSkumatz SERA Independent research

                                                                                                                                  NTG ndash accuracy measurement incomplete

                                                                                                                                  Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                                  NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                                  EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                                  Cost Complicated expensive local changes

                                                                                                                                  RELATIVE Risk from NEBs

                                                                                                                                  Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                                  Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                                  NEBs as accurate as many inputs (all could improve)

                                                                                                                                  SERA

                                                                                                                                  DISCOUNT RATE MATTERS

                                                                                                                                  47

                                                                                                                                  Source Skumatz SERA Research

                                                                                                                                  0

                                                                                                                                  20

                                                                                                                                  40

                                                                                                                                  60

                                                                                                                                  80

                                                                                                                                  100

                                                                                                                                  120

                                                                                                                                  1 2 3 4 5 6 7 8 9 10

                                                                                                                                  PV Ratios vs 1 Discount Rate

                                                                                                                                  10 yr PV 20 yr PV

                                                                                                                                  Discount rate - percent

                                                                                                                                  WACCSocialLTT

                                                                                                                                  Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                                  SERA

                                                                                                                                  EUL VARIATIONS-ADOPTED VALUES

                                                                                                                                  Source Skumatz Economic Research Associates SERA independent research

                                                                                                                                  0

                                                                                                                                  50

                                                                                                                                  100

                                                                                                                                  150

                                                                                                                                  200

                                                                                                                                  250

                                                                                                                                  300

                                                                                                                                  350

                                                                                                                                  400

                                                                                                                                  450

                                                                                                                                  0 10 20 30 40 50 60 70

                                                                                                                                  EUL Variation by Measure

                                                                                                                                  67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                                  Poorly supportedOldTechnology change

                                                                                                                                  SERA

                                                                                                                                  KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                                  Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                                  measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                                  change the BC conclusion

                                                                                                                                  On the cusphellip

                                                                                                                                  If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                                  IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                                  Source SERA all rights reserved

                                                                                                                                  SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                                  SERA

                                                                                                                                  IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                                  Utility Soc Part Conserv Recrsquom

                                                                                                                                  Rationale

                                                                                                                                  Base Percent X X X Program-invariant

                                                                                                                                  Low Income X X X X Multiple sources

                                                                                                                                  Weatherization X X X Substantial Participantimpacts

                                                                                                                                  Measure Program-specific

                                                                                                                                  X Varies by measure sector

                                                                                                                                  Other Recomrsquos Local Research

                                                                                                                                  50

                                                                                                                                  Source Skumatz SERA

                                                                                                                                  Developing values for multiple states amp utilities

                                                                                                                                  SERA

                                                                                                                                  TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                                  Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                                  Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                                  5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                                  6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                                  THANK YOU

                                                                                                                                  Questions

                                                                                                                                  Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                                  SERA

                                                                                                                                  POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                  53

                                                                                                                                  SERA

                                                                                                                                  NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                  Negative NEB values cost of barrier

                                                                                                                                  Solar Water Heat NZ$ Euros

                                                                                                                                  Solar Design NZ$ Euros

                                                                                                                                  Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                  -23 -12 (079)

                                                                                                                                  -11 -6 (06)

                                                                                                                                  Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                  Source Skumatz Economic Research Associates research

                                                                                                                                  SERA

                                                                                                                                  CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                  NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                  -5-4-3-2-1012345

                                                                                                                                  Op Co

                                                                                                                                  st

                                                                                                                                  Init C

                                                                                                                                  ost

                                                                                                                                  Maint

                                                                                                                                  Perfo

                                                                                                                                  rm

                                                                                                                                  Prod

                                                                                                                                  uctiv

                                                                                                                                  Tena

                                                                                                                                  nt Sa

                                                                                                                                  t

                                                                                                                                  Comf

                                                                                                                                  ort

                                                                                                                                  Appe

                                                                                                                                  ar

                                                                                                                                  Light

                                                                                                                                  Qual

                                                                                                                                  SellL

                                                                                                                                  ease

                                                                                                                                  Devp Ownr AampE

                                                                                                                                  Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                  Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                  Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                  Owners more positive ndashUnderinvestment train

                                                                                                                                  • Soup to nuts on non-energy benefits NEIs
                                                                                                                                  • TOPICS
                                                                                                                                  • BACKGROUND NEBS
                                                                                                                                  • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                  • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                  • KEY APPLICATIONS OF NEBS
                                                                                                                                  • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                  • Neb measurement
                                                                                                                                  • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                  • MEASURING PARTICIPANTNEBs
                                                                                                                                  • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                  • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                  • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                  • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                  • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                  • PROGRESS AND GAPS
                                                                                                                                  • PROGRESS AND GAPS
                                                                                                                                  • USING NEB NEI VALUES
                                                                                                                                  • Slide Number 19
                                                                                                                                  • ARE NEBS HIGH VALUE
                                                                                                                                  • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                  • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                  • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                  • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                  • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                  • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                  • AUDIT FINANCIAL PROGRAM
                                                                                                                                  • CampI NEW CONSTRUCTION
                                                                                                                                  • CampI TECHNICAL ASSISTANCE
                                                                                                                                  • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                  • Slide Number 31
                                                                                                                                  • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                  • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                  • UTILITY NEBS
                                                                                                                                  • ADJUSTED PAYBACKS
                                                                                                                                  • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                  • NEBS IN BC
                                                                                                                                  • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                  • APPLICATION TO BC
                                                                                                                                  • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                  • STATE TREATMENT OF NEBS NEBs
                                                                                                                                  • Slide Number 42
                                                                                                                                  • SELECTED STATE PROGRESS
                                                                                                                                  • SELECTED STATE PROGRESS
                                                                                                                                  • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                  • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                  • DISCOUNT RATE MATTERS
                                                                                                                                  • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                  • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                  • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                  • TAKEAWAYSON NEBSNEIS
                                                                                                                                  • THANK YOUQuestions
                                                                                                                                  • Slide Number 53
                                                                                                                                  • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                  • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                    SERA

                                                                                                                                    40

                                                                                                                                    METHODS TO INCLUDE NEBsIN REGULATORY TESTS

                                                                                                                                    Maximize DSM opportunities amp feedback Accuracy tailoring

                                                                                                                                    Minimize Regulatory amp Implementer Risk

                                                                                                                                    Minimize Evaluation Cost

                                                                                                                                    Adder

                                                                                                                                    Readily MeasurableHybrid

                                                                                                                                    All NEBsSource SERA Research

                                                                                                                                    SERA

                                                                                                                                    41

                                                                                                                                    STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                                                    Screening Application Utilities regions

                                                                                                                                    Program Marketing Fairly widespread use in utilities states across the country

                                                                                                                                    MO

                                                                                                                                    RE AGGRESSIVE ==gt

                                                                                                                                    Test Pgm Screen - adder

                                                                                                                                    IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                                                    Test Pgm Screen - readily measured

                                                                                                                                    MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                                                    Test Hybrid (potential adder amp measured)

                                                                                                                                    CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                                                    Test Pgm screen - Broad

                                                                                                                                    With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                                                    Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                                                    SERA

                                                                                                                                    42

                                                                                                                                    Key

                                                                                                                                    States with NEBs in CE tests for At least one type of program

                                                                                                                                    copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                                                    STATES WITH NEBS IN CE TESTS

                                                                                                                                    NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                                                    groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                                                    SERA

                                                                                                                                    SELECTED STATE PROGRESS CA

                                                                                                                                    2001 ndash LIPPT model NY

                                                                                                                                    Mid-2000s measurement amp scenarios not included CO

                                                                                                                                    Adders (10 electric 25 LI 5 gas) VT

                                                                                                                                    15 adder helped by previous research DC

                                                                                                                                    10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                                                    43

                                                                                                                                    Source Skumatz SERA research

                                                                                                                                    SERA

                                                                                                                                    SELECTED STATE PROGRESS Dominos ongoing

                                                                                                                                    Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                                                    Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                                                    Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                                                    Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                                                    44

                                                                                                                                    Source Skumatz SERA research

                                                                                                                                    SERA

                                                                                                                                    APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                                    Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                                    Expansion Concern with risk calculation accuracy

                                                                                                                                    45

                                                                                                                                    Source Skumatz SERA research

                                                                                                                                    SERA

                                                                                                                                    46

                                                                                                                                    NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                                    [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                                    SourceSkumatz SERA Independent research

                                                                                                                                    NTG ndash accuracy measurement incomplete

                                                                                                                                    Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                                    NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                                    EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                                    Cost Complicated expensive local changes

                                                                                                                                    RELATIVE Risk from NEBs

                                                                                                                                    Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                                    Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                                    NEBs as accurate as many inputs (all could improve)

                                                                                                                                    SERA

                                                                                                                                    DISCOUNT RATE MATTERS

                                                                                                                                    47

                                                                                                                                    Source Skumatz SERA Research

                                                                                                                                    0

                                                                                                                                    20

                                                                                                                                    40

                                                                                                                                    60

                                                                                                                                    80

                                                                                                                                    100

                                                                                                                                    120

                                                                                                                                    1 2 3 4 5 6 7 8 9 10

                                                                                                                                    PV Ratios vs 1 Discount Rate

                                                                                                                                    10 yr PV 20 yr PV

                                                                                                                                    Discount rate - percent

                                                                                                                                    WACCSocialLTT

                                                                                                                                    Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                                    SERA

                                                                                                                                    EUL VARIATIONS-ADOPTED VALUES

                                                                                                                                    Source Skumatz Economic Research Associates SERA independent research

                                                                                                                                    0

                                                                                                                                    50

                                                                                                                                    100

                                                                                                                                    150

                                                                                                                                    200

                                                                                                                                    250

                                                                                                                                    300

                                                                                                                                    350

                                                                                                                                    400

                                                                                                                                    450

                                                                                                                                    0 10 20 30 40 50 60 70

                                                                                                                                    EUL Variation by Measure

                                                                                                                                    67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                                    Poorly supportedOldTechnology change

                                                                                                                                    SERA

                                                                                                                                    KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                                    Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                                    measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                                    change the BC conclusion

                                                                                                                                    On the cusphellip

                                                                                                                                    If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                                    IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                                    Source SERA all rights reserved

                                                                                                                                    SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                                    SERA

                                                                                                                                    IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                                    Utility Soc Part Conserv Recrsquom

                                                                                                                                    Rationale

                                                                                                                                    Base Percent X X X Program-invariant

                                                                                                                                    Low Income X X X X Multiple sources

                                                                                                                                    Weatherization X X X Substantial Participantimpacts

                                                                                                                                    Measure Program-specific

                                                                                                                                    X Varies by measure sector

                                                                                                                                    Other Recomrsquos Local Research

                                                                                                                                    50

                                                                                                                                    Source Skumatz SERA

                                                                                                                                    Developing values for multiple states amp utilities

                                                                                                                                    SERA

                                                                                                                                    TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                                    Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                                    Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                                    5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                                    6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                                    THANK YOU

                                                                                                                                    Questions

                                                                                                                                    Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                                    SERA

                                                                                                                                    POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                    53

                                                                                                                                    SERA

                                                                                                                                    NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                    Negative NEB values cost of barrier

                                                                                                                                    Solar Water Heat NZ$ Euros

                                                                                                                                    Solar Design NZ$ Euros

                                                                                                                                    Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                    -23 -12 (079)

                                                                                                                                    -11 -6 (06)

                                                                                                                                    Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                    Source Skumatz Economic Research Associates research

                                                                                                                                    SERA

                                                                                                                                    CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                    NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                    -5-4-3-2-1012345

                                                                                                                                    Op Co

                                                                                                                                    st

                                                                                                                                    Init C

                                                                                                                                    ost

                                                                                                                                    Maint

                                                                                                                                    Perfo

                                                                                                                                    rm

                                                                                                                                    Prod

                                                                                                                                    uctiv

                                                                                                                                    Tena

                                                                                                                                    nt Sa

                                                                                                                                    t

                                                                                                                                    Comf

                                                                                                                                    ort

                                                                                                                                    Appe

                                                                                                                                    ar

                                                                                                                                    Light

                                                                                                                                    Qual

                                                                                                                                    SellL

                                                                                                                                    ease

                                                                                                                                    Devp Ownr AampE

                                                                                                                                    Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                    Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                    Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                    Owners more positive ndashUnderinvestment train

                                                                                                                                    • Soup to nuts on non-energy benefits NEIs
                                                                                                                                    • TOPICS
                                                                                                                                    • BACKGROUND NEBS
                                                                                                                                    • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                    • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                    • KEY APPLICATIONS OF NEBS
                                                                                                                                    • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                    • Neb measurement
                                                                                                                                    • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                    • MEASURING PARTICIPANTNEBs
                                                                                                                                    • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                    • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                    • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                    • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                    • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                    • PROGRESS AND GAPS
                                                                                                                                    • PROGRESS AND GAPS
                                                                                                                                    • USING NEB NEI VALUES
                                                                                                                                    • Slide Number 19
                                                                                                                                    • ARE NEBS HIGH VALUE
                                                                                                                                    • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                    • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                    • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                    • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                    • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                    • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                    • AUDIT FINANCIAL PROGRAM
                                                                                                                                    • CampI NEW CONSTRUCTION
                                                                                                                                    • CampI TECHNICAL ASSISTANCE
                                                                                                                                    • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                    • Slide Number 31
                                                                                                                                    • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                    • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                    • UTILITY NEBS
                                                                                                                                    • ADJUSTED PAYBACKS
                                                                                                                                    • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                    • NEBS IN BC
                                                                                                                                    • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                    • APPLICATION TO BC
                                                                                                                                    • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                    • STATE TREATMENT OF NEBS NEBs
                                                                                                                                    • Slide Number 42
                                                                                                                                    • SELECTED STATE PROGRESS
                                                                                                                                    • SELECTED STATE PROGRESS
                                                                                                                                    • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                    • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                    • DISCOUNT RATE MATTERS
                                                                                                                                    • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                    • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                    • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                    • TAKEAWAYSON NEBSNEIS
                                                                                                                                    • THANK YOUQuestions
                                                                                                                                    • Slide Number 53
                                                                                                                                    • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                    • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                      SERA

                                                                                                                                      41

                                                                                                                                      STATE TREATMENT OF NEBS NEBsRegulatory

                                                                                                                                      Screening Application Utilities regions

                                                                                                                                      Program Marketing Fairly widespread use in utilities states across the country

                                                                                                                                      MO

                                                                                                                                      RE AGGRESSIVE ==gt

                                                                                                                                      Test Pgm Screen - adder

                                                                                                                                      IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)

                                                                                                                                      Test Pgm Screen - readily measured

                                                                                                                                      MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)

                                                                                                                                      Test Hybrid (potential adder amp measured)

                                                                                                                                      CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT

                                                                                                                                      Test Pgm screen - Broad

                                                                                                                                      With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found

                                                                                                                                      Source copySkumatz Economic Research Associates (SERA) may be used with permission of author

                                                                                                                                      SERA

                                                                                                                                      42

                                                                                                                                      Key

                                                                                                                                      States with NEBs in CE tests for At least one type of program

                                                                                                                                      copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                                                      STATES WITH NEBS IN CE TESTS

                                                                                                                                      NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                                                      groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                                                      SERA

                                                                                                                                      SELECTED STATE PROGRESS CA

                                                                                                                                      2001 ndash LIPPT model NY

                                                                                                                                      Mid-2000s measurement amp scenarios not included CO

                                                                                                                                      Adders (10 electric 25 LI 5 gas) VT

                                                                                                                                      15 adder helped by previous research DC

                                                                                                                                      10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                                                      43

                                                                                                                                      Source Skumatz SERA research

                                                                                                                                      SERA

                                                                                                                                      SELECTED STATE PROGRESS Dominos ongoing

                                                                                                                                      Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                                                      Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                                                      Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                                                      Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                                                      44

                                                                                                                                      Source Skumatz SERA research

                                                                                                                                      SERA

                                                                                                                                      APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                                      Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                                      Expansion Concern with risk calculation accuracy

                                                                                                                                      45

                                                                                                                                      Source Skumatz SERA research

                                                                                                                                      SERA

                                                                                                                                      46

                                                                                                                                      NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                                      [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                                      SourceSkumatz SERA Independent research

                                                                                                                                      NTG ndash accuracy measurement incomplete

                                                                                                                                      Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                                      NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                                      EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                                      Cost Complicated expensive local changes

                                                                                                                                      RELATIVE Risk from NEBs

                                                                                                                                      Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                                      Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                                      NEBs as accurate as many inputs (all could improve)

                                                                                                                                      SERA

                                                                                                                                      DISCOUNT RATE MATTERS

                                                                                                                                      47

                                                                                                                                      Source Skumatz SERA Research

                                                                                                                                      0

                                                                                                                                      20

                                                                                                                                      40

                                                                                                                                      60

                                                                                                                                      80

                                                                                                                                      100

                                                                                                                                      120

                                                                                                                                      1 2 3 4 5 6 7 8 9 10

                                                                                                                                      PV Ratios vs 1 Discount Rate

                                                                                                                                      10 yr PV 20 yr PV

                                                                                                                                      Discount rate - percent

                                                                                                                                      WACCSocialLTT

                                                                                                                                      Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                                      SERA

                                                                                                                                      EUL VARIATIONS-ADOPTED VALUES

                                                                                                                                      Source Skumatz Economic Research Associates SERA independent research

                                                                                                                                      0

                                                                                                                                      50

                                                                                                                                      100

                                                                                                                                      150

                                                                                                                                      200

                                                                                                                                      250

                                                                                                                                      300

                                                                                                                                      350

                                                                                                                                      400

                                                                                                                                      450

                                                                                                                                      0 10 20 30 40 50 60 70

                                                                                                                                      EUL Variation by Measure

                                                                                                                                      67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                                      Poorly supportedOldTechnology change

                                                                                                                                      SERA

                                                                                                                                      KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                                      Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                                      measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                                      change the BC conclusion

                                                                                                                                      On the cusphellip

                                                                                                                                      If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                                      IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                                      Source SERA all rights reserved

                                                                                                                                      SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                                      SERA

                                                                                                                                      IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                                      Utility Soc Part Conserv Recrsquom

                                                                                                                                      Rationale

                                                                                                                                      Base Percent X X X Program-invariant

                                                                                                                                      Low Income X X X X Multiple sources

                                                                                                                                      Weatherization X X X Substantial Participantimpacts

                                                                                                                                      Measure Program-specific

                                                                                                                                      X Varies by measure sector

                                                                                                                                      Other Recomrsquos Local Research

                                                                                                                                      50

                                                                                                                                      Source Skumatz SERA

                                                                                                                                      Developing values for multiple states amp utilities

                                                                                                                                      SERA

                                                                                                                                      TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                                      Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                                      Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                                      5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                                      6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                                      THANK YOU

                                                                                                                                      Questions

                                                                                                                                      Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                                      SERA

                                                                                                                                      POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                      53

                                                                                                                                      SERA

                                                                                                                                      NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                      Negative NEB values cost of barrier

                                                                                                                                      Solar Water Heat NZ$ Euros

                                                                                                                                      Solar Design NZ$ Euros

                                                                                                                                      Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                      -23 -12 (079)

                                                                                                                                      -11 -6 (06)

                                                                                                                                      Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                      Source Skumatz Economic Research Associates research

                                                                                                                                      SERA

                                                                                                                                      CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                      NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                      -5-4-3-2-1012345

                                                                                                                                      Op Co

                                                                                                                                      st

                                                                                                                                      Init C

                                                                                                                                      ost

                                                                                                                                      Maint

                                                                                                                                      Perfo

                                                                                                                                      rm

                                                                                                                                      Prod

                                                                                                                                      uctiv

                                                                                                                                      Tena

                                                                                                                                      nt Sa

                                                                                                                                      t

                                                                                                                                      Comf

                                                                                                                                      ort

                                                                                                                                      Appe

                                                                                                                                      ar

                                                                                                                                      Light

                                                                                                                                      Qual

                                                                                                                                      SellL

                                                                                                                                      ease

                                                                                                                                      Devp Ownr AampE

                                                                                                                                      Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                      Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                      Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                      Owners more positive ndashUnderinvestment train

                                                                                                                                      • Soup to nuts on non-energy benefits NEIs
                                                                                                                                      • TOPICS
                                                                                                                                      • BACKGROUND NEBS
                                                                                                                                      • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                      • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                      • KEY APPLICATIONS OF NEBS
                                                                                                                                      • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                      • Neb measurement
                                                                                                                                      • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                      • MEASURING PARTICIPANTNEBs
                                                                                                                                      • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                      • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                      • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                      • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                      • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                      • PROGRESS AND GAPS
                                                                                                                                      • PROGRESS AND GAPS
                                                                                                                                      • USING NEB NEI VALUES
                                                                                                                                      • Slide Number 19
                                                                                                                                      • ARE NEBS HIGH VALUE
                                                                                                                                      • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                      • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                      • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                      • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                      • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                      • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                      • AUDIT FINANCIAL PROGRAM
                                                                                                                                      • CampI NEW CONSTRUCTION
                                                                                                                                      • CampI TECHNICAL ASSISTANCE
                                                                                                                                      • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                      • Slide Number 31
                                                                                                                                      • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                      • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                      • UTILITY NEBS
                                                                                                                                      • ADJUSTED PAYBACKS
                                                                                                                                      • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                      • NEBS IN BC
                                                                                                                                      • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                      • APPLICATION TO BC
                                                                                                                                      • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                      • STATE TREATMENT OF NEBS NEBs
                                                                                                                                      • Slide Number 42
                                                                                                                                      • SELECTED STATE PROGRESS
                                                                                                                                      • SELECTED STATE PROGRESS
                                                                                                                                      • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                      • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                      • DISCOUNT RATE MATTERS
                                                                                                                                      • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                      • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                      • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                      • TAKEAWAYSON NEBSNEIS
                                                                                                                                      • THANK YOUQuestions
                                                                                                                                      • Slide Number 53
                                                                                                                                      • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                      • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                        SERA

                                                                                                                                        42

                                                                                                                                        Key

                                                                                                                                        States with NEBs in CE tests for At least one type of program

                                                                                                                                        copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author

                                                                                                                                        STATES WITH NEBS IN CE TESTS

                                                                                                                                        NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working

                                                                                                                                        groups TRM work etc in states inMidwest mid-Atlantic and elsewhere

                                                                                                                                        SERA

                                                                                                                                        SELECTED STATE PROGRESS CA

                                                                                                                                        2001 ndash LIPPT model NY

                                                                                                                                        Mid-2000s measurement amp scenarios not included CO

                                                                                                                                        Adders (10 electric 25 LI 5 gas) VT

                                                                                                                                        15 adder helped by previous research DC

                                                                                                                                        10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                                                        43

                                                                                                                                        Source Skumatz SERA research

                                                                                                                                        SERA

                                                                                                                                        SELECTED STATE PROGRESS Dominos ongoing

                                                                                                                                        Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                                                        Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                                                        Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                                                        Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                                                        44

                                                                                                                                        Source Skumatz SERA research

                                                                                                                                        SERA

                                                                                                                                        APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                                        Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                                        Expansion Concern with risk calculation accuracy

                                                                                                                                        45

                                                                                                                                        Source Skumatz SERA research

                                                                                                                                        SERA

                                                                                                                                        46

                                                                                                                                        NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                                        [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                                        SourceSkumatz SERA Independent research

                                                                                                                                        NTG ndash accuracy measurement incomplete

                                                                                                                                        Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                                        NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                                        EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                                        Cost Complicated expensive local changes

                                                                                                                                        RELATIVE Risk from NEBs

                                                                                                                                        Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                                        Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                                        NEBs as accurate as many inputs (all could improve)

                                                                                                                                        SERA

                                                                                                                                        DISCOUNT RATE MATTERS

                                                                                                                                        47

                                                                                                                                        Source Skumatz SERA Research

                                                                                                                                        0

                                                                                                                                        20

                                                                                                                                        40

                                                                                                                                        60

                                                                                                                                        80

                                                                                                                                        100

                                                                                                                                        120

                                                                                                                                        1 2 3 4 5 6 7 8 9 10

                                                                                                                                        PV Ratios vs 1 Discount Rate

                                                                                                                                        10 yr PV 20 yr PV

                                                                                                                                        Discount rate - percent

                                                                                                                                        WACCSocialLTT

                                                                                                                                        Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                                        SERA

                                                                                                                                        EUL VARIATIONS-ADOPTED VALUES

                                                                                                                                        Source Skumatz Economic Research Associates SERA independent research

                                                                                                                                        0

                                                                                                                                        50

                                                                                                                                        100

                                                                                                                                        150

                                                                                                                                        200

                                                                                                                                        250

                                                                                                                                        300

                                                                                                                                        350

                                                                                                                                        400

                                                                                                                                        450

                                                                                                                                        0 10 20 30 40 50 60 70

                                                                                                                                        EUL Variation by Measure

                                                                                                                                        67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                                        Poorly supportedOldTechnology change

                                                                                                                                        SERA

                                                                                                                                        KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                                        Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                                        measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                                        change the BC conclusion

                                                                                                                                        On the cusphellip

                                                                                                                                        If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                                        IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                                        Source SERA all rights reserved

                                                                                                                                        SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                                        SERA

                                                                                                                                        IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                                        Utility Soc Part Conserv Recrsquom

                                                                                                                                        Rationale

                                                                                                                                        Base Percent X X X Program-invariant

                                                                                                                                        Low Income X X X X Multiple sources

                                                                                                                                        Weatherization X X X Substantial Participantimpacts

                                                                                                                                        Measure Program-specific

                                                                                                                                        X Varies by measure sector

                                                                                                                                        Other Recomrsquos Local Research

                                                                                                                                        50

                                                                                                                                        Source Skumatz SERA

                                                                                                                                        Developing values for multiple states amp utilities

                                                                                                                                        SERA

                                                                                                                                        TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                                        Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                                        Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                                        5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                                        6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                                        THANK YOU

                                                                                                                                        Questions

                                                                                                                                        Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                                        SERA

                                                                                                                                        POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                        53

                                                                                                                                        SERA

                                                                                                                                        NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                        Negative NEB values cost of barrier

                                                                                                                                        Solar Water Heat NZ$ Euros

                                                                                                                                        Solar Design NZ$ Euros

                                                                                                                                        Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                        -23 -12 (079)

                                                                                                                                        -11 -6 (06)

                                                                                                                                        Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                        Source Skumatz Economic Research Associates research

                                                                                                                                        SERA

                                                                                                                                        CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                        NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                        -5-4-3-2-1012345

                                                                                                                                        Op Co

                                                                                                                                        st

                                                                                                                                        Init C

                                                                                                                                        ost

                                                                                                                                        Maint

                                                                                                                                        Perfo

                                                                                                                                        rm

                                                                                                                                        Prod

                                                                                                                                        uctiv

                                                                                                                                        Tena

                                                                                                                                        nt Sa

                                                                                                                                        t

                                                                                                                                        Comf

                                                                                                                                        ort

                                                                                                                                        Appe

                                                                                                                                        ar

                                                                                                                                        Light

                                                                                                                                        Qual

                                                                                                                                        SellL

                                                                                                                                        ease

                                                                                                                                        Devp Ownr AampE

                                                                                                                                        Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                        Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                        Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                        Owners more positive ndashUnderinvestment train

                                                                                                                                        • Soup to nuts on non-energy benefits NEIs
                                                                                                                                        • TOPICS
                                                                                                                                        • BACKGROUND NEBS
                                                                                                                                        • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                        • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                        • KEY APPLICATIONS OF NEBS
                                                                                                                                        • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                        • Neb measurement
                                                                                                                                        • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                        • MEASURING PARTICIPANTNEBs
                                                                                                                                        • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                        • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                        • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                        • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                        • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                        • PROGRESS AND GAPS
                                                                                                                                        • PROGRESS AND GAPS
                                                                                                                                        • USING NEB NEI VALUES
                                                                                                                                        • Slide Number 19
                                                                                                                                        • ARE NEBS HIGH VALUE
                                                                                                                                        • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                        • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                        • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                        • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                        • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                        • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                        • AUDIT FINANCIAL PROGRAM
                                                                                                                                        • CampI NEW CONSTRUCTION
                                                                                                                                        • CampI TECHNICAL ASSISTANCE
                                                                                                                                        • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                        • Slide Number 31
                                                                                                                                        • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                        • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                        • UTILITY NEBS
                                                                                                                                        • ADJUSTED PAYBACKS
                                                                                                                                        • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                        • NEBS IN BC
                                                                                                                                        • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                        • APPLICATION TO BC
                                                                                                                                        • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                        • STATE TREATMENT OF NEBS NEBs
                                                                                                                                        • Slide Number 42
                                                                                                                                        • SELECTED STATE PROGRESS
                                                                                                                                        • SELECTED STATE PROGRESS
                                                                                                                                        • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                        • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                        • DISCOUNT RATE MATTERS
                                                                                                                                        • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                        • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                        • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                        • TAKEAWAYSON NEBSNEIS
                                                                                                                                        • THANK YOUQuestions
                                                                                                                                        • Slide Number 53
                                                                                                                                        • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                        • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                          SERA

                                                                                                                                          SELECTED STATE PROGRESS CA

                                                                                                                                          2001 ndash LIPPT model NY

                                                                                                                                          Mid-2000s measurement amp scenarios not included CO

                                                                                                                                          Adders (10 electric 25 LI 5 gas) VT

                                                                                                                                          15 adder helped by previous research DC

                                                                                                                                          10 NEB adder 10 risk 10 enviro + NEBs in goals amp measured benchmarks

                                                                                                                                          43

                                                                                                                                          Source Skumatz SERA research

                                                                                                                                          SERA

                                                                                                                                          SELECTED STATE PROGRESS Dominos ongoing

                                                                                                                                          Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                                                          Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                                                          Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                                                          Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                                                          44

                                                                                                                                          Source Skumatz SERA research

                                                                                                                                          SERA

                                                                                                                                          APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                                          Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                                          Expansion Concern with risk calculation accuracy

                                                                                                                                          45

                                                                                                                                          Source Skumatz SERA research

                                                                                                                                          SERA

                                                                                                                                          46

                                                                                                                                          NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                                          [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                                          SourceSkumatz SERA Independent research

                                                                                                                                          NTG ndash accuracy measurement incomplete

                                                                                                                                          Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                                          NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                                          EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                                          Cost Complicated expensive local changes

                                                                                                                                          RELATIVE Risk from NEBs

                                                                                                                                          Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                                          Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                                          NEBs as accurate as many inputs (all could improve)

                                                                                                                                          SERA

                                                                                                                                          DISCOUNT RATE MATTERS

                                                                                                                                          47

                                                                                                                                          Source Skumatz SERA Research

                                                                                                                                          0

                                                                                                                                          20

                                                                                                                                          40

                                                                                                                                          60

                                                                                                                                          80

                                                                                                                                          100

                                                                                                                                          120

                                                                                                                                          1 2 3 4 5 6 7 8 9 10

                                                                                                                                          PV Ratios vs 1 Discount Rate

                                                                                                                                          10 yr PV 20 yr PV

                                                                                                                                          Discount rate - percent

                                                                                                                                          WACCSocialLTT

                                                                                                                                          Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                                          SERA

                                                                                                                                          EUL VARIATIONS-ADOPTED VALUES

                                                                                                                                          Source Skumatz Economic Research Associates SERA independent research

                                                                                                                                          0

                                                                                                                                          50

                                                                                                                                          100

                                                                                                                                          150

                                                                                                                                          200

                                                                                                                                          250

                                                                                                                                          300

                                                                                                                                          350

                                                                                                                                          400

                                                                                                                                          450

                                                                                                                                          0 10 20 30 40 50 60 70

                                                                                                                                          EUL Variation by Measure

                                                                                                                                          67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                                          Poorly supportedOldTechnology change

                                                                                                                                          SERA

                                                                                                                                          KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                                          Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                                          measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                                          change the BC conclusion

                                                                                                                                          On the cusphellip

                                                                                                                                          If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                                          IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                                          Source SERA all rights reserved

                                                                                                                                          SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                                          SERA

                                                                                                                                          IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                                          Utility Soc Part Conserv Recrsquom

                                                                                                                                          Rationale

                                                                                                                                          Base Percent X X X Program-invariant

                                                                                                                                          Low Income X X X X Multiple sources

                                                                                                                                          Weatherization X X X Substantial Participantimpacts

                                                                                                                                          Measure Program-specific

                                                                                                                                          X Varies by measure sector

                                                                                                                                          Other Recomrsquos Local Research

                                                                                                                                          50

                                                                                                                                          Source Skumatz SERA

                                                                                                                                          Developing values for multiple states amp utilities

                                                                                                                                          SERA

                                                                                                                                          TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                                          Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                                          Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                                          5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                                          6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                                          THANK YOU

                                                                                                                                          Questions

                                                                                                                                          Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                                          SERA

                                                                                                                                          POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                          53

                                                                                                                                          SERA

                                                                                                                                          NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                          Negative NEB values cost of barrier

                                                                                                                                          Solar Water Heat NZ$ Euros

                                                                                                                                          Solar Design NZ$ Euros

                                                                                                                                          Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                          -23 -12 (079)

                                                                                                                                          -11 -6 (06)

                                                                                                                                          Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                          Source Skumatz Economic Research Associates research

                                                                                                                                          SERA

                                                                                                                                          CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                          NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                          -5-4-3-2-1012345

                                                                                                                                          Op Co

                                                                                                                                          st

                                                                                                                                          Init C

                                                                                                                                          ost

                                                                                                                                          Maint

                                                                                                                                          Perfo

                                                                                                                                          rm

                                                                                                                                          Prod

                                                                                                                                          uctiv

                                                                                                                                          Tena

                                                                                                                                          nt Sa

                                                                                                                                          t

                                                                                                                                          Comf

                                                                                                                                          ort

                                                                                                                                          Appe

                                                                                                                                          ar

                                                                                                                                          Light

                                                                                                                                          Qual

                                                                                                                                          SellL

                                                                                                                                          ease

                                                                                                                                          Devp Ownr AampE

                                                                                                                                          Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                          Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                          Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                          Owners more positive ndashUnderinvestment train

                                                                                                                                          • Soup to nuts on non-energy benefits NEIs
                                                                                                                                          • TOPICS
                                                                                                                                          • BACKGROUND NEBS
                                                                                                                                          • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                          • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                          • KEY APPLICATIONS OF NEBS
                                                                                                                                          • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                          • Neb measurement
                                                                                                                                          • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                          • MEASURING PARTICIPANTNEBs
                                                                                                                                          • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                          • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                          • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                          • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                          • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                          • PROGRESS AND GAPS
                                                                                                                                          • PROGRESS AND GAPS
                                                                                                                                          • USING NEB NEI VALUES
                                                                                                                                          • Slide Number 19
                                                                                                                                          • ARE NEBS HIGH VALUE
                                                                                                                                          • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                          • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                          • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                          • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                          • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                          • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                          • AUDIT FINANCIAL PROGRAM
                                                                                                                                          • CampI NEW CONSTRUCTION
                                                                                                                                          • CampI TECHNICAL ASSISTANCE
                                                                                                                                          • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                          • Slide Number 31
                                                                                                                                          • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                          • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                          • UTILITY NEBS
                                                                                                                                          • ADJUSTED PAYBACKS
                                                                                                                                          • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                          • NEBS IN BC
                                                                                                                                          • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                          • APPLICATION TO BC
                                                                                                                                          • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                          • STATE TREATMENT OF NEBS NEBs
                                                                                                                                          • Slide Number 42
                                                                                                                                          • SELECTED STATE PROGRESS
                                                                                                                                          • SELECTED STATE PROGRESS
                                                                                                                                          • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                          • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                          • DISCOUNT RATE MATTERS
                                                                                                                                          • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                          • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                          • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                          • TAKEAWAYSON NEBSNEIS
                                                                                                                                          • THANK YOUQuestions
                                                                                                                                          • Slide Number 53
                                                                                                                                          • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                          • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                            SERA

                                                                                                                                            SELECTED STATE PROGRESS Dominos ongoing

                                                                                                                                            Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled

                                                                                                                                            Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage

                                                                                                                                            Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year

                                                                                                                                            Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip

                                                                                                                                            44

                                                                                                                                            Source Skumatz SERA research

                                                                                                                                            SERA

                                                                                                                                            APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                                            Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                                            Expansion Concern with risk calculation accuracy

                                                                                                                                            45

                                                                                                                                            Source Skumatz SERA research

                                                                                                                                            SERA

                                                                                                                                            46

                                                                                                                                            NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                                            [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                                            SourceSkumatz SERA Independent research

                                                                                                                                            NTG ndash accuracy measurement incomplete

                                                                                                                                            Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                                            NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                                            EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                                            Cost Complicated expensive local changes

                                                                                                                                            RELATIVE Risk from NEBs

                                                                                                                                            Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                                            Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                                            NEBs as accurate as many inputs (all could improve)

                                                                                                                                            SERA

                                                                                                                                            DISCOUNT RATE MATTERS

                                                                                                                                            47

                                                                                                                                            Source Skumatz SERA Research

                                                                                                                                            0

                                                                                                                                            20

                                                                                                                                            40

                                                                                                                                            60

                                                                                                                                            80

                                                                                                                                            100

                                                                                                                                            120

                                                                                                                                            1 2 3 4 5 6 7 8 9 10

                                                                                                                                            PV Ratios vs 1 Discount Rate

                                                                                                                                            10 yr PV 20 yr PV

                                                                                                                                            Discount rate - percent

                                                                                                                                            WACCSocialLTT

                                                                                                                                            Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                                            SERA

                                                                                                                                            EUL VARIATIONS-ADOPTED VALUES

                                                                                                                                            Source Skumatz Economic Research Associates SERA independent research

                                                                                                                                            0

                                                                                                                                            50

                                                                                                                                            100

                                                                                                                                            150

                                                                                                                                            200

                                                                                                                                            250

                                                                                                                                            300

                                                                                                                                            350

                                                                                                                                            400

                                                                                                                                            450

                                                                                                                                            0 10 20 30 40 50 60 70

                                                                                                                                            EUL Variation by Measure

                                                                                                                                            67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                                            Poorly supportedOldTechnology change

                                                                                                                                            SERA

                                                                                                                                            KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                                            Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                                            measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                                            change the BC conclusion

                                                                                                                                            On the cusphellip

                                                                                                                                            If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                                            IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                                            Source SERA all rights reserved

                                                                                                                                            SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                                            SERA

                                                                                                                                            IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                                            Utility Soc Part Conserv Recrsquom

                                                                                                                                            Rationale

                                                                                                                                            Base Percent X X X Program-invariant

                                                                                                                                            Low Income X X X X Multiple sources

                                                                                                                                            Weatherization X X X Substantial Participantimpacts

                                                                                                                                            Measure Program-specific

                                                                                                                                            X Varies by measure sector

                                                                                                                                            Other Recomrsquos Local Research

                                                                                                                                            50

                                                                                                                                            Source Skumatz SERA

                                                                                                                                            Developing values for multiple states amp utilities

                                                                                                                                            SERA

                                                                                                                                            TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                                            Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                                            Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                                            5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                                            6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                                            THANK YOU

                                                                                                                                            Questions

                                                                                                                                            Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                                            SERA

                                                                                                                                            POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                            53

                                                                                                                                            SERA

                                                                                                                                            NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                            Negative NEB values cost of barrier

                                                                                                                                            Solar Water Heat NZ$ Euros

                                                                                                                                            Solar Design NZ$ Euros

                                                                                                                                            Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                            -23 -12 (079)

                                                                                                                                            -11 -6 (06)

                                                                                                                                            Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                            Source Skumatz Economic Research Associates research

                                                                                                                                            SERA

                                                                                                                                            CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                            NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                            -5-4-3-2-1012345

                                                                                                                                            Op Co

                                                                                                                                            st

                                                                                                                                            Init C

                                                                                                                                            ost

                                                                                                                                            Maint

                                                                                                                                            Perfo

                                                                                                                                            rm

                                                                                                                                            Prod

                                                                                                                                            uctiv

                                                                                                                                            Tena

                                                                                                                                            nt Sa

                                                                                                                                            t

                                                                                                                                            Comf

                                                                                                                                            ort

                                                                                                                                            Appe

                                                                                                                                            ar

                                                                                                                                            Light

                                                                                                                                            Qual

                                                                                                                                            SellL

                                                                                                                                            ease

                                                                                                                                            Devp Ownr AampE

                                                                                                                                            Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                            Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                            Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                            Owners more positive ndashUnderinvestment train

                                                                                                                                            • Soup to nuts on non-energy benefits NEIs
                                                                                                                                            • TOPICS
                                                                                                                                            • BACKGROUND NEBS
                                                                                                                                            • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                            • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                            • KEY APPLICATIONS OF NEBS
                                                                                                                                            • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                            • Neb measurement
                                                                                                                                            • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                            • MEASURING PARTICIPANTNEBs
                                                                                                                                            • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                            • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                            • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                            • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                            • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                            • PROGRESS AND GAPS
                                                                                                                                            • PROGRESS AND GAPS
                                                                                                                                            • USING NEB NEI VALUES
                                                                                                                                            • Slide Number 19
                                                                                                                                            • ARE NEBS HIGH VALUE
                                                                                                                                            • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                            • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                            • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                            • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                            • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                            • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                            • AUDIT FINANCIAL PROGRAM
                                                                                                                                            • CampI NEW CONSTRUCTION
                                                                                                                                            • CampI TECHNICAL ASSISTANCE
                                                                                                                                            • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                            • Slide Number 31
                                                                                                                                            • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                            • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                            • UTILITY NEBS
                                                                                                                                            • ADJUSTED PAYBACKS
                                                                                                                                            • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                            • NEBS IN BC
                                                                                                                                            • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                            • APPLICATION TO BC
                                                                                                                                            • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                            • STATE TREATMENT OF NEBS NEBs
                                                                                                                                            • Slide Number 42
                                                                                                                                            • SELECTED STATE PROGRESS
                                                                                                                                            • SELECTED STATE PROGRESS
                                                                                                                                            • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                            • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                            • DISCOUNT RATE MATTERS
                                                                                                                                            • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                            • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                            • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                            • TAKEAWAYSON NEBSNEIS
                                                                                                                                            • THANK YOUQuestions
                                                                                                                                            • Slide Number 53
                                                                                                                                            • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                            • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                              SERA

                                                                                                                                              APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values

                                                                                                                                              Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo

                                                                                                                                              Expansion Concern with risk calculation accuracy

                                                                                                                                              45

                                                                                                                                              Source Skumatz SERA research

                                                                                                                                              SERA

                                                                                                                                              46

                                                                                                                                              NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                                              [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                                              SourceSkumatz SERA Independent research

                                                                                                                                              NTG ndash accuracy measurement incomplete

                                                                                                                                              Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                                              NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                                              EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                                              Cost Complicated expensive local changes

                                                                                                                                              RELATIVE Risk from NEBs

                                                                                                                                              Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                                              Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                                              NEBs as accurate as many inputs (all could improve)

                                                                                                                                              SERA

                                                                                                                                              DISCOUNT RATE MATTERS

                                                                                                                                              47

                                                                                                                                              Source Skumatz SERA Research

                                                                                                                                              0

                                                                                                                                              20

                                                                                                                                              40

                                                                                                                                              60

                                                                                                                                              80

                                                                                                                                              100

                                                                                                                                              120

                                                                                                                                              1 2 3 4 5 6 7 8 9 10

                                                                                                                                              PV Ratios vs 1 Discount Rate

                                                                                                                                              10 yr PV 20 yr PV

                                                                                                                                              Discount rate - percent

                                                                                                                                              WACCSocialLTT

                                                                                                                                              Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                                              SERA

                                                                                                                                              EUL VARIATIONS-ADOPTED VALUES

                                                                                                                                              Source Skumatz Economic Research Associates SERA independent research

                                                                                                                                              0

                                                                                                                                              50

                                                                                                                                              100

                                                                                                                                              150

                                                                                                                                              200

                                                                                                                                              250

                                                                                                                                              300

                                                                                                                                              350

                                                                                                                                              400

                                                                                                                                              450

                                                                                                                                              0 10 20 30 40 50 60 70

                                                                                                                                              EUL Variation by Measure

                                                                                                                                              67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                                              Poorly supportedOldTechnology change

                                                                                                                                              SERA

                                                                                                                                              KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                                              Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                                              measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                                              change the BC conclusion

                                                                                                                                              On the cusphellip

                                                                                                                                              If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                                              IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                                              Source SERA all rights reserved

                                                                                                                                              SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                                              SERA

                                                                                                                                              IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                                              Utility Soc Part Conserv Recrsquom

                                                                                                                                              Rationale

                                                                                                                                              Base Percent X X X Program-invariant

                                                                                                                                              Low Income X X X X Multiple sources

                                                                                                                                              Weatherization X X X Substantial Participantimpacts

                                                                                                                                              Measure Program-specific

                                                                                                                                              X Varies by measure sector

                                                                                                                                              Other Recomrsquos Local Research

                                                                                                                                              50

                                                                                                                                              Source Skumatz SERA

                                                                                                                                              Developing values for multiple states amp utilities

                                                                                                                                              SERA

                                                                                                                                              TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                                              Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                                              Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                                              5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                                              6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                                              THANK YOU

                                                                                                                                              Questions

                                                                                                                                              Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                                              SERA

                                                                                                                                              POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                              53

                                                                                                                                              SERA

                                                                                                                                              NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                              Negative NEB values cost of barrier

                                                                                                                                              Solar Water Heat NZ$ Euros

                                                                                                                                              Solar Design NZ$ Euros

                                                                                                                                              Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                              -23 -12 (079)

                                                                                                                                              -11 -6 (06)

                                                                                                                                              Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                              Source Skumatz Economic Research Associates research

                                                                                                                                              SERA

                                                                                                                                              CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                              NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                              -5-4-3-2-1012345

                                                                                                                                              Op Co

                                                                                                                                              st

                                                                                                                                              Init C

                                                                                                                                              ost

                                                                                                                                              Maint

                                                                                                                                              Perfo

                                                                                                                                              rm

                                                                                                                                              Prod

                                                                                                                                              uctiv

                                                                                                                                              Tena

                                                                                                                                              nt Sa

                                                                                                                                              t

                                                                                                                                              Comf

                                                                                                                                              ort

                                                                                                                                              Appe

                                                                                                                                              ar

                                                                                                                                              Light

                                                                                                                                              Qual

                                                                                                                                              SellL

                                                                                                                                              ease

                                                                                                                                              Devp Ownr AampE

                                                                                                                                              Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                              Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                              Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                              Owners more positive ndashUnderinvestment train

                                                                                                                                              • Soup to nuts on non-energy benefits NEIs
                                                                                                                                              • TOPICS
                                                                                                                                              • BACKGROUND NEBS
                                                                                                                                              • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                              • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                              • KEY APPLICATIONS OF NEBS
                                                                                                                                              • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                              • Neb measurement
                                                                                                                                              • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                              • MEASURING PARTICIPANTNEBs
                                                                                                                                              • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                              • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                              • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                              • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                              • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                              • PROGRESS AND GAPS
                                                                                                                                              • PROGRESS AND GAPS
                                                                                                                                              • USING NEB NEI VALUES
                                                                                                                                              • Slide Number 19
                                                                                                                                              • ARE NEBS HIGH VALUE
                                                                                                                                              • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                              • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                              • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                              • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                              • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                              • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                              • AUDIT FINANCIAL PROGRAM
                                                                                                                                              • CampI NEW CONSTRUCTION
                                                                                                                                              • CampI TECHNICAL ASSISTANCE
                                                                                                                                              • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                              • Slide Number 31
                                                                                                                                              • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                              • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                              • UTILITY NEBS
                                                                                                                                              • ADJUSTED PAYBACKS
                                                                                                                                              • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                              • NEBS IN BC
                                                                                                                                              • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                              • APPLICATION TO BC
                                                                                                                                              • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                              • STATE TREATMENT OF NEBS NEBs
                                                                                                                                              • Slide Number 42
                                                                                                                                              • SELECTED STATE PROGRESS
                                                                                                                                              • SELECTED STATE PROGRESS
                                                                                                                                              • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                              • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                              • DISCOUNT RATE MATTERS
                                                                                                                                              • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                              • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                              • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                              • TAKEAWAYSON NEBSNEIS
                                                                                                                                              • THANK YOUQuestions
                                                                                                                                              • Slide Number 53
                                                                                                                                              • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                              • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                SERA

                                                                                                                                                46

                                                                                                                                                NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges

                                                                                                                                                [PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]

                                                                                                                                                SourceSkumatz SERA Independent research

                                                                                                                                                NTG ndash accuracy measurement incomplete

                                                                                                                                                Savings Impact repeatedly amp expensivelymeasured little variation $100K+

                                                                                                                                                NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps

                                                                                                                                                EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation

                                                                                                                                                Cost Complicated expensive local changes

                                                                                                                                                RELATIVE Risk from NEBs

                                                                                                                                                Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment

                                                                                                                                                Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo

                                                                                                                                                NEBs as accurate as many inputs (all could improve)

                                                                                                                                                SERA

                                                                                                                                                DISCOUNT RATE MATTERS

                                                                                                                                                47

                                                                                                                                                Source Skumatz SERA Research

                                                                                                                                                0

                                                                                                                                                20

                                                                                                                                                40

                                                                                                                                                60

                                                                                                                                                80

                                                                                                                                                100

                                                                                                                                                120

                                                                                                                                                1 2 3 4 5 6 7 8 9 10

                                                                                                                                                PV Ratios vs 1 Discount Rate

                                                                                                                                                10 yr PV 20 yr PV

                                                                                                                                                Discount rate - percent

                                                                                                                                                WACCSocialLTT

                                                                                                                                                Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                                                SERA

                                                                                                                                                EUL VARIATIONS-ADOPTED VALUES

                                                                                                                                                Source Skumatz Economic Research Associates SERA independent research

                                                                                                                                                0

                                                                                                                                                50

                                                                                                                                                100

                                                                                                                                                150

                                                                                                                                                200

                                                                                                                                                250

                                                                                                                                                300

                                                                                                                                                350

                                                                                                                                                400

                                                                                                                                                450

                                                                                                                                                0 10 20 30 40 50 60 70

                                                                                                                                                EUL Variation by Measure

                                                                                                                                                67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                                                Poorly supportedOldTechnology change

                                                                                                                                                SERA

                                                                                                                                                KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                                                Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                                                measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                                                change the BC conclusion

                                                                                                                                                On the cusphellip

                                                                                                                                                If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                                                IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                                                Source SERA all rights reserved

                                                                                                                                                SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                                                SERA

                                                                                                                                                IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                                                Utility Soc Part Conserv Recrsquom

                                                                                                                                                Rationale

                                                                                                                                                Base Percent X X X Program-invariant

                                                                                                                                                Low Income X X X X Multiple sources

                                                                                                                                                Weatherization X X X Substantial Participantimpacts

                                                                                                                                                Measure Program-specific

                                                                                                                                                X Varies by measure sector

                                                                                                                                                Other Recomrsquos Local Research

                                                                                                                                                50

                                                                                                                                                Source Skumatz SERA

                                                                                                                                                Developing values for multiple states amp utilities

                                                                                                                                                SERA

                                                                                                                                                TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                                                Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                                                Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                                                5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                                                6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                                                THANK YOU

                                                                                                                                                Questions

                                                                                                                                                Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                                                SERA

                                                                                                                                                POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                                53

                                                                                                                                                SERA

                                                                                                                                                NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                                Negative NEB values cost of barrier

                                                                                                                                                Solar Water Heat NZ$ Euros

                                                                                                                                                Solar Design NZ$ Euros

                                                                                                                                                Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                                -23 -12 (079)

                                                                                                                                                -11 -6 (06)

                                                                                                                                                Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                                Source Skumatz Economic Research Associates research

                                                                                                                                                SERA

                                                                                                                                                CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                                -5-4-3-2-1012345

                                                                                                                                                Op Co

                                                                                                                                                st

                                                                                                                                                Init C

                                                                                                                                                ost

                                                                                                                                                Maint

                                                                                                                                                Perfo

                                                                                                                                                rm

                                                                                                                                                Prod

                                                                                                                                                uctiv

                                                                                                                                                Tena

                                                                                                                                                nt Sa

                                                                                                                                                t

                                                                                                                                                Comf

                                                                                                                                                ort

                                                                                                                                                Appe

                                                                                                                                                ar

                                                                                                                                                Light

                                                                                                                                                Qual

                                                                                                                                                SellL

                                                                                                                                                ease

                                                                                                                                                Devp Ownr AampE

                                                                                                                                                Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                                Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                                Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                                Owners more positive ndashUnderinvestment train

                                                                                                                                                • Soup to nuts on non-energy benefits NEIs
                                                                                                                                                • TOPICS
                                                                                                                                                • BACKGROUND NEBS
                                                                                                                                                • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                                • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                                • KEY APPLICATIONS OF NEBS
                                                                                                                                                • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                                • Neb measurement
                                                                                                                                                • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                                • MEASURING PARTICIPANTNEBs
                                                                                                                                                • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                                • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                                • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                                • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                                • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                                • PROGRESS AND GAPS
                                                                                                                                                • PROGRESS AND GAPS
                                                                                                                                                • USING NEB NEI VALUES
                                                                                                                                                • Slide Number 19
                                                                                                                                                • ARE NEBS HIGH VALUE
                                                                                                                                                • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                                • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                                • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                                • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                                • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                • AUDIT FINANCIAL PROGRAM
                                                                                                                                                • CampI NEW CONSTRUCTION
                                                                                                                                                • CampI TECHNICAL ASSISTANCE
                                                                                                                                                • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                                • Slide Number 31
                                                                                                                                                • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                                • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                                • UTILITY NEBS
                                                                                                                                                • ADJUSTED PAYBACKS
                                                                                                                                                • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                                • NEBS IN BC
                                                                                                                                                • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                                • APPLICATION TO BC
                                                                                                                                                • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                                • STATE TREATMENT OF NEBS NEBs
                                                                                                                                                • Slide Number 42
                                                                                                                                                • SELECTED STATE PROGRESS
                                                                                                                                                • SELECTED STATE PROGRESS
                                                                                                                                                • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                                • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                                • DISCOUNT RATE MATTERS
                                                                                                                                                • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                                • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                                • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                                • TAKEAWAYSON NEBSNEIS
                                                                                                                                                • THANK YOUQuestions
                                                                                                                                                • Slide Number 53
                                                                                                                                                • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                                • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                  SERA

                                                                                                                                                  DISCOUNT RATE MATTERS

                                                                                                                                                  47

                                                                                                                                                  Source Skumatz SERA Research

                                                                                                                                                  0

                                                                                                                                                  20

                                                                                                                                                  40

                                                                                                                                                  60

                                                                                                                                                  80

                                                                                                                                                  100

                                                                                                                                                  120

                                                                                                                                                  1 2 3 4 5 6 7 8 9 10

                                                                                                                                                  PV Ratios vs 1 Discount Rate

                                                                                                                                                  10 yr PV 20 yr PV

                                                                                                                                                  Discount rate - percent

                                                                                                                                                  WACCSocialLTT

                                                                                                                                                  Source Skumatz Electricity Journal IEPEC paper independent research

                                                                                                                                                  SERA

                                                                                                                                                  EUL VARIATIONS-ADOPTED VALUES

                                                                                                                                                  Source Skumatz Economic Research Associates SERA independent research

                                                                                                                                                  0

                                                                                                                                                  50

                                                                                                                                                  100

                                                                                                                                                  150

                                                                                                                                                  200

                                                                                                                                                  250

                                                                                                                                                  300

                                                                                                                                                  350

                                                                                                                                                  400

                                                                                                                                                  450

                                                                                                                                                  0 10 20 30 40 50 60 70

                                                                                                                                                  EUL Variation by Measure

                                                                                                                                                  67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                                                  Poorly supportedOldTechnology change

                                                                                                                                                  SERA

                                                                                                                                                  KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                                                  Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                                                  measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                                                  change the BC conclusion

                                                                                                                                                  On the cusphellip

                                                                                                                                                  If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                                                  IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                                                  Source SERA all rights reserved

                                                                                                                                                  SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                                                  SERA

                                                                                                                                                  IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                                                  Utility Soc Part Conserv Recrsquom

                                                                                                                                                  Rationale

                                                                                                                                                  Base Percent X X X Program-invariant

                                                                                                                                                  Low Income X X X X Multiple sources

                                                                                                                                                  Weatherization X X X Substantial Participantimpacts

                                                                                                                                                  Measure Program-specific

                                                                                                                                                  X Varies by measure sector

                                                                                                                                                  Other Recomrsquos Local Research

                                                                                                                                                  50

                                                                                                                                                  Source Skumatz SERA

                                                                                                                                                  Developing values for multiple states amp utilities

                                                                                                                                                  SERA

                                                                                                                                                  TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                                                  Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                                                  Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                                                  5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                                                  6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                                                  THANK YOU

                                                                                                                                                  Questions

                                                                                                                                                  Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                                                  SERA

                                                                                                                                                  POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                                  53

                                                                                                                                                  SERA

                                                                                                                                                  NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                                  Negative NEB values cost of barrier

                                                                                                                                                  Solar Water Heat NZ$ Euros

                                                                                                                                                  Solar Design NZ$ Euros

                                                                                                                                                  Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                                  -23 -12 (079)

                                                                                                                                                  -11 -6 (06)

                                                                                                                                                  Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                                  Source Skumatz Economic Research Associates research

                                                                                                                                                  SERA

                                                                                                                                                  CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                  NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                                  -5-4-3-2-1012345

                                                                                                                                                  Op Co

                                                                                                                                                  st

                                                                                                                                                  Init C

                                                                                                                                                  ost

                                                                                                                                                  Maint

                                                                                                                                                  Perfo

                                                                                                                                                  rm

                                                                                                                                                  Prod

                                                                                                                                                  uctiv

                                                                                                                                                  Tena

                                                                                                                                                  nt Sa

                                                                                                                                                  t

                                                                                                                                                  Comf

                                                                                                                                                  ort

                                                                                                                                                  Appe

                                                                                                                                                  ar

                                                                                                                                                  Light

                                                                                                                                                  Qual

                                                                                                                                                  SellL

                                                                                                                                                  ease

                                                                                                                                                  Devp Ownr AampE

                                                                                                                                                  Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                                  Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                                  Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                                  Owners more positive ndashUnderinvestment train

                                                                                                                                                  • Soup to nuts on non-energy benefits NEIs
                                                                                                                                                  • TOPICS
                                                                                                                                                  • BACKGROUND NEBS
                                                                                                                                                  • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                                  • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                                  • KEY APPLICATIONS OF NEBS
                                                                                                                                                  • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                                  • Neb measurement
                                                                                                                                                  • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                                  • MEASURING PARTICIPANTNEBs
                                                                                                                                                  • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                                  • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                                  • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                                  • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                                  • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                                  • PROGRESS AND GAPS
                                                                                                                                                  • PROGRESS AND GAPS
                                                                                                                                                  • USING NEB NEI VALUES
                                                                                                                                                  • Slide Number 19
                                                                                                                                                  • ARE NEBS HIGH VALUE
                                                                                                                                                  • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                  • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                                  • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                                  • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                                  • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                                  • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                  • AUDIT FINANCIAL PROGRAM
                                                                                                                                                  • CampI NEW CONSTRUCTION
                                                                                                                                                  • CampI TECHNICAL ASSISTANCE
                                                                                                                                                  • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                                  • Slide Number 31
                                                                                                                                                  • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                                  • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                                  • UTILITY NEBS
                                                                                                                                                  • ADJUSTED PAYBACKS
                                                                                                                                                  • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                                  • NEBS IN BC
                                                                                                                                                  • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                                  • APPLICATION TO BC
                                                                                                                                                  • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                                  • STATE TREATMENT OF NEBS NEBs
                                                                                                                                                  • Slide Number 42
                                                                                                                                                  • SELECTED STATE PROGRESS
                                                                                                                                                  • SELECTED STATE PROGRESS
                                                                                                                                                  • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                                  • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                                  • DISCOUNT RATE MATTERS
                                                                                                                                                  • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                                  • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                                  • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                                  • TAKEAWAYSON NEBSNEIS
                                                                                                                                                  • THANK YOUQuestions
                                                                                                                                                  • Slide Number 53
                                                                                                                                                  • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                                  • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                    SERA

                                                                                                                                                    EUL VARIATIONS-ADOPTED VALUES

                                                                                                                                                    Source Skumatz Economic Research Associates SERA independent research

                                                                                                                                                    0

                                                                                                                                                    50

                                                                                                                                                    100

                                                                                                                                                    150

                                                                                                                                                    200

                                                                                                                                                    250

                                                                                                                                                    300

                                                                                                                                                    350

                                                                                                                                                    400

                                                                                                                                                    450

                                                                                                                                                    0 10 20 30 40 50 60 70

                                                                                                                                                    EUL Variation by Measure

                                                                                                                                                    67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages

                                                                                                                                                    Poorly supportedOldTechnology change

                                                                                                                                                    SERA

                                                                                                                                                    KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                                                    Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                                                    measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                                                    change the BC conclusion

                                                                                                                                                    On the cusphellip

                                                                                                                                                    If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                                                    IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                                                    Source SERA all rights reserved

                                                                                                                                                    SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                                                    SERA

                                                                                                                                                    IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                                                    Utility Soc Part Conserv Recrsquom

                                                                                                                                                    Rationale

                                                                                                                                                    Base Percent X X X Program-invariant

                                                                                                                                                    Low Income X X X X Multiple sources

                                                                                                                                                    Weatherization X X X Substantial Participantimpacts

                                                                                                                                                    Measure Program-specific

                                                                                                                                                    X Varies by measure sector

                                                                                                                                                    Other Recomrsquos Local Research

                                                                                                                                                    50

                                                                                                                                                    Source Skumatz SERA

                                                                                                                                                    Developing values for multiple states amp utilities

                                                                                                                                                    SERA

                                                                                                                                                    TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                                                    Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                                                    Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                                                    5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                                                    6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                                                    THANK YOU

                                                                                                                                                    Questions

                                                                                                                                                    Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                                                    SERA

                                                                                                                                                    POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                                    53

                                                                                                                                                    SERA

                                                                                                                                                    NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                                    Negative NEB values cost of barrier

                                                                                                                                                    Solar Water Heat NZ$ Euros

                                                                                                                                                    Solar Design NZ$ Euros

                                                                                                                                                    Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                                    -23 -12 (079)

                                                                                                                                                    -11 -6 (06)

                                                                                                                                                    Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                                    Source Skumatz Economic Research Associates research

                                                                                                                                                    SERA

                                                                                                                                                    CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                    NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                                    -5-4-3-2-1012345

                                                                                                                                                    Op Co

                                                                                                                                                    st

                                                                                                                                                    Init C

                                                                                                                                                    ost

                                                                                                                                                    Maint

                                                                                                                                                    Perfo

                                                                                                                                                    rm

                                                                                                                                                    Prod

                                                                                                                                                    uctiv

                                                                                                                                                    Tena

                                                                                                                                                    nt Sa

                                                                                                                                                    t

                                                                                                                                                    Comf

                                                                                                                                                    ort

                                                                                                                                                    Appe

                                                                                                                                                    ar

                                                                                                                                                    Light

                                                                                                                                                    Qual

                                                                                                                                                    SellL

                                                                                                                                                    ease

                                                                                                                                                    Devp Ownr AampE

                                                                                                                                                    Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                                    Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                                    Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                                    Owners more positive ndashUnderinvestment train

                                                                                                                                                    • Soup to nuts on non-energy benefits NEIs
                                                                                                                                                    • TOPICS
                                                                                                                                                    • BACKGROUND NEBS
                                                                                                                                                    • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                                    • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                                    • KEY APPLICATIONS OF NEBS
                                                                                                                                                    • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                                    • Neb measurement
                                                                                                                                                    • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                                    • MEASURING PARTICIPANTNEBs
                                                                                                                                                    • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                                    • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                                    • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                                    • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                                    • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                                    • PROGRESS AND GAPS
                                                                                                                                                    • PROGRESS AND GAPS
                                                                                                                                                    • USING NEB NEI VALUES
                                                                                                                                                    • Slide Number 19
                                                                                                                                                    • ARE NEBS HIGH VALUE
                                                                                                                                                    • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                    • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                                    • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                                    • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                                    • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                                    • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                    • AUDIT FINANCIAL PROGRAM
                                                                                                                                                    • CampI NEW CONSTRUCTION
                                                                                                                                                    • CampI TECHNICAL ASSISTANCE
                                                                                                                                                    • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                                    • Slide Number 31
                                                                                                                                                    • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                                    • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                                    • UTILITY NEBS
                                                                                                                                                    • ADJUSTED PAYBACKS
                                                                                                                                                    • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                                    • NEBS IN BC
                                                                                                                                                    • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                                    • APPLICATION TO BC
                                                                                                                                                    • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                                    • STATE TREATMENT OF NEBS NEBs
                                                                                                                                                    • Slide Number 42
                                                                                                                                                    • SELECTED STATE PROGRESS
                                                                                                                                                    • SELECTED STATE PROGRESS
                                                                                                                                                    • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                                    • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                                    • DISCOUNT RATE MATTERS
                                                                                                                                                    • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                                    • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                                    • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                                    • TAKEAWAYSON NEBSNEIS
                                                                                                                                                    • THANK YOUQuestions
                                                                                                                                                    • Slide Number 53
                                                                                                                                                    • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                                    • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                      SERA

                                                                                                                                                      KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests

                                                                                                                                                      Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost

                                                                                                                                                      measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)

                                                                                                                                                      change the BC conclusion

                                                                                                                                                      On the cusphellip

                                                                                                                                                      If NO Yoursquore doneAnd bias addressedsufficiently

                                                                                                                                                      IF YESRefine measurement up to value or cost of ldquowrongrdquo decision

                                                                                                                                                      Source SERA all rights reserved

                                                                                                                                                      SERArsquos lsquoNEB-ItrdquoModel ndash

                                                                                                                                                      SERA

                                                                                                                                                      IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                                                      Utility Soc Part Conserv Recrsquom

                                                                                                                                                      Rationale

                                                                                                                                                      Base Percent X X X Program-invariant

                                                                                                                                                      Low Income X X X X Multiple sources

                                                                                                                                                      Weatherization X X X Substantial Participantimpacts

                                                                                                                                                      Measure Program-specific

                                                                                                                                                      X Varies by measure sector

                                                                                                                                                      Other Recomrsquos Local Research

                                                                                                                                                      50

                                                                                                                                                      Source Skumatz SERA

                                                                                                                                                      Developing values for multiple states amp utilities

                                                                                                                                                      SERA

                                                                                                                                                      TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                                                      Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                                                      Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                                                      5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                                                      6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                                                      THANK YOU

                                                                                                                                                      Questions

                                                                                                                                                      Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                                                      SERA

                                                                                                                                                      POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                                      53

                                                                                                                                                      SERA

                                                                                                                                                      NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                                      Negative NEB values cost of barrier

                                                                                                                                                      Solar Water Heat NZ$ Euros

                                                                                                                                                      Solar Design NZ$ Euros

                                                                                                                                                      Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                                      -23 -12 (079)

                                                                                                                                                      -11 -6 (06)

                                                                                                                                                      Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                                      Source Skumatz Economic Research Associates research

                                                                                                                                                      SERA

                                                                                                                                                      CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                      NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                                      -5-4-3-2-1012345

                                                                                                                                                      Op Co

                                                                                                                                                      st

                                                                                                                                                      Init C

                                                                                                                                                      ost

                                                                                                                                                      Maint

                                                                                                                                                      Perfo

                                                                                                                                                      rm

                                                                                                                                                      Prod

                                                                                                                                                      uctiv

                                                                                                                                                      Tena

                                                                                                                                                      nt Sa

                                                                                                                                                      t

                                                                                                                                                      Comf

                                                                                                                                                      ort

                                                                                                                                                      Appe

                                                                                                                                                      ar

                                                                                                                                                      Light

                                                                                                                                                      Qual

                                                                                                                                                      SellL

                                                                                                                                                      ease

                                                                                                                                                      Devp Ownr AampE

                                                                                                                                                      Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                                      Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                                      Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                                      Owners more positive ndashUnderinvestment train

                                                                                                                                                      • Soup to nuts on non-energy benefits NEIs
                                                                                                                                                      • TOPICS
                                                                                                                                                      • BACKGROUND NEBS
                                                                                                                                                      • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                                      • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                                      • KEY APPLICATIONS OF NEBS
                                                                                                                                                      • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                                      • Neb measurement
                                                                                                                                                      • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                                      • MEASURING PARTICIPANTNEBs
                                                                                                                                                      • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                                      • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                                      • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                                      • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                                      • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                                      • PROGRESS AND GAPS
                                                                                                                                                      • PROGRESS AND GAPS
                                                                                                                                                      • USING NEB NEI VALUES
                                                                                                                                                      • Slide Number 19
                                                                                                                                                      • ARE NEBS HIGH VALUE
                                                                                                                                                      • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                      • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                                      • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                                      • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                                      • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                                      • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                      • AUDIT FINANCIAL PROGRAM
                                                                                                                                                      • CampI NEW CONSTRUCTION
                                                                                                                                                      • CampI TECHNICAL ASSISTANCE
                                                                                                                                                      • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                                      • Slide Number 31
                                                                                                                                                      • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                                      • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                                      • UTILITY NEBS
                                                                                                                                                      • ADJUSTED PAYBACKS
                                                                                                                                                      • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                                      • NEBS IN BC
                                                                                                                                                      • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                                      • APPLICATION TO BC
                                                                                                                                                      • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                                      • STATE TREATMENT OF NEBS NEBs
                                                                                                                                                      • Slide Number 42
                                                                                                                                                      • SELECTED STATE PROGRESS
                                                                                                                                                      • SELECTED STATE PROGRESS
                                                                                                                                                      • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                                      • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                                      • DISCOUNT RATE MATTERS
                                                                                                                                                      • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                                      • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                                      • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                                      • TAKEAWAYSON NEBSNEIS
                                                                                                                                                      • THANK YOUQuestions
                                                                                                                                                      • Slide Number 53
                                                                                                                                                      • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                                      • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                        SERA

                                                                                                                                                        IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES

                                                                                                                                                        Utility Soc Part Conserv Recrsquom

                                                                                                                                                        Rationale

                                                                                                                                                        Base Percent X X X Program-invariant

                                                                                                                                                        Low Income X X X X Multiple sources

                                                                                                                                                        Weatherization X X X Substantial Participantimpacts

                                                                                                                                                        Measure Program-specific

                                                                                                                                                        X Varies by measure sector

                                                                                                                                                        Other Recomrsquos Local Research

                                                                                                                                                        50

                                                                                                                                                        Source Skumatz SERA

                                                                                                                                                        Developing values for multiple states amp utilities

                                                                                                                                                        SERA

                                                                                                                                                        TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                                                        Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                                                        Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                                                        5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                                                        6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                                                        THANK YOU

                                                                                                                                                        Questions

                                                                                                                                                        Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                                                        SERA

                                                                                                                                                        POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                                        53

                                                                                                                                                        SERA

                                                                                                                                                        NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                                        Negative NEB values cost of barrier

                                                                                                                                                        Solar Water Heat NZ$ Euros

                                                                                                                                                        Solar Design NZ$ Euros

                                                                                                                                                        Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                                        -23 -12 (079)

                                                                                                                                                        -11 -6 (06)

                                                                                                                                                        Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                                        Source Skumatz Economic Research Associates research

                                                                                                                                                        SERA

                                                                                                                                                        CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                        NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                                        -5-4-3-2-1012345

                                                                                                                                                        Op Co

                                                                                                                                                        st

                                                                                                                                                        Init C

                                                                                                                                                        ost

                                                                                                                                                        Maint

                                                                                                                                                        Perfo

                                                                                                                                                        rm

                                                                                                                                                        Prod

                                                                                                                                                        uctiv

                                                                                                                                                        Tena

                                                                                                                                                        nt Sa

                                                                                                                                                        t

                                                                                                                                                        Comf

                                                                                                                                                        ort

                                                                                                                                                        Appe

                                                                                                                                                        ar

                                                                                                                                                        Light

                                                                                                                                                        Qual

                                                                                                                                                        SellL

                                                                                                                                                        ease

                                                                                                                                                        Devp Ownr AampE

                                                                                                                                                        Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                                        Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                                        Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                                        Owners more positive ndashUnderinvestment train

                                                                                                                                                        • Soup to nuts on non-energy benefits NEIs
                                                                                                                                                        • TOPICS
                                                                                                                                                        • BACKGROUND NEBS
                                                                                                                                                        • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                                        • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                                        • KEY APPLICATIONS OF NEBS
                                                                                                                                                        • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                                        • Neb measurement
                                                                                                                                                        • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                                        • MEASURING PARTICIPANTNEBs
                                                                                                                                                        • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                                        • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                                        • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                                        • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                                        • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                                        • PROGRESS AND GAPS
                                                                                                                                                        • PROGRESS AND GAPS
                                                                                                                                                        • USING NEB NEI VALUES
                                                                                                                                                        • Slide Number 19
                                                                                                                                                        • ARE NEBS HIGH VALUE
                                                                                                                                                        • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                        • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                                        • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                                        • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                                        • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                                        • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                        • AUDIT FINANCIAL PROGRAM
                                                                                                                                                        • CampI NEW CONSTRUCTION
                                                                                                                                                        • CampI TECHNICAL ASSISTANCE
                                                                                                                                                        • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                                        • Slide Number 31
                                                                                                                                                        • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                                        • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                                        • UTILITY NEBS
                                                                                                                                                        • ADJUSTED PAYBACKS
                                                                                                                                                        • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                                        • NEBS IN BC
                                                                                                                                                        • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                                        • APPLICATION TO BC
                                                                                                                                                        • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                                        • STATE TREATMENT OF NEBS NEBs
                                                                                                                                                        • Slide Number 42
                                                                                                                                                        • SELECTED STATE PROGRESS
                                                                                                                                                        • SELECTED STATE PROGRESS
                                                                                                                                                        • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                                        • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                                        • DISCOUNT RATE MATTERS
                                                                                                                                                        • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                                        • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                                        • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                                        • TAKEAWAYSON NEBSNEIS
                                                                                                                                                        • THANK YOUQuestions
                                                                                                                                                        • Slide Number 53
                                                                                                                                                        • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                                        • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                          SERA

                                                                                                                                                          TAKEAWAYSON NEBSNEIS 1) Valuable ndash can exceed savings matters bias 2) Defensible methods available estimates 3) Transferable depending ldquoadd-onrdquo analysis 4) Uncertainties acceptable

                                                                                                                                                          Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]

                                                                                                                                                          Other sources of bias to research (EUL NTG etc) bring high variation

                                                                                                                                                          5) States are incorporating NEBs dominos input in deliberations in multiple states

                                                                                                                                                          6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author

                                                                                                                                                          THANK YOU

                                                                                                                                                          Questions

                                                                                                                                                          Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                                                          SERA

                                                                                                                                                          POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                                          53

                                                                                                                                                          SERA

                                                                                                                                                          NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                                          Negative NEB values cost of barrier

                                                                                                                                                          Solar Water Heat NZ$ Euros

                                                                                                                                                          Solar Design NZ$ Euros

                                                                                                                                                          Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                                          -23 -12 (079)

                                                                                                                                                          -11 -6 (06)

                                                                                                                                                          Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                                          Source Skumatz Economic Research Associates research

                                                                                                                                                          SERA

                                                                                                                                                          CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                          NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                                          -5-4-3-2-1012345

                                                                                                                                                          Op Co

                                                                                                                                                          st

                                                                                                                                                          Init C

                                                                                                                                                          ost

                                                                                                                                                          Maint

                                                                                                                                                          Perfo

                                                                                                                                                          rm

                                                                                                                                                          Prod

                                                                                                                                                          uctiv

                                                                                                                                                          Tena

                                                                                                                                                          nt Sa

                                                                                                                                                          t

                                                                                                                                                          Comf

                                                                                                                                                          ort

                                                                                                                                                          Appe

                                                                                                                                                          ar

                                                                                                                                                          Light

                                                                                                                                                          Qual

                                                                                                                                                          SellL

                                                                                                                                                          ease

                                                                                                                                                          Devp Ownr AampE

                                                                                                                                                          Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                                          Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                                          Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                                          Owners more positive ndashUnderinvestment train

                                                                                                                                                          • Soup to nuts on non-energy benefits NEIs
                                                                                                                                                          • TOPICS
                                                                                                                                                          • BACKGROUND NEBS
                                                                                                                                                          • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                                          • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                                          • KEY APPLICATIONS OF NEBS
                                                                                                                                                          • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                                          • Neb measurement
                                                                                                                                                          • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                                          • MEASURING PARTICIPANTNEBs
                                                                                                                                                          • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                                          • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                                          • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                                          • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                                          • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                                          • PROGRESS AND GAPS
                                                                                                                                                          • PROGRESS AND GAPS
                                                                                                                                                          • USING NEB NEI VALUES
                                                                                                                                                          • Slide Number 19
                                                                                                                                                          • ARE NEBS HIGH VALUE
                                                                                                                                                          • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                          • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                                          • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                                          • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                                          • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                                          • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                          • AUDIT FINANCIAL PROGRAM
                                                                                                                                                          • CampI NEW CONSTRUCTION
                                                                                                                                                          • CampI TECHNICAL ASSISTANCE
                                                                                                                                                          • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                                          • Slide Number 31
                                                                                                                                                          • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                                          • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                                          • UTILITY NEBS
                                                                                                                                                          • ADJUSTED PAYBACKS
                                                                                                                                                          • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                                          • NEBS IN BC
                                                                                                                                                          • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                                          • APPLICATION TO BC
                                                                                                                                                          • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                                          • STATE TREATMENT OF NEBS NEBs
                                                                                                                                                          • Slide Number 42
                                                                                                                                                          • SELECTED STATE PROGRESS
                                                                                                                                                          • SELECTED STATE PROGRESS
                                                                                                                                                          • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                                          • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                                          • DISCOUNT RATE MATTERS
                                                                                                                                                          • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                                          • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                                          • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                                          • TAKEAWAYSON NEBSNEIS
                                                                                                                                                          • THANK YOUQuestions
                                                                                                                                                          • Slide Number 53
                                                                                                                                                          • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                                          • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                            THANK YOU

                                                                                                                                                            Questions

                                                                                                                                                            Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom

                                                                                                                                                            SERA

                                                                                                                                                            POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                                            53

                                                                                                                                                            SERA

                                                                                                                                                            NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                                            Negative NEB values cost of barrier

                                                                                                                                                            Solar Water Heat NZ$ Euros

                                                                                                                                                            Solar Design NZ$ Euros

                                                                                                                                                            Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                                            -23 -12 (079)

                                                                                                                                                            -11 -6 (06)

                                                                                                                                                            Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                                            Source Skumatz Economic Research Associates research

                                                                                                                                                            SERA

                                                                                                                                                            CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                            NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                                            -5-4-3-2-1012345

                                                                                                                                                            Op Co

                                                                                                                                                            st

                                                                                                                                                            Init C

                                                                                                                                                            ost

                                                                                                                                                            Maint

                                                                                                                                                            Perfo

                                                                                                                                                            rm

                                                                                                                                                            Prod

                                                                                                                                                            uctiv

                                                                                                                                                            Tena

                                                                                                                                                            nt Sa

                                                                                                                                                            t

                                                                                                                                                            Comf

                                                                                                                                                            ort

                                                                                                                                                            Appe

                                                                                                                                                            ar

                                                                                                                                                            Light

                                                                                                                                                            Qual

                                                                                                                                                            SellL

                                                                                                                                                            ease

                                                                                                                                                            Devp Ownr AampE

                                                                                                                                                            Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                                            Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                                            Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                                            Owners more positive ndashUnderinvestment train

                                                                                                                                                            • Soup to nuts on non-energy benefits NEIs
                                                                                                                                                            • TOPICS
                                                                                                                                                            • BACKGROUND NEBS
                                                                                                                                                            • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                                            • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                                            • KEY APPLICATIONS OF NEBS
                                                                                                                                                            • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                                            • Neb measurement
                                                                                                                                                            • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                                            • MEASURING PARTICIPANTNEBs
                                                                                                                                                            • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                                            • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                                            • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                                            • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                                            • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                                            • PROGRESS AND GAPS
                                                                                                                                                            • PROGRESS AND GAPS
                                                                                                                                                            • USING NEB NEI VALUES
                                                                                                                                                            • Slide Number 19
                                                                                                                                                            • ARE NEBS HIGH VALUE
                                                                                                                                                            • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                            • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                                            • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                                            • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                                            • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                                            • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                            • AUDIT FINANCIAL PROGRAM
                                                                                                                                                            • CampI NEW CONSTRUCTION
                                                                                                                                                            • CampI TECHNICAL ASSISTANCE
                                                                                                                                                            • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                                            • Slide Number 31
                                                                                                                                                            • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                                            • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                                            • UTILITY NEBS
                                                                                                                                                            • ADJUSTED PAYBACKS
                                                                                                                                                            • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                                            • NEBS IN BC
                                                                                                                                                            • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                                            • APPLICATION TO BC
                                                                                                                                                            • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                                            • STATE TREATMENT OF NEBS NEBs
                                                                                                                                                            • Slide Number 42
                                                                                                                                                            • SELECTED STATE PROGRESS
                                                                                                                                                            • SELECTED STATE PROGRESS
                                                                                                                                                            • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                                            • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                                            • DISCOUNT RATE MATTERS
                                                                                                                                                            • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                                            • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                                            • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                                            • TAKEAWAYSON NEBSNEIS
                                                                                                                                                            • THANK YOUQuestions
                                                                                                                                                            • Slide Number 53
                                                                                                                                                            • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                                            • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                              SERA

                                                                                                                                                              POWER OF NEBS ndashBARRIERS DISCONNECTS

                                                                                                                                                              53

                                                                                                                                                              SERA

                                                                                                                                                              NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                                              Negative NEB values cost of barrier

                                                                                                                                                              Solar Water Heat NZ$ Euros

                                                                                                                                                              Solar Design NZ$ Euros

                                                                                                                                                              Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                                              -23 -12 (079)

                                                                                                                                                              -11 -6 (06)

                                                                                                                                                              Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                                              Source Skumatz Economic Research Associates research

                                                                                                                                                              SERA

                                                                                                                                                              CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                              NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                                              -5-4-3-2-1012345

                                                                                                                                                              Op Co

                                                                                                                                                              st

                                                                                                                                                              Init C

                                                                                                                                                              ost

                                                                                                                                                              Maint

                                                                                                                                                              Perfo

                                                                                                                                                              rm

                                                                                                                                                              Prod

                                                                                                                                                              uctiv

                                                                                                                                                              Tena

                                                                                                                                                              nt Sa

                                                                                                                                                              t

                                                                                                                                                              Comf

                                                                                                                                                              ort

                                                                                                                                                              Appe

                                                                                                                                                              ar

                                                                                                                                                              Light

                                                                                                                                                              Qual

                                                                                                                                                              SellL

                                                                                                                                                              ease

                                                                                                                                                              Devp Ownr AampE

                                                                                                                                                              Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                                              Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                                              Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                                              Owners more positive ndashUnderinvestment train

                                                                                                                                                              • Soup to nuts on non-energy benefits NEIs
                                                                                                                                                              • TOPICS
                                                                                                                                                              • BACKGROUND NEBS
                                                                                                                                                              • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                                              • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                                              • KEY APPLICATIONS OF NEBS
                                                                                                                                                              • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                                              • Neb measurement
                                                                                                                                                              • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                                              • MEASURING PARTICIPANTNEBs
                                                                                                                                                              • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                                              • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                                              • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                                              • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                                              • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                                              • PROGRESS AND GAPS
                                                                                                                                                              • PROGRESS AND GAPS
                                                                                                                                                              • USING NEB NEI VALUES
                                                                                                                                                              • Slide Number 19
                                                                                                                                                              • ARE NEBS HIGH VALUE
                                                                                                                                                              • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                              • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                                              • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                                              • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                                              • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                                              • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                              • AUDIT FINANCIAL PROGRAM
                                                                                                                                                              • CampI NEW CONSTRUCTION
                                                                                                                                                              • CampI TECHNICAL ASSISTANCE
                                                                                                                                                              • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                                              • Slide Number 31
                                                                                                                                                              • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                                              • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                                              • UTILITY NEBS
                                                                                                                                                              • ADJUSTED PAYBACKS
                                                                                                                                                              • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                                              • NEBS IN BC
                                                                                                                                                              • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                                              • APPLICATION TO BC
                                                                                                                                                              • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                                              • STATE TREATMENT OF NEBS NEBs
                                                                                                                                                              • Slide Number 42
                                                                                                                                                              • SELECTED STATE PROGRESS
                                                                                                                                                              • SELECTED STATE PROGRESS
                                                                                                                                                              • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                                              • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                                              • DISCOUNT RATE MATTERS
                                                                                                                                                              • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                                              • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                                              • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                                              • TAKEAWAYSON NEBSNEIS
                                                                                                                                                              • THANK YOUQuestions
                                                                                                                                                              • Slide Number 53
                                                                                                                                                              • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                                              • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                                SERA

                                                                                                                                                                NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS

                                                                                                                                                                Negative NEB values cost of barrier

                                                                                                                                                                Solar Water Heat NZ$ Euros

                                                                                                                                                                Solar Design NZ$ Euros

                                                                                                                                                                Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)

                                                                                                                                                                -23 -12 (079)

                                                                                                                                                                -11 -6 (06)

                                                                                                                                                                Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip

                                                                                                                                                                Source Skumatz Economic Research Associates research

                                                                                                                                                                SERA

                                                                                                                                                                CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                                NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                                                -5-4-3-2-1012345

                                                                                                                                                                Op Co

                                                                                                                                                                st

                                                                                                                                                                Init C

                                                                                                                                                                ost

                                                                                                                                                                Maint

                                                                                                                                                                Perfo

                                                                                                                                                                rm

                                                                                                                                                                Prod

                                                                                                                                                                uctiv

                                                                                                                                                                Tena

                                                                                                                                                                nt Sa

                                                                                                                                                                t

                                                                                                                                                                Comf

                                                                                                                                                                ort

                                                                                                                                                                Appe

                                                                                                                                                                ar

                                                                                                                                                                Light

                                                                                                                                                                Qual

                                                                                                                                                                SellL

                                                                                                                                                                ease

                                                                                                                                                                Devp Ownr AampE

                                                                                                                                                                Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                                                Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                                                Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                                                Owners more positive ndashUnderinvestment train

                                                                                                                                                                • Soup to nuts on non-energy benefits NEIs
                                                                                                                                                                • TOPICS
                                                                                                                                                                • BACKGROUND NEBS
                                                                                                                                                                • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                                                • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                                                • KEY APPLICATIONS OF NEBS
                                                                                                                                                                • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                                                • Neb measurement
                                                                                                                                                                • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                                                • MEASURING PARTICIPANTNEBs
                                                                                                                                                                • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                                                • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                                                • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                                                • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                                                • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                                                • PROGRESS AND GAPS
                                                                                                                                                                • PROGRESS AND GAPS
                                                                                                                                                                • USING NEB NEI VALUES
                                                                                                                                                                • Slide Number 19
                                                                                                                                                                • ARE NEBS HIGH VALUE
                                                                                                                                                                • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                                • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                                                • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                                                • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                                                • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                                                • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                                • AUDIT FINANCIAL PROGRAM
                                                                                                                                                                • CampI NEW CONSTRUCTION
                                                                                                                                                                • CampI TECHNICAL ASSISTANCE
                                                                                                                                                                • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                                                • Slide Number 31
                                                                                                                                                                • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                                                • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                                                • UTILITY NEBS
                                                                                                                                                                • ADJUSTED PAYBACKS
                                                                                                                                                                • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                                                • NEBS IN BC
                                                                                                                                                                • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                                                • APPLICATION TO BC
                                                                                                                                                                • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                                                • STATE TREATMENT OF NEBS NEBs
                                                                                                                                                                • Slide Number 42
                                                                                                                                                                • SELECTED STATE PROGRESS
                                                                                                                                                                • SELECTED STATE PROGRESS
                                                                                                                                                                • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                                                • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                                                • DISCOUNT RATE MATTERS
                                                                                                                                                                • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                                                • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                                                • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                                                • TAKEAWAYSON NEBSNEIS
                                                                                                                                                                • THANK YOUQuestions
                                                                                                                                                                • Slide Number 53
                                                                                                                                                                • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                                                • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                                  SERA

                                                                                                                                                                  CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                                  NEBs Influence on Incorporating HPEE Direction amp Importance by Actor

                                                                                                                                                                  -5-4-3-2-1012345

                                                                                                                                                                  Op Co

                                                                                                                                                                  st

                                                                                                                                                                  Init C

                                                                                                                                                                  ost

                                                                                                                                                                  Maint

                                                                                                                                                                  Perfo

                                                                                                                                                                  rm

                                                                                                                                                                  Prod

                                                                                                                                                                  uctiv

                                                                                                                                                                  Tena

                                                                                                                                                                  nt Sa

                                                                                                                                                                  t

                                                                                                                                                                  Comf

                                                                                                                                                                  ort

                                                                                                                                                                  Appe

                                                                                                                                                                  ar

                                                                                                                                                                  Light

                                                                                                                                                                  Qual

                                                                                                                                                                  SellL

                                                                                                                                                                  ease

                                                                                                                                                                  Devp Ownr AampE

                                                                                                                                                                  Note AgtE (~4 vs ~2) for these NEBs

                                                                                                                                                                  Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip

                                                                                                                                                                  Source Skumatz Economic Research Associates Inc 2002

                                                                                                                                                                  Owners more positive ndashUnderinvestment train

                                                                                                                                                                  • Soup to nuts on non-energy benefits NEIs
                                                                                                                                                                  • TOPICS
                                                                                                                                                                  • BACKGROUND NEBS
                                                                                                                                                                  • 20+ YEARS OF NEBS PROGRESShellip
                                                                                                                                                                  • NEB PERSPECTIVES CATEGORIES BEST PRACTICES
                                                                                                                                                                  • KEY APPLICATIONS OF NEBS
                                                                                                                                                                  • USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
                                                                                                                                                                  • Neb measurement
                                                                                                                                                                  • NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
                                                                                                                                                                  • MEASURING PARTICIPANTNEBs
                                                                                                                                                                  • HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
                                                                                                                                                                  • PARTICIPANT MEASUREMENT METHODS COMPARISON
                                                                                                                                                                  • PROS AND CONS OF NEBS MEASUREMENT METHODS
                                                                                                                                                                  • PROGRESS TRANSFERABILITY OF NEBS
                                                                                                                                                                  • RANGES TYPICAL NEB VALUES FOR WxRETROFI
                                                                                                                                                                  • PROGRESS AND GAPS
                                                                                                                                                                  • PROGRESS AND GAPS
                                                                                                                                                                  • USING NEB NEI VALUES
                                                                                                                                                                  • Slide Number 19
                                                                                                                                                                  • ARE NEBS HIGH VALUE
                                                                                                                                                                  • RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                                  • TOP NEBS FOR WEATHERIZATION PROGRAM
                                                                                                                                                                  • RESULTS FOR ENERGY STARreg APPLIANCES
                                                                                                                                                                  • INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
                                                                                                                                                                  • ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
                                                                                                                                                                  • CampI PROGRAMS ldquoNEBBEDrdquo
                                                                                                                                                                  • AUDIT FINANCIAL PROGRAM
                                                                                                                                                                  • CampI NEW CONSTRUCTION
                                                                                                                                                                  • CampI TECHNICAL ASSISTANCE
                                                                                                                                                                  • EXPRESSING NEBS VALUEndashCxe
                                                                                                                                                                  • Slide Number 31
                                                                                                                                                                  • JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
                                                                                                                                                                  • EMISSIONS (SOCIETAL NEBS)
                                                                                                                                                                  • UTILITY NEBS
                                                                                                                                                                  • ADJUSTED PAYBACKS
                                                                                                                                                                  • NEBS IN COST-EFFECTIVENESS APPLICATIONS
                                                                                                                                                                  • NEBS IN BC
                                                                                                                                                                  • CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
                                                                                                                                                                  • APPLICATION TO BC
                                                                                                                                                                  • METHODS TO INCLUDE NEBs IN REGULATORY TESTS
                                                                                                                                                                  • STATE TREATMENT OF NEBS NEBs
                                                                                                                                                                  • Slide Number 42
                                                                                                                                                                  • SELECTED STATE PROGRESS
                                                                                                                                                                  • SELECTED STATE PROGRESS
                                                                                                                                                                  • APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
                                                                                                                                                                  • NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
                                                                                                                                                                  • DISCOUNT RATE MATTERS
                                                                                                                                                                  • EUL VARIATIONS- ADOPTED VALUES
                                                                                                                                                                  • KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
                                                                                                                                                                  • IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
                                                                                                                                                                  • TAKEAWAYSON NEBSNEIS
                                                                                                                                                                  • THANK YOUQuestions
                                                                                                                                                                  • Slide Number 53
                                                                                                                                                                  • NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
                                                                                                                                                                  • CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo

                                                                                                                                                                    top related

                                                                                                                                                                    Soup to Nuts: Rapid Iteration for Innovative Customer...
                                                                                                                                                                    Technology
                                                                                                                                                                    SEQUENCE COMPLIANCE SOUP TO NUTS
                                                                                                                                                                    Documents
                                                                                                                                                                    SEO: From Soup to Nuts (part 2)
                                                                                                                                                                    Documents
                                                                                                                                                                    Blog Monetization: Soup to Nuts
                                                                                                                                                                    Documents
                                                                                                                                                                    Soup to Nuts Data Repository 102 - Acmeware to nuts - data.....
                                                                                                                                                                    Documents
                                                                                                                                                                    SEMINAR ON STATUS OF MEASURE LIFETIME WORK RTF EUL Kickoff.....
                                                                                                                                                                    Documents
                                                                                                                                                                    Windows Azure Role Cloud Computing Soup to Nuts
                                                                                                                                                                    Documents
                                                                                                                                                                    Assessment: Soup to Nuts - hets.org · Assessment: Soup to....
                                                                                                                                                                    Documents
                                                                                                                                                                    Guerrilla Podcasting Soup to Nuts Using WordPress and...
                                                                                                                                                                    Technology
                                                                                                                                                                    APIs: A Soup to Nuts Analysis
                                                                                                                                                                    Technology
                                                                                                                                                                    Blog Monetization: Soup to Nuts Stephan Spencer, Founder &.....
                                                                                                                                                                    Documents
                                                                                                                                                                    Row Management INDOT's Expectations: From Soup to Nuts
                                                                                                                                                                    Documents
                                                                                                                                                                    Soup to Nuts Guide
                                                                                                                                                                    Documents
                                                                                                                                                                    The Jungle - going global in 2016 from soup to nuts
                                                                                                                                                                    Law
                                                                                                                                                                    PHP from soup to nuts Course Deck
                                                                                                                                                                    Internet
                                                                                                                                                                    Get Started with SQL Azure Cloud Computing Soup to Nuts
                                                                                                                                                                    Documents