2003 Statewide Nonresidential Audit Program Evaluation

Electric / Gas / Water

Summary of Final Evaluation Report

Prepared by:

Rafael Friedmann, PG&EKris Bradley & Christie Torok, Quantum Consulting

2003 Statewide Nonresidential Audit Program Evaluation

Knowledge to Shape Your Future 2

• Tracking System Assessment– Verify tracking system accomplishments versus goals and

evaluate tracking system content• Small Company Impact and Satisfaction Assessment

– Measure first year program impact and customer satisfaction with program components

• Large Company Impact Assessment– Measure program-related energy efficiency actions taken

over the first two years following the Audit• Best Practices Assessment

– Provide an up-to-date, prospective recommendation for the best approach to on-site audits for different sized customers.

Evaluation Objectives


• Statewide the Audit program delivered audit services to over 29,000 utility customers, with over ½ of the completed audits provided to customers defined as hard-to-reach (HTR)

Q4 Report Goals Q4 Report GoalsPG&E 11,865 6,500 9,059 3,500 SCE 8,533 7,100 3,963 2,840 SDG&E 5,063 5,000 2,621 800 SCG 4,283 4,000 475 400 Total 29,744 22,600 16,118 7,540

Total Participation Hard-to-Reach Participation

Program Targets and Accomplishments


Data Collection Summary

• Smaller Customer Surveys (Less Than 100 kW)– 259 Participant / 261 Nonparticipant– Customers with Less Than 100 kW – PY 2003 Participants– Phone, Mail, On-Site, CD-ROM, On-Line

• Larger Customer Surveys (More Than 100 kW)– 84 Participant / 86 Nonparticipant – Customers with More Than 100 kW– PY 2002 Participants– On-Site Only

• Program Manager and Implementation Staff Interviews


• 25% of small and very small nonparticipating customers are aware of the audit program, versus about 40% of medium/large customers

25% 25%

37%

42%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Very Small(N=129)

Small(N=132)

Medium(N=37)

Large(N=49)

Rates of Audit Program Awareness in the General Population


• Awareness of the Audit Program in the general population is driven by the IOUs, who account for 70 percent of overall awareness

34%

9%

84%

50%

24%

18%12%

22%

7%

3%

3%

23%29%

10%

28%

38%

--2%--1%

--1%--2%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Very Small and SmallParticipants

(N=259)

Very Small and SmallNonparticipants

(N=59)

Medium and LargeParticipants

(N=84)

Medium and LargeNonparticipants

(N=33)

Other

RespondentContacted Utility

Bill Inserts

Utility Brochure inMail

Contacted by UtilityRepresentative

*Medium and large participants are PY2002 and onsite; very small and small participants are PY2003 and include all delivery mechanisms

Utility Marketing Sources of Program Awareness in the General Population


• Satisfaction levels with the Audit program are reasonably high, but participants desire a more useful audit product (report)

65%70% 74%

46%

27%19% 15%

32%

5% --2%

16%

--2%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Overall Audit Program Quality of Report Credibility ofRecommendations

Usefulness

Unsatisfied(1-3)SomewhatSatisfied (4-7)Very Satisfied(8-10)

N=259

Satisfaction with Audit Program Elements Among Small and Very Small Participants


(N=86)

(N=261)(N=84)

(N=255)

(N=84)(N=259)

1

2

3

4

5

6

7

Small and Very Small Medium and Large

Mea

n Kn

owle

dge

Ratin

g (1

-10

scal

e)

Nonparticipants

Participants - Before Audit

Participants - After Audit


Self-Reported Knowledge of Energy Efficiency

• Participants show a measurable increase in energy efficiency knowledge resulting from the audit


• Audit participants purchase rebated equipment at much higher rates than nonparticipants

0%

10%

20%

30%

40%

50%

60%

Small/Very Small Participant(N=259)

Small/Very SmallNonparticipant (N=261)

Medium/Large Participant(N=84)

Medium/LargeNonparticipant (N=86)

SPCExpress EfficiencyUnspecified Rebate Program

Rebate Program Participation Levels


• Participants want more customization in the audit reports (and recommendations), and want measure costs and rebates for use in ROI calculations

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Very Small and Small Participants(N=85)

Medium and Large Participants(N=15)

Less information

More communication &information

More personal contact

Provide results faster

Email info on rebates, EE

Give costs to replaceequipment

More customization


Participant Suggestions for Program Improvement


32%

24%

9%

20%

17%

12%

9%

12%

0%

5%

10%

15%

20%

25%

30%

35%

Lighting Cooling Gas Other

Adop

tion

Rate

Participant (N=259)Nonparticipant (N=261)

Small and Very Small Customer Equipment Adoption Rates

• Among smaller customers, participant adoption rates for lighting and cooling, as well as ‘other’ miscellaneous equipment, are considerably higher than for a nonparticipant control group


N=51

N=33

N=84

N=84N=84

N=39

N=47 N=86

N=86N=86

0%

10%

20%

30%

40%

50%

60%

Lighting Cooling Gas Other Industrial Process

Adop

tion

Rate

ParticipantNonparticipant

Medium and Large Customer Equipment Adoption Rates

• Among larger customers, when comparing participants and a nonparticipant control group, marked differences in adoption rates exist for industrial process, lighting, cooling, and gas equipment


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%Ve

ry S

mal

lan

d Sm

all

(N=6

6)

Med

ium

and

Larg

e(N

=31)

Very

Sm

all

and

Smal

l(N

=52)

Med

ium

and

Larg

e(N

=29)

Very

Sm

all

and

Smal

l(N

=17)

Med

ium

and

Larg

e(N

=6)

Very

Sm

all

and

Smal

l(N

=44)

Med

ium

and

Larg

e(N

=8)

Med

ium

and

Larg

e(N

=24)

Lighting Cooling Gas Appliances Other Equipment ProcessEquipment

Perc

ent o

f Equ

ipm

ent A

dopt

ions

Don't knowNoYes


Percent of Equipment Adoptions Recommended in the Audit Report

• Recommendations in the audit reports clearly leads to equipment adoptions, especially for lighting measures


N=52

N=32

N=69N=190

N=49

N=37N=132N=129

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Very Small Small Medium Large

Perc

ent I

mpl

emen

ting

Cons

erva

tion

Mea

sure

s

ParticipantNonparticipant

Implementation of Conservation Measures by Customer Size

• Counter to PY2002 evaluation results, Audit influence on conservation measure adoptions is substantial


N=139

N=51 N=38

N=35

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Very Small Small Medium Large

Perc

ent o

f Con

serv

atio

n M

easu

res

Beg

un a

s a

Res

ult o

f the

Aud

it


Percent of Conservation Measures Begun as a Result of the Audit

• Self-reported conservation practices begun as a result of the audit ranges from 46 percent among the very small to about 25 percent for the largest customers


Customer Size

On-Site Audit ApproachVery Small(<20 kW)

Small(20 - <100 kW)

Medium(100 - <500 kW)

Large(>500 kW)

Set goals based on number of completed audits • •Implement audit outreach to Express Efficiency participants • •Set goals based on energy saving targets • •

Tailor On-Site Audit Delivery by Customer Size

• Program outreach – program goals should drive the focus of the outreach effort to obtain the desired mix of small, medium and large customers


Customer Size


Small(20 - <100 kW)

Medium(100 - <500 kW)

Large(>500 kW)

Deliver the audit report during the site visit • •Provide an Express Efficiency application; filled-in where possible • •Furnish a list of qualified service providers • • •

Tailor On-Site Audit Delivery by Customer Size (Cont’d)

• Site Visit – for smaller customers all audit services should seek to maximize energy efficient equipment adoptions and Express Efficiency participation


Customer Size


Small(20 - <100 kW)

Medium(100 - <500 kW)

Large(>500 kW)

Smaller customers need educational information on relevant measures • •Smaller customers need measures ranked by the cost to the bottom line • •Larger customers need customized information relevant to their building • •Larger customers need measures ranked by ROI to justify investment • •


• Audit Report and Recommendations – tailor audit products to customer wants and needs by customer size


Customer Size


Small(20 - <100 kW)

Medium(100 - <500 kW)

Large(>500 kW)

Track account rep. performance using completed Express applications • •In-depth follow-up efforts with larger customers to maximize savings • •


• Audit Follow-up – motivate account representatives to follow-up using job performance incentives for achieving program savings and equitable spending targets


Program Implementation for Smaller Customers

• Overall, a direct install program may be a preferable delivery channel for bringing energy efficiency to the under 20 kW market

• Alternatively, providing a turnkey program that combines Express Efficiency and Audit services may improve program results for very small customers

• Outreach efforts to small customers should emphasize remote Audits

• Emphasize low-cost and no-cost recommendations in Audits directed at smaller customers


Program Implementation for Larger Customers

• Continue to direct on-site audits to larger customers • Provide in-depth follow-up services to maximize

recommended measure uptake and support for the Express Efficiency and SPC rebate programs

• Provide well-documented, detailed return-on-investment (ROI) calculations in the Audit report

• Include significant analytic customization and site-specific research in Audit reports for large customers


Other Program Implementation Recommendations

• Provide Audit program outreach services using previous years’ Express Efficiency participants

• With the exception of SoCalGas, all other IOUs should increase Audit emphasis on the gas appliance end-use category

• Use outreach or follow-up strategies to improve customer utilization of the CD-ROM Audit tool

• Assess program marketing effectiveness and emphasize best practices

• Consider tracking goals based on downstream energy efficiency uptake

2003 Statewide Nonresidential Audit Program Evaluation

Documents