NSW Energy Efficiency Programs 2012 Evaluation Report Final Report to Office of Environment and Heritage, NSW Department of Premier and Cabinet August 2012 Includes OEH responses at June 2013
NSWEnergyEfficiencyPrograms
2012EvaluationReport
FinalReportto
OfficeofEnvironmentandHeritage,
NSWDepartmentofPremierandCabinet
August2012IncludesOEHresponsesatJune2013
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
ARTDPtyLtdABN75003701764Tel0293739900Fax0293739998
Level4,352KentStSydneyPOBox1167QueenVictoriaBuildingNSW1230Australia
ii
©2014StateofNSWandOfficeofEnvironmentandHeritage
TheStateofNSWandOfficeofEnvironmentandHeritagearepleasedtoallowthismaterialtobereproducedinwholeorinpartforeducationalandnon‐commercialuse,providedthemeaningisunchangedanditssource,publisherandauthorshipareacknowledged.
PreparedbyARTDConsultantsfortheOfficeofEnvironmentandHeritage.
DisclaimerThisreporthasbeencompiledingoodfaith,exercisingallduecareandattention.Norepresentationismadeabouttheaccuracy,completenessorsuitabilityoftheinformationinthispublicationforanyparticularpurpose.OEHshallnotbeliableforanydamagewhichmayoccurtoanypersonororganisationtakingactionornotonthebasisofthispublication.Readersshouldseekappropriateadvicewhenapplyingtheinformationtotheirspecificneeds.
Publishedby:OfficeofEnvironmentandHeritage59GoulburnStreet,SydneyNSW2000POBoxA290,SydneySouthNSW1232Phone:(02)99955000(switchboard)Phone:131555(environmentinformationandpublicationsrequests)Phone:1300361967(nationalparks,generalenvironmentalenquiries,andpublicationsrequests)Fax:(02)99955999TTYusers:phone133677,thenaskfor131555Speakandlistenusers:phone1300555727,thenaskfor131555Email:[email protected]:www.environment.nsw.gov.auReportpollutionandenvironmentalincidentsEnvironmentLine:131555(NSWonly)orinfo@environment.nsw.gov.auSeealsowww.environment.nsw.gov.auISBN9781743593868OEH2014/0052February2014
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
ARTDPtyLtdABN75003701764Tel0293739900Fax0293739998
Level4,352KentStSydneyPOBox1167QueenVictoriaBuildingNSW1230Australia
iii
Acknowledgments
ThisworkwascompletedwiththeassistanceoftheDataandEvaluationTeamintheOfficeofEnvironmentandHeritage,DepartmentofPremierandCabinet.
WewouldalsoliketothanktheOEHmanagersandstafffortheirtimeandinsightsandtrustthattheirviewsareadequatelyrepresentedinthisreport.
ARTDConsultancyteam
ChrisMilneFlorentGomez‐Bonnet
OEHCommentBoxes
TheOfficeofEnvironmentandHeritage(OEH)hasprovidedcommentboxesattheendofeachsectioninthisreport.
Thesecommentboxes: providedetailsonthestatusofOEHenergyefficiencyprogramssincethecompletion
ofthereportuptoJune2012;and commentontheevaluationfindings.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
iv
Contents
Tablesandfigures...........................................................................................................................v
Acronymsandglossary...............................................................................................................vii
Executivesummary.....................................................................................................................viii Scopeoftheprograms......................................................................................................................viii The2012evaluation.........................................................................................................................viii Overallfinding........................................................................................................................................ix Energysavingsarebeingachieved.................................................................................................x Implementationhasbeeneffective............................................................................................xiii
1. Theprogramsandtheevaluation................................................................................16 1.1 Theprogramsinscope..........................................................................................................16 1.2 Directenergysavingprogramsarethemajorcomponent....................................17 1.3 ThepolicycontextforenergyefficiencyinNSW.......................................................18 1.4 Programlogic............................................................................................................................19 1.5 Evaluationoftheenergyefficiencyprograms.............................................................21
2. Overalloutcomesandachievements..........................................................................23 2.1 Programoutcomes..................................................................................................................23 2.2 Achievementsandissueswithimplementation.........................................................30
3. Progressofindividualprograms..................................................................................35 3.1 HomePowerSavingsProgram(HPSP)..........................................................................35 3.2 EnergyEfficiencyforSmallBusinessProgram(EESBP)........................................39 3.3 EnergySaverProgram(ESP)..............................................................................................44 3.4 EnergyefficiencycomponentofNSWGovernmentSustainabilityPolicy
(EEGSP)........................................................................................................................................48 3.5 EnergyEfficiencyTrainingProgram(EETP)...............................................................56 3.6 EnergyEfficiencyCommunityAwarenessProgram(EECAP)..............................61 3.7 DataandEvaluationProgram(DEP)...............................................................................66
4. Cross‐programcoordinationandstrategy‐widegovernance............................73 4.1 Theinitialdesignhadlimitedscopeforintegration................................................73 4.2 Organisationalsiloswereabarriertointegration....................................................74 4.3 Coordinationwouldhavebeenimprovedbymorestructured
opportunities.............................................................................................................................75 4.4 Managersseekmoreoverarchinggovernance...........................................................76
Appendix1. Evaluationreportsusedforthesynthesis............................................77
Appendix2. Estimatesofactualenergysavings‐preliminaryresults................79
Appendix3. Keyreferences................................................................................................86
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
v
Tablesandfigures
Table1‐1. Theprogramsbyapproach,sector,completionandallocatedOEHexpenditure.......16
Table1‐2. Programsgroupedbyapproachesandgovernance................................................................17
Table2‐1. Energyefficiencyprograms–estimatedannualsavingsfromactivitiesimplementedtoJune2012.................................................................................................................26
Table2‐2. Cost‐effectivenessforthedirectenergysavingsprograms..................................................29
Table3‐1. HomePowerSavingsProgram–estimatedannualsavingsfromactivitiesimplementedtoJune2012.................................................................................................................37
Table3‐2. EnergyEfficiencyforSmallBusinessProgram–estimatedsavingsforoneyearfromactivitiesimplementedtoJune2012..................................................................................42
Table3‐3. ESPuptakeandimplementationovertimetoendofJune2012........................................45
Table3‐4. EnergySaverProgram–estimatedsavingsforoneyearfromactivitiesimplementedtoJune2012.................................................................................................................46
Table3‐5. NABERSratingsobtainedfrom2007/08to2010/11............................................................51
Table3‐6. GovernmentBuildingRetrofitProgram–smallsitescomponent‐estimatedsavingsforoneyearfromactivitiesimplementedtoJune2012........................................55
Table3‐7. Reporteddemandanduptakecomparedtoexpectations....................................................58
Table3‐8. SummaryofmethodsusedtoestimateenergysavingsforeachprogramtoendofJune2012..............................................................................................................................................68
Table4‐1.Evaluationreportsandmaindatacollectionmethodsforeachreport.............................77
Table4‐2. HPSPbillingdataanalysispreliminaryresults..........................................................................79
Table4‐3. ESSMeasurement&Verificationpreliminaryresults............................................................80
Table4‐4. ESPMeasurement&Verificationpreliminaryresults............................................................81
Table4‐5. EEGSPMeasurement&VerificationpreliminaryresultsfortheWestmeadHospitalcasestudy................................................................................................................................82
Table4‐6. EEGSPMeasurement&VerificationpreliminaryresultsfortheCentralPassageoftheSydneyOperaHousecasestudy..........................................................................................83
Table4‐7. EESBPMeasurement&Verificationpreliminaryresultsbybusinessactivity.............84
Table4‐8. EESBPMeasurement&Verificationpreliminaryresultsbytechnology........................85
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
vi
Figure1‐1. ProgramlogicoftheinitialEnergyEfficiencyStrategy..........................................................20
Figure2‐1. Improvedreliabilityofenergysavingsestimates.....................................................................24
Figure2‐2. AllocationofoverallelectricitysavingsachievedtoendofJune2012acrossprograms....................................................................................................................................................25
Figure3‐1. TreasuryLoanFunduptake...............................................................................................................53
Figure4‐1. OrganisationalchartofOEHunitsinvolvedinenergyefficiency.......................................74
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
vii
Acronymsandglossary
ACP AccreditedCertificateProviderundertheESSthatcreatetradeableenergysavingscertificates(ESCs)
CALD CulturallyandLinguisticallyDiverse
DEAS DECCWEnergyAssessmentSystem
DEC NSWDepartmentofEducationandCommunities
DECCW NSWDepartmentofEnvironment,ClimateChangeandWater(until2011,thenOEH)
DTRIS DepartmentofTradeandInvestment,RegionalInfrastructureandServices
EEAP EnergyEfficiencyActionPlan,theNSWGovernmentapproachtoenergyefficiencyproposedfrom2012
EECAP EnergyEfficiencyCommunityAwarenessProgram,NSWgovernmentprogram
EES EnergyEfficiencyStrategy,theNSWGovernment’sapproachtoenergyefficiency2008‐2012
EESBP EnergyEfficiencyforSmallBusinessProgram,NSWgovernmentprogramtargetingsmallbusinesses
ESC EnergySavingsCertificate,atradablecertificatecreatedthroughtheESSfromthemeasurementofenergysavings
ESP EnergySavingsProgram,NSWgovernmentprogramtargetingmediumtolargebusinesses
EETP EnergyEfficiencyTrainingProgram,NSWgovernmentprogram
EEGSP EnergyefficiencycomponentofNSWGovernmentSustainabilityPolicy
ESS EnergySavingsScheme,NSWgovernmentwhitecertificatescheme
FY FinancialYear
HPSP HomePowerSavingsProgram,NSWgovernmentprogramtargetinglowincomehouseholds
HVAC Heating,VentilationandAir‐conditioning
IAB InternalAuditBureau,NSWgovernmenttradingenterprisethatprovidesauditservices
IPART IndependentPricingandRegulatoryTribunal
IRA InstituteforRuralAffairs
ISF InstituteforSustainableFutures
KPI KeyPerformanceIndicator
M&V MeasuringandVerificationtheprocessofusingmeasurementtoreliablydetermineactualenergysavingsbycomparingmeasuredusebeforeandafterimplementationofaproject
NGO Non‐governmentorganisation
NABERS NationalAustralianBuiltEnvironmentRatingSystem
OEH OfficeofEnvironmentandHeritage, NSWDepartmentofPremierandCabinet
PD4VET ProfessionalDevelopmentforVocationalEducationandTrainingpractitioners
RTO RegisteredTrainingOrganisation
VET VocationalEducationandTraining
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
viii
Executivesummary
Thisreportisthe2012evaluationofsevenprogramsmanagedbytheOfficeofEnvironmentandHeritage(OEH)thatwerepartoftheformerNSWEnergyEfficiencyStrategy(ESS).ItassessestheprogramsataninterimpointtoJune2012andisbasedonevaluationfindingsavailablebyJune2012.
Scopeoftheprograms
TheNSWGovernmenthasanongoingcommitmenttoimprovingenergyefficiencyacrossthehousehold,businessandgovernmentsectors.Thesesevenprogramscommencedin2009andrepresentaninvestmentinenergyefficiencybyOEHof$161.4million.ThereareotherNSWGovernmentandOEHenergyefficiencyprogramswhicharenotpartofthisevaluation.
Threeoftheseprogramsaredirectenergysavingprograms(HPSP,ESP,andEESBP)targetingspecificsectors(lowincomehouseholds;smallbusinesses;mediumandlargebusinesses)andaccountfor61percentoftheOEHexpenditure(HPSPaccountsfor39percent).Twoarebroadercapacitybuildingprogramsthataddresscommunityawareness(EECAP)andtheworkforceforenergyefficiency(EETP).EEGSPisapolicymixtargetingthegovernmentsectorthroughfiveprograms.
Theuncertaintyaroundenergysavingsfromenergyefficiencyprograms,highlightedbythe2007OwenInquiryintoElectricitySupplyinNSW,wasanimportantdriverforevaluation,andledtheestablishmentoftheDataandEvaluationProgram(DEP).
ThedesignoftheEESwasinfluencedbytheapproachinCalifornia,consideredtheworldleaderinenergyefficiencypolicy,whichhasamixofprogramstargetingmultiplemarketsectors;programsforeducation,trainingandstate‐widemarketing;andasubstantialevaluationprogram.
The2012evaluation
ThisevaluationassessestheprogramsataninterimpointtoJune2012.Itexamines
howeffectivelyprogramsarebeingdeliveredandreachingtheirtargetgroups theoutcomesproducedbytheprogramstodateincludingreducedenergyuseand
increasedcapacityformanagingenergyuseacrossthesectors
Theprograms HomePowersavingsProgram
(HPSP) EnergyEfficiencyforSmall
BusinessProgram(EESBP) EnergySaverProgram(ESP) energyefficiencycomponentof
NSWGovernmentSustainabilityPolicy(EEGSP)
EnergyEfficiencyCommunityAwarenessProgram(EECAP)
EnergyEfficiencyTrainingProgram(EETP)
DataandEvaluationProgram(DEP)
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
ix
improvementsinthereliabilityofenergysavingsestimates
Theevaluationisbasedonasynthesisofevaluationsandreportsontheprograms.Quantitativedataonreach,uptakeandenergysavingswascollectedbytheprogramsandanalysedbytheOEHStrategyandAnalysisteam.Eachprogramundertookarangeofevaluationandreporting.MethodstoverifyenergysavingsweredevelopedandimplementedthroughtheDataandEvaluationProgram.
Weareconfidentthatthefindingsofthisevaluationrepresenttheoverallpatternsofdeliveryandoutcomestodate.Thefindingsfromdifferentsourceswereconsistent,andmorereliabledataonenergysavingswasavailable(seebelow).Somecautionisneededassomeoftheevaluationsandthemeasurementprojectsareworksinprogressatthisstageandsomeuncertaintiesremain.
Overallfinding
Theseenergyefficiencyprogramsarelargelybeingdeliveredeffectively.Thishasrequiredmanyadaptationswithinprogramswhichhaveledtoincreasinglystrategicandeffectiveapproaches.Alltheevidenceindicatesthattheprogramsaregenerallywellreceivedbyparticipants.
OEHnowhasmorereliablemeasuresofenergysavings,withtheapplicationofnewmethodssubstantiallyimprovingestimatesovertheperiod.
Energysavingsarebeingachievedandtheseareexpectedtoincrease.Alldirectenergysavingsprogramshavedeliveredsubstantialenergysavings,thoughthereisstillvariationinthereliabilityofestimates.Theseenergysavingsarecost‐effective,deliveredatalowercostthanthecostofprovidingthesameamountofelectricity.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
x
OEHComment
TheCouncilofAustralianGovernments(COAG)calledonjurisdictionstoreviewthecomplementarityofclimatemitigationprograms,includingNSWenergyefficiencyprograms,withtheCommonwealth’sCleanEnergyFuturepackage.
InDecember2012,thePremierannounceddecisionstocease,reformandretainNSWenergyefficiencyprogramsasaresultofthiscomplementarityreview.
TheNSWProfessionalServicesActionPlananddraftNSWRenewableEnergyActionPlancommittedtheMinisterfortheEnvironmenttobringforwardanEnergyEfficiencyActionPlanwithanimprovedsuiteofenergyefficiencyprograms.
BoththefindingsofthisindependentevaluationandgovernmentdecisionsonthecomplementarityreviewinformedtheNSWEnergyEfficiencyActionPlan.
Energysavingsarebeingachieved
Expectationsattheinterimstage
Mostoftheprogramsstartedintheircurrentformin2009andarefundeduntil2013or2014.Theyarebeingdeliveredinacontextofchangeanduncertaintyaroundelectricitycosts,energyefficiencyandclimatechangewithingovernmentsandthewidercommunity.MostoftheprogramsarenewforNSW—HPSPandEESBParethefirsttimelowincomehouseholdsandsmallbusinesseshavebeendirectlytargetedinthisway.Inthesecircumstancesevidenceofinitialoutcomesandeffectiveimplementationiscritical.
OEHhasmorereliablemeasuresofenergysavings
Theestimatesofenergysavingshavesubstantiallyimprovedovertheperiod,progressingfromprojecteddeemedsavingstoverifiedestimatesbasedonmeasuresofactualenergyuse.Asignificantachievementhasbeenthedevelopmentofmethodstoestimateactualenergysavingsinlinewithinternationalbestpractice.OEHreachedagreementswithenergydistributorstoaccesslarge‐scalebillingdataforhouseholds(HPSP)andsmallbusinesses(EESBP)—thefirsttimesuchcomprehensivedatahasbeenavailabletoagovernmentagencyinAustralia—whichformedthebasisforindependentstudiesusingrigorousstatisticalanalysis.Formediumtolargesites(ESP,EEGSP)OEHdevelopedaguideformeasurementandverification(M&V)basedontheinternationalprotocolandhascommissionedarangeofindependentstudies.
AtthisstagetheevidenceforHPSPandEESBPisbasedonactualmeasuresofenergyuse,andforESPonengineeringestimates,withM&Vprojectstobeunderwaylaterthisyear.Whilethescopeformeasurementhasvariedbetweentheprograms(dependingonthenumbersandcharacteristicsofparticipantsaswellasontechnologiesinstalled),thisworkhasandwillcontinuetoprovideincreasinglyreliablemeasures.Withmorerobust
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
xi
evidenceandreduceduncertaintyaroundenergysavings,OEHhasanincreasinglysoundbasisforassessingeffectivenessandcost‐effectiveness,forrefiningdeliveryandforplanningfutureenergyefficiencyprograms.
Energysavingsarebeingachievedandexpectedtoincrease
Alldirectenergysavingsprogramshavedeliveredsubstantialenergysavings,thoughasdescribedabovethereliabilityofestimatesvaries.Totalannualenergysavingsare118,834MWhofelectricityand187,716Gjofgas,leadingtoanestimatedreductioninparticipants’energybillsof$28,496,340toendofJune2012(thisincludestheGovernmentBuildingRetrofitProgramforsmallsites).Theestimatedenergysavingswouldresultin136,697tonnesofavoidedCO2emissions.Eachoftheprogramshasbeenasignificantsourceofenergysavings,withESPcontributing38%ofelectricitysavings,EESBP32%andHPSP27%.
FortheESPthesavingsatthisinterimstageareexpectedtoaccelerateoverthenextfewyears.Itisclearthattheprogramhasamodelthatcanachievesubstantialenergysavingsandcutenergybillsatmediumtolargesites.Theextentofsavingsislikelytobeclearerasmorebusinessesimplementauditrecommendationsovertime,andtheprogrammeasuresandreportsonthis,withincreasinglyreliabledatafromM&V.
ForEESBPthehighlevelofreachprovidesastrongfoundationforsavings(17,185smallbusinessesregisteredtoendofJune2012comparedtotheinitialtargetof6,000).Thekeychallengehasbeenliftingtherateofconversionfromregistrationtotheimplementationofrecommendations,andmajorrefinementstotheprogramdesignarebeingsuccessfulinaddressingthis,withevidenceofanincreasingconversionrateandmostactionsareattributabletotheprogram.
HPSPhasbeenauniqueprogramforOEHinseveralways.Itisaverylarge‐scaleintermsofbudgetandnumberofparticipants,andbeganwithrelativelyuntestedassumptions.Itiscomplicatedbyhavingsocialaswellasenergyefficiencyobjectives,whichfocusonlowincomehouseholds,specifictargetgroups(CALDandAboriginalhouseholds),andgeographicequityofaccess.InitialresultsforHPSPhavebeenpromising.Withextensivereachacrossitstargetgroupoflowincomehouseholds(115,508completedassessmentstoJune2012),ithasafirmbasistodeliversavings.EnergysavingswereestimatedwithahighlevelofreliabilitybythebillingdataanalysisconductedwithDEPinMay2012.Itfoundaveragesavingsof0.22megawatt‐hoursofelectricityperannum(or4percentreduction)foreachparticipatinghousehold,butdifferedwiththekititemsreceived(householdsreceivingtheshowerheadachieveonaverage6percentsavings).Theenergysavingsareencouraginggiventhelimitedpossibleimpactofthesmallkititems.Further,theelectricitysavingfromthelargestiteminstalledbyHPSP(efficientshowerheads)matchedthemeasuredsavingsfromSydneyWater’sefficientshowerheadretrofitprojects.Inadditiontheprogramhasachievedthesesavingswhileeffectivelyaddressingitssocialobjectiveoftargetinglow‐incomehouseholds.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
xii
WhiletheenergysavingspresentedinthisreportareprimarilylimitedtothedirectenergysavingprogramsandmeasuredbyOEH,therearelikelytobemorediffuseimpacts.Forexamplethereisevidencethatthedirectprogramsleadtowiderenergysavingswithintheirsectorsthroughmultipliereffectsasindividualsandbusinessespromoteenergyefficiencymorebroadly.Thecapacitybuildingprograms(EETPandEECAP)arealsoexpectedtohavepositivebutmorediffuseimpactsonenergysavingsnowandintothefuture.
TheESPhashadanimportantpositiveimpactforNSWbypositioningparticipatingorganisationstoattractfundingfortheirenergyefficiencyactivities.ForexampleinJune2012fourESPparticipantswongrantsinthefirstroundoftheCommonwealthCleanTechnologyInvestmentProgram(CTIP),receiving86%ofthenationalallocation,andafurther20ESPsitesareapplying.AkeysuccessfactorwasthecomprehensiveESPaudit.
OEHComment
ThecomplementarityreviewinformedgovernmentdecisionstomergetheEESBPintoESPtoimproveadministrativeefficiencyandtoretainHPSPasitmetallofthecomplementarityprinciples.
NSW2021includesatargettosupport220,000lowincomehouseholdstoreduceenergyuseby“upto20%”by2014.TheEnergyEfficiencyActionPlanseekstoincreasetheenergysavingsofparticipatinghouseholds.
Energysavingsarebeingproducedatalowercostthanthecostofprovidingthesameamountofelectricity
Onthecurrentevidenceallthedirectenergysavingsprogramareprovingtobecost‐effectiveovertheirlifetimewhenthecostpermegawatt‐houriscomparedwiththebenchmarkretailcostofprovidingelectricitytoeachcustomergroup.
OEHcalculatedcost‐effectivenessusingthemethodinthe2009IPARTreviewofNSWClimateChangeMeasures.BasedonthesavingsestimatesdescribedaboveforeachprogramtoendofJune2012,cost‐effectivenessisexpressedasthelevelisedcostpermegawatt‐hourthatreflectsthepresentvalueofcostsoverthelifetimeofprojects,takingintoaccountthevaryingpersistencefactorsoftheretrofits,equipmentinstalledandbehaviourchanges.Onthisbasisthelevelisedcostofonemegawatt‐hoursavedis$138forHPSP,$66forEESBP,$34forESPand$179forGBRP.
Comparisonofcost‐effectivenessbetweentheseprogramsgiveslimitedinsightbecausetheyhavedifferentobjectivesandinparticulartargetdifferentsectorsandcustomergroups.Itismorerelevanttocomparethecostofamegawatt‐hoursavedforeachprogramtoitsrelevantbenchmarkcost,inthiscasetheretailcostinresourcesofprovidingelectricityforthesametypeofcustomers.Thisbenchmarkcostvariesfrom$184forGovernmentcontractsto$281forhouseholds.Onthisbasisallthedirectenergysavingsprogramarecost‐effectiveovertheirlifetime.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
xiii
Communityawarenessisincreasing
TheSavePowermassmediacampaigntelevisionadvertisinginitsfinalphasereachedalmosttwo‐thirdsofNSWadults(63percent),withmost(77percent)findingitconvincingandathirdfeelingmotivatedtotheactionsinthemessages.Approvalofthecampaignhasbeenpositiveandresearchsuggeststhatitiscontributingtoimprovedknowledgeofenergyefficiencyandhaspositivelyinfluencedenergyusebehaviours.Thewidereachofthecampaignoffersgreatpotentialforenergysavingsfrombehaviourchangeatthisscale.
Implementationhasbeeneffective
Adaptiveandincreasinglyeffectiveimplementation
Akeyachievementtothispointisthattheprogramsarelargelybeingdeliveredasintended,arereasonablyinlinewithdeliverytargets,andhavedevelopedincreasinglystrategicandeffectiveapproaches.Forthedirectenergysavingsprogramsinparticularthishasrequiredmanyadaptationstotheirinitialsettingsanddeliveryprocesses,summarisedinthisreport.
Highparticipantsatisfactionandpositivestakeholderengagement
Alltheevidenceindicatesthattheprogramsweregenerallywellreceivedbyparticipants.Highlevelsofparticipantsatisfactionhavebeenrecordedforthedirectenergysavingprograms.Theprogramshavealsobeeneffectiveinengagingexternalstakeholdersincludingparticipantsas‘multipliers’topromotetheprogram.Assessorshavebeenakeyassetnotjustindeliveringassessmentsbutincontributingtopromotionandfurtherrecruitmentofparticipants.
Regionalfocusachieved
TheprogramshavehadasignificantregionalfocusinlinewiththeGovernment’scommitmenttosupportregionalcommunities.HPSPisaddressinganexplicittargetforgeographicequityofaccess,andtheothertwodirectenergysavingsprogramshaveover50%ofparticipatingbusinessesfromoutsideSydney.
Workingwiththegovernmentsectorhadmixedresults
EffectivenessofimplementationofthefiveprogramsundertheNSWGovernmentSustainabilityPolicyvariedsignificantly:somehadsomegoodresultsliketheGovernmentRetrofitBuildingProgram,whilesomeothershadlittleimpact.Commonbarrierstoengageeffectivelywithagenciesincludedtheirlackoftechnicalknowledge,andtimeandresourcestoapplyforthesupportoffered.
Communityawarenessandworkforcecapacityforenergyefficiencyisbeingdeveloped
Twoprogramsaddressedtheenablersandbarriersforgreaterenergyefficiency,andhavebothbeenlargelyimplementedasintended.TheEETPtargetskeyindustrysectors
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
xiv
andoccupationswhichareabletoimplementenergyefficiencyopportunities,theeducationandtrainingworkforce,andpolicymakersandprogrammanagers.IthasdevelopedclearformalpartnershiparrangementswiththeDepartmentofEducationandCommunities(DEC)thatnowprovidesasoundplatformforsharedprogramdelivery.Amajorachievementhasbeenengaging5,290participantsinenergyefficiencytraining,networkingorprofessionaldevelopmentbetweenJuly2009andApril2012.
TheEECAPsuccessfullyimplementedacomprehensive,multi‐strategyandongoingsocialmarketingcampaignoverthreeyears.TheSavePowermassmediacampaignwasdeliveredinfivephasesofadvertisingactivityovertwowintersandtwosummers,supportedbytwophasesofqualitativeresearchandsevenroundsoftrackingresearch.
TheEECAPeffectivelydeliveredarangeofothereducationalprogramstoreachdifferentsegmentsoftheNSWcommunity.Bi‐lingualeducatorsweretrainedanddelivered57energyworkshopsdeliveredtoCALDparticipants.TheCSIRO’sEnergymarkprojectwastrialledwith112groupstoApril2012.SavePowerlibrarykitswereborrowedby6,400householdsintwelvemonthswithevidenceofpositiveimpactontheirknowledgeandbehaviour.Majorelectricalapplianceretailersweresuccessfullyengagedwithtrainingdeliveredtostaffin142storestoassistcustomerstoconsiderenergyefficiencyintheirpurchasedecisions.Theexperienceoftheseprograms,thetrackingresearchandtheevaluationsareprovidingOEHwithasubstantialevidencebasetoinformfuturesocialmarketingforenergyefficiency.
OEHComment
ThecomplementarityreviewinformedgovernmentdecisionstoceasethevocationaltrainingelementsofEETPastheyduplicatefundingfromtheCommonwealth.Thenon‐duplicativeaspectsofEETParetobemergedintotheESP.
TheEECAPendedin2012asplanned.
Cross‐impactsofprogramshavebeenlimited
Therehavebeenonlylimitedcross‐programimpactsatthisstage.ThedeliveryofthedirectenergysavingsprogramsandtheEEGSPwererelativelyseparateandfocussedontheirspecifictargetgroups.LinkagesbetweentheseprogramsandtheNSWEnergySavingsScheme(ESS)havebeenlimited.
Thetwocapacitybuildingprogramshavescopetosupporttheotherprogramstosomedegreethroughincreasingawarenessacrossthestate,andbuildingtheenergyefficiencyworkforce.TheSavePowercampaignandtheEESwebsitedeliveredprovidedcommonbrandingfortheprogramsandinfluencedattitudesinthecommunity.EETPisplanningformoreemphasisoncustomisingexistingcoursesandresourcesandcomplementingimplementationofESPprojects.Inpracticecross‐pollenisationhasbeenlimitedbythedifferenttimeframes.Long‐termimpactsofEETPontheenergyefficiencyworkforcewillbemorediscernibleinthefuture.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
xv
MostOEHmanagersinvolvedwiththeseenergyefficiencyprogramswantedamoresystematicandstructuredprocessforoverallgovernanceandinformationsharing,whilerecognisingtheneedtoavoidprocessburdensandtomaintainprogramflexibility.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
16
1. Theprogramsandtheevaluation
1.1 Theprogramsinscope
TheNSWGovernmenthascommittedtoimprovingenergyefficiencyacrossthehousehold,businessandgovernmentsectors.InJune2008itlaunchedasuiteofeightprogramsknownastheNSWEnergyEfficiencyStrategy.NotallNSWGovernmentorOEHenergyefficiencyprogramswereincludedintheEES.ThisevaluationcoverstheprogramsundertheEEStoJune2012butdoesnotincludetheNSWEnergySavingsScheme(ESS)whichisnotmanagedbyOEHandhasnotbeenfullyevaluatedtodate.
TheprogramsrepresentaninvestmentinenergyefficiencybyOEHof$161.4million,withtheHomePowerSavingProgram(HPSP)accountingfor39percent.Mostoftheprogramsstartedintheircurrentformin2009andarecurrentlyfundeduntil2013or2014(Table1‐1).
Table1‐1. Theprogramsbyapproach,sector,completionandallocatedOEHexpenditure
Program Sector/initialtarget Fundeduntil $million %
HomePowerSavingProgram(HPSP)
Households‐ 220,000lowincomehouseholds
June2014 $63 39%
EnergyEfficiencyforSmallBusinessProgram(EESBP)
Smallbusinesses‐ 6,000 December2012 $15 9%
EnergySaverProgram(ESP) Mediumandlargesites‐ 800 June2014 $20 12%
EnergyefficiencycomponentofNSWGovernmentSustainabilityPolicy(EEGSP)
NSWGovernmentagencies December2012 $26.4 16%
EnergyEfficiencyCommunityAwarenessProgram(EECAP)
NSWcommunity‐ domesticandworkplace
June2012 $15 9%
EnergyEfficiencyTrainingProgram(EETP)
Educationandtrainingorganisations
June2013 $20 12%
DataandEvaluationProgram Energyefficiencyprograms June2013 $2 1%
Total $161.4 100%
Therationaleforthestrategyincludedafocusonintensiveandhighlyvisibleprogramsthatwouldkick‐startenergyefficiencyactionsinNSW,andwouldcomplementtheanticipatednationalemissionstradingscheme.AninitialinfluencewastheapproachinCalifornia,consideredtheworld’sleaderinenergyefficiencypoliciesandprograms,althoughatamuchlargerscaleandwithaverydifferenthistory,regulatorysettingandmarketcontextfromNSW.LikeNSW,Californiahasamixofprogramstargeting
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
17
multiplemarketsectors(residential,commercial,industrialandagricultural),monetaryincentivesandotherprogramssuchaseducationandtraining,state‐widemarketingandasubstantialevaluationprogram.1
1.2 Directenergysavingprogramsarethemajorcomponent
Forthisevaluationitisusefultogrouptheprogramsbasedontheirmainstrategiesforchangeandtheirgovernancearrangements(Table1‐2):
Threedirectenergysavingprogramstargetdifferentsectors,allmanagedbyOEH,aimtodeliveryenergyandcostsavingsforprogramparticipants.Togethertheyarethemostsignificantcomponentoftheprogramsandaccountfor61percentofOEHexpenditure.
Twoprogramsaddresstheunderlyingcapacityforchangetowardsgreaterenergyefficiency.OneismanagedbyOEH,theotherisaformalpartnershipbetweenOEHandNSWDepartmentofEducationandCommunities(DEC).
TheenergyefficiencycomponentofNSWGovernmentSustainabilityPolicyinvolvesmultipleagenciesandhasdistributedgovernancearrangementswhereOEHhaspolicyresponsibilitybutnodirectmanagementrole.
TheOEHDataandEvaluationProgramwhichaimstoimprovecapacityformeasuringenergyefficiencyacrosstheprogramsandtoprovideevaluation.
Table1‐2. Programsgroupedbyapproachesandgovernance
DirectenergysavingprogramsmanagedbyOEH
ThreeOEHprogramsaimtoachieveoutcomesattwolevels(1)participantswithincreasedcapacity(assessments,retrofitrebates,energyefficientitems),skillsandawarenessformanagingtheirenergyleadingto(2)energysavingsandtheassociatedsavingsinparticipants’energybills.Theyuseanassessmentofenergyusewhereassessorsprovideparticipantswithadviceandtoolstoimprovetheirenergyefficiencyusingan‘assess‐plan‐act’model.
Theprogramstargetenergyusersindifferentsectorsandhavedistinctivestrategiesforengagingwithparticipantsandsupportinghouseholdororganisationalbehaviourchange. HomePowerSavingProgram(HPSP)offersfree
assessmentandasavepowerkittolowincomehouseholds.
SmallBusinessEnergyEfficiencyProgram(EESBP)offersasubsidisedenergyassessmenttosmallbusinessesandmatchedfundingtosupporttheimplementationofrecommendations.Upto4hoursofsupportisalsoavailableassistthebusinesseswiththeirupgrades.
EnergySaverProgram(ESP)providesmediumtolargebusinesseswithsubsidisedenergyassessments,includingbusinesscasesforinvestmentinenergyefficiency.Upto20hoursoftechnicalandprojectsupportisalsoavailabletoassistwiththeinstallationofequipment.
1SeeCalifornianPublicUtilitiesCommission,EnergyDivision(2010)2006‐2008EnergyEfficiencyReport.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
18
Capacitybuildingprograms
Twoprogramsaddressunderlyingcommunityandworkforcecapacityforaddressingenergyefficiency.OneismanagedbyOEH(EECAP);theotherisaformalpartnershipbetweenOEHandDEC(EETP).
EnergyEfficiencyCommunityAwarenessProgram(EECAP)usesmainstreamandbelow‐the‐linecommunicationscampaignsunderthe“SavePower”banner.
EnergyEfficiencyTrainingProgram(EETP),multiplestrategiestoimprovethecapacityoftheworkforceforenergyefficiency
OEHsupportingaprogramimplementedbyothergovernmentagencies
OEHcoordinatestheenergyefficiencycomponentofNSWGovernmentSustainabilityPolicy,whichcoversseveralprogramsthatareultimatelytheresponsibilityoftheindividualagencies
TargetsforthewholeofgovernmentweresetinthePolicy.TheenergyefficiencycomponentofNSWGovernmentSustainabilityPolicy(EEGSP)coversfiveprograms:theSchoolsEnergyEfficiencyProgram,NABERSratingofofficebuilding,Greenleaseschedules,TreasuryLoanFund‐SustainableGovernmentInvestmentProgramandtheGovernmentBuildingRetrofitProgram.OEHisprovidingsupporttoparticipatingagenciesthroughfacilitation,technicalsupportandinformationresources.
Dataandevaluation
ThisOEHprogramisbuildingcapacitytomeasureandevaluateenergyefficiency.
DataandEvaluationProgram(DEP)hasprojectstoimprovecapacityformeasuringandverifyingenergyefficiency,forevaluatingprogramsandforassessingoverallbenefits.
1.3 ThepolicycontextforenergyefficiencyinNSW
EnergyefficiencyisamajoraspectofNSWGovernment10‐yearstrategicplan‘NSW2021’;itflowsacrossseveralgoals,especiallythoserelatedtothecostoflivingandthenaturalenvironment.Rapidlyincreasingelectricitypricesarehighlightingthevalueofenergyefficiencyforthepublicandthegovernment.
Inthewiderpolicycontext,energyefficiencyisreceivingmoreandmoreattentionacrossAustralianstatesandterritoriesandatthenationallevel,especiallyinthecontextoftheCommonwealthGovernment’scarbontax.AspartoftheCleanEnergyFutureplan,theCommonwealthGovernmenthasdevelopedenergyefficiencyinitiativesthatmayimpactonNSWprogramsinthisarea.
Thecontextfortheseprogramsoverrecentyearshasbeenchangeanduncertainty.TheyaremainlynewprogramswithsomeapproachesbeingimplementedforthefirsttimeinNSWandevenAustralia.TheyhaveinvolvedOEHinnewstrategies,increasedscalesofdeliveryandnewstakeholderrelationships.Theyarebeingdeliveredina
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
19
policyandprogramcontextforenergyefficiencyandclimatechangecharacterisedbyuncertaintywithinstateandCommonwealthgovernments.
1.4 Programlogic
AnEvaluationFrameworkwasdevelopedfortheEESin2010andrevisedwiththeEESEvaluationAdvisoryGroup.ItissummarisedintheprogramlogicdiagraminFigure1‐1.Theevaluationframeworkandprogramlogichelpstocommunicatetheprograms’objectives,formulateevaluationquestions,assesstheextentofimplementationatdifferentlevelsofimplementation,andformthebasisofevaluativeargumentsabouttheoveralleffectivenessoftheprograms.2
TheprogramlogicshowshowtheEESisbasedonthesuccessfuldeliveryoftheprogramstothetargetedsectors(theverticaldimension),toachievetheintermediateoutcomesofimprovedcapacitytomanageenergyuseinthatsector.Thedirectenergysavingsprogramsareexpectedtodeliverenergysavingsfortheirparticipants.Ultimatelytheprogramsareexpectedtocontributetotheintendedenvironmental,economicandsocialbenefitstoNSW.Oneoftheseoutcomesisreliableinformationonenergysavingsandcosts.
Theprogramlogicalsoshowstheassumedhorizontalinfluences:twoprograms(EECAPandEETP)aretobuildwidercapacitywithinthecommunityandtheindustrythatwillsupporttheoutcomesoftheotherprograms,andtheperformanceofthewholestrategyisinfluencedbyeffectivegovernanceandmanagementprocesses,andtheDataandEvaluationProgram(DEP).
ThelogicillustrateshowtheEnergySavingSchemeisoutsideofOEHandrunsparallelwiththeotherprograms.ItalsohighlightstheroleofotherfactorsthatmayimpactuponthepotentialachievementsoftheEESandinfluencetheintendedoutcomes.
2Identifyingtheprogramlogicisimportantintheevaluationofenergyefficiencyprogramsbecauseitrepresentstheunderlyingtheoryofchange,assumptionsandintermediateoutcomes—seeReed,J.H.,Jordan,G.,&Vine,E.(2007).Impactevaluationframeworkfortechnologydeploymentprograms,USDepartmentofEnergyhttp://www.cee1.org/eval/impact_framework_tech_deploy_2007_main.pdf;MirjamHarmelink,LarsNilssonandRobertHarmsen(2008),Theory‐basedpolicyevaluationof20energyefficiencyinstruments,EnergyEfficiencyVolume1,Number2,Springer
FinalReport– August2012 2012evaluationofNSWenergyefficiencyprograms
20
Otherenergyefficiencyinstruments
(Cwlth,NSW)
ESS
Activitiesandoutputs
Intermediateoutcomes
Environmental,economicandsocialbenefitstoNSW Reducedcoststotheeconomyandenvironment(electricitydemand;emissions;needforinfrastructure;impactofrising
pricesoncustomers;costofemissionsreduction) Transformedmarketforenergyefficiency Sustainedbehaviourchangesbypeopleandorganisations Reliableinformationonenergysavingsandcosts
Ultimateoutcomes
Appropriatemixofprogramsdesigned,resourced,established.Targetsset.Appropriategovernancearrangements&processesestablished.Dataandevaluationsystemsdeveloped.
Reducedenergyuseacrosskeysectors
Increasedcapacity,skillsandawarenessformanagingenergyuseinkeysectors
Otherinfluences(positiveornegative)
Changingeconomicconditions
Changingcommunityattitudes
Newtechnologies
Factorsarisinginothersectorse.g.building
Populationchanges
Weatherandclimate
Targetgroupsreached/accessprograms
Immediateoutcomes
EESProgramsdeliveredefficientlyandasplanned
Households‐lowincomefamiliesHPSP
SmallBus.EESBP
Med&LargeBus.ESP
CommunityEECAP
NSWGovagenciesEEGSP
WorkforcesEETP
economicandenvironmentalcostsofenergywasteandhighdemandduetomarketfailures costandgreenhousegasemissionsofmoregenerationandnetworkinfrastructure risingelectricitypricesforNSWbusinesses,householdsandgovernmentagencies. uncertaintyandlackofreliableinformationaboutenergysavingsandtheircost
Design
Effectivegovernance,management,dataandevaluation
Serviceprovidersentermarketandseekaccreditation
Suitablesettings,management
Energyusersupgradeequipmentandimproveenergyefficiency
ACPs reachenergyusers,generateESCs
Suitableserviceprovidersareaccredited(ACPs)
Needidentified
Figure1‐1. ProgramlogicoftheinitialEnergyEfficiencyStrategy
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
21
1.5 Evaluationoftheenergyefficiencyprograms
AnimportantdriverforevaluationofenergyefficiencyprogramsinNSWwastheuncertaintyaroundreportedenergysavings,highlightedbythe2007OwenInquiryintoElectricitySupplyinNSW.
Enhancedenergyefficiencycoulddelaytheneedfornewbaseloadcapacity,butitwouldnotbeprudenttorelyonthisbeingthecase,particularlyinviewofthelackofreliableinformationabouttheactualelectricitysavingstodatefromexistingenergyefficiencyprograms,andtheuncertaintiessurroundingfutureelectricitysavingsfromexistingandpotentialenergyefficiencymeasures.
Thishasledtoafocusonbettermeasurementofenergysavingsandevaluation,andtheestablishmentofDataandEvaluationProgram(DEP)in2009(seesection3.7).
OEHdevelopedtheevaluationframeworkbasedontheprogramlogic(above)andanevaluationstrategywhereeachprogramcollectsdataonimplementationandoutcomesandconductsitsownevaluationactivities.ThisiscomplementedbydataanalysisbytheStrategyandAnalysisunitandthemeasurementandevaluationprojectsundertheDEP.
The2012evaluation(thisreport)isaDEPprojectundertakenbyARTDConsultantsincollaborationwithOEH.Theevaluationispartimpactevaluationandpartimplementationevaluation3,andexamines
– howeffectivelyprogramsarebeingdeliveredandreachingtheirtargetgroups– theoutcomesproducedbytheprogramstodateincludingreducedenergyuse
andincreasedcapacityformanagingenergyuseacrossthesectors– improvementsinthereliabilityofenergysavingsestimates
Thisevaluationhasbeenconductedasasynthesisoftheavailableevaluationinformationstructuredaroundtheprogramlogic.Themainsourceswere:
evaluationsandreportsfromtheprograms.Eachprogramconductedevaluationandresearchactivities,includingcomprehensiveindependentevaluationsorevaluationactivitieslikeparticipantsurveys,and/orinternalevaluationreports(seeAppendix1).
datafromtheStrategyandAnalysisteamcollectedforClimateChangeFundreporting
3ForacomprehensiveoverviewofevaluationforenergyefficiencyintheUSthatisrelevanttoNSW,seeReed,J.H.,Jordan,G.,&Vine,E.opcit;andNationalActionPlanforEnergyEfficiency(2007)ModelEnergyEfficiencyProgramImpactEvaluationGuide.StevenR.Schiller,SchillerConsulting,Inc.www.epa.gov/eeactionplan
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
22
reportsfromtheDEPmeasurementandverificationprojects
interviewswithOEHmanagersbyARTDinMay‐June2012tocollectfeedbackontheapproachtogovernance,coordinationandinformationsharing
feedbackfromOEHstakeholdersonearlierreports.
Weareconfidentthatthefindingsofthisevaluationrepresenttheoverallpatternsofdeliveryandoutcomestodate.Thedatafromdifferentsourceswereconsistent.Atthisinterimstage,somegapsanduncertaintiesremain,particularlywhereevaluationreportsfocussedoninitialphasesoftheprogramorwerestillindraftform.Similarlytheestimationsofenergysavingsarestillbeingdevelopedandapplied.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
23
2. Overalloutcomesandachievements
Mostoftheprogramsstartedintheircurrentformin2009andarecurrentlyfundeduntilin2013or2014.Atthisinterimpointthischapterexaminestheoveralloutcomesandachievementsinmid‐2012,drawinguponthefindingsfromtheevaluationsoftheindividualprogramssummarisedinchapterthree.
2.1 Programoutcomes
2.1.1 Estimatesofenergysavingsareincreasinglyreliable
SincetheprogramsstartedOEHhassubstantiallyimprovedthereliabilityofestimatesofenergysavingsachievedbyprogramsdirectlysupportingparticipantstoreducetheirenergyuse.Intheearlyphaseofeachprogram,energysavingsestimatesweredesktopcalculationsbasedontargetuptakeandplanningassumptions.Theseinitialestimateswereinformedbytheavailableinformationineachareawhichvariedbetweenprograms.Forinstancelimitedinformationwasavailableaboutenergysavingsachievedthroughdistributionofsmallenergyefficiencyitemstohouseholds,whereasnumerousstudieshadalreadytestedassumptionstoestimatetheimpactofretrofittingbuildingsandequipment.
OEHhasprogressivelyrefinedthesecalculationsfirstlybasedonactualuptakeofprogramsandsecondlybasedonrefinedengineeringestimatesasnewevidenceavailableaboutthetargetgrouporthetypesofchangesrecommended(retrofit,newequipmentorbehaviourchange).Inparticular,theyincludethepersistenceofenergysavings.ForexamplewithHPSPpersistenceisassumedtovarywiththeequipmentinstalled(e.g.10yearssavingsfromshowerheads,5yearsfromotherkitcomponents)andwithbehaviourchange(oneyearpersistence).
AsrepresentedinFigure2‐1belowthemainachievementhasbeentomovefromprojectedtoverifiedenergysavingsbasedon‘before‐and‐after’analysisinlinewithinternationalbestpractice.TheDataandEvaluationProgramhasprovidedallenergyefficiencyprogramswithvarioustoolsforrigorousestimationofenergysavings:after1.5yearsofnegotiationsagreementswerereachedwithenergydistributorstoaccesslarge‐scalebillingdataforhouseholds(HPSP)andsmallbusinesses(EESBP);DEPdevelopedaguideformeasurementandverification(M&V)basedontheinternationalprotocol4thathasbeenusedbyEESBP,ESP,GBRPandtheESS;arangeofindependent
4Measurementandverificationistheprocessofusingmeasurementstoreliablydetermineactualsavingscreatedforanindividualfacilityorproject.IPMVPistheInternationalPerformanceMeasurementandVerificationProtocol(http://www.evo‐world.org).
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
24
studieshavebeencommissionedtoanalysemeasurementdatacollected.Section3.7providesdetailedinformationonmethodsusedtoestimategrosssavingsforeachprogram.
Figure2‐1. Improvedreliabilityofenergysavingsestimates
Deemedsavings(projected) Grosssavings(verified) Netsavings(verified)
Engineeringassessment…
...basedontargetuptake
…basedonactualuptake
Beforeandafteranalysis…
...basedoncasestudiesorasmallsample
…basedonarepresentativesample
Excludingfree‐ridership
Accountingforpersistence
…coveringallprojects
HPSP
EESBPESP
GBRP
2009‐10 2011‐12
Reliability
HPSP
EESBP
ESP
GBRP
Source:OEHandARTDbasedonUSNationalActionPlanforEnergyEfficiency(2007)
AtthisstageestimatesofenergysavingsarebasedonactualmeasuresofenergyusefromrepresentativesamplesforHPSPandEESBP;reportingforGBRPandESPstillreliesondeemedsavingsbasedonactualuptake,withM&Vprojectson‐goingorplannedfor2013.Thedifferencesinmethodsandtimeframeformeasurementmainlydependontheparticipants’characteristicsandtheprojecttimeframe.InparticularESPandGBRPprojectsareofmuchbiggersizeandhavelongertimeframeswhichleadstodelaysin‘before‐and‐after’analysis.Asaresultmonitoringtheimplementationoftheseprojectsovertimeisakeychallenge.
Thenextstepinthiscontinuumtowardsmorereliableenergysavingsestimatesistoassessnetsavings,ortheportionofenergysavingsthatisattributabletotheprogram.Estimatingnetsavingsinvolvesexcludingfree‐ridersandreboundeffects.Someofthemethodsusedinthisevaluationalreadyassessnetsavings.All‘beforeandafter’analysisofenergyuse(e.g.forEESB)includesanyreboundeffectaftertheenergyefficiencyaction.Further,theverificationofenergysavingsfromHPSPincludeduseofanon‐participantcontrolgroup.Thisshowedgrossreductionsinenergyuseforprogramparticipantsof10%,with4%netenergysavingsafterremovingcommunitywideeffectsevidentinthecontrolgroup.
TheworkinitiatedbyDEPhasreduceduncertaintyaroundenergysavingsfromtheseprogramsandestablishedasoundbasistoassesstheeffectivenessandcost‐effectivenessoftheprograms.Thiscaninformimprovementstoprogramdeliveryanddesignoffutureprograms.DEPresultshavebeenincorporatedinestimatesofprojectedsavingsoverthelifetimeofprojectsthatareinformingthedesignoftheEnergyEfficiencyActionPlan.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
25
2.1.2 Energysavingsarebeingachievedandexpectedtoincrease
Thedirectprogramsaimtoachieveenergysavingsandtheassociatedsavingsinparticipants’energybillsthroughprovidingthemwithincreasedcapacity(energyassessments,retrofitrebates,energyefficientitems)skillsandawarenessformanagingtheirenergyuse.
Onthebestavailableestimates,allthedirectenergysavingsprogramshaveeitherdemonstratedorindicatedsubstantialenergysavingstodate.Acrossallfourprograms(includingtheGovernmentBuildingRetrofitProgramforsmallsites,adirectenergysavingscomponentoftheNSWGovernmentSustainabilityPolicy),totalestimatedannualenergysavingsare118,434MWhofelectricityand187,716Gjofgas,leadingtoareductioninparticipants’energybillsof$28,496,340peryear.Theestimatedenergysavingswouldalsoresultin136,697tonnesofavoidedCO2emissions(Table2‐1).
Theprogramswillproducebenefitsoverthelifecyclesoftheinstalledtechnologieswhiletheyremaininplaceandoperable.Thesesavingswillvarywiththetechnologiesandwithunderlyingtechnical,marketandeconomicchangesineachsector.
Eachoftheseprogramshascontributedtotheenergysavings,withatthisstagemostsavingscomingfromthemediumtolargebusinesssectorthroughtheESP.
Figure2‐2. AllocationofoverallelectricitysavingsachievedtoendofJune2012acrossprograms
27%
32%
38%
3%
Home Power Savings Program
Energy Efficiency for Small Business Program
Energy Saver Program
Government Building Retrofit Program
.
FinalReport– August2012 2012evaluationofNSWenergyefficiencyprograms
26
Table2‐1. Energyefficiencyprograms–estimatedannualsavingsfromactivitiesimplementedtoJune2012
Fundingtodate$ Reach Electricitysaved(MWh)pa
Gassaved(Gj)pa CO2tonnessaved Participants $savings
electricitybillspa
Participants $savings
gasbillspa
TotalParticipants$savingspa
HomePowerSavingProgram
$23,924,019 115,508completedassessments
31,595 32,346 $8,372,675 $8,372,675
EnergyEfficiencyforSmallBusinessProgram(EESBP)
$17,958,385 17,185businessesregistered
37,396 39,640 $9,124,627 $9,124,627
EnergySaverProgram
$11,095,056 366audits 45,432 186,155 60,357 $7,996,068 $2,345,552 $10,341,620
EnergyefficiencycomponentofNSWGovernmentSustainabilityPolicy(GovernmentBuildingRetrofitProgram)
$5,249,631 105sites,567projects
4,011 1,561 4,354 $637,749 $19,669 $657,418
Total 118,434 187,716 138,273 $26,131,119 $236,5221 $28,496,340
Source:StrategyandAnalysis,OEHbasedonprogramdata
Notes:
MethodsforcalculatingsavingsarebasedonthemostrecentdatafromevaluationreportsandtheDEPmeasurementandverificationprojects,usingconservativeestimates.Detailsareinnotestotheindividualprogramtablesinsection3.
Thebillsavingsusethetariffsdevelopedforthe2010‐11ClimateChangeFundannualreport.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
27
FortheESPthe31,595MWhofestimatedenergysavingsatthisstageareexpectedtoaccelerateoverthenextfewyears.Itisclearthattheprogramhasamodelthatcanachievesubstantialenergysavingsandcutenergybillsatparticipants’sites.Casestudieshavedemonstratedenergysavingsofupto70percentforindustrialrefrigerationatvarioussites,andsimilarresultsforlightingretrofits.Theextentofsavingsislikelytobeclearerasmorebusinessesimplementauditrecommendationsovertime,andtheprogrammeasuresandreportsonthis.TheincreaseduseofM&VshouldprovidefirmevidenceofenergyandcostsavingsandallowESPtomoreaccuratelyextrapolatefindingsacrossparticipatingbusinessesandsub‐sectors.
EESBPhasachievedahighlevelofreach(17,185smallbusinessesregisteredtoJune2012comparedtotheinitialtargetof6,000)andgeneratedanestimated37,400MWhofannualelectricitysavingsleadingtoasavingsinparticipantselectricitybillsofover$9.1million.Thekeychallengehasbeenliftingtherateofconversionfromregistrationtorebatesforretrofits.Majorrefinementstotheprogramdesignhaveincreasedtheconversionratetoover50percenttotheendofJune2012,withevidencethatmostactionsareattributabletotheprogram.ThescaleofenergysavingswasdemonstratedthroughfindingsfrommeasurementandverificationanalysisconductedbyDEPover331projects(accountingforaround17%ofalltherebatesatthattime).Onaverageeachbusinesssaved5.64MWhor9.3percentrelativetoitsbaselineforthefirstyear.
InitialresultsforHPSPhavebeenpromising.Ithasachievedextensivereachacrossitstargetgroupoflowincomehouseholds(115,508completedassessmentstoJune2012),andhasdelivered31,595MWhofannualelectricitysavingsleadingtoasavingsinparticipants’electricitybillsofalmost$8.4millionTheseenergysavingsweremeasuredwithahighlevelofreliabilitythroughthebillingdataanalysisconductedinMay2012whereaveragesavingsof0.22megawatt‐hourofelectricityperannum(or4percentreduction)foreachparticipatinghouseholdwerefound.Savingsdifferedaccordingtothekititemsreceived(householdsreceivingtheshowerheadachieveonaverage6percentsavings).Theenergysavingsareencouraginggiventhelimitedpossibleimpactofthesmallkititems.Further,theelectricitysavingfromthelargestiteminstalledbyHPSP(efficientshowerheads)matchedthemeasuredsavingsfromSydneyWater’sefficientshowerheadretrofitprojects.Inadditiontheprogramhasachievedthesesavingswhileeffectivelyaddressingitssocialobjectiveoftargetinglow‐incomehouseholds,equityofaccessacrossthestate,andworkingwithspecifictargetgroupsinCALDandAboriginalhouseholds.
2.1.3 Otherbenefitsareemerging
Theenergysavingspresentedinthisreportareprimarilylimitedtothedirectenergysavingprograms.Howevertherearelikelytobemorediffuseimpacts.Forexamplethereisevidencethatthetheseprogramsleadtowiderenergysavingswithintheirsectorsthroughmultipliereffectsasindividualsandbusinessespromoteenergyefficiencymorebroadly.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
28
Furthermorethecapacitybuildingprogramswillalsohaveapositiveimpactonenergysavings.WhileEETPhasnodirectmeasurableenergysavingsoutcomes,atthisstagethelimitedpreandposttrainingdatashowstrainingparticipantsreportingchangedorganisationalpractices,operationsand/ortechnologies,andanumberestimatedormeasuredresultingenergysavingsattributedtotheprogram.TheEECAPsocialmarketingcampaignfocussedonwiderawarenessraisingandcapacitybuildingbutanumberofcomponentsindicateddirectenergysavings.Peoplewhohadraisedawarenessfromthemassmediacampaignreportedincreasedenergysavingbehaviours,sustainedovertime.ParticipantsintheEnergymarkprojectreportedreductionsintheirhousehold’scarbonemissionsandelectricityuse.
AnotherlongertembenefitforNSWisimprovingthecapacityororganisationstofurtherachieveenergysavings.ForexampletheESPhashadanimportantpositiveimpactbypositioningparticipatingorganisationstoattractfundingtoimplementtheirenergyefficiencyactivities.InJune2012fourESPparticipantswongrantsinthefirstroundoftheCommonwealthCleanTechnologyInvestmentProgram(CTIP),receiving86%ofthenationalallocation,andafurther20ESPsitesareapplying.AkeysuccessfactorwasthecomprehensiveESPaudit.AnotherexampleislinkingeachsitetoanAccreditedCertificateProvider(ACP)withintheESS,theNSWenergyefficiencytradingscheme,togeneratefundsforbusinessestoimplementprojects.
2.1.4 Thedirectenergysavingsprogramsarecost‐effectiveoverthelifetimeofsavings
OEHassessedthecost‐effectivenessofitsenergy‐efficiencyprogramsusingthemethodsetoutinthe2009IPARTreviewofNSWClimateChangeMeasuresonthebasisofcomplementaritytotheCommonwealthcarbonpricescheme.Undertherecommendedframework,acost‐effectiveenergysavingprogramisdefinedasonethatdeliversitssavingsatalowercostthanifthesamebenefitsweremadeundertheCarbonPollutionReductionScheme(CPRS).Thebenchmarkcostisdefinedasthefuturecarbon‐inclusiveretailpriceofelectricitythatisrelevanttothetargetedcustomergroup,whichreflectsnotonlytheusagecomponentbutthefullcostinresourcesofprovidingelectricity.5
Cost‐effectivenesswascalculatedby:
1. Expressingtheestimatedcostsandsavingsinpresentvaluetermsusinga7percentdiscountrate
2. Convertingthepresentvalueofthecostsandbenefitsintoaanaveragecostperunit—levelisedcost—inthiscasethecostpermegawatt‐hour
3. Calculatingtherelevantbenchmarkcostaccordingtothetargetcustomergroup.
5IPART,ReviewofNSWClimateChangeMitigationMeasures,May2009
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
29
TheTable2‐2belowpresentstheresultsofthiscost‐effectivenessanalysisforthefourprogramsproducingdirectenergysavings.Thekeypartofthisanalysisistoestimatesavingsachievedbyparticipantsbasedonthedatetheyenteredtheprogram,assumptionsforannualsavingsandlifeofsavingsaccordingtotheretrofit,equipmentinstalledorbehaviourchange.
Table2‐2. Cost‐effectivenessforthedirectenergysavingsprograms
HPSP EESBP ESP GBRP
Discountrate 7% 7% 7% 7%
ProgramcoststoendofJune2012(realcosts,$m)
$27.296 $18.794 $11.622 $5.381
Electricitysavingsperyear(MWh)
31,595MWh 37,396MWh 45,432MWh 4,011MWh
Lifetimeelectricitysavings 252,763MWh 373,960MWh 454,322MWh 40,110MWh
Calculatedlevelisedcost $138/MWh $66/MWh $34/MWh $179/MWh
Benchmarklevelisedcost $281/MWh $245/MWh $187/MWh $184/MWh
Source:OEH,WaterandEnergyProgramDivision,StrategyandAnalysissection,August2012
Onthisbasisallfourprogramsprovetobecost‐effective,withlowercostpermegawatthoursavedthantheirrelevantbenchmarkcost(Table2.2).Eachprogramisdeliveringenergysavingsatalowercostthanthecostofprovidingthesameamountofelectricity.Comparingcost‐effectivenessbetweentheseprogramsgiveslimitedinsightbecauseprogramshavedifferentobjectivesandinparticulartargetdifferentcustomergroups.
Theaccuracyoftheseresultsissubjecttothereliabilityofenergysavingsestimatesasdescribedinsection2.1.1withhigherreliabilityforHPSPandEESBPwhereestimatesarebasedonverifiedsavings.
2.1.5 Markettransformationandeconomicbenefitswillbeassessed
Afullerpictureofthelongertermimpactsoftheprogramswillbeavailableby2013,withtwostudiesconductedbytheDEP.Amarkettransformationstudyhascommencedtoevaluatetheimpactoftheprogramsontheenergyefficiencyproductandservicemarkets(seesection3.7).SecondDEPiscommissioningaprojecttoestablishcost‐benefitmetricsoftheenergyefficiencyprogramsasawholefortheelectricitymarketandthestateeconomy,coveringpublicaswellasprivatecost.Itwillinvolveenergymarketandeconomy‐widemodellingandwilltransfersomeofthemodellingcapacitytoOEHatthecompletion(currentlyunderprocurement).
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
30
2.2 Achievementsandissueswithimplementation
2.2.1 Directenergysavingprogramsareadaptiveandbeingdeliveredeffectively
Thethreedirectenergysavingprograms(HPSP,EESBP,andESP)arethemostsignificantcomponentoftheprogramsandaccountfor61percentofOEHexpenditure.Theyaimtodirectlyproduceenergysavings,andusebroadlysimilarstrategiestoworkwithenergyusersintheirtargetsectors,lowincomehouseholds,smallbusinessesandlargeenergyusingsites.Eachprogramhasmethodstoassesstheparticipants’energyuseandoffersupportforimprovingtheirenergyefficiency.
HPSPandEESBPwerenewprogramsforNSWandthefirsttimelowincomehouseholdsandsmallbusinesseshadbeendirectlytargetedinthisway,whileESPwasadevelopmentofthepre‐existingSustainabilityAdvantageProgram.HPSPinparticularisaverylarge‐scaleprogramthatbeganwithrelativelyuntestedtheoriesandassumptions.HPSPisfurthercomplicatedinthatinadditiontoitsenergyefficiencyobjective,italsohasasocialequityobjectivetofocusonlowincomehouseholds,reachspecifictargetgroups(CALDandAboriginalhouseholds),andtoensuregeographicequityofaccessbymakingtheprogramavailableacrossallofNSWatalltimes.Thishashadsignificantimplicationsforprogramdelivery.
AstrengthofthedesignphasefortheseprogramswasthedevelopmentofrobustITsystemstomanagedeliveryandrecordimplementationofassessments.Eachprogramdevelopedandrefinedassessmenttoolsforusebyassessors.Theprogramsalsoestablishedrobustauditingprocesses,totacklepotentialriskstoimplementationashighlightedbytheCommonwealth’shomeinsulationprogram.
Foreachprogram,amajorachievementwasestablishinglarge‐scalecontractualarrangementswithserviceproviders,overcominginitialchallengesandcapturingalargepartoftheenergyefficiencycontractingmarket.Theprogramscontributedtotheup‐skillingofauditors/assessorsandjobcreation(theextentwillbeassessedthroughthemarkettransformationstudy,anothercomponentoftheoverallevaluation).
Challengesinimplementationincludedfindingthebestchannelstopromotetheprogram,gettingthesignalsrightforthedifferenttargetgroups,andengagingwithparticipantsinthelongerruntosupportsustainedbehaviourchange.Anotherchallengehasbeenusingdata.Programscollectedawealthofdataonimplementationandwhilesomeprogramsmadelimiteduseofthisfortheanalysisofimplementation,ithasbeenincreasinglyusedthistoguideimprovements.
Theprogramsunderwentmanyadaptationstoreachandbetterrespondtotheirtargetgroups(seechapter3)andby2012wereonmorestrategicfootings.Programshadtoexpandandrefinestaffingtoaddresstheemergingrangeofdeliverytasks.Theprogramshadtheflexiblytochangetheirsettingswithinoveralltargetsandobjectives.WhileHPSPandESPhadinitialslowuptakes,allthreeprogramshavehadtheir
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
31
timeframesextended,andareallnowontracktoreachtheirdeliverytargets.Asthevolumeofdeliveryhasincreased,efficiencyofimplementationhasimprovedovertime.
Theprogramsmadechangestomanageboththedemandforandthesupplyofenergyefficiencyservices.Allthreeprogramsshiftedfocusfromusingtheassessmenttoidentifypracticalmeasurestoimproveenergyefficiency,tosupportingtheimplementationofrecommendations,whetheritwasHPSPhouseholdersinstallingkititemsorESPbusinessesmakingdecisionsaboutinvestinginnewtechnology.EESBPfurtherrefinedtheprocesstohaveassessorstoputmorefocusonsupportingimplementationandlessondeliveringanassessmentonly,resultinginadramaticliftinbusinessesproceedingtorebate.
Datafromprogramdeliveryisbeingincreasinglyusedtoguidestrategiestotargethigherpotentialforsavings.Forexample,EESBPistargetingbusinesstypeslikebutchers,poultryfarmers,smallsupermarketsandfastfoodoutletsashighenergyusers,withapproachesarebeingputinplaceinpartnershipwithpeakindustry.EESBPisalsodevelopingindustrybenchmarkingbasedonthedatacollectedtodate.Forexampleinthedairyindustry,datafrom440assessmentshasestablishedabenchmarkofenergyuseper1,000litresofmilk.
2.2.2 Highparticipantsatisfactionandpositivestakeholderengagement
Alltheevidenceindicatesthatprogramsweregenerallywellreceivedbyparticipants.Thethreedirectenergysavingprogramshaverecordedhighlevelsofparticipantsatisfaction.HPSPmonitorssatisfactionthroughitsreportingandauditprocesses,whichfoundover95%ofhouseholdshighlysatisfiedwiththeprogram,especiallywiththefreekititemsandthetailoredassessmentsthathelpthemtosaveenergy.ForEESBPindependentsurveyresearchinMay‐June2012foundthat78percentofbusinessesweresatisfiedorverysatisfiedwiththeassessorand80percentwiththerecommendations(n=301),inlinewithsimilarfindingsin2010.SimilarlyforESP,independentsurveyresearchin2011foundparticipatingbusinessesverysatisfiedwiththeESP(80percentwithauditors,92percentwiththeESPteam).MorebroadlyapprovaloftheSavePoweradvertisingcampaignhasbeenpositive(over80percentofpeopleapprovedorstronglyapproved).
Akeyachievementofdeliveryofthedirectenergysavingprogramswasengagingexternalstakeholdersincludingparticipantsas‘multipliers’topromotetheprogramthroughtheirnetworks,andusingearlyadoptersasadvocates.Assessorswerealsoakeyassetnotjustindeliveringassessmentsbutalsoincontributingtopromotionandfurtherparticipantrecruitment.Programsbuiltthisintotheirdeliveryindifferentwayssuchasdevelopingtrainingorincludingincentivesintheirpaymentmodel.Theprogramsdevelopedstrongsupplychainrelationshipswithassessorsandotherproviderstobuildpartnerships,usingmethodssuchasforums,requiredattendanceatprogramupdates,webinarsandemailcommunication.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
32
2.2.3 Regionalfocusachieved
TheprogramshavehadasignificantregionalfocusinlinewiththeGovernment’scommitmenttosupportregionalcommunities.HPSPisaddressinganexplicittargetforgeographicequityofaccess,whileEESPandESPhaveover50%ofparticipatingbusinessesfromoutsideSydney.
2.2.4 Workingwiththegovernmentsectorhadmixedresults
ThefiveprogramsundertheenergyefficiencycomponentofNSWGovernmentSustainabilityPolicy(EEGSP)hadvaryingsuccessinreachingthegovernmentagenciesandeffectivelyengagethemintheprogramrationale.Settingtargetswasnotsufficient.‘Outclauses’includedinthepolicy,suchas“wherepractical”or“whererelevant”meanttargetwerenotmandatory.Basedonfeedbackfromagencies,OEHstaffidentifiedthathavingaclearpolicymandate,agency‐specifictargetsandaccountabilityrequirementspossiblywithpublicreportingwouldsupportthesuccessfulimplementationofprograms.
Effectivenessofimplementationoftheprogramsvariedsignificantly:somehadsomegoodresultsliketheGovernmentRetrofitBuildingProgram,thedirectenergysavingscomponent,wherethesmallsitescomponenthadgeneratedestimatedsavingsof4,011MWhofelectricityand1,561Gjofgasleadingtoasavingsinparticipants’energybillsofover$.657million.OtherssuchastheGreenLeaseprogramdidnotproduceanyimpact.Allprogramshadcommonbarrierstoengageeffectivelywithagencies:thelackoftechnicalknowledge,timeandfinancialresourcesforauditstoapplyforthesupportoffered.
2.2.5 Awarenessandcapacityforenergyefficiencyisbeingdeveloped
Twoprogramsaddressedtheenablersandbarriersforgreaterenergyefficiency,targetingthewidercommunity,keyindustrysectorsandoccupationswhichareabletoimplementenergyefficiencyopportunities,theeducationandtrainingworkforce,andpolicymakersandprogrammanagers.Eachoftheseprogramshasbeenlargelyimplementedasintended,andsummariesareinchapter3.
TheEnergyEfficiencyTrainingProgram(EETP)usesmultiplestrategiestoimprovethecapacityoftheworkforceforenergyefficiency.AkeyfeatureoftheEETPiscross‐agencygovernancebetweenOEHandNSWDepartmentofEducationandCommunities(DEC).EETPhasdevelopedclearformalpartnershiparrangementsandwhilethepartnershipfacedsomechallengesinitsdevelopment,itisnowvaluedandsupportedbyinternalstakeholdersacrossbothagenciesandprovidesasoundplatformforsharedprogramdelivery.Industryandtrainingstakeholdersalsovaluedthepartnershiparrangementsbetweendifferentgovernmentagenciesandfoundthiscontributedtothesuccessoftheprogram.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
33
AmajorachievementofEETPhasbeenengaging5,290participantsinenergyefficiencytraining,networkingorprofessionaldevelopmentbetweenJuly2009andApril2012,including
2,695committedplacesintheVocationalEducationandTraining(VET)system 179participantsintheProfessionalDevelopmentforVETPractitioners(PD4VET)
workshops 629participantsinotherVETProfessionalDevelopment(PD)trainingor
networkingevents 1,233participantsin30IndustryPartnershipsprojects 544participantsinHigherEducationcourses
TheEnergyEfficiencyCommunityAwarenessProgram(EECAP)isacommunicationandeducationprogramtargetingthebroadcommunitytoimprovecommunityknowledge,understandingandmotivationtoactinrelationtoenergyefficiency.Itusedacomprehensive,multi‐strategyandongoingapproachthatreflectsbestpracticeinsocialmarketingtoreacharangeofdifferentaudiencesthroughmainstreamandbelow‐the‐linecommunicationscampaignsunderthe“SavePower”banner,andspecificeducationprojects.TheEECAPmassmediacampaigntargetedanoverallpopulationof5.8millionpersons(approximateNSWadultpopulation)
TheSavePowermassmediacampaigntelevisionadvertisinginitsfinalphasereachedalmosttwo‐thirdsofNSWadults(63percent),withmost(77percent)findingitconvincingandathirdfeelingmotivatedtotheactionsinthemessages.Trackingresearchdemonstratedthatapprovalofthecampaignremainedhighovertime,fromwinter2009tosummer2010:morethan80percentofpeopleapprovedorstronglyapprovedgovernmentadvertisingwithmessagesaboutsavingpower.Researchsuggeststhatitcontributedtoimprovedknowledgeofenergyefficiencyissuesandhashadaninfluenceinchangingenergyusebehaviours.Thewidereachprovidesgreatpotentialforenergysavingsfrombehaviourchangeatthisscale.
TheEECAPeffectivelytrialledarangeofothereducationalapproachestoreachdifferentsegmentsoftheNSWcommunity.InpartnershipwiththeEthnicCommunitiesCouncil,bi‐lingualeducatorsweretrainedandthendelivered57energyworkshopsdeliveredto1,044participants.Anewprogramin2012willrecruit20EnergyChampionsand50LeadersfromacrossCALDcommunitiestoencourageparticipationandactiontoreduceenergy.TheCSIRO’sEnergymarkprojectwastrialledinNSW,with112groupsand517participantstoApril2012(another19werestillinprogress).SavePowerlibrarykits(841)wereplacedwith62CentralLibraryServicesinNSW(withupto260branchlibraries)andborrowedby6,400householdsintwelvemonthswithevidenceofpositiveimpactontheirknowledgeandbehaviour.TheRetailerEngagementProjectsuccessfullyengagedmajorelectricalapplianceretailersandprovidedtrainingtostaffin142storestoassistcustomerstoconsiderenergyefficiencyintheirpurchasedecisions.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
34
2.2.6 Cross‐impactsofprogramshavebeenlimited
Theoverallstructureanddesignhasnotfocussedoncross‐programlinksorlentitselftomoreintegrateddelivery,andtherehavebeenonlylimitedcross‐programimpactsatthisstage.Thisisfurtherdiscussedinchapter4oncoordinationandgovernance.
ThedeliveryofthethreedirectenergysavingsprogramsandtheEEGSPwererelativelylinear,separateandfocussedontheirdifferenttargetgroupssectors.However,ESPandEESBPthatworkoutofthesamebranchhavecollaboratedinseveralways:recommendingbusinessestotheotherprogram,developingtechnologyandsectorreports,designingsectorapproachesthatcoverbusinessesofdifferentsizes,andutilisingthesamepanelsofserviceproviders.
Thetwocapacitybuildingprograms(EECAP,EETP)wereexpectedtosupporttheotherprogramstosomedegree(throughincreasingcommunityawareness,andbuildingtheenergyefficiencyworkforce).TheSavePowercampaignandwebsitedevelopedthroughEECAPprovidedcommonbrandingforthestrategy.EECAPmadegrantsto36councilstopromotetheHPSPandawarenessofSavePower.
Inpracticecross‐pollenisationhasbeenlimitedbythedifferenttimeframes.Forexample,long‐termimpactsofEETPontheenergyefficiencyworkforcewillbemainlydiscernibleintothefuture.TheDEPmarkettransformationstudywillassessprogresswithchangesinenergyefficiencyservicesinNSW.Forthisevaluation,verylittleifanydatawascollectedbytheprogramsontheinfluenceofEECAP(orpotentiallyEETP)ontheirparticipants.
TheNSWEnergySavingsScheme(ESS)offersasynergywiththethreedirectenergysavingprogramsandtheEEGSP,whereparticipantscanuseEnergySavingsCertificates(ESC)tofundenergyefficiencydevelopments.RecentlyeachoftheseprogramshasbeendevelopingtheseopportunitiesdirectlywithIPARTasadministratoroftheESS.However,coordinationisneededtoavoidriskssuchasinconsistentmarketsignals,oversubscription,andshortagesofcapacityfromserviceproviders.
OEHComment
ThecomplementarityreviewinformedgovernmentdecisionstomergetheEESBPandthenon‐duplicativeaspectsofEETPintoESPandtoincreaseuseoftheEnergySavingsScheme(ESS)tofundenergyefficiencyactions.
Thesedecisionsprovidethebasisformoreintegrateddeliverye.g.practicaltraininglinkedtolow‐costtoolstoaccesstheESSforspecifictechnologiesandsectors.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
35
3. Progressofindividualprograms
Thischapterassesseshowtheprogramshavebeendesignedanddelivered,theadaptationsmadetoaddressbarriersandchangingcontexts,andtheinitialevidenceofenergyefficiencyoutcomes.Thesourceistheevaluationsandreportsfromeachprogram,andinterpretiveinterviewswithprogrammanagers.
3.1 HomePowerSavingsProgram(HPSP)
3.1.1 Programdesignanddevelopment
HPSPaimstohelp220,000low‐incomehouseholdsreducetheirpoweruseandbillsbytheendofJune2014.Thefreeprogramincludesthreemaincomponents:akitofenergyefficientitems,ahomepowerassessmentandatailoredactionplanidentifyingfreeandlow‐costwaysforthehouseholdtosavepower.TheprogramismanagedbyOEH,andFieldforceisthecontractordeliveringenergyassessmentservices.
FromtheinitialpolicybriefinJune2008totheendofDecember2011,HPSPhasbeenthroughalongjourney,withchangesmadealongthewaytobetterfitcircumstancesandthetargetaudience.TheinitialpolicydesignwasverybroadanddidnotspecifyaclearstrategicframeworkforaprogramthatwasuniqueforOEHinitsobjectivesaswellasitssizewitha$63millionbudget.Asaconsequence,twoofthefiveyearsinitiallyplannedfordeliveryofthe220,000energyefficiencyassessmentswereusedtodevelopamorerobustprogramdesignandrunthreepilots.Theprogramhasalsoanequityobjectivethathassignificantimplicationsforprogramdelivery.Thisobjectivemeansnotonlyreachingspecifictargetgroups(CALDandAboriginalhouseholds),butalsoensuringgeographicequityofaccessbymakingtheprogramavailableacrossallofNSWassplitintopostcodeclusters(PCC)—specificgeographicareascreatedtomanagethedeliveryoftheprogram—atalltimes.
Inthedevelopmentphase,OEHinitiatedwhathaveproventobesomeoftheprogram’skeystrengths.Theseincluded:involvingexternalstakeholdersfromthebeginning,pavingthewayfortheirstrongengagementinthepromotionoftheprogram;developingcomprehensiveITsystems,suchastheassessmenttoolandtherelatedassessmentsdatabase(DEAS),tosupportthedeliveryofassessments;andestablishingrobustauditandreportingsystemstoensurepropermonitoringandaccountability.
Theinitialdeliverymodel,however,didnotproperlycopewithallofthechallengesraisedbytheprogramimplementationtomeettheprogramobjectives.ThemainchallengesfacedintheinitialstagesofimplementationweregeneratingenoughdemandtoreachthequantitativetargetandclarifyingresponsibilitiesformarketingandcommunicationbetweenOEHandFieldforce.TheHPSPteamdidnothavethenumbersandskillstofullysupportthepromotionoftheprogramintheinitialphase.Increased
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
36
staffingandimprovedprocesseswithFieldforcehelpedtoleveragethepromotionaleffortsofallstakeholdersthroughanewlyestablishedmarketingstrategy.
3.1.2 Deliveryandreach
AssessmentsarearrangedbyFieldforcethroughaneffectiveschedulingprocessanddeliveredbyassessorsinthefield.Assessorsareattheforefrontandthemainassetoftheprogram;theirskillsandengagementarehighlyvaluable.However,theoriginalpricingmodelestablishedthroughthetenderprocessundervaluestheirkeyroleinprogrampromotionthroughlackofclearandsufficientincentives.
ByJune2012HPSPhadcompleted115,508assessmentsor58%ofitsfinaltarget.NowthattheprogramhasbeenextendedforanotheryeartoJune2014(inthecontextofNSW2021)andwiththesuccessofthemini‐campaignsthatfocusintensivepromotiononafewpostcodeclusters(PCCs)foralimitedtimeframe,theprogramappearstobeinreachofitsambitioustarget.
UptakeinsomePCCshasbeenmoresuccessfulthanothers,butthereisnoclearpatterninthesuccessfactorsthatcouldbevalidacrossNSW.SomePCCswithsimilarcharacteristicsperformdifferently.Inthiscontext,thereisno‘one‐size‐fits‐all’approach:localpromotionalstrategiesmustplayakeyroleinsuccessfuluptake.ThesestrategiesshouldbedesignedaccordingtothemainfeaturesandchallengesofeachPCC,forexample,stallsinmallsmayworkinPCCswithahighproportionofseniors,andpartneringwithCALDcommunityorganisationsforPCCswithahighproportionofthesehouseholds.
Overall,participantsarehighlysatisfiedwiththeprogram.HPSPmonitorssatisfactionthroughreportingandauditprocesses.Fieldforceconductsamonthlypost‐assessmentphonesurveyof100randomlyselectedparticipants–inDecember2011,95percentsaidtheywereextremelysatisfiedand5percentsatisfiedwiththeservice.IABauditresultsforthelastquarterof2011found97percentreportedapositiveexperience(n=153).Participantswereespeciallypleasedwiththekititemsbecausetheywerefreebutalsobecausetheymadetheideaofsavingsconcrete,andwiththetailoredassessmentsthatwouldhelpthemtosaveenergy.Themaincriticismfromparticipantswasthattheactionplanmaynotgivearealisticviewofthechangesandsavingstheycanachieve.Anecdotalevidenceofinconsistenciesinthewayassessmentsoritemsaredeliveredindicatesariskofnegativeperceptionsandfrustrationwhennotproperlyjustified(e.g.noshowerheadinstalledforHousingNSWtenants).
Programcostsarelowerthanbudgetedwithanaveragecostperassessmentof$270predictedfortheendoftheprogramcomparedwiththe$286initiallybudgeted,reflectingtheconstantattentiongiventoefficientprogramdelivery.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
37
3.1.3 Outcomes
ByJune2012theprogramhadgeneratedanestimated31,595MWhofannualelectricitysavingsleadingtoasavingsinparticipants’electricitybillsofalmost$8.4million.Theenergysavingshavealsoresultedin32,346tonnesofavoidedCO2emissionsinoneyear(Table3‐1).Basedonthesesavingsestimatesthelevelisedcostpermegawatt‐hourcomparesverypositivelywiththebenchmarkcostinresourcesforprovidingretailelectricitytohouseholds.Takingintoaccountdifferentpersistencefactorsforsavingsfromkititemsandbehaviourchange,thecostofmegawatt‐hoursavedbyHPSPparticipantsis$138comparedto$281resourcescostforretailelectricity.
Table3‐1. HomePowerSavingsProgram–estimatedannualsavingsfromactivitiesimplementedtoJune2012
Reach Electricitysaved(MWh)pa
CO2tonnessaved Participants $savingselectricitybillspa
Participants watersavingsperyear(ML)
115,508completedassessments
31,595 32,346 $8,372,675 92.4
Source:StrategyandAnalysis,OEH
Notes:
ThesavingswerecalculatedbyapplyingthepreliminaryresultsofregressionanalysisofbillingdataundertakenbyindependentconsultantsinMay2012throughtheDEPmeasurementandverificationproject;estimatingsavingsbyassigningsavingstohouseholdsbasedonwhetherthehouseholdreceivedashowerheadornot;andassigningapersistencefactorof10yearstosavingsfromshowerheads,5yearstosavingsfromotherkitcomponentsand1yeartosavingsfrombehaviourchange
ThisestimateisconservativebecauseitadoptsthelowerestimatesofsavingsfromthedraftHPSPevaluationreport;appliesverylowpersistencefactorstosavingsfrombehaviourchange;andapplieslowerboundestimatesofpersistencefactorssavingsattributedtokititems.
Thebillsavingsusethetariffsdevelopedforthe2010‐11ClimateChangeFundannualreport($265/MWh),andusedthroughout2011‐12.
Electricitysavingsfromasampleof23,500homesweremeasuredbythebillingdataanalysisconductedwiththeDEPmeasurementandverificationprojectinMay2012.TheanalysisprovidedpreliminaryfindingsinMay2012withaveragesavingsof0.6kWhperhouseholdperday(or0.22megawatt‐hours(MWh)peryear,a4percentsaving),Thisrepresentsatotalfirstyearsavingof8979MWhfromthe41,000householdsthathadparticipatedintheprogrambyJune2011.Savingsvariedaccordingtothekititemsreceived:6percentsavingsforparticipantsthatreceivedthewholesavingkitagainst3.5percentforhouseholdersthatreceivedallitemsexceptshowerheads,whichhighlightsthesubstantialimpactofshowerheadsonenergysavings.Savingfromlightingrefitwasalsoestimatedfromthebillingdataanalysistobe0.35kWhperhouseholdperday(or2.3percentreduction).
Theseresultsareencouraginginthattheyprovedthehardwarecomponentsoftheprogrami.e.showerheadandCFLrealisedtheexpectedsavingsunderrealisticuseconditions.Forexample,theelectricitysavingfromthelargestiteminstalledbyHPSP
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
38
(efficientshowerheads)matchedthemeasuredsavingsfromSydneyWater’sefficientshowerheadretrofitprojects6.Ithighlightsthedistributionandinstallationofshowerheadsasamajorfactorforenergysavings.Oneissueisthatthemajorityofexistingparticipants,includingnon‐socialhousingtenants,arealreadyequippedwithenergyefficientshowerheads.
Furtheropportunitiesexisttoincreaseenergysavingsespeciallythroughthetipsintheactionplan,thathaveverybroadscopefromtakingshortershowerstosubstantialretrofitslikeahotwatersystem.Theevaluationrecommendedafollow‐upsurveyofparticipantstobetterunderstandtheirbehaviourandtheresultingoutcomes.HPSPisalsodevelopingastrategytoengageparticipantsinacontinuousrelationshipthatshouldbeinformedbyresearchonbehaviourchange,learningfromgoodpracticeinareaslikepreventivehealth.
3.1.4 Conclusion
HPSPhasbeenaverychallengingprogramtoimplementbasedonanambitiouspolicybriefandbothenergyefficiencyandsocialobjectives.Despiteinitialdifficultiesindefiningappropriatesettings,theprogramisontracktoreachthetargetof220,000participantsbyJune2014with115,508completedassessmentsattheendofJune2012.Theenergysavingsareencouraginggiventhelimitedpossibleimpactofthesmallkititems,andanecdotallyareconsistentwithcomparableprograms.Themainchallengeforthenextphaseoftheprogramwillbetoenhancethesavingsachievedbyparticipants,andtoensurethesustainabilityofbehaviourchange.
6SydneyWaterCorporation,InstituteforSustainableFutures(ISF),2004,‘EveryDropCounts’ResidentialRetrofits‐AnalysisofWaterSavingsStudyReport,April2004,ISFISF,2001,EvaluationoftheShellharbourResidentialRetrofitProgram:Part2(draft),June2001
OEHComment
ThecomplementarityreviewinformedgovernmentdecisionstoretainHPSPasitmetallofthecomplementarityprinciples.
TheNSWEnergyEfficiencyActionPlanincorporatesthesegovernmentdecisionsandevaluationfindings.Forexample,theevaluationshowedthatsmall,lowcostitemsdeliververifiableenergysavingsinhouseholdsofaround6%.Newmechanismsarerequiredtoenablelowincomehouseholdstoinvestinlarger,highercostitemsthatproducemoresignificantenergysaving.
Lowincomehouseholdsvalueface‐to‐faceadviceprovidedthroughenergyassessors,althougheffectivenesscanbeimprovedbybettertargetingofactionstothespecificneedsofthehouseholdandefficiencycouldbeimprovedbyuseofmarketdelivery.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
39
3.2 EnergyEfficiencyforSmallBusinessProgram(EESBP)
3.2.1 Designanddevelopment
TheEESBPwaslaunchedinFebruary2009andtargetssmallandmediumsizedbusinesses(thatuseelectricityuptoabout$20,000peryearor160megawatt‐hours;orhaveupto10fulltimestaff).The$15millionprogramhadaninitialtargetofatleast6,000smalltomediumsizedbusinessestomakeenergyefficiencyimprovements.TheEESBPwasoriginallyduetobecompletedinJune2011buthasbeenextendedtoDecember2012withanadditional$6.6millioninfunding.
TheEESBPprovidesaparticipatingbusinesswithasubsidisedenergyassessmentthatidentifieswhereelectricityisbeingused,andatailoredactionplanwithelectricityandcostsavingrecommendationsandtheinformationneededtoclaimrebates.Businessesthatimplementrecommendationswithgreaterthanatwo‐yearpaybackperiodhaveaccessmatchedfundingtocoverhalfoftheretrofitcostsforlighting,HVAC,motors,aircompressors,commercialrefrigeration,boilers,hotwatersystemsorinsulation.Fourhoursoffreesupportfromtheassessorisalsoprovidedtoassistwiththeinstallationofnewequipment.
Fromtheinitialpolicybrieftodatetheprogramhassignificantlyevolvedinresponsetochallengesemergingfromtheearlystagesoftheprogramimplementation,especiallythelowconversionratefromassessmentstorebates.Accordingtofeedbackfromprogramstaffthetimeframetodeveloptheprogramdesignwasveryshort.AprogrammanagerwashiredonlyinSeptember2008.Formativeresearchonbusinessesknowledgeandattitudesandsomepilotassessmentswereconductedtotesttheconcept.
Variousresearch,monitoringandevaluationactivitiesinformedchangestotheprogramdesign,inparticularregularmonitoringofassessmentsdata,externalauditsofservicesprovidedbyassessorsandparticipantssurveys.OEHcontractedDatabuildtoconducttwoparticipantssurvey,oneinNovember2010(230participants)andoneinJune2012(around300participants)toassessbusinesses’satisfaction,identifybarrierstoimplementrecommendationsandfinallyestimatetowhatextentdecisionstoretrofitbusinessescanbeattributedtoEESBP.Thesecondparticipants’surveyincludes
TheEnergySavingsSchemehasthepotentialtobothpartiallyfundsmallandlargeenergyefficiencyitemsandprovideabusinesscaseforthirdpartiestoconductassessmentsofhouseholds.
Energysavingsfromaone‐offbehaviourchangeactiondonotlastsotheEnergyEfficiencyActionPlanwillseeksnewwaystodeliversustainedsavingsfrombehaviourchangethroughtheEnergySavingsScheme.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
40
interviewswithprogramstakeholders(programstaff,assessors,serviceprovidersandexternalauditors).
3.2.2 Deliveryandreach
AlthoughEESBPwasoriginallyintendedtobea30monthprogram,itreacheditstargetof6,000participatingbusinessesinMay2010,only15monthsfromtheofferingofassessmentinFebruary2009.ByJune2012EESBPhad17,185registeredbusinesses,15,214ActionPlanspreparedandaround2,510retrofitsimplemented.Themainchangestotheprogramdesignhavesincefocusedonincreasingtheconversionratefromassessmentstorebate.
Changeswerealsomadeintheapproachtorecruitingbusinesses.ForreasonsofequityEESBPwasinitiallyadvertisedbyOEHandthroughcouncils.Theapproachthenmovedtoassessorsmainlyresponsiblefortheprogrampromotion,usingcommunicationmaterialprovidedbyOEH.Fixedpriceservicesprovidedbyorganisationsselectedbycompetitivetender(andplacedonaNSWPanel)meansthatassessorsareguaranteeda$600or$300assessmentforeverybusinesstheyrecruit.Thisinnovativeapproachensuresthatassessorshavea‘profitmotive’torecruitbusinessestoEESBPandithasbeenafoundationoftheoutstandingparticipationnumbers.
Duringthefirstandsubsequentyearsassessorswereofferedfurtherguidelinesandtraining.Theassessmenttoolwasrefinedonfouroccasionstoimprovethequalityofdataenteredandoffermorereportingfunctionalities.Furtherriskassessmentidentifiedadditionalrecommendations,includingcontractingoutauditstoimprovethequalityofassessments,implementedfromJanuary2011.
Programstaffidentifiedthekeychallengeasthelowerthanexpectedconversionratefromassessmenttotheimplementationof‘rebatable’recommendations.Programdatashowedatwopercentcumulativeconversionrateinmid‐2010.Afacilitationpilotwasconductedinearly2010wherebusinesseswereprovidedwithfourhoursofassistancetoimplementrecommendationsfromtheiractionplans,whichshowedanincreaseinconversionrateto51percent.Inresponse,significantchangestoprogramdesign(Phase2)weremadefromJuly2010:
reductionofassessmentscostforbusinesseswithasimplifiedone‐offfee fourhoursoffreesupporttohelpregisteredbusinessestoimplementtheaction
plan(facilitationtimepaidtoassessors) optionforbusinessestopayonly50percentoftheretrofitcostsupfront,OEH
payingtheremaining50percent 50percentconversionratetargetforassessorsbeforebeingallocatedmore
businessestoassess.
AnewpanelofassessorswascontractedinAugust2010toalignwiththeserequirements,especiallyhavingassessorsthathadrelationshipswithsupplierssotheycouldcoordinatemoreeffectivelytheretrofitcomponent.Thosechangesappeartohave
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
41
significantlyimprovedtheconversionratewhichprogressivelyincreasedtoaround30percentinearly2011anddidnotfallbelow50percentfromAugust2011.
Sincethattimetherehasbeenanimprovementintheconversionratesasaresultoftheprogram.Therecentindependentsurveyofparticipants(Databuild,June2012,n=301)examinedthisindetail.Itfoundthat93%ofbusinesseshadatleastpartiallyimplementedoneormorerecommendationofanytypeinbothphasesoftheprogram,dominatedbythehighlevelofactionfornocost/lowcostrecommendations.Theproportionofbusinessestakingnocost/lowcostactionshasfallenfrom93percentinPhase1to84percentinPhase2butthisshortfallhasbeenmadeupbyincreasedactionintheothertwocategories:
Justundertwothirdsofbusinesses(62%)haveatleastpartlyimplementedrebateeligiblerecommendationscomparedto40percentinPhase1.
47percentofbusinessesreceivingworthwhilelowcostrecommendationsatleastpartiallyimplementedoneormoreoftheseinPhase2comparedto37percentinPhase1.
Thestudyassessedtheattributionoftheseactionstotheprogramandfoundthatwhilethishasnotincreasedoverall,theproportionthatisfullyattributedhasincreasedfrom64to83percent.Fornocost/lowcostrecommendationsattributionoverallhasincreasedfrom55to78percent.Forworthwhilelowcostrecommendationsthelevelofattributionhasnotchanged.
Participantsatisfactionhasbeenhigh.IndependentsurveyresearchbyDatabuildinMay‐June2012foundthat78percentofbusinessesweresatisfiedorverysatisfiedwiththeassessorand80percentwiththerecommendations(n=301).Thisisinlinewithsimilarfindingsforphase1oftheprograminNovember2010(n=230)whereatleast90percentofrespondentsweresatisfiedwitheachaspectoftheprogram(theassessor,theactionplanrecommendationsandthefollow‐upsupport).The2010researchshowedthatthemajorityofinterviewedbusinesses(68percent)weremotivatedtotakepartintheprogrambythepotentialenergycostsavings.Barrierstoimplementingrecommendationsoccurredinsomecaseswheretheywerenotseenasappropriatebythebusinessesthemselves,orthelackofup‐frontcapitalformorecostlymeasures.Asaresultprogramchangesweremadetoimprovetheappropriatenessofrecommendationsintheactionplananddevelopbettercommunicationtobusinessesaboutfinancialassistance.
Auditsofassessmentsconductedsincemid‐2011havealsocontributedtoimprovequalityandaccuracyofenergyassessmentsaswellasverifyingretrofits.ToendofMarch2012,367assessmentsacross33assessorcompanieshavebeenauditedandfindingsprovidedtoassessorstoadoptcorrectingmeasuresifneeded.Theoverallqualityofassessmentsimprovedfrom‘average(70)’to‘aboveaverage(75)’onthe100pointsauditscale.Interestinglytheassistanceoflocalcouncils(inlessthan5%ofcases)improvedthequalityofthefacilitationforimplementingrecommendations.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
42
EESBPisincreasinglyusingquantitativeanalysisofthedataonparticipatingbusinessestoidentifypatternsofuptakeandconversionacrossindustries,sizeorlocationthatmayguidestrategiestotargetbusinesseswithhigherpotentialforsavings.Thisisunderwaywithkeybusinesstypeslikebutchers,poultryfarmers,smallsupermarketsandfastfoodoutlets.Inthesecaseshighenergyusersarebeingtargetedforspecialistaction,suchascoolroomsinbutchersandotherformsofrefrigerationinsupermarkets.Inthecaseofpoultryfarmers,commonopportunitiesarebeingidentified,withimplementationsupporttofocusontheseopportunities.Inallcases,theselocationandsectorwideapproachesarebeingputinplaceinpartnershipwithindustryassociationssuchasNSWFarmersortheAustralianMeatIndustryCouncil,andfranchisessuchasFoodworks.
Toencouragebestpractice,industrybenchmarkingisbeingestablishedbasedontheEESBPdatacollectedtodate.Inthedairyindustryforexample,datafrom440assessmentsacrossNSWhasenabledtheestablishmentofabenchmarkofenergyuseper1,000litresofmilk.Thisenablesdairyfarmerstocomparetheirenergyefficiencyagainstfarmerswithsimilarsizedherd.Anecdotalfeedbacksuggestsitisprovingtobeanimportantmotivatorforaction.
3.2.3 Outcomes
ByJune2012theprogramhadgeneratedanestimated37,400MWhofannualelectricitysavingsleadingtoasavingsinparticipantselectricitybillsofover$9.1million.Theenergysavingshavealsoresultedin39,640tonnesofavoidedCO2emissions(Table3.2).ConsideringtheprogramcoststoendofJune2012,theprogramisprovingtobeverycost‐effectivewithacostpermegawatthoursavedbyprogramparticipantsat$66comparedtoabenchmarkcostof$245(costinresourcesforretailelectricitytosmallbusinesses).
Table3‐2. EnergyEfficiencyforSmallBusinessProgram–estimatedsavingsforoneyearfromactivitiesimplementedtoJune2012
Reach Electricity saved(MWh)pa
CO2tonnessaved
Participants$savingselectricitybillspa
17,185businessesregistered15,214ActionPlans
37,396 39,640 $9,124,627
Source:StrategyandAnalysis,OEH
Notes:Thesavingswerecalculatedbyextrapolatingtheknownrateofrebateclaims(16.5%ofbusinesses)andaveragesavingsperrebate(5.2MWhperbusiness)tothetotalnumberofparticipantbusinesses;andestimatingimplementationofnon‐rebateactionsusingtheratesfromthe2012Databuildsurveyandextrapolatingtheaveragesavingsperbusiness(5MWh)fromnon‐rebateactionstothetotalnumberofparticipantbusinesses(32%ofworthwhilelowcostactionsarefullyimplementedduetotheinterventionofEESBP,basedon37%ofworthwhilelowcostactionsfullyimplemented,with87%ofthosecanbeattributedpartiallyorfullytotheEESBProgram).Thisestimateisconservativebecausethe2012Databuildsurveyshowsthatbusinesseshaveonlyclaimedrebatesfor51%ofimplementedrebatableactions.Thissuggeststhatthesavingsfromtherebatescouldbeashighas200%ofthoseclaimed.Thebillsavingsusethetariffsdevelopedforthe2010‐11ClimateChangeFundannualreport($244/MWh),andusedthroughout2011‐12.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
43
EESBPistakingpartina‘before‐and‐after’projectcoordinatedbyDEPtomeasureactualsavingsachievedbyparticipants.Thelatestresults(August2012)providedfindingsbasedon331projectsthatweresubjecttoM&Vanalysis.Onaverage5.64megawatt‐houror9.3percentenergysavingswereachievedfromeachproject.Thisanalysisalsoidentifiedsignificantdifferencesintheamountofsavingsaccordingtothetechnology:lightingretrofitsresultedinthebiggestsaving(13percent)comparedtorefrigerationthathadthelowest(3.2percent).BecauseofthebroadbasistheseestimatescanbeconsideredasrepresentativeforallEESBPparticipants.TheyconfirmpreviousestimatesofsavingsthatwereusedtoinformthereviewofdeemedsavingsaspresentedinTable3‐2.
Anecdotalevidencefromprogramstaffshowsthatbetterresultsinimplementingrecommendationsareachievedforsometypesofbusinesseslikehairdressers,cafes,dairiesorfranchises.
Feedbackfromprogramstakeholderssuggeststhatotherimpactsarebeingachievedforparticipatingbusinesses,includingincreasedknowledgeandexpertiseintheenergyefficiencyarea,butthesearemoredifficulttomeasure.
3.2.4 Conclusion
EESBPhasdevelopedaneffectiveapproachforworkingwiththesmallbusinesssector.ItbeganasanewprogramforNSWandthefirsttimesmallbusinesseshadbeendirectlytargetedinthisway.Itsreachhasexceededexpectationswith17,185registeredbusinessestoendofJune2012comparedwiththeinitialtargetof6,000participants,suggestingthatithasreachedbeyondtheearlyadopters.Initsearlystagesakeychallengewasthelowconversionratefromparticipationtorebatesforretrofits.However,significantchangestotheprogramdesignhadaverypositiveimpactonthecumulativeconversionratethatwentfrom2percentofregisteredbusinessesrequestingrebatesasattheendofJune2010to15percenttotheendofJune2012,withevidenceofcost‐effectiveenergysavings.Theprogramhasprovedtobeeffectiveandcost‐effectiveinworkingwiththesmallbusinesssector,withthepotentialforgeneratingsimilarresultsonalargerscale.
OEHComment
ThecomplementarityreviewinformedgovernmentdecisionstoreformtheEESBPbymergingitwiththeESPandceasingrebatesforenergyefficiencyactions.ThereviewemphasisedgreateruseoftheESSasafinancialincentiveforsmallbusinessestoimplementenergyefficiencyactions.
TheNSWEnergyEfficiencyActionPlanincorporatesthesedecisionsandevaluationfindings.Forexample,implementationratesincreasedramaticallywhenauditsarefollowedupwithtechnicalsupportandfinancialincentives.TheEnergySavingsSchemewillneedsimplemethodstostreamlineaccessforserviceproviderstosmallbusinesses,
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
44
3.3 EnergySaverProgram(ESP)
3.3.1 Designanddevelopment
TheESPtargetsmediumandlargesitesusingbetween160megawatts‐hourto10GWhperannuminelectricity.ItispositionedbetweentheSmallBusinessProgram(EESB)thattargetsbusinessesthatusebelow160megawatts‐hour(wheremostelectricitymeterscannotrecordintervaldata)andlargesitesusingabove10GWhthatareobligedtotakepartintheNSWEnergySavingsActionPlan(ESAP)program.
TheESPhadanoriginaltargetof800siteauditsbyJune2013,nowextendedtoJune2014.TheESPbudgetis$20millionoverthefiveyears2008/09to2013/14.
TheESPprovidessubsidisedenergyauditsandfacilitationtoguide,promoteandpreparebusinesscasesforinvestmentinenergyefficiency.Italsosetupapanelofenergyefficiencyauditorswhorecommendupgradesandimprovedprocedures,identifypotentialEnergySavingSchemefundingandsupportbusinessestoactontherecommendations.
Overtheperiodtodate,thestoryoftheESPhasbeenacontinuousadaptationoftheinitialsettingstoaddresschallengeswithreach,delivery,timelinessandturningauditsintoenergysavings.ThishasbeeninformedbyprogramexperienceandresearchcommissionedbyESPonotherenergyauditingprograms(InstituteforRuralFutures2011),andontheengagementofbusinessesintheESP(Databuild2011).
3.3.2 Reachanddelivery
Theprogrambeganinearly2009andhadaslowstart‐up,butbyJune2012itreportedtrackingwelltotargetwith366auditscompleted,approximately50percentatregionalorruralsites(Table1).Programexpenditureof$11.1millionwasonforecastfortheJune2014completion.
assite‐specificengineeringmeasurementofenergysavingsisnotfeasibleforsmallbusinesses.
Basedonthe15,000assessmentsundertakenbyJune2012,thetypesofenergyusageandopportunitiesforenergyefficiencywithinsectorsareknowntoberelativelyconsistent.Hence,identificationofopportunitiesandcalculationofsavingscanbestreamlined.
Akeyfindingisthatverifiedbillsavingsacrossalargenumberofsmallbusinessesdeliverequivalentorgreaterpeakdemandsavings.Futureprogramscanbuildonthisfindingtoimprovealignmentbetweenenergyefficiencyprogramsandmanagementofpeakdemand.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
45
Table3‐3. ESPuptakeandimplementationovertimetoendofJune2012
2009 2010 2011 2012 *
AuditsCompleted Yearly 14 82 119 151
Cumulative 14 96 215 366
Organisationsimplementedoneormoreprojects
Yearly 6 40 15 ‐
Cumulative 6 46 61 ‐
Implementationrate(implementedsavings/totalestimatedsavings)
Yearly 5.5% 11.3% 4.6% ‐
Cumulative 0.3% 5.0% 7.5% ‐
Source:OEH,ESPmonitoringdatabase,June2012;*toendofJune2012
AnearlyadaptationwastheeasingofrequirementsforEnergySaverparticipantstoundertakethewiderSustainabilityAdvantageProgram,whichincludedreplacingthe$2,000‐$3,000joiningfeewitha$500feeatsign‐up.In2010eligibilitywaswidenedtoincludegovernmentsites.Toaddressbarrierstoimplementingenergyefficiencyprojects,theESPaddedImplementationSupportin2010(below).Marketingmaterialswereconstantlyimprovedandgoodrelationsbuiltwithindustrybodies.Oneofthelargestdelayswithanenergyauditisthecollectionofenergydata,soinearly2010ESPaddeda‘kick‐off’meeting,facilitatedbytheenergyauditor,andmadethisapaymentmilestonefortheauditor.
Amajoradaptationwasmovingthefocusfromtargetingbusinessesingeneraltoalsoincludingasectortransformationapproachfor1)selectedtechnologies(lighting,HVACandindustrialrefrigeration,andin2012multi‐generationandsub‐metering);and2)keyindustrysectors(agedcare,registeredclubs,hotels,foodprocessors,schools).Thisallowsmorespecificmarketing,targetinghighenergyusers,andspecialisedauditsforsimilarbusinessesortechnologies.In2012ESPisfurtherfocussingtechnologyprojectsbyofferingtrainingontechnologicalsolutions,andhasreportedahighdemand.
In2011,ESPformedacollaborationwithCityofSydneyfortheSmartGreenApartmentsproject(target30audits)andthemediumbusinesspilot(12audits–tobeevaluated2012).
OnthesupplysidetheESPhasalmostdoubledtheauditpanelto46organisationswithover300individualauditors,asignificantproportionofenergyauditingconsultantsinNSW.ESPleveragesofftheauditorsthroughgrantingexclusiverightswhereanauditorbringsabusinesstotheprogram,providingpricingisconsistentwiththeirtenderapplicationandinlinewiththeaveragepriceforsimilaraudits.Businessesrecruitedbyauditorsnowaccountsforthemajorityofnewaudits.
Accordingtothe2011Databuildsurvey,mostparticipatingbusinesseshavebeensatisfiedtoverysatisfiedwiththeESPprocess(auditors80percent,ESPteam92percent,n=38‐39).Thisstudyalsofoundthatwhilebusinessessawsubstantialbarriersto
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
46
investinginenergyefficiencyprojects,theimportantfeaturesofESPwere:(i)credibilityofagovernmentprogram(ii)subsidisedaudits;and(iii)theresultingbusinesscasethatwasneededtosecureabudgetforinvestment.
Siteswithinthetargetgroupcanvaryinenergyusebyafactoroveraround60,sowhilethenumberofauditsisaKeyPerformanceIndicator(KPI)itdoesnotreflectthedegreeofuptake,themegawatt‐hourvalueofenergyassessedorlikelysavings.TheESPcanprovidemoredetaileddatabyindustrysectorsonreachanddeliveryfordifferentprojecttypesfromtheESPdatabase,butnofurtheranalysishasbeencontractedyet.
3.3.3 Outcomes
ByJune2012theprogramhadgeneratedestimatedannualsavingsof45,432MWhofelectricityand186,155Gjofgasleadingtoasavingsinparticipantsenergybillsofover$10.3million.Theenergysavingshavealsoresultedin60,367tonnesofavoidedCO2emissionsforoneyear(Table3.4).
Table3‐4. EnergySaverProgram–estimatedsavingsforoneyearfromactivitiesimplementedtoJune2012
Reach Electricitysaved
(MWh)pa
Gassaved(Gj)pa
CO2tonnessaved
Participants$savingselectricitybillspa
Participants$savingsgasbillspa
Participants$total
savingspa
386entitiesaudited
45,432 186,155 60,357 $7,996,068 $2,345,552 $10,341,620
Source:StrategyandAnalysis,OEH
Notes:Thesavingswerecalculatingby:identifyingacohortof113auditsthatwerecompletedmorethan18monthsearlier(datasetextracted19June2012)andhavereportedatleastonceweretakenasconservativelyindicativeofimplementationundertheprogram.Includingbusinessesthatreportednoimplementation,thisgroupachievedreportedaverageelectricitysavingsof5%andnaturalgassavingsof6%againsttheirbaselines(note:the6%figureincludesbusinessesthathadnonaturalgasuse,thiswasdonetoallowthefiguretobeappliedtoallbusinessesthathadparticipated).Theaveragebaselineforallauditscompletedmorethan6monthsearlierwastakentobetheaveragebaselineforallparticipants(averagebaselinesare2,401MWhand8,477GJ).The5%and6%savingswereextrapolatedtoallbusinessesthathavereceivedanauditundertheprogram(366at30June2012).Thisfigureisconservativebecause:theimplementationrateistakenfromearlierintheprogramfrombusinessesthatdidnotprovidemultiplereports.Itdoesnotreflecttheimpactofimplementationsupport,otherimprovementstoauditlayout,auditmanagementprocessesandbusinesscasequality,ortheintroductionofCommonwealthfundingforimplementation.Thebillsavingsusethetariffsdevelopedforthe2010‐11ClimateChangeFundannualreport(naturalgas,business‐$12.60/GJ;electricity,businesseswithmediumsizebaselines‐$176/MWh).
TheESPevaluationreportusesdatafromthecurrentprojectsandassumesa45percentimplementationratetoestimatesavingsinelectricityuseof122,400megawatts‐hourperyearacrossallthebusinesses,equivalenttosavingsoverthe10yearlifeoftheimplementedprojectsof1,224,000megawatts‐hour,leadingtoacost‐effectivenessof$16.33permegawatt‐hourofenergysaved.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
47
CasestudiesdemonstratethattheESPapproachcandeliversubstantialenergysavings.TheESPevaluationreportdescribesthesuccessoftheIndustrialRefrigerationTechnologyProjectwheresavingsupto70percentenergyconsumptionforrefrigerationwereidentifiedatvarioussites.Forexampletwolargeprojects(TamburlaineWinesandSwireColdStorage)aresavingatotalofover$123,500peryear.SimilarlytheLightingTechnologyProjecthasalreadyledtotwomajorbusinesses(SydneyMasonicCentre&Diageo)committingover$500,000incapitalinvestmentforlightingprojectswhichwillsaveover$275,000annually.TheM&VcasestudyofthelightingretrofitprojectinSydneyMasonicCentreisintheAppendix2.
Itisclearthattheprogramhasdevelopedamodelthatcanachievesubstantialenergysavingsandcutenergybillsatparticipants’sites.Butatthisstagethereisalowreportedrateofimplementationafteranaudit(10percentatMarch2012,13percentatJune2012),comparedwiththeexpectedrateofbetween40and50percentachievedoncompletionbyotherauditingprograms(ISF2011).Onefactorisgreatersupportforimplementation,whichtheESPaddressedbyaddingupto30hoursfortheauditteamstoassistbusinessesinimplementingtheauditrecommendations.
Moreimportantlythelowrateislargelyanartefactofreportingduetothelagtimeforenergyefficiencyprojectswhichtakebetween6‐24monthsormoretoimplement(forexampleprojectsundertheNSWEnergySavingActionPlanstook3yearsformaximumimplementationofauditopportunities).Aroundhalfofthe250completedESPauditshadonlyfinishedwithin6monthsofthe2012ESPreport.TheESPexpectsthattheimplementationratewillrisedramaticallyoverthecoming12monthsasbusinessesactontheexistingaudits,anditisontracktoreachthe40‐50percentachievedbyotherprograms.Reportingonimplementationisarelatedchallenge.Participantscommittoreportprogresspost‐auditbutitisavoluntaryself‐reportwithnoexternalverificationofcostsorsavings.OEHcreatedaRelationshipOfficerrolewithintheteaminmid‐2011tofillinthisgap,andgatherpost‐auditinformation.The95organisationswhohavesubmittedreports12monthsafterthe‘sign‐off’oftheirauditrevealaverageannualsavingsof$45,800.Thisisexpectedtoriseconsiderablyoverthenext1‐2yearsasthetimeandcapitalisfoundtoimplementadditionalcost‐effectiveenergyefficiencymeasures.
ESPisalsodevelopingmorerobustdataonsavingsusingM&V,withapilotinearly2011at10sites.Inmid‐2012ESPusedtheOEHM&Vguideandanindependentspecialistat10‐20sitestoconfirmimplementation,actualenergyandcostsavings,andcomparerealsavingswithinitialfindingsintheauditreports.TheexpectedfindingsshouldallowESPtomoreaccuratelyextrapolatefindingsacrossparticipatingbusinessesandsub‐sectors.
AsignificantpositiveimpactoftheESPhasbeenpositioningNSWbusinessestoattractfundingforfurtherenergyefficiencyactivities.ForexampleinJune2012fourESPparticipantswongrantsinthefirstroundoftheCommonwealthCleanTechnologyInvestmentProgram(CTIP),receiving86%ofthenationalallocation,andafurther20sitesareapplying.AccordingtoAusIndustryakeyfactorwasthecomprehensiveESPauditthatcoveredtheCTIPcriteriaanddeterminedthebestoptions.Anotherexample
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
48
islinkingeachsitetoanACPwithintheEnergySavingsSchemetogeneratingEnergySavingCertificates(ESCs)forbusinesseswhichimplementprojects.
3.3.4 Conclusion
TheESPprogramhasbeenoperatingeffectivelydespiteaslowstart‐up;growthandprogramparticipationhasbeensteadyandestimatesindicatethattheprogramisontracktocost‐effectivelymeettargets.Manychangeshavebeenmadetomaketheprogrammorestrategicandtargeted.
Itisclearthattheprogramhasacost‐effectivemodelthatcanachievesubstantialenergysavingsandcutparticipants’energybills.Theextentofsavingsislikelytobeclearerasmorebusinessesimplementauditrecommendationsovertime,andtheprogrammeasuresandreportsonthis.TheincreaseduseofM&VshouldprovidefirmevidenceofenergyandcostsavingsandallowESPtomoreaccuratelyextrapolatefindingsacrossparticipatingbusinessesandsub‐sectors.
OEHComment
ThecomplementarityreviewinformedgovernmentdecisionstoreformtheESPbyintegratingitwiththeEESBPandthenon‐duplicativecomponentsoftheEETP.ThereviewalsoemphasisedgreateruseoftheESSasafinancialincentiveforallbusinessestoimplementenergyefficiencyactions.
TheNSWEnergyEfficiencyActionPlanincorporatesthesedecisionsandevaluationfindings.Someofthesearesimilartothefindingsforsmallbusiness,forexample,theconsistencyofenergysavingsopportunitieswithinbusinesssectorsandtheopportunitytostreamlineaccesstotheEnergySavingsScheme.Streamlinedmethodsalsoprovideanopportunitytoovercomethedelaysinprojectimplementationfollowingstand‐aloneauditswithoutfinancialincentives.
Animportantdifferencetothesmallbusinesssectoristhatagreateruseofmeasurementandverification(M&V)isrequiredtoprovideevidenceofenergysavingsandanongoingbusinesscaseforinvestmentinenergyefficiency.
3.4 EnergyefficiencycomponentofNSWGovernmentSustainabilityPolicy(EEGSP)
3.4.1 Designanddevelopment
TheEEGSPisdifferentfromotherenergyefficiencyprogramsinthatitisnotaprogrambutapolicycoveringarangeofprogramsthatsettargetsforNSWGovernmentagenciesfor:greenhousegasemissionsfrombuildingenergyuse,water,environmentalperformanceofbuildings,cleanergovernmentfleet,waste,recyclingandpurchasing.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
49
FiveprogramsrelatingtoenergyconsumptionwereconsideredtobepartoftheoverarchingEnergyEfficiencyStrategy:
TheSchoolsEnergyEfficiencyprogram NABERSratingofofficebuilding Greenleaseschedulesfortenantedbuildings TreasuryLoanFund–SustainableGovernmentInvestmentProgram(TLF‐SGIP) GovernmentBuildingRetrofitProgram(GBRP)
Whileindividualprogramsweredevelopedtoaddressspecificpurposesanddifferinsettings,targetgroup,timeframeandrequirements,theyhadsomecommonfeaturesandweregroupedundertheEEGSPumbrellawithcoordinationandfacilitationthroughOEH.AllfiveprogramsaretargetingNSWGovernmentagenciesthatwererequiredtomeetwholeofgovernmenttargetstoreducegreenhousegasemissions.OEHprovidedexperttechnicaladvicetogovernmentagencies,tohelpthemtakecosteffectiveactionstomeetthetargets.
ThissynthesisoffindingsforEEGSPprogramsismainlybasedonaself‐assessmentreportdraftedinMarch2012byOEH.Whileinformedbystakeholders’feedbackandmonitoringdataavailableforeachprogramthismainlyreflectsaninternalperspectiveonthestrengthsandweaknesses.Therewouldbebenefitinamorecomprehensiveindependentevaluationthatcouldbuildupexistingmonitoringandevaluationinformationexistingatprogramlevel.
3.4.2 Progressofindividualprograms
SchoolsEnergyEfficiencyProgram
TheSchoolsEnergyEfficiencyProgramprovidesfundingtohelpreducegreenhousegasemissionsfromNSWpublicschools(primaryandsecondary).The$20millionprogramisopento225NSWpublicschoolsuntil2012.Theprogramofferslightingretrofitforeachparticipatingschoolandupto$18,000perschoolforstudentstoselectandfundtheirownenergyefficiencyprojectsthroughtheStudentSavingsFund.7ImplementationiscoordinatedbytheDepartmentofEducationandCommunities(DEC)withtheassistanceofOEH.Deliveryisconductedthroughsub‐contractors.
ThedesignisaturnkeyprogramtoaddresstheageinglightinginfrastructureinNSWGovernmentschools,particularlyforlightfittingswithlampsnolongermanufactured.ParticipatingschoolswerechosenbytheDEC,basedontheirneedforalightingupgrade,thoughthosealreadyearmarkedforupgradesthroughforwardcapitalprogramswereexcluded.
7http://www.environment.nsw.gov.au/grants/ssep.htm
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
50
AccordingtoOEHfeedback,theprogramwasdeliveredin‐timeandonbudget.Atotalof126schoolswillhavereceivedlightingretrofitsbyJune2012,whichrepresentshalfoftargetedschools.Deliveryfacedsomedifficulties,especiallyinrelationtotheinstallationofnewlights.Installationhadtooccurduringschoolholidaysandatthebeginningoftheprogramitwasnotpossibletousethefirstsetofholidaysbecauseanumberofapprovalshadtobesoughtbeforehand.Forthefinalyear(2012)theprogrammadesuretoobtainstakeholderapprovalearlierthanfundingapprovalinordertobeabletostartinstallationduringthefirstschoolholidays.Feedbackfromparticipatingschoolssuggeststhatparticipantsaresatisfiedwiththeprogram.OnebarrierwaswhereimplementationoverlappedwiththeCommonwealth’sBuildingtheEducationRevolutionprogramwhichresultedinsomesitesbeingpartiallyretrofittedbyeachprogram.ImprovedcommunicationbetweenCommonwealthandStateprogramdeliverersreducedthisoverlap.
Intermsofoutcomes,estimatedannualsavingsacrossthe126schoolsare$1,330,000offpowerbills(schoolsareonlyinuse200outof365daysayear,whichincreasestheprojectpaybackperiod),6,150megawatts‐hourofelectricityandmorethan6,500tonnesofcarbonpollution.Howevernoevidenceisavailableonactualsavings.
OEHComment
TheSchoolsEnergyEfficiencyProgramisnowcompleted.ResultsfromitwillinformtheDepartmentofEducationandCommunitiesontheviabilityoffutureschoolenergyefficiencyprojectstobefundedundertheSGIPorthroughcapitalexpenditure.
NABERSratingsofofficebuildings
TheEEGSPsetcleartargetsinrelationtoenvironmentalperformanceofbuildingsforgovernmentownedortenantedofficebuildingsover1000m²to
obtainaNABERSratingby31December2008, achieveandmaintainaNABERSratingof4.5starsforenergyandwaterby1July
2011,wherecosteffective,and whereneworrefurbished,achieveandmaintain2011targetsfrom18monthsof
thefirstoccupancy,wherecost‐effective.
OEHidentifiedofficesrequiringNABERSrating(NationalAustralianBuiltEnvironmentRatingSystem)buteachagencywasresponsibleforobtainingtheirNABERSratings.Tomeetthosetargets,agencieswereofferedsupport.TheStatePropertyAuthority(SPA)establishedapaneloftwoNABERSassessorsthroughatender,whichwasaccessibletoallthesites,andOEHprovidedadviceandfacilitation.AccordingtoOEHfeedback,effortsremainedfocusedonhavingallthesitesobtainaNABERSrating.HoweverattheendDecember2011only77percentofthesitehaddonesodespitecontinuousimprovementfrom2007/08(seefiguresacrossfinancialyearsuntilendofJune2011in
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
51
Table3‐5).OEHstaffidentifiedseveralreasonsthatcouldexplaintheslowuptakeinrequiredsitestoobtainratings.
DelaysintheinitialreleaseoftheEEGSPincludingtheNABERStargetmeantthatthedeadlinewasunrealistic.
AnumberofagenciesinitiallybelievedthatSPAwasgoingtoorganisetheirratings.Clearandearlycommunicationdirectlywiththesitesrequiredtoobtainaratingwouldhavemitigatedthis.
Table3‐5. NABERSratingsobtainedfrom2007/08to2010/11
NABERSratings 2007/08 2008/09 2009/10 2010/11
Numberofratings‐cumulative 35 66 75 100
ProportionofeligiblesitesNABERSrated 18% 40% 45% 69%
Source:OEH,NABERSextract,June2012
OEHidentifiedsomekeysuccessfactorsthatcontributedtoimprovetheuptakeaccordingtothesitescharacteristics.
LocalsitecontactshaveproventobethemosteffectiveatfacilitatingNABERSratingsforsmallagencies
CentralcontactswerethekeycontactforlargeagenciessuchasNSWPolice Letterstoexecutivesofagencieswhohadfailedtoobtainratingsresultedinan
increaseinactivity.
WhenconsideringthesecondtargetthatshouldhavebeenachievedbyJuly2011,only37percentofthe100ratedsiteshadmetorexceededthe4.5startargetasofDecember2011.
Thereisverylittleevidenceofoutcomesintermsofreducedenergyconsumption.Theratingmayhaveprovidedmotivationforeachsitetoreducetheirenergyconsumptionbutanecdotalevidencesuggeststhattheratingalonedoesn’tprovideapathwayforimprovement.
GreenLeaseSchedules(GLS)fortenantedbuildings
TheEEGSPincludedanothertargetrelatedtoenvironmentalperformance:allgovernmenttenantedbuildingshadtoincludeaGreenLeaseScheduleinallnewornegotiatedleasesorwhenexercisingaleaseoption,wherepractical.AGreenLeaseScheduleisaformalcommitmenttoenergyefficiencyandsetsaminimumongoingoperationalbuildingenergyperformancestandardmeasuredbytheNABERS.
TheStatePropertyAuthority(SPA)hadresponsibilityundertheEEGSPto‘implement’GreenLeaseSchedulesatthosetenanciesinwhichSPAwastheheadlessee.OthertenantedsiteswereabletoaccesstheGreenLeaseschedule.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
52
AtMarch2012,noGreenLeaseScheduleshadbeenattachedtotheleaseofaNSWGovernmenttenant.OEHstaffidentifiedsomekeyreasonsforthis.
AgaindelaysintheinitialreleaseoftheEEGSPincludingNABERStargetsmeantthatthedeadlinewassignificantlycondensed.ThiswasalsoduetothedevelopmentofaNationalGreenLeaseScheduleatthesametime.SPAdelayedthereleaseoftheNSWscheduletoensureitalignedwiththeNationalGreenLeaseSchedule,whichintheendhadlowerrequirementsthantheNSWschedule.
TheGreenLeasesufferedfromthecompetinggoalofreducingrentsforcontractmanagers:becausebuildingownersperceivedGLSasasourceofadditionalcoststheydroppeditearlyinleasenegotiations.
Inlate2011,SPAsentalettertoOEHsayingthatbycreatingtheGreenLeaseScheduleithadfulfilleditscommitmentto‘implement’,althoughnoGreenLeaseScheduleshadbeenattachedtoNSWGovernmentleases.
AccordingtoOEHstaffanimportantareaforimprovementforthisprogramisthecommunicationandcollaborationwiththeSPAthatcouldhavebeenbettertoensureanappropriateprogramdesignandeffectiveimplementation.
TreasuryLoanFund–SustainableGovernmentInvestmentProgram(TLF‐SGIP)
TheTreasuryLoanFundhasbeendesignedtohelpagenciestomeettheup‐frontcostsofimplementingenergyefficiencymeasures.Itprovidesgovernmentagenciesaccesstofundingtoimplementcosteffectiveenergyandwaterefficiencyupgradeswithborrowingsfromthefundrepaidwiththesavingsgenerated.Eligibleagenciesarebudgetdependentagencieswithbuildingupgradeprojectstotalling$10,000‐$500,000($1,000,000forNSWHealth).
OEHprovidessupporttoagenciesintheformoftechnicalassistancetoidentifycosteffectiveactions,auditsandbusinesscasedevelopment.OEHorganisesworkshopsandone‐on‐oneassistancetoagencies.Acontractorwasalsoengagedin2009tomaketheapplicationformeasiertofollow.TLF‐SGIPapplicationsroseslightlyfrom2010‐11asshownintheFigure3‐1below.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
53
Figure3‐1. TreasuryLoanFunduptake
0
5
10
15
20
25
30
1998‐99 1999‐00 2000‐01 2001‐02 2002‐03 2003‐04 2004‐05 2005‐06 2006‐07 2007‐08 2008‐09 2009‐10 2010‐11 2011‐12(so far)
2012‐13
$ m
illions
Funds drawn from the Treasury Loan Fund for retrofits (as at 1 May 2012)
Actual $(Million)
Projected
Source:OEH,TLFexpenditure,May2012
Feedbackfromagenciesidentifiedtwomainbarrierstoloantakeup:thelackoftechnicalknowledge(engineeringandfinancial)andtimeorstafftopreparethecomplexapplicationform,butespeciallythelackofresourcestoidentifyappropriateopportunitiestobefunded.ToaddressthisOEHdevelopedtheGovernmentBuildingRetrofitProgramfromJuly2010tosupportthisfirststepforagencies.UptakeoftheTLF‐SGIPbyNSWGovernmentagenciesforbuildingretrofitswentfromlessthan$84,000overJuly2007‐June2010toover$2.5millionin2011/12asaresultoftheGBRP.AccordingtofinancialdatafromprojectsthatwereatanadvancedstageofplanninginMay2012itisexpectedthatagencieswilldraw$23.9millionfromthefundin2012/13(Figure3‐1).
OEHstaffidentifiedanotherkeyrequirementtoimprovetheuptakeoftheTreasuryLoanFund:theagencyfinanceapprovalchainhastobeactivatedasearlyaspossibletoensurethatprojectsidentifiedinafinancialyearcanprogressinthatsameyear.
GovernmentBuildingRetrofitProgram(GBRP)
In2010,theMinisterfortheEnvironmentapprovedthe2yearGovernmentBuildingRetrofitProgram(GBRP).Theprogramhastwostreams,onethatcoversauditcostsforlargesitesontheprovisiontheagencyagreestoapplytoTLFfunding;andtheotherthatcoverbothauditandimplementationcostsforsmallfrontlineservicedeliverysites(thathasbeenpilotedintheLowerHunter,IllawarraandCircularQuayprecinctuntiltheendofJune2012).Underthelargesitesstream,OEHcoordinatesandfundsanaudittoidentifycost‐effectiveprojects.TheresultingauditreportprovidestheagencywiththespecificationsandbusinesscaserequiredintheTLF‐SGIPapplicationform.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
54
Facingtheriskthatparticipatingagenciesmaynotprogresstoimplementation,OEHlearnedfromtheexperienceoftheWaterandEnergySavingActionPlan(WESAP)thatwascompulsoryforlargeenergyandwaterconsumersinNSWfrom2007.FifteenNSWagencieswererequiredtodevelopaWESAP;theyidentified77cost‐effectiveactionsbutonly8wereimplemented.Themainreasonwasthattherewasnorequirementtoimplementthoseactionsandannualreportingrequirementscouldstillbemetbyreportingnoactivity.ToavoidsuchresultsOEHchangedtheinitialprogramdesigntorequiretheagencytocommittoapplyforaTLF–SGIPloantoreceivetheaudit.Theprogramalsoinitiallyunderestimatedthetimerequiredforagencyapprovalswhichresultedinsomedelaysincommencingworksandimplementation.Thisprocessisnowlaunchedasearlyaspossible.
AsattheendofMarch2012,112smallsiteshadbeenaudited,butsevenauditsdidnotprogresstoapprovedretrofits;implementationhadcommencedorbeencompletedin7;20applicationsforlargesiteswerealsointheapplicationprocessfortheTreasuryLoanFundatthattime.Feedbackfromagencyrepresentativesisverypositive:theyappreciatedtheexpertiseandfundingprovidedtoundertakebuildingretrofitsandultimatelysavetheorganisationmoney.
Keysuccessfactorsofthisprogramhavebeentoprovideagencieswithaturnkeyprograminordertocompensatetheirlackoftechnicalknowledge,timeandbudgetforauditsaswellastofocusonlyonasmallnumberofactionsofferingopportunitiesforgreatersavings.Identifyingthekeysitecontacthasalsogreatlycontributedtosuccessfulimplementationastheyhavethesiteknowledgesuchasmaintenanceandcontractrequirements.
TheGovernmentRetrofitProgramistheonlyEEGSPprogramthathasresultsavailableonactualsavingsachievedbyparticipantsandverifiedthroughsbeforeandaftermeasurement.MeasurementandVerification(M&V)hasbeenappliedtotwoprojects,chillerretrofitinWestmeadHospitalandlightingretrofitinSydneyOperaHouse.SavingsfortheWestmeadHospitalrepresented12.4percentreductioninchilledwaterplantenergyuseanda3.6percentreductionoftotalsiteelectricityuse.TotalelectricitysavingsfortheSydneyOperaHousewasestimatedintherangeof3to5percentoverallbyaddingupsavingsachievedfromsixspecificareas.TheCentralPassageisoneoftheseareasandithassaved32percentinelectricityconsumptionasconfirmedintheM&Vtrial. WhiletheexamplesarenotrepresentativeofallGBRPprojects,theyprovideanindicationofsavingsparticipatingagenciescanachieve.
ByJune2012thesmallsitescomponentoftheGovernmentBuildingRetrofitProgramhadgeneratedestimatedsavingsof4,011MWhofelectricityand1,561Gjofgasleadingtoasavingsinparticipants’energybillsofover$.657million.Theenergysavingshavealsoresultedin4,354tonnesofavoidedCO2emissions(Table3.6).
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
55
Table3‐6. GovernmentBuildingRetrofitProgram–smallsitescomponent‐estimatedsavingsforoneyearfromactivitiesimplementedtoJune2012
Reach Electricitysaved(MWh)pa
Gassaved(Gj)pa
CO2tonnessaved
Participants$savingselectricitybillspa
Participants$savingsgasbillspa
Participants$totalsavingspa
105sites,567projects
4,011 1,561 4,354 $637,749 $19,669 $657,418
Source:StrategyandAnalysis,OEH,fromprogramdataprovided22June2012
Notes:Thesavingswerecalculatedforthe105governmentownedsitesreceivedasiteassessmentanddetailedspecificationsforworksandpotentialenergysavings.Siteassessmentsidentified567energysavingmeasures.Implementationisunderway/expectedforallidentifiedmeasures.Thebillsavingsusethetariffsdevelopedforthe2010‐11ClimateChangeFundannualreport(naturalgas,government‐$12.60/GJ;electricity,government‐$159/MWh).
3.4.3 Overalllearnings
Fromtheperspectiveofcoordinatorandfacilitator,OEHstaffidentifiedoveralllessonsfromtheimplementationoftheGovernmentSustainabilityPolicyasanensembleofenergyefficiencyprogramstargetinggovernmentagencies.
Thedesignofthefiveprogramsunderthisumbrellaprovedtobenotequallyappropriatetoreachthegovernmentagenciesandeffectivelyengagethemintheprogramrationale.Settingtargetswasnotsufficient.‘Outclauses’includedinthepolicy,suchas“wherepractical”or“whererelevant”meanstargetwerenotmandatory.Basedonfeedbackfromagencies,OEHstaffidentifiedthathavingaclearpolicymandate,agency‐specifictargetsandaccountabilityrequirementspossiblywithpublicreportingwouldsupportthesuccessfulimplementationofprograms.
Effectivenessofimplementationoftheprogramsvariedsignificantly:somehadsomegoodresultsliketheGovernmentRetrofitBuildingProgramwhilesomeothersdidnotproduceanyimpactliketheGreenLeaseprogram.Allprogramshadcommonbarrierstoengageeffectivelywithagencies:thelackoftechnicalknowledge,timeandfinancialresourcesforauditstoapplyforthesupportoffered.Somestrategiesprovedtobemoresuccessful.
Targetandengagewiththerightcontactsoneachsiteandwithinthefinancialchainhasbeenakeyrequirementforprogram’suptake.
Getearlyapprovalfromdecisionmakersateachlevelisessentialforsmoothdelivery.Theapprovalprocessescantakealongtimeandshouldbeanticipatedintheprogramdeliverytimeframe.
EEGSPmanagementidentifiedanotherkeyareaforimprovement:EEGSPprogramswouldbenefitfrombettercoordinationandalignmentwithotherNSWenergyefficiencyprograms,forexampletheGreenSkillsTrainingProgram,tobuildcapacityofagencies
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
56
staffortheEnergySavingsSchemetominimisetheprogram’supfrontcostsforparticipatingagencies.
OEHComment
ThecomplementarityreviewinformedgovernmentdecisionstoretaintheEEGSP.TheGBRPandSchoolEnergyEfficiencyProgramwereceasedasplannedin2012.
TheNSWEnergyEfficiencyActionPlanincorporatesthesedecisionsandevaluationfindings.
Thefindingshighlightedbarriersthatmayslowuptakeofgovernmentenergyefficiencyprojects.Forinstance,whiletheTreasuryLoanFundcanprovidefinanceforgovernmentenergyefficiencyprojects,itonlyappliestoasmallnumberofagencies,hasrulesthatforceagenciestorepayloansbeforeprojectsareimplemented,andhascomplexadministrativeprocedures.
Thefindingsindicatedthatuptakeofenergyefficiencywithingovernmentcouldbeimprovedthroughearlycommunicationbetweengovernmentagenciestoensureinternalapprovalsareobtainedandthatthereisexecutivebuy‐inforprojects.Existingprogramsthathavenomandatedtargetsorpenaltiesalsosufferfromsignificantnon‐compliance,indicatingthatagenciesneedacleardriverforprojectimplementation.
AswiththeEnergySaverProgram,thefindingsindicatedthatthereisaneedforstrongmeasurementandverificationofenergyefficiencyprojectsinthegovernmentsectortoestablishevidenceoffinancialbenefitforongoingparticipation.
3.5 EnergyEfficiencyTrainingProgram(EETP)
3.5.1 Designanddevelopment
TheEETPisjointlymanagedandimplementedbyOEHandtheNSWDepartmentofEducationandCommunities(DEC).TheEETPhasabudgetof$20millionacrossfiveyearsandcommencedinJuly2009.Theprogramaimsatbuildingtheknowledgeandskillsofkeytradespeopleandprofessionalswhofacilitateanddeliverenergyefficiencypractices,productsandservicesinNSW.LiketheCommunityAwarenessprogram,EETPisalsointendedtosupportthedeliveryofotherNSWenergyefficiencyprogramsinensuringthattheappropriateskilledworkforcewillbeavailabletodeliverthoseprogramsintothefuture.
Theprogramwasdesignedtoaddressskillsshortagesthatcouldcontributetobottlenecksinthemarketforenergyefficiencyservicesandcreateupwardcostpressuresandsupplydelays,obstructingtheuptakeofenergyefficiencymeasuresinNSW.EETPwasintendedtotacklefurtherkeyissues.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
57
Thecurrenttrainingofferforenergyefficiencyisfragmented,andsupplyanddemandaremismatched.
Energyefficiencytrainingcontentisoftennotindustry‐relevantorreachingqualitystandards.
Industryawarenessofandsupportfornewmarketopportunitiesassociatedwithenergyefficiencyarelimited.
ToaddresssuchissuesEETPtargetedprioritysectors,rolesandprofessionsasidentifiedbynationalresearch(NationalCentreforSustainability,2009,MatchingEnergyEfficiencyPolicyandTrainingandJobs).Theseindustrysectors,rolesandprofessionshavebeenconfirmedaskeytoimprovingenergyefficiency(TheAllenConsultingGroup,2012ReviewofEnergyEfficiencySkillsDemandsandTrainingProvisionacrosstheTradesandProfessions.Asaresultofthenationalresearchandconsultation,theEnergyEfficiencyTrainingProgramisstructuredaroundfivecomponents:
– VocationalEducationandTraining(VET)– ProfessionalDevelopmentforVETeducatorsandtrainers– IndustryPartnershipProjects– HigherEducationprojects– ResearchandEvaluation.
TheprogramcontractedUrbistoconductacomprehensiveindependentevaluationoverthreeyears.Yearoneevaluationreport(July2011)focusedonthedevelopmentoftheevaluationframeworkandtheassessmentofEETPpartnershiparrangements.Yeartwoevaluationreport(draftversioninJune2012)presentsfindingsandrecommendationsabouttheprogramdeliveryandoutcomestodate.Themainmethodsusedwerestrategicworkshopswithprogramstaff,interviewswithexternalstakeholders,documentaryreviewandanalysisofdataprovidedbyOEHandDEC.
3.5.2 Deliveryandreach
ThefirstninemonthsoftheprograminvolveddevelopingthepartnershiparrangementbetweenOEHandDECasasoundplatformforsharedprogramdelivery.InthisinitialstageEETPexperiencedchallenges,especiallyinrelationtodifferencesbetweenthetwomanagingagenciesinprogramfocusandoperatingphilosophy.Asanexample,DEC’scentralmandateistosupportaccreditedtrainingthatleadstorecognisedqualifications,whilstOEHhassoughttotargetdirecttrainingtomeetbusinessrequirements.ExtensivediscussionsandnegotiationsresultedinasharedunderstandingandagreedprioritiesfortheProgram.
Internalstakeholdersnowagreethatthepartnershiparrangementhascontributedkeystrengthsoftheprogram:combinedcapability,expertiseandnetworks,formalstructuretofocusopportunities,resourcesharingandcapacitybuilding,strengthenedrelationshipsandunderstandingacrossagencies,aswellaseconomicandstrategic
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
58
benefitssuchasgoodwillandcooperationingovernment,oraccesstofundingandresourcesthatwouldnothavebeenavailableotherwise.
ByApril2012stakeholdersreportedthattheprogramhadincreasedthecapacityoftheeducationsectortoprovideenergyefficiencytrainingtoprioritysectors,tradesandprofessions.Fundingsupporthasfacilitatedthedevelopmentofasuiteoftrainingcoursesandmaterials,andtheProfessionalDevelopmentcomponenthastargetedskillsdevelopmentfortheeducationworkforcetodeliverthetraining.Theprogramassistedinthedevelopmentofanumberofpilotcoursesandtrainingresourcesthathavebeensuccessfullytakenupbyindustryassociations,individualcompanies/employersandthevocationaltrainingsectorforongoingdeliveryand/orintegrationintoexistingtrainingcourses.Throughwell‐designeddisseminationandknowledge‐sharingstrategies,includingplacingontheOEHwebsite,EETPresourceshavebecomeavailableacrossindustrygroups,individualcompanies/employers,RegisteredTrainingOrganisations(RTO)andpractitionersnotdirectlyinvolvedinpilottraining.
EETPmanagementnotedthelargeeffortneededtoensurehighlevelsofquality,butlimitedcontrolonimplementation.Thebiggestimpactonthequalityofcourseswasthelevelofinvolvementandexpertisebyindustryandtrainersandenergyefficiencycontentspecialistandindustry/businessconsultation.Thisvariedstronglyfromlowtoveryhighcommitment,expertiseandresources.
BetweenJuly2009andApril2012,atotalof5,290participantshadundertakenenergyefficiencytraining,networkingorprofessionaldevelopment,including:
2,695committedplacesintheVETsystem 179participantsinthePD4VETworkshops 629participantsinotherVETPDtrainingornetworkingevents 1,233participantsin30IndustryPartnershipsprojects 544participantsinHigherEducationcourses
ThefollowingtablegivesanoverviewofthereportedlevelofinterestanddemandforthetrainingsandcoursesassummarisedintheEETPyear2evaluation.
Table3‐7. Reporteddemandanduptakecomparedtoexpectations
Component Reporteddemandanduptake
VET Demandappearstohavegrownstronglyinanumberofcourses,suchascarbonaccounting(105coursesfromJuly2009toApril2012)andcarbonmanagement(499courses).
Instabilityinregulatorydriversandgovernmentprogramshasreducedtheongoinglevelofdemandinothercourses(suchassolarPV).
Spendandenrolmentdatashowsstronggrowthoverthelasttwoyears,withDECmanagingtoexhaustthefullbudgetfortrainingsubsidies.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
59
Component Reporteddemandanduptake
VETPD DemandforsubsidedplacesintheVocationalGraduateCertificateatSwinburnehasoutnumberedtheavailableplaces.
UptakeanduseofthePD4VETresourcesbypractitionersandRTOsmaynothavebeenasstrongasanticipated.
IndustryPartnerships
Thelevelofindustryandemployeeinterestinthetrainingprojectshasvaried. Anumberofprojectsreportedhighlevelsofindustryengagementand
participation. Othersindicatedthatfillingthecoursehadprovedchallenging,includingcases
wherethosewhoinitiallysignedupbutwereunabletoattendduetoworkcommitments.
HigherEducation
AnumberoftheengineeringPDcourseshavebeendelayedduetoinsufficientenrolments.
Demandhasbeenhighestamongundergraduateandpostgraduatestudents(evenforcoursesdesignedprimarilyforworkingengineers)
Thefirstsetofaccountingcoursespilotedhavehadreasonablystronguptake,withflexibledeliverymodes(interactiveseminars)reportedtobemorepopularthanthelongercourse.
Source:Urbis,EvaluationoftheEnergyEfficiencyTrainingProgram–Year2Annualreport,Draftreport,June2012
Theevaluationidentifiedthetimerequiredtobeawayfromworkasthemainbarrierforuptakeandparticipation.Instabilityintheregulatoryenvironmentandothergovernmentprogramshasalsoreducedthedemandinsomesectors.Programstaffandprojectpartnersdevelopedstrategiestoaddressthosechallenges,especiallyindevelopingflexiblemodesoftraininglikeonlinemodulesorthroughcloseengagementwiththeindustrytoalignthecoursestotheexistingindustrycultureandtrainingmechanisms,e.g.accreditedtrainingorContinuingProfessionalDevelopment(CPD)requirements.
Theprogramstartedwithabroadapproachandatthisinterimstageislookingtohavemorecontroloverthedeliverymechanismsandmorestrategicfocuswiththestakeholdersthathavebeeninvolvedintheprogram.
3.5.3 Outcomes
TheevaluationfoundthattheIndustryPartnershipandUniversityProjectshavecontributedtowardsastructuralchangewithinindustryandtheeducationsectorwithrespecttoawarenessandunderstandingoftheopportunitiespresentedbyenergyefficiency.FeedbackfromparticipantsontherelevanceandqualityoftrainingdeliveredwaslargelypositiveforIndustryPartnershipprojectsandmixedforVETcourses.
Themajorityofparticipantsforbothtypesofprojectsintendtoapplytheirlearninginpractice.AcommonlynotedcoretrainingoutcomeacrossallEETPcomponentsisparticipants’increasedconfidencetoengagewithenergyefficiencyandcapacitytoworkwithenergyefficiencytechnicalexpertstoensureadviceisappliedappropriatelytotheirbusiness.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
60
SomebarrierstotheapplicationoflearningacrossVETandIndustryPartnershipshavebeenidentified:lackofmanagementsupportintheformofcommitmentfromseniorlevelsoftheorganisationandprioritisationofenergyefficiencyinitiatives;timeandresourcinglimitations;challengestodemonstratethebusinesscaseforchange;andashort‐horizonforbusinessdecision‐making.Possiblemechanismstoaddresssuchbarriershavebeenidentifiedsuchasensuringsufficientcustomisationtothebusinesscontext,includingworkplaceprojectsinthetraining,involvingdirectlythemanagementinthetraining,continuingasystemsapproachtotraining(targetingbothtechnicalskillanddecision‐makingrequirements),andprovidingend‐to‐endtrainingandimplementationsupport.
TheevaluationdidnotreportontheextentthatEETPhasatthisstagebeenabletosupportadevelopmentofaskilledworkforceinvolvedthedeliveryofotherNSWenergyefficiencyprogramsasittakesyearstoincreaseskillsandseetheresultsintheworkforce.
Atthisstage,whilemanyofthenewtrainingcourseswereonlydeliveredinthepast18months,thereissomeevidencethatanumberoftrainingparticipantshavedemonstratedchangedorganisationalpractices,operationsand/ortechnologiesandsomewereabletoestimateormeasureresultingenergysavings.ForexampleoneIndustryPartnershipsprojectproponentsaid‘We’reintheprocessofmeasuringsavings–nominallyexpectthesetobearound$1,200amonthbutmightactuallyrealisealotmorethanthis.We’rethenhopingwecanexpandthistootherplantsaroundNSW.’
Otherexamplesofsavingshavebeendocumentedincasestudies,forexampletheGPTtrainingandthatconductedforCommercialKitchens(ontheOEHwebsite).Outcomesfrommorerecentprojectshavehighlightedtheenergyefficiencyinitiativesundertakenandtheresultingenergyreductionse.g.UniversityofWollongongreportedsavingalmost$200,000perannumfromanITshutdownprocedure;CompassHousingeducationfortenantsandstaffledtolightingupgradesandtenantsinstallingshadesandfans;andtheCasinoClubreportedpowerusagedownbyalmost20%asaresultofmonitoringappliances.Theseorganisationsreportedthatthetrainingwascrucialtotheseequipmentandbehaviourchanges.
3.5.4 Conclusion
Theprogram’spartnershipanditsstrategicapproachtostakeholderengagementhaveputenergyefficiencyontheskillsdevelopmentagendaoftheindustry.Basedonastrongframeworkforcollaborationbetweengovernment,industryandtheeducationsector,EETPhascontributedtostrengthenthequalityandrelevanceofenergyefficiencytrainingforprioritysectors.Participantsreportedthattrainingwasuseful,relevanttotheirroleandcontributedtogrowingconfidenceandengagementwithenergyefficiency.Thereisemergingevidenceofenergysavingsfromthetrainingprojects.Thelongertermsavingsfromchangedpracticesresultingfromthetrainingwillonlybecomediscernibleinfutureyears.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
61
OEHComment
ThecomplementarityreviewinformedthegovernmentdecisiontoceasethevocationaltrainingelementsoftheEETPasitduplicatedfundingfromtheCommonwealthforenergyefficiencyvocationaltrainingaspartoftheCleanEnergyFuturepackage.
Thenon‐duplicativecomponentsoftheEETPhavebeenmergedintotheESP.
3.6 EnergyEfficiencyCommunityAwarenessProgram(EECAP)
3.6.1 Designanddevelopment
TheEECAPisacommunicationandeducationprogramtargetingthebroadcommunitytoimprovecommunityknowledge,understandingandmotivationtoactinrelationtoenergyefficiency.InitialresearchindicatedthatNSWconsumerslackedinformationonhowtheirchoicesoftechnologiesandbehavioursrelatetotheirelectricityuse.TheprogramwasdesignedtoaddressthisandtosupportthebroaderobjectivesoftheEnergyEfficiencyStrategyusingacomprehensive,multi‐strategyandongoingapproachthatreflectsbestpracticeinsocialmarketing.ThedesigndrewupontheexperienceofpreviousNSWGovernmentbehaviourchangeprograms,suchasroadsafety,healthandwaste.Itaimedtoreacharangeofdifferentaudiencesincludinghouseholds,culturallyandlinguisticallydiversecommunities,smallbusinesses,andyoungpeople,andundertookformativeresearchtounderstandtheirneedsandwantsandhowbesttopitchthebehaviourchangemessage.Thecampaigndesignalsohadtoaccountforanumberofissues:startingfromvirtuallyazerobase(therehadbeennopriormasseducationrelatingtoenergyefficiency);energyusewascomplexandseasonal(requiringtwophasesperyear);sustainedandregularengagementwasneededtobegintoestablishanewsocialnorm.
This$15millionprogramhadthreemaincomponentswithmultipleprojects:
Massmediacommunication(80percentofthebudget)usingtheSavePowermass‐mediaadvertisingcampaign(blackballoons)throughavarietyofcommunicationchannels(bothaboveandbelowtheline)andpublicrelations,sponsorship,onlinemediaandpublicengagementactivities.Thewebsitewasdesignedtobeanongoinginformationhubforenergyefficiencytips,apledgetool,e‐news,SMStipssubscriptionandaportaltootherenergyefficiencyinitiativessuchastheEESBP,EETPandEnergySaverandHPSP.
Educationandtraining(15percent)coveredthreekeyinitiatives:
– TheNSWEnergymarkproject,apartnershipwithCSIROthatgatheredsmallgroupsofhouseholderseachweekinpeople’shomestolearnabouttheenvironmentalimpactsofenergyusage,andhowtobemoreenergyefficientintheirlives.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
62
– TheSavePowerlibrarykitswhichenabledborrowerstoself‐assesstheirhouseholdenergyuse,costsandgreenhousegasproduction.
– TheRetailerEngagementProject(REP),apartnershipwithmajorapplianceretailersthatincludedtrainingoftheirsalesstaffandprovisionofpointofsaleinformationtoassistcustomerstoincludeconsiderationofenergyefficiencyintheirpurchasingdecisionmakingprocesses
Researchandevaluation(5percent).Thiscomponentwascriticaltoinformtheinitialprogramdesign.Italsocontributedtorefinetheprogramdeliveryalongtheway,inparticularthemassmediacampaignthroughtrackingresearch.
UndertheresearchandevaluationcomponentOEHcontractedInstinctandReasontoconductcomprehensivemonitoringandtrackingresearchforthemassmediacampaign.In2011OEHcommissionedDatabuildtoundertakeacomprehensiveevaluationofalltheprogramcomponentswiththebroadaimofascertainingtheimpactofEECAPontheknowledge,attitudesandbehavioursoftheNSWcommunity,drawinguponthetrackingresearch.TheDatabuildreportisnotcompleteatthetimeofwriting,however,preliminaryfindingshavebeenobtainedandarepresentedbelow.ThemethodsbeingusedbyDatabuildinthereportincludeatelephonesurveyof200NSWresidentsidentifiedthroughthetrackingresearchandanonlinefollow‐upsurvey.OEHalsohadthefinalreportbytheCSIROonNSWEnergymarkaswellasinternalevaluationreportsfortheSavePowerKitsprojectandtheRetailEngagementProject.
ThepreliminaryresultsindicatethattherationaleforEECAPandtheholisticprogramlogicwasclear,logicalandalignedwithgoodpracticesdrawinguponpreviousexperienceandextensiveresearch.The$15millionbudgetoverthreeyearswasconsideredcomparablewithbudgetsforsimilarprograms(e.g.$5‐6millionannuallyfortheVictorianGovernment’s‘blackballoons’energyefficiencycampaign),althoughresourceswereoneofthekeychallengesinthedeliveryoftheprogram.WhilethereweresomeissuesinmanagingpartnersandtheCSIROworkfortheEnergymarkproject,EECAPengagedeffectivelywithaverybroadrangeofstakeholders.EECAPprogrammanagersmentionedtwodifficultieswithimplementation,firstlythedifferenttimeframesofthecomponentsandsecondlythedualprogramstructurewithtwomanagers,oneforprogramsupportandoneforcommunityeducation.
3.6.2 Reachanddelivery
Themass‐mediacampaignachievedahighlevelofreach.Ittargetedanoverallpopulationof5.8millionpersons(approximateNSWadultpopulation),andoverthefivecampaignsthetrackingresearchestimatedover15millionadultviewingsoftheadvertisingontelevisionatleastonce(dependingonthephase,between35and53percentofthosewhosawitsaytheysawtheadvertisingsixormoretimes).Atthefinalmeasure(postsummer,2010/11),3.7millionadultsor63percentoftheNSWpopulationhadseentheSavePoweradontelevision,andofthese,44percentsawitsixormoretimes.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
63
TheSavePowerwebsitealsoachievedasubstantiallevelofexposure.Inthreeyearsithadover246,750uniquevisitors(72%fromNSW);over1millionpagesviewed(1,040,267)andover3,500SavePowerwebsitemembers;650SMSsubscribersforseasonaltips;andover3,800subscriberstoinformationupdates.Over9,000actionswerepromisedfrom819individuals,withpotentialtosave$966,600offpowerbillsand4,935tonnesofcarbon.EECAPreportthatitwasrankedinthetop10ofenvironmentalwebsitesinJanuary2011.
Inadditiontothemainstreamadvertising,thecampaignachievedarangeofmediaexposureandengagement,throughfeaturesinconsumerlifestyleandhomepublicationsandradio;theSavePowerChallengepartnershipwithNewsLimited;directengagementwithconsumersthroughe‐news,SMSandothermedia;distributionofpromotionaltools;grantsto36councilstopromotetheHPSPandawarenessofSavePower;andgrantstoninecouncilstopromotethemessagewithafocusonAboriginalandCALDcommunities.
AnillustrationofoutreachunderSavePoweristheworkthroughtheSustainableLivingProgram,apartnershipwiththeEthnicCommunitiesCouncil.15bi‐lingualeducatorsreceivedtrainingonthekit,and13receivedthe1½daystrainingprogram,resultingin57energyworkshopsdeliveredto1044participants.Anewresource(flipchart)isbeingfinalisedwithSavePowerinformationandmessagingdesignedforeducators,councilstaff,andmigrantcentrestaff.Anewprogramin2012willrecruit20EnergyChampionsand50LeadersfromacrossCALDcommunitiestoencourageparticipationandactiontoreduceenergy.
TheEnergymarkprojectwasaNSWtrialofanationalprogramdevelopedandmanagedbytheCSIRO.SinceactivecommencementinNovember2010,112convenorscompletedgroups(withanother19convenorsandgroupsstillinprogress).Withanaverageoffivepeopleineachgroup,517participantshadcompletedagroupbyApril2012.TheNSWEnergymarkFinalReport(CSIRO,June1012)notedthattheprogramdisproportionatelyattractedmiddle‐aged,university‐educated,professionalwomen,andmostparticipantsalreadyheldpositiveattitudestowardstheenvironment,typicalofAustraliansingeneral.Evenwiththismorealignedandreceptivegroup,theprogramwasabletoachieveenergysavings.
ThereachofotherEECAPprojectswerealsoveryencouraging.SavePowerlibrarykits(841)wereplacedwith62CentralLibraryServicesinNSW(withupto260branchlibraries).Over6,400householdsborrowedakitinthetwelvemonthsbetweenMarch2011andMarch2012,and2,200peopleattendedalocallibraryworkshoponhowtouseit.Theborrowersweresatisfiedwiththekits(96percentoftherespondentstotheonlineborrowersurvey)andtherewasevidenceofpositiveimpactontheirknowledgeandbehaviour(e.g.75percentoftherespondentssaidthatsinceborrowingthekittheywillnowpurchaseanelectricalappliancethatuseslesspower,comparedwithonly31%beforeusingthekit).
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
64
TheRetailerEngagementProjecthassuccessfullyengagedmajorelectricalapplianceretailers(HarveyNorman,TheGoodGuys,DavidJones,Myer).Ithasbeeninthefieldfor18monthsandprovidedtrainingtostaffin142storesthrough1,282storevisits,resultingin78percentofstorestafftrainedand92percentsatisfactionfromstoremanagers.
3.6.3 Outcomes
PeoplereachedbythemassmediaandeducationalcomponentsofEECAPhavereportedraisedawareness,positiveresponsesandinmanycasesincreasesintheirenergysavingbehaviours.
Trackingresearchdemonstratedthatapprovalofthecampaignremainedhighovertime,fromwinter2009tosummer2010:morethan80percentofpeopleapprovedorstronglyapprovedtheNSWGovernmentadvertisingwithmessagesaboutsavingpower.Thecampaignappearstohavesomesuccessinraisingawarenessofissuesinvolvingenvironmentalconcernsaboutenergyuseandmotivatingbehaviourchange.
Thosewhohaveseenthecampaignaremorelikelytohavechangedbehaviourtosaveelectricitycomparedtothosewhohavenotseenthecampaign(46percentversus36percent).TrackingresearchfoundsignificantincreasesinmanyofthebehavioursportrayedintheSavePowercampaigncomparedwiththe2009benchmarks(suchasinstallingthickcurtains,blindsorshutterstoreduceheat;checkingthefridgeisworkingefficiently).Atthefinalmeasure(Postsummer,2010/11),almosttwo‐thirdsofNSWadults(63percent)hadseenthetelevisionadvertisement,with77percentfindingit“very”or“fairly”convincing,andathirdfeelingmotivatedtotheenergysavingactionsinthemessages.
Atthesametimeissuesaroundenergyefficiencyandparticularlyrisingelectricitycostswerefeaturingwidelyinthemedia.Toexaminetheextentbehaviourchangesrelatedtothecampaign,theDatabuildtelephonesurvey(N=200)examinedtheattributionofthechangedbehaviour,andfoundthatthecampaignhadaninfluenceuponhalfofthosewhohavetakenactionsincethecampaignandrecallit.RespondentstendedtociteSavePowerinitiativesasbeingastronginfluenceonbehaviourrelatedtoelectricalappliancesandairconditioning,lesssowashingclothes(mostrespondentswhocarriedoutthesebehaviourssaidtheyhadalwaysdoneso).
Thetrackingresearchalsosuggeststhatthebehaviourchangesaresustainedovertimeandthatthecampaignisbuildingsocialnormsaroundenergyefficiency.Notonlydopeopleperceivetheirownmindfulnessaboutelectricityuseincreasingbuttheynoteothersaroundthemarealsobecomingmoremindfulovertime.Almostsixinten(58%)claimtheywere‘very’or‘extremely’mindfulabouttheirelectricityuse(inpost‐summer2010/11measure).
WhilethefocusofEECAPwaswiderawarenessraisingandcapacitytoleadtochangingbehavioursandlongertermenergysavings,anumberofcomponentsindicateddirect
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
65
impactsonenergysavings.InparticulartheEnergymarkprogramappearedtoinducepositivechangesinindividualbehaviour,andreportedreductionsincarbonfootprint,andelectricityconsumptioncomparedtopre‐projectlevels.PreliminarybillinganalysisbyCSIROindicatesanaveragereductioninelectricityuseof12.3percent.OEHisintheprocesstoobtainmorecomprehensivedatatosubstantiatetheestimates.CSIROestimatesabroaderreductionincarbonemissionsof6.4%onaverageforelectricityandnaturalgasuse(combined),petrolanddieselconsumption(combined),extentofairtravel,useofwoodfires,householdexpenditureonconsumergoodsandservices,andfoodconsumption.InterestinglythereportnotesthatamongparticipantsexposedtotheEnergymarkexperience,eventhosewhocameoutoftheprocesswithoutenhancedunderstandingandsupportofsustainability,generallyreducedtheiremissionsandelectricityconsumptionasmuchasthosewhose‘green’preferencesandknowledgehadbeengreatlyaugmented.ThisfindinghasbeenreplicatedinanumberofdifferentEnergymarktrials.
TheSavePowercampaignmayalsohaveimpactedontheparticipantsintheotherenergyefficiencyprograms.Theremaybescopetoexplorethiscollectingfeedbackfromparticipants;andthroughanalysingthepatternofhitsonpagesrelatedtospecificprogramsonthewebsite.
3.6.4 Conclusion
TheEECAPsuccessfullyimplementedacomprehensive,multi‐strategyandongoingsocialmarketingcampaignoverthreeyears.InitsfinalphasetheSavePowertelevisionadvertisingreachedalmosttwo‐thirdsofNSWadults(63percent),withmost(77percent)findingitconvincingandathirdfeelingmotivatedtotheactionsinthemessages.ApprovaloftheadvertisingcampaignhasbeenpositiveamongtheNSWpopulationandresearchsuggeststhatitcontributedtoimproveknowledgeofenergyefficiencyissuesaswellassomeinfluenceinchangingenergyusebehaviours.Thepowerofthemassmediacampaigncomesfromthewidereachwhichoffersgreatpotentialforenergysavingsfrombehaviourchangesatthisscale.
TheEECAPeffectivelytrialledarangeofotherapproachestoreachdifferentaudiences.Theexperienceofdelivery,thetrackingresearchandtheevaluationsareprovidingOEHwithasubstantialevidencebasetoinformfuturesocialmarketinginthisarea.
OEHComment
TheEECAPwasceasedinJune2012asplanned.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
66
3.7 DataandEvaluationProgram(DEP)
3.7.1 Designanddevelopment
In2009OEHestablishedtheDataandEvaluationProgram(DEP)toreportonenergysavingsandbenefitsoftheprogramsandinformfuturedirections.TheDEPdirectlyaddressesthelackofreliableinformationonenergysavingsfromenergyefficiencyprogramsthatwerehighlightedbythe2007OwenInquiryintoElectricitySupplyinNSW.The$2millionDEPisground‐breakinginAustralia—nootherjurisdictionhasappliedsuchaclearcommitmenttoenergyefficiencymeasures.
Thedesigncamefromthe2010EvaluationFrameworkandProjectPlanfortheEESbutnowaimsmorebroadlytoinformpolicyanddecisionmakingforthefutureprogramsincludingtheEnergyEfficiencyActionPlan(EEAP).Ithasfourmainelementsthatareinlinewithbestpracticeinthearea(seeboxattheendofthissectionontheevaluationofenergyefficiencyinCalifornia):
Measurementandverificationofactualenergysavings Collectionofprogramdeliveryinformationandprogramevaluations(processand
impact) Assessmentofmarketandeconomicbenefits Integration,interpretationandreporting.
3.7.2 Measurementandverificationofactualenergysavings
Thisprojectaimstoprovidereliablemeasuresofenergysavingsdirectlyachievedthroughhome,businessandgovernmentprogramsbyusingevidenceofactualelectricityusetocomplementestimatesbasedondeemedsavings.8Currentfindingsonenergysavingsforprogramsareinprevioussections.
Whenthesedirectenergysavingsprogramsweredesignedtheexpectedsavingswereusuallydeemedusingengineeringcalculationsandassumptionsabouttheextentofuptakeandeffectivenessoftheuptakeinrealisingsavings.Deemedsavingsare
8Thebiggestchallengeinevaluatingenergyefficiencyprogramsisalackofdirectmeasurement.Energysavingsarewhatdidnothappen,butenergyconsumptionisactuallywhatismeasured.Thedifferencebetweenenergyconsumptionandwhatenergyconsumptionwouldhavebeenhadenergyefficiencymeasuresnotbeeninstalledprovidesameasureofenergysavings.Savingscomputationthereforeinvolvescomparingmeasuredenergydataandacalculationof“adjustments”toconvertbothmeasurementstothesamesetofoperatingconditions(i.e.abaseline).Bothmeasurementandadjustmentprocessesintroduceuncertainty.Theseprocessesproducestatistical“estimates”withreportedorexpectedvaluesandsomelevelofvariability.NationalActionPlanforEnergyEfficiency(2007).ModelEnergyEfficiencyProgramImpactEvaluationGuide.StevenR.Schiller,SchillerConsulting,Inc.www.epa.gov/eeactionplan
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
67
uncertainbecausetheiraccuracydependsupontheinitialassumptionswhichcanbeunderminedbymanyreal‐worldfactors.
Theprogramaddressedbarriersrelatedtomeasurementofenergysavingssuchas:
nomeasurementnomanagement‐financiers,decision‐makersandadministratorsofenergyefficiencyprojectsarehinderedbylackofevidenceshowingsavings
hightransactionalcost‐foranenergyefficiencyprojecttosourcefinancefromthemarketplace(e.g.throughESS),thelackofaconsistentandagreedmethodstoclaimandcertifysavingsincurshightransactionalcosts
lackofskills–businessesinvolvedinenergyefficiencyprojectsgenerallylackofM&Vskillswhichinturnexacerbatespreviousbarriers.
Inthefollowingboxarelistedthethreemainmethodsusedtodeterminegrossprogramenergysavingsasdescribedin2007bytheU.S.EnvironmentalProtectionAgency.
Thethreemajormethodstoestimategrossenergysavings
OneormoreM&VmethodsfromtheInternationalPerformanceMeasurementandVerificationProtocol(IPMVP)areusedtodeterminethesavingsfromasampleofprojects,andthesesavingsarethenappliedtoalloftheprojectsintheprogram.
Deemedsavingsbasedonhistorical,verifieddataareappliedtoconventionalenergyefficiencymeasuresimplementedintheprogram.
Statisticalanalysesoflargevolumesofenergymeterdataareconducted.NationalActionPlanforEnergyEfficiency(2007).ModelEnergyEfficiencyProgramImpactEvaluationGuide.StevenR.Schiller,SchillerConsulting,Inc.www.epa.gov/eeactionplanpage98.
AsenergyefficiencyprogramsinNSWarerelativelynew,thenecessaryenergysavingsdata,measurementandverificationmethodsandtoolshavehadtobedeveloped.Theactualsavingscanonlybeadequatelyestimatedusing“beforeandafter”approaches,inwhichasavingisdeterminedasthedifferenceoftheenergyusemeasuredaftertheinterventiontookplaceandabaselinerepresentingtheenergyuseiftheinterventionhadnottakenplace.Thisapproachreliesontwofactors(1)theavailabilityofmeasurementdata,and(2)acounterfactualbaselinewhichcanbeonlyestablishedthroughstatisticalmodellingwithhistoricalmeasurementdata.BothfactorsposedachallengetotheDEP.
Measurementdatacouldeithercomefrommeteredenergyuse(billingdata)regularlycollectedbytheenergynetworkserviceproviders,orfromtask‐orientedon‐sitedatalogging.OEHdoesnotownandpossessbillingdata,whiledataloggingincurssignificantcostandneedstechnicalskills/qualificationforwiring.ToaddressthisDEPhasnegotiatedarrangementswithNSWnetworkserviceproviderstosecureparticipantsbillingdatathathadneverbeenreleasedpreviously.
Thesecondfactor—establishingacounterfactualbaseline—requiresstatisticalandregressionmodelling,andthesespecialistskillswerenotavailablewithinprogramstaff
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
68
orparticipants.Toaddressthisgap,DEPdevelopedprogram‐specificmeasurementandverificationapproachesthatbestfittedthesituationsofeachprogram,takingaccountofrobustness,capacitybuilding,andcost‐effectiveness.
ThefollowingtablepresentsasummaryofthemethodsusedformeasuringsavingsasperJune2012.Detailsofthemethodsforeachprogramareprovidedinthefollowingparagraphs.
Table3‐8. SummaryofmethodsusedtoestimateenergysavingsforeachprogramtoendofJune2012
Deemedsavings Grosssavings Netsavings
Basedontargetuptake
Basedonactualuptake
Casestudy GBRP
Smallsample ESP
Representativesample
HPSPEESBP
HPSP
Allprojects ESPGBRP
Source:OEH,DataandEvaluationProgram,August2012
Twoprimarymethodshavebeendevelopedandtrialled:
Analysisofbillingdataforrelativelysmalluserssuchashouseholdsandsmallbusinesses
Measurement&Verification(M&V)thatusesindustrialmeasuresandismostsuitedformediumtolargesites.
Billingdataanalysisisamethodtomeasureparticipants’energyusebasedonstatisticalanalysisofbillingdataprovidedbyenergydistributors.Thislowcostmethodismostsuitableforrelativelysmallandhomogeneoususerssuchashouseholdsandsmallbusinesseswheremoderatesavingsareexpected,typicallyoverlargenumbersofusers.
NegotiationsbetweenDEPandNSWenergydistributorsfordatasharingtook1.5years,withtheneedtosatisfytheirconcernsaboutcustomerprivacyandprovidethemwithbusinessbenefits.ThisresultedinarrangementsforvariousenergysavingsquantificationtaskstoaccessthemostcomprehensiveenergyusedataevermadeavailabletoagovernmentagencyinAustralia.Theapplicationsinclude:
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
69
HomePowerSavingsProgram‐thehighparticipantnumbersanddiversityinhouseholdcharacteristicsmakehousedatalogginginadequate.Participantsbillingdataisusedtodeterminesavingsthroughcomparingenergyusewithacontrolgroupusingrigorousstatisticalmethods.Preliminarysavingsanalysisforhasbeencompletedandfullanalysisfor2012isunderway
EESBP‐projectstypicallyhaverelativelylowinvestment,straightforwardtechnology,lackofintervalmeterdata,lackoftimeandhigherpriorityissues.WhilecomprehensiveM&Visnotfeasible,thespecificityofeachbusinessmakesithardtoidentifyasuitablecontrolgroup.TheapproachdevelopedbyDEPusesestimatedsavingsfromparticipantsbillingdataandregressionmethodsundertherigorousprinciplesofM&V.Thisprovesacost‐effectivemethodologytoverifysavingsandassistingsmallbusinessestosourcefinancefromtheESS.
QuantificationofactualsavingsisalsounderwayfortheHousingNSW’sHomeInsulationProgram(aCCFproject),andunderplanningforthecompletedNSWHomeRebatesProgram.
Measurement&Verification(M&V)‐thismethodusesbestindustrialpracticetoverifysavings,andismostsuitedformediumtolargesiteswhereenergyuseisheterogeneous,measuresarecomplexandrelativelyhighcost,andlargesavingsareexpected.
Guideforlargebusinessandgovernmentagencyprograms(industrialprojectsinESS,ESP,EEGSP)‐heavierinvestment,complextechnology,andaccessibilitytointervalmeterdatacallsforrobustsite‐specificM&Vfortheselargeparticipantsprograms.Thisrequiresspecialskillsandconsistentapproaches,soDEPdevelopedanOperationalGuideforM&Vforpractitionerstostandardiseapproachesalignedwithinternationalbestpractice.ThedraftGuidewastestedusingsupervisedtrialswithselectedretrofitprojectsineachofthethreelargeparticipantsprograms.TheprojectswillprovidecasestudiesintherevisedGuide,whichisduetobeweb‐publishedinAugust2012.TheprojectwithESPhasbeenextendedforafullM&VaimingatcreationofEnergySavingsCertificates.
EnergySavingsScheme–whileESSislegislatedandadministeredoutsidetheOEH,DEPtooktheinitiativetoworkwithIPARTtodevelopatechnicalguidebookfortheESSRulesaccreditationprocedure,basedontheM&VGuide.ThisapproachbenefitsOEHevaluationbutalsoIPART’scertificateaccreditationwhichhasbeenhamperedbythelackofstandardandeasy‐to‐followguidelines;
EnergySaverProgram–participantsactonenergysavingmeasuresidentifiedinenergyauditslargelyonavoluntarybasiswithoutgovernmentfinancialsupport,soanM&Vrequirementwouldbeviewedasanextraburden.TheESPteamstartedwithanM&Vprojectforasmallsampleofbusinessesusingtheleastcostingoption.DEPisplanningtofollowthiswithamorecomprehensiveM&VtrialtunedtohelpclaimsavingsfromtheEnergySavingsScheme;
GovernmentSustainabilityPolicy–Thisprogramcomprisesofaclusteroffundschemesdifferinginsetting,targetgroup,timeframeandrequirement.ThefundingandtheinvestmentscalemakethoseschemesparticularlysuitabletoM&Vand
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
70
someschemeshaveM&Vinthecontractualconditions.Thechallengetocommitmentismainlyresourceconstraintandlackoftechnicalknowledge.Apartfromthetrialprojectsabove,DEPisplanningtohelpprojectmanagerstakeupM&Vbyprovidingexpertsupervisionandassistanceformoredemonstrationprojects;
3.7.3 Collectionofprogramdeliveryinformation
Thiscomponentfacilitatesthecollectionofreports,data,viewsandlessonsfromprograms,andcomplementsquantitativeprogramreporting.Akeyelementisfacilitationofevaluationreportsfromthesevenprograms.
ThedirectenergysavingprogramsandthecapacitybuildingprogramsrecorddetailedimplementationdataindatabasesandITsystems.TheStrategyandAnalysisUnitbuiltcomprehensivedatabases,analysedtrendsandprovidedreportsfortheprograms,aswellasthebroaderClimateChangeFundAnnualReportsthatincludetheseprograms.Threeprogramshavecommissionedmulti‐stageindependentevaluations(HPSP,EECAP,EETP)withdraftorinterimreportsavailableinJune2012whichhaveeachprovidedasoundbasisforconclusionsabouttheprogram’sachievementsandlessons.ESPandEESBPhavecommissionedvariousstudiesandsurveysandinJune2012providedinternalevaluationreportswhichcapturesomeoftheprogram’sexperience,achievementsandbarriers,withscopeformoredetailedanalysisoftrendsandpatternsfromtheirdata.
InJune2012ARTDcollectedtheviewsandexperiencefrommanagersoncoordinationandgovernanceacrosstheenergyefficiencyprogramstoinformcontinuousimprovement.
3.7.4 Assessmentofmarket&economicbenefits
DEPisusingmarketstudiesandeconomicmodellingtoassessimpactsoftheprogramsasawholeonenergyefficiencymarket,thenationalelectricitymarketandthestateeconomy.
Marketbenefits–theprojectaimstoevaluatetheimpactoftheenergyefficiencyprogramsasawholeontheenergyefficiencyproductandservicemarkets.Itstartedin2011withabaselineanalysisassumingnoprograminplace,andwillundertakeafinalanalysisincorporatingalltheenergyefficiencyprograms.DEPistosupplementthisdesktopstudywithamarketsurveyintoenergyefficiencyserviceandproductmarket.
Economicbenefits‐thisprojectistoestablishcost‐benefitmetricsoftheenergyefficiencyprogramsasawholefortheelectricitymarketandthestateeconomy,coveringpublicaswellasprivatecosts.ItinvolvesenergymarketandeconomywidemodellingandwilltransfersomeofthemodellingcapacitytoOEHatthecompletion(currentlyunderprocurement).
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
71
3.7.5 Integration,interpretationandreporting
Theprojectwasdesignedtointegrateandinterpretprograminformationandevaluationfindingsofthesevenenergyefficiencyprogramsandotherprojectsintoanoverallassessment.
Integrationandreporting‐theprojecthasevolvedtofocusonannualoverallreportsontheprogressandoutcomesoftheprogramsandinterpretingthefindingsintopolicymessages.DEPengagedARTDin2010astheevaluationpartnertoundertakethisrole.
Thepresentreportisakeyoutputofthisproject,integratingfindingsattheprogramlevelintoanassessmentofthedeliveryoftheinitialEnergyEfficiencyStrategy.Thiswillbepartoftheevidencebaseforconsultationandplanningonfuturedirectionswithenergyefficiencypoliciesandprograms.EvaluationAdvisoryGroup(EAG)‐thegroupwassetuptocoordinateandreviewevaluationprocessesacrossfourbranches,andcomprisesprogrammanagersofthesevenenergyefficiencyprograms,withDEPassecretarytotheGroup.SixEAGmeetingshavebeenheldsinceDecember2010withtheseventhinJune2012.AsatisfactionsurveybyARTDin2011foundmemberslargelysatisfiedwithsharinginformationbutsuggestedimprovementsformeetingpreparation,schedulingandlocation,
3.7.6 Otherdatadevelopmentforenergyefficiency
OEHandCSIROhaveacollaborativeprojectaimingtodevelopbroad‐basedenergyusedataandenergyefficiencyscenariomodellingtosupportevidencebasedpolicydevelopmentintheresidentialsector,inparallelwiththeevaluationoftheenergyefficiencyprograms.Itcommencedin2010inpartnershipwithAusgrid,EndeavourEnergy,EssentialEnergy,JemenaGasNetwork,HousingNSWandDepartmentofPlanning.Phase1isagranularNSWhomeenergyusedatadevelopmentprojectcompletedinJune2011.CSIROiscurrentlyfinalisingPhase2dueJuly2012–modellingforthreepilotscenarios:solarphotovoltaic(PV)update,hotwatersystemreplacementandhighstarwashmachineuptake.
3.7.7 Conclusion
TheDEPisprovidingmorereliableinformationonenergysavingsandtellingthestoryoftheOEHinvestmentsinenergyefficiency,leadingtolessuncertaintyandbetterinformationtoinformfuturedirections.
DEPhasmadesignificantprogressindevelopingandapplyingbestpracticemethodstoproducemorereliableenergysavingsestimatethroughbillingdataanalysisforsmallusersandM&Vformediumtolargesitesbasedontheinternationalprotocol.Whiletheprocesshasbeenrelativelyslow—owinglargelytotheengagementofexternalparticipantsonavoluntarybasis—itisresultinginhighlycrediblemeasuresthatwillincreasinglygeneratemorereliabledataonactualenergysavings.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
72
Theindividualprogramsarerecordingdetaileddeliverydataincomprehensivedatabases,supportedbytheanalyticalcapacityoftheStrategyandAnalysisUnit.Programsareincreasinglyusingthisdatatoinformimplementationandforevaluation.Theprogramswithmulti‐stageindependentevaluations(HPSP,EECAP,EETP)haveprovidedcomprehensiveassessmentsoftheirprogressandlessons.Otherprogramsundertookinternalevaluationreports,supportedbyindependentlycollectedparticipantdata.
Theintegrationandreportingofevaluationfindingsacrossthesevenenergyefficiencyprograms—asperthepresentreport—appearstobefillingagapintheOEHevidencebaseandcanprovideOEHwithaformalrecordofprogressandachievementsaswellasasourceofinformationforplanningfuturedirections.The2012evaluationprocesshasgeneratedgreaterinterestinandcommitmenttoongoingevaluationoftheprograms,andprovidesanumberoflessonsforconsideringthefutureevaluationstrategyforinvestmentsinenergyefficiency.
EvaluationofenergyefficiencyinCalifornia
TheDEPprogramhasmanysimilaritieswiththeevaluationprogramforenergyefficiencyinCalifornia,thoughatmuchsmallerscale.Californiaisaleaderinenergyefficiencyandsetsthebenchmarkforevaluation.Relevantfeaturesofthe2006‐2008evaluationcompletedin2010bytheEnergyDivision(withabudgetof$97million)were:
thestrongfocusonverifyingreportedenergysavings useofevaluationtoolsandmeasurestoapplyacrossmultipleprogramsandsites asetofrobustindependentevaluationscoveringalltheprograms focusonimpactevaluationssupportedbyprocessevaluations useofprogramtheory(programlogic)forevaluationframeworks includingcost‐effectivenessandmarkettransformationstudies increasingthefocusonbehaviouralissuesinenergyefficiencyprograms productionofastate‐wideoverallevaluationsynthesisreportwithaggregatedresults
The2010evaluationreportmadesomerecommendationsinlinewithfindingsofthisevaluation,forexample:
‐ programimplementersimproveprogramtrackingdatacollection‐ resultsfromtheevaluationsareusedforcontinuouslyimprovingsavingsestimates
andinformingfutureprogramdesign.
Sources:CalifornianPublicUtilitiesCommission,EnergyDivision(2010)2006‐2008EnergyEfficiencyReport.http://www.cpuc.ca.gov/PUC/energy/Energy+Efficiency/EM+and+V/2006‐2008+Energy+Efficiency+Evaluation+Report.htm
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
73
4. Cross‐programcoordinationandstrategy‐widegovernance
Thissectionreviewsthestrategy‐widegovernanceandthecross‐programcoordinationandinformationsharingfortheNSWenergyefficiencystrategyinordertoidentifylessonsforthefuture.Itisbasedonareviewoftheprogramevaluationreports;andinterviewsinMay‐June2012withmanagersinvolvedintheEESabouttheirexperiencewiththeinternalgovernancearrangementsandtheirviewsaboutitsstrengthsandweaknesses.
Effectivegovernanceprocessesarealwaysachallenge.Theyneedtobalanceformalandinformalarrangements,andthetimeandresourcesforstrategy‐widegovernancewiththoseforprogrammanagement.Theidealapproachwillvarywiththestageofthestrategy,issuesaroundintegration,andthestructureandcultureoftheorganisation.
4.1 Theinitialdesignhadlimitedscopeforintegration
TheEnergyEfficiencyStrategyisacross‐organisationandtosomeextentcross‐agencysetofprogramsandpolicies.Theinitialdesignallowedforadegreeofcross‐impact:directenergysavingprogramswithsimilarapproachesthattargeteddistinctsectors(lowincomehouseholds,businesses,andgovernment);programsprovidingcapacity‐buildingsupporttotheindustryandthecommunity;andthemarket‐basedinstrumentoftheEnergySavingsScheme(ESS).TherewereexpectationsthatthedirectenergysavingprogramswouldutilisetheESSovertime,andthatthecapacity‐buildingprograms—EECAP,EETPwouldcontributeindirectlytothedeliveryofotherprogramsbyprovidingafavourableenvironmentforenergyefficiencyinitiatives.
Inpracticethescopeforintegrationwaslimited.TheEESitselfwasdevelopedandapprovedquickly,beforeadetailedunderstandingofhowprogramswouldfunction.NotallOEHenergyefficiencyprogramswereincluded,andinternalstakeholdershaddifficultieswiththerationaleforwhichprogramswereinandwhichprogramswerenot.Programsinthestrategyrarelyhaddirectlinkstootherprogramsembeddedintheirrespectiveobjectivesandsettings.InparticulartheESSwasmanagedbyIPARTnotOEHandinvolvementwiththeOEHprogramshasonlycommencedrecently.Thetwocapacitybuildingprograms(EECAP,EETP)wereexpectedtosupporttheotherprogramstosomedegree,butinpracticethishasbeenlimitedastheirtimeframesdidnotalwaysmatchtheimplementationtimeframeoftheotherprograms.EETPinparticularwhichissupportingthedevelopmentofaskilledworkforceforenergyefficiencywilltakesmanyyearstofullyrealisetheseresults.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
74
4.2 Organisationalsiloswereabarriertointegration
AnimportantpointisthattheEESsatontopoftheexistingOEHprogrammanagementandreportingarrangementsandtheseexistingarrangementswerelargelyrelieduponforthegovernanceofthestrategy.Thepolicyandoperatingenvironmentoverthisperiodwasamajorfactor,withaveryvolatilepolicycontext,theorganisationwithacontinuinghistoryofstructuralchange,andthetransitionofmanyseniormanagers.
Inthiscontextakeyconstraintonstrategy‐widecoordinationidentifiedbyanumberofmanagerswastheorganisationalstructurethathadsilosformanagementofenergyefficiencyprograms.AsrepresentedintheFigure4.1below,twosiloswerethetwomainbrancheswhereenergyefficiencyprogramsweresitting—WaterandEnergyprogramsontheonehandandBusinessandCommunityprogramsontheother.ThiswasheightenedbythegeographicsplitbetweenthefirstbranchlocatedintheCityandtheotherinParramatta.Intheabsenceofmoreformalarrangements,coordinationandinformationsharingreliedheavilyoninformalandpersonalrelationshipswherephysicalproximityisakeyfactor.
Theothersiloswerethesplitbetweenpolicyandprogramimplementation.Someprogrammanagersregrettedthelackoffeedbackdownthereportinglinethatwouldhaveofferedopportunitiesforlearningfromotherprograms.
Figure4‐1. OrganisationalchartofOEHunitsinvolvedinenergyefficiency
Environment&HeritagePolicyand
Programs
SustainabilityPrograms
Water&EnergyPrograms
HomePowerSavingsProgram
NSWGovernmentSustainabilityPolicy
Othersections:ClimateChangeFunding,Communications,
strategy&analysis,etc.
Business&CommunityPrograms
BusinessPartnership
EnergyEfficiencyforSmallBusiness
Program
EnergySaverProgram
CommunityEducation
EnergyEfficiencyCommunityAwareness
Program
EnergyEfficiencyTrainingprogram
Othersections:beachwatch,Localgov&wastestrategy,etc.
ClimateChange,AirandNoise
Emissionsreductions
DataandEvaluationProgram
Othersections:AirPolicy,NoisePolicy,etc.
Otherdivisions:GroupCoordination&Strategy,Heritage
Branch,etc.
Residential‐Govt /City
Business‐Community/Parramatta
Policy
Programimplementation
Source:ARTDaccordingtoOEHorganisationalchart,June2012
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
75
4.3 Coordinationwouldhavebeenimprovedbymorestructuredopportunities
EEShadanumberofarrangementstosupportstrategy–widecoordinationandinformation‐sharingincluding:
– SavePowercampaign(blackballoons)andtheEESwebsite– ProgramdatacollectionandreportingthroughtheClimateChangeFund– Aninformation‐sharingforumforOEHprogramsin2010– ComprehensiveEESevaluationframework– DEPactivitiesandthemeetingsoftheEvaluationAdvisoryGroup(EAG)
TheDEPinparticularhasanexplicitmandateforcross‐programsupportandintegration.OEHstaffwelcomedtheopportunityofferedbyEAGmeetingstosharelearningsfromotherprograms.Howevercoordinationofevaluationactivitiesdidnotfeedintoawiderstructureforcoordinationorgovernance.
Accordingtomanagerstherewaslittleattentiontocross‐programcoordinationorinformationsharingduringtheinitialdevelopmentanddeliveryphase,whereeachprogramwaslargelyfocusedonrefiningitsmethodsanddeliveringitsoutputs,
Howeverprogramsidentifiedopportunitiesforcoordinatedactivitiesalongtheway.ForexampleHPSPandEECAPhadcoordinatedinitiatives,forinstanceinregardtoacommonCALDstrategyandcouncilgrants.Whiledevelopingtheseinitiativestooktimeandcouldhavegonefurther,bothprogramsacknowledgedthebenefitsandwishedthereweremoresimilarinitiatives.Intheresidentialsector,aninternalhouseholdreferencegroupwascreatedtopromotecoordinationincludingaresearchcomponentthatprovidedsomekeyinsightintobehaviourchangesissues,butthishasnotcontinued.
Otherexamplesofsuccessfulcoordinationincludecross‐programinitiativeswithinthebusinesssectorbetweenEESBPandESP,bothworkingoutofthesamebranch.Theycollaboratedonthepanelsofserviceprovidersusedforretrofitsandtechnologyseminars.EETPhasalsoworkedcollaborativelywiththeESPprogramandthedevelopmentoftechnologyguides.Thetwobusinessprogramsalsohavearrangementsforbusinessesthatfallinthe‘grey’areabetweenthem.TheseprogramsareworkingtodevelopsynergieswiththeEnergySavingsSchemewhichisapotentialsourceofprojectcapitalforparticipatingbusinessesthroughthegenerationofESCs.ProgramstaffregrettedthelackofformalarrangementsforcoordinationwiththeESSthoughrecentlyprogramshavebeendevelopingtheseopportunities.
Oneoftheconsequencesoflimitedcoordinationhasbeenriskssuchasinconsistentmarketsignalsfromdifferentinitiativesormarketsaturation.Therewerealsocaseswhendifferentprogramsworkedquiteseparatelywiththesameexternalstakeholderssuchasgovernmentagencies,councilsorNGOs.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
76
Atthesametimemanagershighlightedthegoodwilloftheircolleaguesandtheinstancesofinformalsharingthatoccurred.ForinstanceEEGSPhascalledonthetwobusinessprogramstoassistittoaddressimplementationissuesexperiencedbyparticipantagencies.Butinpractice,mostmanagerssawlostopportunitiesforworkingmorecloselytogetherandsharinginformation,resources,andevenstaff.Feedbackfromprogrammanagershighlightedthepotentialforimprovingimplementationthroughamorestructuredapproachtocoordination.
4.4 Managersseekmoreoverarchinggovernance
TheEnergyEfficiencyStrategydidnothaveanoverallgovernanceframeworkoracross‐programmanagementgroup.Themainmanagementfocuswasondeliverybyeachprogram,throughthenormalprogrammanagementcontrols,andreportingthroughtheClimateChangeFunddatasystem.
Managershadmixedviewsaboutthecommunicationoftheoveralldirectionforthestrategy,withseniormanagersmorepositivethantheothers.Programmanagerswantedmoreinformationontheoverallstrategicagenda,howtheprogramswerepositioned,andlikelyfuturedevelopments,particularlyintherapidlychangingenvironment.
Overallmostmanagerswantedamoresystematicandstructuredprocessforstrategy‐widegovernanceandinformationsharingatthestrategicandprogramdeliverylevels.Theyalsorecognisedthepotentialburdenassociatedwithexcessivegovernancearrangementsandthevalueofretainingprogramflexibility.
OEHComment
OEHhasestablishedtheEnergyEfficiencySteeringGrouptocoordinateenergyefficiencypolicydevelopment,programdeliveryandevaluation.
ImprovedgovernanceandcoordinationisalsoafeatureofOEH’snewcorporatestructureasofearly2013.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
77
Appendix1. Evaluationreportsusedforthesynthesis
EachNSWEnergyEfficiencyprogramhasbeenconductingevaluationactivitiesaspartoftheoverallevaluationstrategyand/ortheirownevaluationstrategy.ToinformthepresentreportARTDrequestedinformationfromeachprogramaboutthemainachievementstoearly2012,progresstowardsandlikelysuccessinmeetingitsobjectivesandtargets,andthestoryofhowtheprogramhasadaptedtoovercomebarriersandmakeimprovements.
Inlinewiththeirrespectiveevaluationarrangements,programsprovidedthisinformationasanindependentevaluationreportorasaninternalreportbasedonfeedbackanddatacollectedoverthecourseoftheprogram,includinginsomecasesindependentevaluationactivitiessuchassurveysofstakeholders.
ThefollowingtablepresentsanoverviewofprogramevaluationsandreportsthatARTDhasusedasinputforthisevaluation.Foreachprogramsitspecifiesmaindatacollectionmethods.
Table4‐1.Evaluationreportsandmaindatacollectionmethodsforeachreport
Program Evaluationreports Maindatacollectionmethods
HomePowerSavingsProgram(HPSP)
2012InterimEvaluationoftheHomePowerSavingsProgram(ARTD2012)
Reviewofprogramdocumentation,monitoringandauditdata
Quantitativeanalysisoftheassessmentsdatabase
Assessors’survey 3focusgroupswithhouseholds Interviewswithinternalandexternal
stakeholders Onepilotcasestudy
EnergyEfficiencyforSmallBusinessProgram(EESBP)
Internalinterimevaluationreport(ISF2012)EvaluationoftheProgramPhaseI(DatabuildNovember2011)EvaluationoftheProgramPhaseII(DatabuildJune2012)
Analysisofassessmentsandretrofitdatabases
Auditfindings
participantssurveyandinterviews(Databuild)
EnergySaverProgram(ESP)
Internalinterimevaluationreport(2012)EvaluationoftheDECCWESPII(DatabuildMarch2011)
Analysisoftheprojectsdatabase Participantssurvey(Databuild)
NSWGovernmentSustainabilityPolicy(GSP)
Internalinterimevaluationreport(2012)
Regularreviewofmonitoringandauditdata
Informalfeedbackfromprograms’participants
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
78
Program Evaluationreports Maindatacollectionmethods
EnergyEfficiencyTrainingProgram(EETP)
EvaluationoftheEnergyEfficiencyTrainingProgram–Year2Annualreport(draft)(UrbisJune2012)
Reviewofprogramdocumentation Workshopsanddiscussionswith
programstaff Interviewswithexternalprogram
stakeholders Analysisofprojectdata
EnergyEfficiencyCommunity‐AwarenessProgram(EECAP)
PreliminaryfindingstoinformtheevaluationoftheEnergyEfficiencyCommunityAwarenessProgram(DatabuildJune2012)NSWEnergymarkFinalReport(CSIROJune2012)
Literatureanddocumentaryreview Interviewswithinternalandexternal
stakeholders Telephonesurveywith200NSW
residents
Pre‐Project,InterimandPost‐ProjectQuestionnaires(knowledge,attitudesandbehaviour)
Pre‐ProjectandPost‐ProjectCarbonFootprintQuestionnaires
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
79
Appendix2. Estimatesofactualenergysavings‐preliminaryresults
HomePowerSavingsProgram(HPSP)
Savingswereestimatedusingtwomethodsfrom7,000homeswhichaccountforabout18per cent of the total programparticipants at the timeof commencing the analysis.Two analysing methods were used: paired comparison with a control group, andregression.Theresultissummarisedinthefollowingtable.
Table4‐2. HPSPbillingdataanalysispreliminaryresults
Pairedcomparisonwithacontrolgroup
Regression
Averagesavingsacrossallhomes 0.6kWh/home/day(or4.0%reduction)
0.57kWh/home/day(or3.8%reduction)
Averagesavingsfromhomesthatreceivedthewholesavingkit 0.9kWh/home/day
(or6.0%reduction)
1.1kWh/home/day(or7.3%reduction)
Averagesavingsfromhomesthatreceivedthesavingkititemsexceptshowerheads
0.52kWh/home/day(or3.5%reduction)
0.45kWh/home/day(or3.0%reduction)
Relativelyspeaking,theresultfromPairedComparisonwithaControlGrouptendstobemorereliable.Howevertheregressionmethodhasgreaterpowertoprobeindetails.
The analysis was unable to detect any statistically significant saving attributable tobehaviouralchange.
EnergySavingsScheme
Thefollowingtable isacase‐studyfromtheM&VGuideDevelopmentprojectshowingthesavingsfromamajorretailerverifiedusingInternationalPerformanceMeasurement& Verification Protocol Option B/C9 for the purpose of claiming Energy SavingsCertificates(ESC)undertheESS.AsnoprojectsunderOEHprogramshaveclaimedESC,thiscasestudywasprovidedbyEnergetics.
9OptionA–Keyparametermeasurementfortheretrofitmeasure,OptionB–Fullparametermeasurementfortheretrofitmeasure,OptionC–Measurementattheboundaryofthewholefacility
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
80
Table4‐3. ESSMeasurement&Verificationpreliminaryresults
Program EnergySavingsScheme Site MajorRetailer
ProjectDescription Arangeofretrofitstostorelighting,HVACandrefrigerationsystems.
M&VMethodology IPMVPOptionB/C– Regressionbasedanalysisofwholefacility,analysinginputelectricityuseagainstambienttemperature(informofcoolingdegreedays).
BaselinePeriod 12monthperiodsbetween2007and2010.
SavingsanalysisPeriod Latest12monthstoAugust2011
Store BAUUsage(kWh)
ActualUsage(kWh)
EnergySavings(kWh)
GHGSavings(@0.89kgCO2‐e/kWh)
CostSavings($@$0.125/
kWh)
Savings%againstBAU
Store1 4,013,657 3,535,775 477,882 425,315 $59,735 12%
Store2 3,058,474 2,870,810 187,664 167,021 $23,458 6%
Store3 2,629,012 2,408,866 220,147 195,931 $27,518 8%
Store4 2,886,362 2,808,521 77,841 69,278 $9,730 3%
Store5 3,059,178 2,808,955 250,222 222,698 $31,278 8%
Store6 2,831,738 2,707,505 124,233 110,568 $15,529 4%
Store7 3,008,221 2,823,100 185,121 164,757 $23,140 6%
Store8 3,259,854 2,886,208 373,646 332,545 $46,706 11%
Store9 3,274,362 2,517,750 756,612 673,385 $94,577 23%
Store10 2,747,018 2,534,275 212,743 189,341 $26,593 8%
Store11 3,108,976 2,724,971 384,005 341,764 $48,001 12%
Store12 2,511,443 2,303,222 208,221 185,316 $26,028 8%
Store13 2,208,725 2,045,200 163,525 145,537 $20,441 7%
Store14 2,212,504 1,812,628 399,877 355,890 $49,985 18%
Store15 2,767,678 2,622,685 144,993 129,044 $18,124 5%
Store16 2,964,527 2,753,039 211,488 188,224 $26,436 7%
Store17 2,506,977 2,454,609 52,368 46,608 $6,546 2%
Store18 2,332,957 2,100,988 231,969 206,452 $28,996 10%
Store19 3,034,522 2,419,343 615,180 547,510 $76,897 20%
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
81
Store20 2,163,252 1,579,611 583,641 519,441 $72,955 27%
Store21 2,415,475 2,339,668 75,808 67,469 $9,476 3%
Store22 3,055,294 2,693,323 361,970 322,154 $45,246 12%
Store23 2,330,794 2,000,496 330,299 293,966 $41,287 14%
Store24 2,283,027 1,972,611 310,417 276,271 $38,802 14%
Store25 2,033,274 1,685,498 347,776 309,521 $43,472 17%
Store26 1,083,103 980,395 102,708 91,410 $12,838 9%
EnergySaverProgram(ESP)
Thefollowingtable isacase‐studyfromtheM&VGuideDevelopmentprojectshowingthesavingsfromamajorlightingretrofitprojectinSydneyMasonicCentre.TheprojectwasprovidedbyEnergySaverasatrialprojectforM&V.
Table4‐4. ESPMeasurement&Verificationpreliminaryresults
Program EnergySaver Site SydneyMasonicCentre
ProjectDescription LightingupgradefromincandescenttoLEDswithinGrandLodgeRoomandBanquetHallfunctionrooms.Theoldandretrofittedlightingsystemscanbedimmedfrom0to100percent.
M&VMethodology IPMVPOptionA1. Conductloadtesttomeasureinputpowerdraw(amps)foreachlightingcircuit
andmeasurethecorrespondinglightoutput(lux).2. Developbaselineandpost‐retrofitmodelsforpowerdraw.3. EstimateoperatinghoursfromfunctiondatabetweenJanuaryandMay2012.4. Extrapolatesavingsacross12months.
BaselinePeriod December2011
SavingsanalysisPeriod
01January2012to31May2012(5months)offunctionsdataJune2012forpostretrofitpowerdraw/lightlevelreadings
Actual Savings
Electricity (kWh) Cost ($) Greenhouse Gas Emissions (t.CO2-e) (@0.89 kgCO2/kWh)
Savings %
GrandLodgeRoom 35,310 $5,296 31 78%
BanquetHall 27,253 $4,088 24 66%
Total 62,563 $9,384 56 72%
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
82
Context Savingsanalysiswasbasedontheassumptionthattheretrofittedlightingsystemiscontrolledusingsimilarstepsaswiththeoldsystem.However,aslightinglevelshavesignificantlyimproved,differentcontrolstepsmaybeusedwhichmayaffectthesavingsestimates.
GovernmentSustainabilityPolicy(EEGSP)
Thefollowingtable isacase‐studyfromtheM&VGuideDevelopmentprojectshowingthesavingsfromchillerretrofitinWestmeadHospital.ThisprojectwasprovidedbytheGovernmentSustainabilityProgramasatrialprojectforM&V.
Table4‐5. EEGSPMeasurement&VerificationpreliminaryresultsfortheWestmeadHospitalcasestudy
Program GovernmentBuildingRetrofitProgram
Site WestmeadHospital
ProjectDescription
Upgradeoftwoexistingchillers(installed1978and1983)to2x4000kWvariablespeeddrivecentrifugalchillers.Additionaltwocoolingtowerstomeetincreasedcapacityandprovideimprovedcondenserwatertemperaturecontrol.Newcondenserwaterandchilledwaterpumpswithvariablespeeddrives.Newintegratedcontrolsystemwithautomaticsystemoptimisationcapability.
M&VMethodology
IPMVPOptionB/C–Regressionbasedanalysisofchilledwaterplant(chillers,pumpsandcoolingtowers),analysinginputelectricityuseagainstambienttemperature(informofcoolingdegreedays).
BaselinePeriod 1July2010to30June2011(12months)
Savings analysis Period
01August2011to31March2012(8months)
ActualSavings
Electricity(kWh) Cost($) GreenhouseGasEmissions(t.CO2‐
e)(@0.89kgCO2/kWh)
Uncertainty(%)at95%confidence
1,196,680 $138,760 1,065 ±29%
Context Savingsrepresenta12.4%reductioninchilledwaterplantenergyuse(measurementboundary)anda3.6%reductionoftotalsiteelectricityuse.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
83
0
500,000
1,000,000
1,500,000
2,000,000
2,500,000M
on
thly
Ele
ctri
city
Use
(kW
h)
HVAC kWh
Modelled HVAC kWh
Savings Period:1,196 MWh$138,0001,280 t.CO2-e12.4%
The followingtable isacase‐studyfromtheM&VGuideDevelopmentprojectshowingthesavingsfromastartingphaseoftheCentralPassageofSydneyOperaHouselightingretrofit.ThisprojectwasprovidedbytheGovernmentSustainabilityProgramasatrialprojectforM&V.
Table4‐6. EEGSPMeasurement&VerificationpreliminaryresultsfortheCentralPassageoftheSydneyOperaHousecasestudy
Program GovernmentRetrofitProgram
Site SydneyOperaHouse,CentralPassage
ProjectDescription
Lightingupgradeinvolvinglamp/fixturereplacementandintroductionofcontrols.
M&VMethodology IPMVPOptionB1. Measurebaselineandpost‐retrofitenergyuseusingadataloggerforthe"Central
Passage".2. Determinesavingsforpost‐retrofitperiodbyaligningdaysandadjustingfor
changesinSunrise/Sunsettimesduetophotoelectriccellcontroloflighting.3. Determineaveragehourlyusageandsavingsfordayandnightperiods.4. Useannualdataforsunrise/sunsettoextrapolatethe2weeksofdataacross12
months.5. Assumesteadystateoperationoneverydayoftheyear.
BaselinePeriod 12thto24thJuly2011(12days)
SavingsanalysisPeriod
25thOctoberto6thNovember2011(12days). Savingsextrapolatedusing2011solardata.
ActualSavings Electricity (kWh) Cost ($) Greenhouse Gas
Emissions (t.CO2-e) (@0.89
kgCO2/kWh)
Saving %
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
84
58,087 $8,713 52 32%
Context ThesavingscalculatedaboveapplytotheCentralPassagepartoftheupgrade.
EnergyEfficiencyforSmallBusinessProgram
Of the1,259 rebatedbusinesses in theAusgrid andEssential Energy’s network areas,509 were matched with billing data for over 12 months both before and after theretrofit.Of the509businesses,331weresuccessfullymodelled toestablishabaselinewhich enabled an M&V analysis under the Option C (whole building) of the IPMVP(InternationalPerformanceMeasurement&VerificationProtocol).
For the 331 businesses included in theM&V analysis, a total of 1,867MWhhas beensavedinthefirstyear,or8.3%relativetothebaseline.Onaverage,everybusinesssaved5.64MWhor9.3%relativetoitsbaselineforthe1styear.Thisishighlycomparabletothe initial estimated saving by the auditors of 5.2 MWh per business per year. Thefollowing tables show theprojects groupedbybusiness types and retrofit technology,respectively.
Table4‐7. EESBPMeasurement&Verificationpreliminaryresultsbybusinessactivity
BusinessActivity Number ActualUsage(kWh)
EnergySavings(kWh)
CostSavings($@$0.20/kWh)
EnergySavings%(toBAU)
Accommodation 12 1,363,717 158,488 31,690 10.4%
Agriculture,forestry,fishing 16 905,360 61,240 12,248 6.3%
Café/Restaurant 90 7,155,684 314,870 62,974 4.2%
Communication 2 112,483 ‐1.579 ‐319 ‐1.4%
Construction 1 25,970 3,496 699 11.9%
Cultural/RecreationalService
7 774,476 14,606 2,921 1.9%
Education 2 31,257 2,969 594 8.7%
Electricity,gas&water 2 13,314 951 190 6.7%
Finance/InsuranceServices 3 41,369 9,578 1,916 18.8%
Health/Communityservice 23 1,183,512 143,246 28,649 10.8%
Manufacturing* 7 607,952 ‐145,537 ‐29,107 ‐31.5%
Personal/Otherservices 34 952,225 130,863 26,173 12.1%
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
85
Property/Businessservices 9 131,723 24,038 4,808 15.4%
Retailtrade 105 5,592,693 753,109 150,622 11.9%
Transport/Storage 15 1,700,393 365,616 73,123 17.7%
Wholesaletrade 3 62,278 30,843 6,169 33.1%
GrandTotal 331 20,654,406 1,866,739 373,348 8.3%
*Oneofthemanufacturingsitehasseena74%increaseinconsumption.Thisincreaseismostlikelytoberelated to some non‐EESBP factor e.g. production growth.However,without further investigation, it isbettertoleaveitintoobservetheobjectivityofM&V.
Table4‐8. EESBPMeasurement&Verificationpreliminaryresultsbytechnology
Technologysummary Number ActualUsage(kWh)
EnergySavings(kWh)
CostSavings($@$0.20/kWh)
Savings%
Aircompression 2 65,904 11,236 2,247 14.6%
Boilers 4 284,955 28,417 5,683 9.1%
Hotwater 21 966,406 68,615 13,723 6.6%
HVAC 56 2,921,560 245,469 49,094 7.8%
Insulation 3 86,777 ‐78 ‐$16 ‐0.1%
Lighting 124 6,865.519 1,030,375 206,075 13.0%
MotorsandVSDs 5 344,493 49,272 9,854 12.5%
Refrigeration 86 7,670,508 254,056 50,811 3.2%
Multiple 30 1,447,847 179,379 35,876 11.0%
GrandTotal 331 20,654,406 1,866,739 373,348 8.3%
EstimatesoutsidethecurrentOEHprograms
Billingdataanalysiswasundertakenfor1,000lowincomehomeswithceilingsinsulatedbytheHousingNSW.Unfortunatelytheprogramdidnotcollectparticipantconsentsfordisclosingtheirbillingdata.TheanalysishadtousethedataobtainedforHPSP.AstheinsulationdateandHPSPassessmentdataareentirelydis‐aligned,itsignificantlydegradedthequalityofthedataforanalysis.Asaresult,nosignificantresultcouldbeestablished.
FinalReport–August2012 2012evaluationofNSWenergyefficiencyprograms
86
Appendix3. Keyreferences
CaliforniaPublicUtilitiesCommission,EnergyEfficiencyEvaluationReport,July2010
CaliforniaPublicUtilitiesCommission,EnergyDivisionScenarioAnalysisReport,July2010
IPART,ReviewofNSWClimateChangeMitigationMeasures,May2009
IPART,Changesinregulatedpricesfrom1July2012,June2012
MirjamHarmelink,LarsNilssonandRobertHarmsen,Theory‐basedpolicyevaluationof20energyefficiencyinstruments,EnergyEfficiencyVolume1,Number2,Springer,2008
U.S.NationalActionPlanforEnergyEfficiency,ModelEnergyEfficiencyProgramImpactEvaluationGuide.PreparedbyStevenR.Schiller,SchillerConsulting,Inc,2007
U.S.NationalActionPlanforEnergyEfficiency,UnderstandingCost‐EffectivenessofEnergyEfficiencyPrograms:BestPractices,TechnicalMethods,andEmergingIssuesforPolicy‐Makers.EnergyandEnvironmentalEconomics,Inc.andRegulatoryAssistanceProject,2008
Reed,J.H.,Jordan,G.,&Vine,E.,Impactevaluationframeworkfortechnologydeploymentprograms,USDepartmentofEnergy,2007