Oklahoma Gas and Electric Company 2017 Oklahoma Demand Programs Annual Report In Accordance with Annual Reporting Requirements Title 165: Oklahoma Corporation Commission Chapter 35. Electric Utility Rules Subchapter 41. Demand Programs 165:35-41-7. Reporting July 2018 Revised October 2018
308
Embed
· 3 Oklahoma Gas and Electric Company Contents 1.0 Executive Summary
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
4.2 Program Evaluations ......................................................................................................................... 57
4.3 Program Ratios .................................................................................................................................. 59
5.6 Water Savings Calculation Methodology .......................................................................................... 73
5.7 Implementer Information ................................................................................................................. 75
5.8 Letter to PUD .................................................................................................................................... 77
4
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
Figures Table 1 - Budget vs Actual Program Costs ............................................................................................................ 9
Table 2 - Actual Program Costs by Spend Type .................................................................................................... 9
Table 3 - Program Savings (Net) .......................................................................................................................... 10
Table 4 - Lifetime Savings and Program Net to Gross Ratios .............................................................................. 10
Table 5 - Program Participation .......................................................................................................................... 10
Table 6 - Cost-Effectiveness – TRC Net Benefits ................................................................................................. 10
Table 39 - Program Evaluations .......................................................................................................................... 57
Table 40 - Program Cost-Effectiveness Ratios .................................................................................................... 59
5
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
1.0 Executive Summary Oklahoma Gas and Electric Company (“OG&E” or “Company”) is submitting its Comprehensive Demand
Program Portfolio Annual Report for 2017. This report is required to be submitted to the Oklahoma
Corporation Commission (“OCC” or “Commission”) by July 1, 2018, pursuant to the Annual Reporting
Requirements in OAC 165:35-41-7, and with the extension granted by the OCC.
OG&E began implementation of its Quick Start Demand Programs in July 2008. The Quick Start effort
was followed by OG&E’s first Comprehensive Demand Portfolio for Program Years (“PY”) 2010-2012.
OG&E’s 2013-2015 Comprehensive Portfolio was approved on December 20, 2012, by the Commission
in Order No. 605737 in Cause No. PUD 201200134.
The 2016-2018 approved 3-year portfolio is expected to provide a reduction of 90,508 kW of demand
and 284,867,210 kWh of energy savings over the life of the measures installed through the programs, at
a cost of $116,008,163. Below are the results from the 2017 program year.
2017 Results:
Net kW 28,876
Net kWh 147,479,258
Actual Expenses $37,587,422
Performance Incentive Earned $ 5,315,710
Lost Net Revenues $ 5,920,000
Levelized Cost per kWh* $0.047
*Levelized Cost per kWh assumptions are at a Discount Rate of 8.32% and Line Loss of 7.76%
Program Implementation in 2017:
OG&E administered the Energy Efficiency (“EE”) and Demand Response (“DR”) Programs in 2017. The
Company uses third-party implementers to execute certain programs. Please see attachment 5.7 for
information about these implementers.
Evaluation, Measurement, and Verification:
Evaluation, Measurement, and Verification (“EM&V”) of the Energy Efficiency and Demand Response
Programs was conducted by ADM Associates. For a complete report on their evaluation protocols and
their results, please refer to attachment 5.1.
Lost Net Revenue (“LNR”):
In accordance with Title 165 Chap. 35 sub chap. 41, and Order Number 648327, OG&E can recover Lost
Net Revenues resulting from the successful implementation of all Demand Programs, except for the
Education Program, and 40% of the IVVC verified savings. The LNR of $5,920,000 is the amount stated in
the Joint Stipulation and Settlement Agreement in Cause No. PUD 201700496, Final Order 679358,
approved by the OCC on June 19, 2018, for Program Years 2016 and 2017.
Incentives:
In accordance with Title 165 Chap. 35, sub chap. 41, OG&E is also allowed to collect a performance
6
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
incentive based on the amount of energy reductions achieved through the Demand Programs. The
incentive is based on the performance of the Total Resource Cost (“TRC”) test and the Utility Cost Test
(“UCT”). OG&E can collect 15% of the net benefits capped at 15% of total program expenditures. In
2017, OG&E earned $5,315,710 in incentives.
Program Highlights in 2017:
Home Energy Efficiency Program (“HEEP”)
The HEEP program, supporting residential customers throughout OG&E’s Oklahoma service territory,
achieved 45,590,079 kWh, or 164% of the stipulated 2017 net annual savings goal of 27,857,573 kWh.
This marks an all-time high energy savings performance since the start of this program offering in 2010.
Positive Energy – New Home Construction Program (“PE-NHC”)
The PE-NHC program improved savings measurement and calculation methodologies for 2017, yielding
higher savings than the previous methods. PE-NHC generated a net annual savings of 4,251,672 kWh
which is 214% of the 2017 goal of 1,991,220 kWh.
Weatherization Residential Assistance Program (“WRAP”)
The WRAP weatherized 3,453 homes in 2017. The savings were 147% of goal at 11,733,069 kWh. WRAP
partnered with Oklahoma Natural Gas Company (“ONG”) and Central Oklahoma Habitat for Humanity to
help provide weatherization services to OG&E customers.
Commercial Energy Efficiency Program (“CEEP”)
The CEEP program, supporting energy efficiency projects for commercial and industrial customers
throughout OG&E’s Oklahoma service territory, generated savings of 83,802,238 kWh, or 157% of the
stipulated 2017 net annual savings goal of 53,222,962 kWh.
Education Program (“EP”)
The Education program provided energy education to help customers make informed decisions about
their energy use. The EP participated in twenty-three community events and presented fifteen internal
meetings to inform customers and employees of the energy efficiency programs available. The EP also
held six rural school Education events involving 550 kids, called Kids Energy Education Project.
Integrated Volt-Var Control Program (“IVVC”)
The IVVC program provides savings at the meter for both residential and commercial customers. Thirty-
nine banks were added in 2017 and studied using a continuous method of operation. Delayed
installations and operational complexities in 2017 affected the savings results of the study. The savings
from the continuous method resulted in 2,102,200 kWh, which is 35% of the stipulated goal. The ADM
evaluation can be seen in attachment 5.2.
Research and Development (“R&D”)
The R&D program researched four vital aspects of energy efficiency in 2016. They were: residential in-
home technology and Smart Hours upgrades, commercial and small industrial price response
participation, load disaggregation technology, and LED lighting services. For 2017-2018 R&D revised
7
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
their research to include Direct Solar Powered Hot Water, Energy Storage for low income, Electric
Vehicle Charging, and the Midwest City Geo-Targeting Pilot. The full report can be seen in section 3.7
Portfolio Savings by Program: Figure 1 - kWh Savings by Program
Figure 2 - kW Savings by Program
Cost Effectiveness:
OG&E engaged ADM Associates to provide an analysis of the cost-effectiveness of each of the Demand
Programs. The portfolio is cost effective, with a portfolio TRC ratio of 1.67. For a full list of program cost
effectiveness results, see section 2.0. For the full analysis, please see attachment 5.1.
HEEP31%
PE-NHC3%
WRAP8%
CEEP57%
IVVC1%
KWH SAVINGS BY PROGRAM
HEEP26%
PE-NHC4%
WRAP11%
CEEP45%
IVVC14%
KW SAVINGS BY PROGRAM
8
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
Reduced Emissions and Water Consumption:
In accordance with Title 165 Chap 35, sub chap. 41-7, Reporting, OG&E is required to report emissions
reductions and generation water saved due to portfolio kWh savings. In 2017, OG&E saved 193.8 tons of
Sulfur Dioxide, 79.1 tons of Nitrous Oxides, 115,935.8 tons of Carbon Dioxide, and 51.1 million gallons of
fresh water. For a list of reduced emissions and water consumption see Table 10 in section 2.0; for
calculation methodology see Attachment 5.6.
Conclusion:
OG&E spent 93.8% of its proposed budget and achieved 152% and 80% of its energy and demand goals,
respectively.
9
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
2.0 Portfolio Impact The following tables summarize the portfolio statistics, including:
Program costs
Program savings (energy and demand)
Program participation
Inducements by program
Portfolio cost-effectiveness
Historical portfolio performance 2011-2016
Reduced emissions and water consumption
Table 1 - Budget vs Actual Program Costs
Program Cost
Budget Actual % of Budget % of Portfolio
HEEP $11,214,053 $10,801,446 96% 29%
PE-NHC $1,247,249 $1,326,946 106% 4%
WRAP $5,400,915 $5,312,511 98% 14%
CEEP $17,836,397 $17,112,624 96% 46%
Education $800,000 $787,145 98% 2%
IVVC $2,884,148 $2,113,607 73% 6%
R&D $565,000 $132,880 24% 0%
Planning & Regulatory $120,000 $263 0% 0%
Total $40,067,762 $37,587,422 93.81% 100.00% *The HEEP, WRAP, and CEEP budgets were adjusted per the 6/18/2017 letter to PUD. See attachment 5.8 for entire letter.
Budget Variance: The program cost variance was 27% under budget. Some of the 2017 deployments
were constructed during 2016 and the associated costs were paid during 2016.
Deployment Year 2017
The summer of 2017 was a challenging test season for the IVVC deployment at OG&E. Seventy-seven (77)
circuits deployed in 2017 yielded a total of 4.04 MW of peak demand reduction and 2,102 MWh of
annualized energy savings based on the continuous method of analysis as discussed in the ADM 2017 IVVC
EM&V Report. IVVC is operated on substation transformer banks (a substation bank typically serves two
to three circuits), with results aggregated to the substation bank level. An adjustment was added to
account for substation and transmission losses using the substation and transmission loss factors of
0.9137 for peak demand and 0.96 for energy.
The table below provides a summary of the 2017 banks and their demand (MW) and energy (MWh)
reduction contributions the overall OG&E demand reduction and energy efficiency programs. Red
highlight indicates banks that were enabled in loss reduction mode only.
Table 33 - IVVC Demand Reductions and Energy Savings
Substation/Bank
Peak Demand
Reduction (MW)
Peak Demand
Reduction (%)
Summer Demand
Reduction Energy Savings
(MWh)
Annualized Loss
Minimization Energy Savings (MWh)
Total Energy Savings (MWh)
Ardmore Bank 4 0.12 0.01 14.10 9.30 23.40
Ardmore Bank 6 0.22 0.02 8.00 25.20 33.20
Ardmore West Bank 3 0.21 0.02 14.70 7.10 21.80
Beeline Bank 4 0.00 0.00 0.00 0.70 0.70
Bowden Bank 1 0.01 0.00 19.90 35.10 55.00
Cedar Ave Bank 1 0.12 0.01 36.20 39.80 76.00
Cedar Ave Bank 2 0.00 0.00 0.00 0.80 0.80
Cedar Ave Bank 3 0.08 0.01 47.20 8.40 55.60
Cherry Creek Bank 1 0.03 0.00 35.60 9.70 45.30
Cherry Creek Bank 2 0.16 0.01 61.20 15.90 77.10
Cherry Creek Bank 3 0.24 0.02 60.80 23.90 84.70
Plan Actual Variance % Variance
IVVC $2,884,148 $2,113,607 ($770,541) (27)
ProgramBudget
39
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
Chestnut Bank 1 -0.16 -0.01 30.70 18.80 49.50
Cleveland Ave Bank 1 -0.07 -0.01 34.00 16.50 50.50
Cleveland Ave Bank 2 -0.04 0.00 53.80 9.60 63.40
Crescent Bank 1 0.00 0.00 0.00 137.50 137.50
Glenwood Bank 2 0.23 0.02 53.50 4.30 57.80
Glenwood Bank 3 0.04 0.00 12.70 2.20 14.90
Hancock Bank 1 0.31 0.02 48.10 54.20 102.30
Hancock Bank 2 0.25 0.02 42.00 13.30 55.30
Hancock Bank 3 0.08 0.01 -3.80 6.60 2.80
Hemlock Bank 1 0.14 0.01 40.50 3.60 44.10
Hemlock Bank 2 0.29 0.02 39.60 7.90 47.50
Inglewood Bank 1 0.00 0.00 0.00 87.70 87.70
Inglewood Bank 2 0.00 0.00 0.00 88.10 88.10
Lone Grove Bank 1 0.00 0.00 0.00 3.50 3.50
NE Enid Bank 1 -0.10 -0.01 11.30 12.10 23.40
Park Lane Bank 3 0.41 0.03 91.70 29.20 120.90
Remington Park Bank 2 0.12 0.01 45.40 0.60 46.00
Riverside Bank 1 -0.04 0.00 12.40 3.40 15.80
Riverside Bank 2 0.11 0.01 11.60 10.60 22.20
SE 15th St Bank 1 0.28 0.02 79.70 8.50 88.20
South Ada Bank 2 0.00 0.00 0.00 17.20 17.20
South Ada Bank 3 -0.02 0.00 10.00 1.80 11.80
St Gregory Bank 2 -0.02 0.00 40.40 81.10 121.50
St Gregory Bank 3 0.31 0.02 26.70 4.90 31.60
Tower Heights Bank 1 0.16 0.01 8.80 16.60 25.40
Tower Heights Bank 2 0.16 0.01 6.40 49.90 56.30
Valley View Bank 1 0.03 0.00 74.90 3.00 77.90
Valley View Bank 2 0.03 0.00 53.40 27.90 81.30
Sub Totals/Averages 3.69 0.01 1121.50 896.50 2018.00 Trans. & Sub Loss Reduction
84.10 84.10
Total incl. Trans & Sub Loss Reduction 980.60 2102.10
OG&E utilized the Open System International (“OSI”) VVC system, which improves distribution system
visibility, provides better accuracy and granularity of data, and allows for better forecasting and
integration of real-time data.
Challenges and Opportunities
The energy and demand savings from the IVVC program were measurably less than planned values for
40
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
2017. Several factors contributed to the underperformance:
1) OG&E encountered significant deployment and operational challenges. In particular, several circuits included line voltage regulators that were not fully commissioned for the majority of the summer testing period, which contributed to operational constraints and reducing the measured demand and energy impacts.
2) The OSI VVC System was not fully commissioned until June 15, 2017, adversely affecting 2017 overall savings estimates. Further, seven of the thirty-nine deployed banks were not enabled in demand reduction mode due to operational constraints and construction delays. However, these seven banks were enabled in loss reduction mode.
3) Reporting and measurement in the year of deployment presented challenges. In previous deployment years, OG&E has delayed the reporting of benefits to allow for fine tuning of the VVC control scheme to accommodate unexpected field conditions that adversely impact IVVC operation.
OG&E expects the overall operation of VVC to improve with the fine tuning of VVC controls, which
should contribute to significantly improved results from the 2017 deployment going forward.
The following is a general overview of lessons learned:
Refining analytic processes can yield a larger reduction
Additional reduction can be achieved through minor capital improvements
Deploying a new IVVC system can result in commissioning delays of VVC operation, short-term operational constraints, and changes in control schemes
Information technology, effective processes and trusted partners are keys to program success
Outlook for Continuation, Expansion, Reduction, or Termination
2018 is the sunset year for IVVC as a Demand Program.
Table 34 - IVVC Circuit Deployment Schedule
Circuits 2013 2014 2015 2016 2017
Plan 52 60 60 60 72
Actual 61 63 62 68 77
Program Benefits
A key benefit derived from IVVC is shaving peak demand which, if objectives are achieved, will
contribute to OG&E’s effort to avoid new fossil fuel generation until 2020. Also, by shaving peak
demand, OG&E customers avoid the additional incremental energy costs associated with power
generation or purchase during peak periods. Optimizing the voltage profile along the length of the
circuit and reducing voltage at the circuit head creates energy savings that will be passed along to OG&E
customers through a reduction in consumption. IVVC also provides the added benefit of reducing line
losses throughout the year, as well as increased visibility into the condition of IVVC circuits.
41
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
3.7 Research & Development (“R&D”) Introduction
Research to test and validate new features of program offerings is vital to the success of OG&E's ability to
meet program metrics and customer expectations that will, or may, emerge in Oklahoma. The intent of
R&D activities is to develop knowledge related to potential new EE processes, products, or services.
Findings will be incorporated into future demand portfolio planning. When designing its R&D programs,
OG&E has intentionally focused on the potential of measures to positively impact demand programs and
end-use energy efficiency in general for its customers in Oklahoma.
Technology advancements and cost reductions are creating opportunities to accomplish demand program
goals and objectives with new measures that are increasingly cost-effective. OG&E’s desire with the R&D
programs is to stay ahead of the curve to be prepared for application of emerging technologies in future
energy efficiency and DR programs. OG&E seeks to develop programmatic opportunities with new
technologies now, while their cost-effectiveness under existing demand program cost-effectiveness tests
is marginal or below benefit to cost ratios of one. This will enable OG&E to evaluate the viability of
emerging technologies in practical use cases while simultaneously studying cost trends to develop
forecasts of cost reductions that will help identify the timeframes within which such measures are market-
ready and sufficiently cost-effective to be included in demand programs.
OG&E’s R&D projects seek to strike a balance of multiple considerations: peak demand reduction, energy
savings, capacity savings, customer and market education, and market development and transformation.
In addition, OG&E endeavors with its R&D projects to learn how application of these tools and
technologies can empower customers to make the most efficient use of utility capacity and energy, thus
reducing energy waste.
Program Budget Table 35 - R&D Costs
Table 36 – R&D Budget Variance
Budget Variance: 2017 R&D budget needs were not as extensive as originally estimated. R&D was 76%
under budget.
Findings and Analysis
For the 2016-2018 DPR Research and Development program, OG&E originally planned 4 studies.
Specifically, they were:
Delivery Marketing
R&D $132,880 $0 $0.00 $0.00 $0 $132,880
ProgramProgram Administration
Inducements EM&VUtility
AdministrationTotal
Plan Actual Variance % Variance
R&D $565,000 $132,880 ($432,120) (76)
ProgramBudget
42
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
Residential In-Home Premium Technology products and services
Commercial and Small Industrial Price Response Participation
Load Disaggregation / Technology
LED Lighting Services
As described in the “OG&E 2016 Demand Portfolio R&D report”, OG&E recommended that no further R&D
investment be made in pursuing the research projects of Residential In-Home Premium Technology and
fee paid products and services, Commercial and Small Industrial Price Response Participation, and Load
Disaggregation / Technology, due to research findings indicating less than favorable impact on EE / DR.
The LED Lighting Services project continued to move forward and results of the Moore and OKC R&D
programs were discussed in the OG&E Demand Portfolio Stakeholder meeting held October 26th, 2017
and are described below.
As also communicated in the Stakeholder meetings held March 10th, 2017 and October 26th, 2017, OG&E
proposed alternative research and development to replace the planned studies that were discontinued.
These updated programs are:
2017-2108 DR / EE Revised Research Studies
• New Technologies
• Direct Solar Panel Powered Hot Water Efficiency
• Storage for Low Income / Senior Multi-Family Demand Response / Energy Efficiency
• Electric Vehicle Charging
• Electric Vehicle charging with Batteries for Demand Response
• Electric Vehicle Charging Equipment Efficiency
In 2017 for each of the DR/EE revised research studies, OG&E identified and defined the problem,
researched viable solutions and designed the experiment with hypotheses related to the expected
outcomes. Results of any of the research and development undertaken in 2017 on these revised studies
is documented below.
Direct Solar Panel Powered Hot Water Efficiency
Water heaters are energy intensive, whether used in the home or for commercial purposes. As an
example, for homeowners, heating water is the 2nd largest energy expense behind heating and cooling.
Recently, technology for hot water heaters that use energy generated by rooftop photovoltaic (“PV”) solar
panels has become commercially available.
It is important for OG&E to understand the potential for this renewable energy technology to save energy
in both residential and commercial settings.
43
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
There are two hypotheses for this project:
1. Solar PV powered water heating is practical in a single-family home application in OG&E’s service
territory and can produce energy savings when compared to conventional water heating
methods.
2. There are several specific commercial applications where Solar PV powered water heating may
be appropriate and can produce energy savings when compared to conventional water heating.
This project is intended to quantify the efficiencies of this technology in comparison to traditional water
heating systems in real-world usage scenarios and understand if it can result in a positive cost / benefit
energy efficiency measure for inclusion in a future demand program.
OG&E initiated the design of the research in October of 2017, based upon the following high-level
requirements:
Requirement Objective Measurement
Install residential and commercial test systems
Install testing systems for evaluation of future DR
/ EE program measures / programs
Measure install time / cost /
issues / risks
Meter and collect data on energy use
Understand impacts / savings opportunity of solar PV systems
Understand sizing of systems for residential and commercial product design
Capture data for usage in study
Measure forecasted consumption vs actual consumption of energy to heat water
Measure energy production by PV vs energy consumption to completely heat water
Document benefits from developing this study
Capture benefits as a product of the study as an input to future programs / measures utilizing this technology
Measure actual vs hypothesized benefits
Document and track all costs related to developing and deploying this R&D project
Fully understand all costs related to development and installation of this project to have inputs to future DR / EE program measures / programs
[All] Cost metrics
Understand, document and track all related / ongoing O&M
Understand what this will cost to own / maintain in order to provide this benefit to customer segment
All O&M data / information documented and tracked
OG&E began the implementation in late December 2017 at its North District Service Department. This
location was selected to test the technology for several reasons. First, this is an existing OG&E owned
facility which resulted in no additional cost to procure or prepare the research site. Next, this site had
more than sufficient available space to perform the installation and engineering of the test technology
without effect on the facility or the personnel. Finally, this location is isolated from any customer induced
variations and allows for a controlled experiment upon which to test the hypotheses. The completion of
the project will be accomplished in 2018. This includes the conclusion of implementation, conducting the
research, testing the hypothesis, analyzing the data and then formulating conclusions by reviewing the
44
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
data and determining if it confirms or disproves the hypothesis.
Storage for Low Income / Senior Multi-Family Demand Response / Energy Efficiency
As the cost of electric energy storage (batteries) continues to decline, the use of this technology to manage
energy use and shift peak demand will become more prevalent. It is important that OG&E gain a deeper
understanding of how this technology works in a real-world application. This project is intended to
quantify the efficiencies and effectiveness of this technology and understand if it can result in a positive
cost / benefit energy efficiency measure for inclusion in a future demand program.
There are two hypotheses for this project:
1. Battery technology can be effectively applied on a multi-family, low-income (or senior / fixed-
income) housing facility and operated in a way that improves load factor of the facility by storing
energy in the battery during off-peak periods and dispatching energy from the battery during peak
periods, thus reducing stress on grid infrastructure assets. In doing so, it will also reduce
customers’ energy bills at the facility through charging the batteries with low cost (off peak)
energy and discharging the batteries during the (on peak) higher priced energy times of the day.
As an added benefit, the batteries will provide improved reliability especially for this sensitive
customer segment that have life-sustaining medical devices or critical needs that require energy
to keep the devices functioning.
2. Applying battery technology in a low-income (or senior / fixed-income) neighborhood will
increase the penetration of the broader energy efficiency programs with these customers and
others.
This technology is applicable to most residential customers however, this project is targeted to low-
income and senior / fixed-income customers who may not have financial access to battery technology
unless specific programs are put in place to engage this traditionally hard-to-reach market. This may be
particularly true with low-income or senior customers living in multi-family dwelling units where there
may be even less incentive for facility owners and property managers to participate in technology
programs such as energy storage/batteries. In an effort to address this concern, a number of utilities
around the country have piloted low-income electric energy storage (battery) projects integrated with
energy efficiency programs in order to gain the benefits of this technology and, at the same time, increase
the penetration of EE programs with these customers.
OG&E initiated the design of the research in 2017, based upon the following high-level requirements:
Requirement Objective(s) Measurement
Install battery and control technology at a multi-family / low income (and/or senior) housing facility
Deploy DER technology
Create model for future low-income DR / EE programs / measures
Understand grid impacts with this type of system / install
Measure install time / cost / issues / risks
Measure impacts on Grid / systems
45
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
Meter and collect data on storage system utilization and customer energy use
Capture data for usage in study Measure forecasted storage utilization vs actual storage utilization
Measure forecasted customer usage vs actual customer usage
Study customer behavior and usage in relationship to implementation of technology
Understand behavioral changes due to having DER technology
Understand sizing of DER installation in relationship to forecasted consumption
Study changes in customer consumption in comparison to forecasted and actual storage availability
Document benefits from developing this study
Capture benefits as a product of the study as an input to future programs / measures utilizing this technology
Measure actual vs hypothesized benefits
Document and track all costs related to developing and deploying this R&D project
Fully understand all costs related to development and installation of this project to have inputs to future DR / EE program measures / programs
[All] Cost metrics
Understand, document and track all related / ongoing O&M
Understand what this will cost to own / maintain in order to provide this benefit to customer segment
All O&M data / information documented and tracked
The completion of project is targeted to be accomplished in 2018. This includes the conclusion of
experimental design, conducting the research, testing the hypothesis, analyzing the data and then
formulating conclusions by reviewing the data and determining if it confirms or disproves the hypothesis.
Electric Vehicle Charging with Batteries for Demand Response
The adoption of electric vehicles (“EV”) in OG&E’s service territory will drive the installation of EV charging
stations at homes, businesses and public locations. The impact of EV charging stations on the OG&E grid
could be significant due to the potential for
usage during system peak periods. This
may be especially true for the use of fast
chargers (also known as Level 3 chargers)
located on the Interstate Highway system
where drivers are likely to charge during
peak drive times, including afternoons
which coincide with OG&E’s system peak.
This is in contrast to home EV charging,
which is more likely to be scheduled
overnight during the off peak. Additionally,
Level 3 chargers draw a greater demand on
grid capacity when they are in use when
compared with level 1 or level 2 charging.
A potential solution to lessen the impact of
Level 3 charging on OG&E’s grid capacity is
46
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
to use electric energy storage (batteries). The batteries would store energy prior to OG&E system peak
and then discharge the energy during EV charging on the system peak, offsetting customer energy demand
and reducing the strain of the grid infrastructure.
There is one hypothesis for this project:
1. Electric energy storage (battery) technology can be effectively operated to support a Level 3
charging station on a highway corridor in a manner that offsets customer energy demand and
associated demand charges and improves the load factor on the associated circuit.
This project will quantify the Level 3 charging station load in relationship to the opportunity to discharge
the electric energy storage during the system peak. This will measure the offset of customer energy
demand and associated demand charges as well as improvement to the load factor and capacity reduction
on the associated circuit. This project is intended to quantify the efficiencies and effectiveness of this
technology and understand if it can result in a positive cost / benefit energy efficiency measure for
inclusion in a future demand program.
This project will benefit all customers as it tests whether electric energy storage (i.e. battery) technology
is effective in managing OG&E system impact of an emerging, potentially significant load (i.e. Level 3
charging stations).
OG&E initiated the design of the research in 2017, based upon the following high-level requirements:
Requirement Objective(s) Measurement
Install level 3 EVSE,
battery and control
technology
Deploy technology
Create model for future DR / EE programs / measures
Understand grid impacts with this type of system / install
Measure install time / cost / issues / risks
Measure impacts on Grid / systems
Meter and collect data on storage system utilization and customer energy use
Capture data for usage in study
Understand sizing of DER installation in relationship to forecasted consumption
Measure forecasted storage utilization vs actual storage utilization
Measure forecasted customer usage vs actual customer usage
Document benefits from developing this study
Capture benefits as a product of the study as an input to future programs / measures utilizing this technology
Measure actual vs hypothesized benefits
Document and track all costs related to developing and deploying this R&D project
Fully understand all costs related to development and installation of this project to have inputs to future DR / EE program measures / programs
[All] Cost metrics
Understand, document and track all related / ongoing O&M
Understand what this will cost to own / maintain in order to provide this benefit to future customers
All O&M data / information documented and tracked
47
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
Completion of the project is targeted to be accomplished in 2018. This includes the conclusion of
experimental design, conducting the research, testing the hypothesis, analysing the data and then
formulating conclusions by reviewing the data and determining if it confirms or disproves the hypothesis.
Electric Vehicle Charging Equipment Efficiency
EV charging equipment is available in varying voltage and electric current configurations, referred to in
the marketplace as Level 1 (110-120 V AC), Level 2 (220-240 V AC) and Level 3 (480 V DC +). These three
different chargers are reported to require different amounts of grid energy to produce the same level of
battery charge in an EV.
As EVs become more prevalent in the OG&E service territory, it becomes important for OG&E to
understand the demand program/energy efficiency potential for each type of EV charger under real world
conditions (e.g., temperature, EV battery charge state, length of charge, type of vehicle, etc.).
There are two hypotheses for this project:
1. EV Level 2 charging equipment is more energy
efficient than EV Level 1 charging equipment and is
practical in home and office charging applications.
2. EV Level 3 charging equipment is more energy
efficient than EV Level 2 charging equipment and is
practical in office, retail and other charging applications.
This project will quantify the efficiencies of each of these
technologies in comparison to each other and model the
optimized energy efficiency situational and customer
application of each of these levels of EV charging.
This project will benefit all customers as it tests whether higher level EV charging equipment can reduce
the use of OG&E grid energy. This project will also benefit EV customers as it will demonstrate the
potential cost savings that can be achieved by owning more efficient charging equipment.
OG&E initiated the design of the research in 2017, based upon the following high-level requirements:
Requirement Objective(s) Measurement
Install level 1, level 2 and
level 3 equipment
appropriate for test
Have level 1, 2 and 3 in a controlled environment to measure
Reuse of existing level 2 in study
Measure install time / cost / issues / risks
Measure impacts on Grid / systems
48
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
Meter and collect data on storage system utilization and customer energy use
Capture data for usage in study Measure forecasted storage utilization vs actual storage utilization
Measure forecasted customer usage vs actual customer usage
Document benefits from developing this study
Capture benefits as a product of the study as an input to future programs / measures utilizing this technology
Measure actual vs hypothesized benefits
Document and track all costs related to developing and deploying this R&D project
Fully understand all costs related to development and installation of this project to have inputs to future DR / EE program measures / programs
[All] Cost metrics
Understand, document and track all related / ongoing O&M
Understand what this will cost to own / maintain in order to provide this benefit to customer segment
All O&M data / information documented and tracked
Completion of the project will be accomplished in 2018. This includes the conclusion of experimental
design, conducting the research, testing the hypothesis, analyzing the data and then formulating
conclusions by reviewing the data and determining if it confirms or disproves the hypothesis.
LED Lighting Services
In 2017, OG&E successfully deployed two technical demonstrations in the Oklahoma City Metro area to
confirm that LED technology was viable, provided additional energy efficiencies to their customers, and
advanced controls and management to the utility. To implement this research project, OG&E installed 102
LED luminaires in three locations (Buck Thomas Park in Moore,
Adventure and Innovation Districts in OKC). These LED luminaires
are equipped with 102 sample smart photo controls each with
integrated Silver Springs Network (SSN) network viability. These
luminaires were tested with 1-10v dimming (10% to 100%), GPS
reporting, and on/off functionality. They were tested as well for
communications utilizing both a local communication test software
and Street Light Vision (SLV).
49
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
The findings of the study are that the efficiency of the LED compared to an HPS (HID) light is approximately
a 50% energy savings based on the installed wattage of the LED's. Most LED fixtures will have a 40-60%
savings based on comparison to past configurations. As to the fixtures, no outages or failures of any type
has been reported. A few of the controls had minimal issues due to wiring and installation problems, but
otherwise the 3 systems have worked flawlessly.
The SLV software has
performed as projected
and has more features
than expected.
Midwest City Geo
Targeting Pilot
The Midwest City geo-targeting pilot was undertaken to determine if a focused energy efficiency effort
targeted to a specific geographic area could 1) increase participation rates in energy efficiency
programs, 2) potentially lower the transformer loading, and 3) reduce billing related call volumes to the
OG&E call center in the target area. Internal research identified an area in Midwest City as a good
candidate for this pilot. The targeted area has relatively low participation in OG&E’s energy efficiency
programs, peak summer transformer loading at or just above capacity, and a high volume of billing
related calls. In addition to the target neighborhood in Midwest City, a neighborhood in The Village,
with a similar customer mix, energy efficiency participation, transformer loading and call volume, was
identified as a good candidate for a control neighborhood. The purpose of the control neighborhood
was to provide data on a geographic area receiving energy efficiency services on business as usual basis
for comparison to the Midwest City target neighborhood. The pilot was designed so only the marketing
effort was increased in the target neighborhood; there were no changes in the existing programs or the
inducement structure.
The Midwest City geo-targeting area below shows the overloaded transformers identified in the specific
targeted area. This area is located just north of Tinker Air Force Base.
50
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
Once both the target and control areas were identified, marketing plans were developed to focus on the
Midwest City neighborhood. The control neighborhood in The Village was planned to receive the same
energy efficiency marketing as the rest of the OG&E service territory. The increased marketing activity
and outreach in the Midwest City target area over the summer is shown below.
51
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
After the summer promotion period, the data collected indicate the marketing and outreach were
successful when comparing participation uptake between the target and control neighborhoods. The
results are shown below.
0
100
200
300
400
500
600
700
800
Test Control Test Control Test Control
Weatherization HEEP Small Business
Completions Enrollments Inquiries
52
Oklahoma Gas and Electric Company
Oklahoma Gas and Electric Company
While the data gathering for energy efficiency program uptake was relatively straightforward, the data
for transformer loading and call center billing call volumes proved to be more difficult to capture.
Equally challenging was developing models to adjust the data for weather, determining other variables
that may impact results and ensuring robust statistical models. Although data gathering and modeling
were planned to be complete early in the summer, this work continued through the end of 2017.
Despite the extended data collection modeling period, some anecdotal results proved positive,
however, more work needs to be completed before more meaningful results can be determined.
As a result of efforts in 2017 there were some lessons learned the will help guide efforts in 2018. These
include:
Start Early
o Plan marketing in advance with geo-targeting in mind
o Secure buy-in and alignment among project team
o Establish certainty that data is available, and any issues have been identified
Data is Key
o Cement understandings about who is collecting what and how
o Secure alignment on data assembly and exchange logistics
o Agree upon systems and processes to be used and support to be provided
Modeling Challenges
o Weather normalization is not trivial and can be done several ways
o Model specification, selection of explanatory terms drives results
o Statistical robustness is the key, more than just an R-Squared and alignment around
“what is good enough”.
To better analyze the data to determine if the desired outcomes are being achieved, the pilot study and
analysis will continue into 2018.
Next Steps
Results of any of this research undertaken will be discussed in the next OG&E Demand Portfolio
Table 4-68: How Customer learned of program....................................................................... 100
Table 4-69 Reasons for program participation......................................................................... 101
Table 4-70 HEEP HVAC Tune-Up and Replacement: Satisfaction by Program Component ... 104
Table 4-71 Familiarity with benefits of energy efficiency improvements prior to program participation ..................................................................................................................... 106
Table 4-72 Familiarity with benefits of energy saving activities prior to program participation.. 107
Table 4-73 As a result of program, how much more knowledgeable about energy efficiency? 108
Table 4-75 Could you correctly identify a typical incandescent light bulb, a CFL, and a LED light bulb if placed in front of you, outside of their packaging? ................................................. 112
Table 4-76 OG&E program participants who have purchased LEDs ....................................... 113
Table 4-77 Type of bulbs LEDs replaced ................................................................................ 113
Table 4-78 Name of retailer where LED bulbs were purchased in past six months ................. 114
Table 4-79 Deciding factors when selecting bulbs for purchase .............................................. 115
Table 4-80 Likelihood of purchasing LEDs if they cost more than $2.50 per bulb .................... 116
Table 4-81 Aware OG&E discounted certain LED bulbs in 2017 ............................................. 116
Table 4-82 Bulb Types Purchased Prior to 2017 ..................................................................... 116
Table 5-1 Energy Savings Summary for PE-NHC in PY2017 .................................................. 120
Table 5-2 Demand Reduction Summary for PE-NHC in PY2017 ............................................ 120
Table 5-3 Participants and Savings per Program Strata .......................................................... 121
Table 5-4 Participating and Savings per Builder ...................................................................... 122
Table 5-5 Participation and Savings per HERS Rater ............................................................. 123
2 The Arkansas Technical Reference Manual (TRM) version 6.0 can be found here:
http://www.apscservices.info/EEInfo/TRM6.pdf 3 Lifetime impacts are the sum of energy savings over the course of the measure’s effective useful life (EUL) and the
weighted average demand reduction across the lifetime of the measure divided by the EUL (in years).
Table 1-5 Cost Benefit Estimates and Net Benefits Summary
Program TRC UCT RIM PCT SCT TRC Net Benefits
HEEP 3.37 3.34 0.63 8.65 4.95 $23,541,213.57
PE-NHC 2.48 4.04 0.61 2.67 3.48 $3,196,816.28
WRAP 2.83 2.83 0.83 3.77 4.30 $8,490,182.12
CEEP 1.58 3.42 0.69 2.62 2.21 $21,407,491.24
IVVC 0.52 0.52 0.47 1.11 0.73 -$8,375,970.02
Energy Education Res 0.00 0.00 0.00 0.00 0.00 -$354,215.40
Energy Education C&I 0.00 0.00 0.00 0.00 0.00 -$432,929.93
Planning 0.00 0.00 0.00 0.00 0.00 -$262.58
Regulatory 0.00 0.00 0.00 0.00 0.00 $0.00
R&D 0.00 0.00 0.00 0.00 0.00 -$132,880.08
Total 1.65 2.32 0.65 2.85 2.37 $47,339,445.19
The portfolio, and all energy efficiency programs pass all cost tests, except for the RIM
test.
1.5 Summary of Process Evaluation Findings
This section presents the high-level findings and recommendations developed through
process evaluation activities for the PY2017 portfolio. The results of the process
evaluation research are largely positive. Program participants, service providers and
program staff were largely satisfied with the PY2017 Demand Program offerings. The
programs gained traction in PY2017, with increasing participation from service providers
and customers as compared to previous years. Key process related findings and
recommendations from the PY2017 evaluation are summarized below:
Key Process Findings for HEEP
High Participant Satisfaction: Results from the participant survey effort
indicate that customers highly value the services offered by HEEP, and recall
very positive experiences with the assessment and measure installation
process. Very few participants expressed dissatisfaction with any program
elements, and several participants provided open-ended commentary that
praised the program for the services it provides and the professionalism of its
trade ally staff. Satisfaction scores on nearly all program components were
consistently rated 4 out of 5 or higher, for both trade allies and participants
Demonstrated Education Effects: Feedback from the participant survey
suggests that the program is increasing customer knowledge of energy
efficiency equipment and energy efficiency behaviors that can be employed to
conserve energy and lower utility bills. Some customers have learned about
other utility offerings through the HEEP, potentially leading to additional energy
savings.
OG&E Oklahoma Demand Program Portfolio Evaluation
21
Smooth Enrollment Process: Regardless of how they requested an
assessment to begin participating in the HEEP, customers had an easy time
communicating with the program and scheduling their home assessment.
Among those who called the Customer Service center, 93 percent indicated
their experience was ‘very easy’, while the remaining customers rated their
experience as “easy”.
Adequate Database Quality: Except for the ex ante savings issues described
below, the Evaluators found the ex ante savings values within the database to
be accurate for most measures. Additionally, CLEAResult was very consistent
in responding to data requests and correcting errors when necessary.
Effective Trade Ally Portal: An elevated level of satisfaction was reported by
trade allies in relation to the portal that CLEAResult utilizes to share information
with trade allies.
Key Process Findings for PE-NHC
Simplified Marketing Approach: Staff simplified the program marketing
approach. The new approach focuses on key benefits of efficient new
construction. All builders were provided brochures and yard signs to promote
the program to their customers. Some builders also received counter signs for
use at home shows.
Duplication of Records Resulted in Overpayment: The evaluator found
duplicate records in the program tracking records. CLEAResult staff identified
the underlying cause and implemented a procedural change to prevent the
issue from re-occurring in the future. Rather than require builders to repay the
additional incentive payment, staff decided to double the incentive payment for
all builders to maintain goodwill. The evaluator concurs that this was the best
approach to resolving the issue.
Builders Interested in Additional Communication: There was a general
consensus that overall communication had decreased, and while builders
agreed that the program was running smoothly and they were receiving their
checks, they would appreciate more in-person communication from staff.
Builders offered suggestions such as personal outreach when the program
launches to ensure builders are aware of the appropriate point of contact for
the implementation contractor, host a short format lunch meeting, or visit in
person to present checks.
Key Process Findings for WRAP
Program Functioning Well: WRAP was first offered as a full program in 2010.
Over the years staff has continually made improvements and at this time
OG&E Oklahoma Demand Program Portfolio Evaluation
22
believe that it is running very well. No changes were made to the delivery and
participation process in 2017.
High Participant Satisfaction: Results from the participant survey effort
indicate that customers highly value the services offered by WRAP, and recall
very positive experiences with the assessment and measure installation
process. Very few participants expressed dissatisfaction with any program
elements, and several participants provided open-ended commentary that
praised the program for the services it provides and the professionalism of its
contractor staff.
Adequate Database Quality: With the exception of the ex ante savings issue
described below, the Evaluators found the ex ante savings values within the
EnerTrek database to be accurate for nearly all measures. Additionally, Frontier
Associates was very consistent in responding to data requests and correcting
errors when necessary. There were a few inconsistencies regarding home level
savings and measure-level claimed savings but these were very infrequent.
Key Process Findings for CEEP
Program communications and marketing: The wide variety of program
marketing tactics employed under the CEEP program umbrella were apparent
within the participant data we collected for this evaluation. Fifty percent of
participants reported hearing about this program directly from a contractor or
vendor, and another 19 percent confirmed that they heard about the program
through word of mouth – a colleague, other business, etc. Fifteen percent cited
a Business Energy Advisor or Program Representative and 14 percent cited an
account representative. Between six to seven percent each cited a
conference/trade show/expo, a mailing from OG&E, the OG&E website, and
previous experience with an OG&E program. Respondents generally reported
having received enough information about the program when they first heard
about it. Ninety-three percent of CEEP respondents overall confirmed they
received enough information about the program through their communication
channel; this is a slight increase from PY16 (88 percent).
Program participant satisfaction: Among all participants, the highest
satisfaction ratings were for the incentive amount compared to the total project
cost (8.5 out of 10), the technical assistance from a contractor or vendor (8.4),
and the energy efficient measures that OG&E provides through the program
(8.4). Satisfaction with program components varied somewhat by program
channel. CEI and SAGE participants were the most satisfied with the various
program components. HVAC Tune-up customers were least satisfied with the
communication from program representatives (7.2), the technical assistance
OG&E Oklahoma Demand Program Portfolio Evaluation
23
from a contractor/vendor (7.5), the materials they received describing the
program requirements and benefits (7.6), and the technical assistance from
OG&E or CLEAResult (7.7).
Trade ally and Midstream Lighting distributor satisfaction: Trade allies
were asked to rate their satisfaction with the OG&E program channel(s) they
worked with. Over two-thirds (69 percent) of the trade ally ratings given for the
Large C&I, SBDI, and HVAC Tune-up program channels were an 8 through 10
on a scale of 0 to 10, where 0 means “very dissatisfied” and 10 means
“extremely satisfied.” Midstream Lighting distributors rated their overall
satisfaction with the program on a 5-point scale—very satisfied, somewhat
satisfied, neither satisfied or dissatisfied, somewhat dissatisfied, and very
dissatisfied. Ten of 12 lighting distributors were very satisfied with the program
overall, while two were somewhat satisfied.
Additional detail regarding these and other process evaluation findings and
recommendations can be found in the body of this report.
1.6 Structure of the Report
This report is structured as shown below:
Section 1 Executive Summary;
Section 2 Introduction;
Section 3 Methodology;
Section 4 HEEP Program Findings;
Section 5 PE-NHC Program Findings;
Section 6 WRAP Program Findings;
Section 7 CEEP Program Findings;
Appendix A – Portfolio Cost-effectiveness;
Appendix B – CEEP Participant Survey Response Rate; and
Appendix C – HEEP Participant Survey Response Rate.
OG&E Oklahoma Demand Program Portfolio Evaluation
24
2. Introduction
This report presents an evaluation of the performance of the energy efficiency and
demand response programs offered by Oklahoma Gas and Electric (OG&E) in 2017.
OG&E is submitting this report to fulfill the requirements outlined in Title 165: Oklahoma
Corporation Commission Chapter 35. Electric Utility Rules Subchapter 41. Demand
Programs 165:35-41-7.
2.1 Program Descriptions
2.1.1 Residential Portfolio Programs
Home Energy Efficiency Program (HEEP): HEEP is a multipronged
residential offering designed to incentivize OG&E’s Oklahoma customers to
reduce their energy consumption by performing energy efficient upgrades to
their homes. Designed to provide homeowners with multiple options, the
proposed program combines Residential Solutions, HVAC Replacement and
Tune-up, and Consumer Products channels. Providing homeowners with
increased choices to participate is expected to result in increased customer
engagement, greater measure adoption, and increased program savings.
o Residential Solutions (RSOL) and Schools Outreach: The
Residential Solutions channel of the HEEP program is a market-driven
approach that promotes energy efficiency by providing homeowners
with low-cost home assessment, direct install measures, community
educational outreach, and incentives on home retrofits. Incentives are
provided to encourage participation and decrease the upfront costs of
energy efficient upgrades.
The Schools Outreach channel includes a direct outreach channel
targeted at reaching residential customers through partnerships with
local schools and education for school aged children. This school portion
provides 5th grade students an educational opportunity to learn about
how they can affect the energy efficiency of their home. Teachers work
directly with the program team to obtain the teaching materials and
energy efficient direct install kits to distribute to students.
o Residential HVAC Replacement & Tune-up: The A/C Tune up and
HVAC Replacement channel of the HEEP program focuses on
improving the energy efficiency of the HVAC systems of residences. It
provides incentives to improve operating efficiency of the existing HVAC
unit or to replace it with a higher efficiency unit.
OG&E Oklahoma Demand Program Portfolio Evaluation
25
o Consumer Products: The consumer products channel promotes the
purchase of energy efficient LED Lighting. To help customers offset a
portion of the incremental cost associated with higher efficiency
products, the program utilizes upstream incentives. There was also a
Food Bank component which distributed CFLs and later LEDs to low-
income customers across the territory.
Positive Energy – New Home Construction Program (PE-NHC): The PE-
NHC program is designed to work with builders and contractors and incentivize
them to include energy efficient practices and measures when constructing new
homes within the OG&E Oklahoma territory.
Weatherization Residential Assistance Program (WRAP): The
Weatherization program has been designed to achieve energy savings by
helping to improve the comfort and reduce energy costs for OG&E Oklahoma’s
residential customers. This is achieved by providing free energy efficiency
upgrades for residential ratepayers with an annual income at or less than
$50,000 per year or LIAP customer, and is available to rental properties if an
eligible customer lives in the home and receives approval from the property
owner.
2.1.2 Commercial & Industrial Programs
Commercial Energy Efficiency Program (CEEP): CEEP is like HEEP, in that
it is a multipronged program approach designed to address the needs of
OG&E’s commercial and industrial customer base. Specifically, the program
provides an umbrella participation channel for all Commercial and Industrial
(“C&I”) customers to participate through either a custom or performance path.
Within this larger program offering there are targeted outreach channels to
further engage with specific commercial customers. The outreach channels
available through CEEP are:
o Large C&I: The Large C&I channel offers direct installation of low-cost
measures and both a performance and custom participation path for
customers to perform energy upgrades. Technical support is also
provided to assist in project identification and development.
Performance: The performance path provides incentives based
on the estimated energy savings achieved with the measures
installed primarily as defined by regionally accepted deemed
savings calculations.
OG&E Oklahoma Demand Program Portfolio Evaluation
26
Custom: The custom path gives participants an opportunity to
achieve their energy efficiency goals by proposing measures that
are outside of the deemed measure list. Proposed measures are
evaluated for savings and costs, and an appropriate incentive
amount is approved if the project is deemed cost-effective.
o SAGE: This channel offers assistance to the institutional and public
sectors to overcome barriers to energy improvement that are unique to
their market segment, such as conflicting organizational goals, outdated
specifications, limited technical knowledge, and counterproductive
energy budgeting. The program is also designed to provide some energy
management guidance and support services to qualifying customers.
o SBDI: The SBDI channel offers direct installation of low cost energy
efficiency measures, facility walkthroughs and incentives for a suite of
energy efficiency measures. This offer is targeted at, but not limited to
business customers with peak demand less than or equal to 150 kW, or
with multiple accounts less than or equal to 250 kW. Direct install
measures include LEDs and other low-cost lighting, low flow devices for
electric water heating, HVAC upgrades, vending misers and low cost
refrigeration measures. This targeted channel is also eligible to
participate in the Large C&I performance or custom channels if the
customer’s needs are beyond the scope of services outlined within the
outreach approach.
o Midstream Lighting: The commercial lighting midstream channel is
designed to encourage C&I customers to purchase and install high
efficiency lighting by providing point of sale (POS) discounts on selected
products through local distributors.
o Commercial HVAC Tune-up: The A/C Tune up channel of the CEEP
program focuses on improving the energy efficiency of the HVAC
systems of commercial facilities. It provides incentives to improve
operating efficiency of the existing HVAC unit through duct sealing,
cleaning, and refrigerant charge adjustments.
o CEI: The Continuous Energy Improvement (CEI) channel is a new
program offering for PY2017. CEI is a multiyear behavioral offering that
provides energy conservation training to all levels of employees within a
customer’s organization with a focus on low/no cost savings
opportunities. CEI also offers customers a facility wide energy
assessment of energy usage and provides customers with continuous
energy usage monitoring.
OG&E Oklahoma Demand Program Portfolio Evaluation
27
2.2 High-Volume Electricity User Opt Out
The Oklahoma Title 165:35-41-4 Rules allow for High-Volume Electricity Users “to opt out
of some or all energy efficiency or demand response programs by submitting notice of
such decision to the director of the Public Utility Division and to the electric utility.” A High-
Volume Electricity User represents a customer within a utility company’s service territory
whose annual consumption is 15 million kWh of electricity or greater regardless of the
number of meters or service locations.
In the table below is the number of customers eligible for High-Volume Electricity User
opt out, the number of such customers that opted out of energy efficiency programs for
PY2017, and the percentage of total energy sales that they comprise.
Table 2-1 High Volume Electricity User Opt Out – Demand Programs
Metric C&I
Customers Chose to opt
out -EE % Opt-Out
Number of accounts 110,394 1,450 1.3%
2017 electric sales (MWh) 15,536,323 4,816,638 31%
2.3 Program Implementation & Strategic Alliances
OG&E had eight fulltime employees dedicated to the implementation of demand
programs in 2017. Additionally, OG&E entered contracts with several energy services
companies and contractors to aid in program implementation. The total FTE count for
those is 37 FTEs.
CLEAResult (CR) was contracted to implement HEEP, PE-NHC, and the entirety of the
CEEP. The WRAP is managed in-house by OG&E, with Skyline Energy Solutions
conducting audits and measure installation, and Frontier Associates being responsible
for hosting the program database and calculating ex ante savings for program measures.
OG&E contracted with Resource Action Programs (RAP) to provide energy efficiency kits
distributed to students through the Schools Outreach channel in HEEP.
2.4 Evaluation Measurement and Verification
OG&E engaged ADM Associates, Inc. (ADM), with collaboration from Tetra Tech, to
conduct an evaluation, measurement, and verification (EM&V) of its Demand Program
portfolio. The ADM and Tetra Tech staff, collectively referred to as ‘the Evaluators,’
evaluated each program within the OG&E portfolio for PY2017.
OG&E Oklahoma Demand Program Portfolio Evaluation
28
2.5 Summary of Evaluation Effort
The table below summarizes the EM&V expenditures by the Evaluators, total EM&V
expenditures by all parties, and total program budgets.
Table 2-2 OG&E’s Portfolio EM&V Expenditures
Total EM&V Expenditures
Program Expenditures
EM&V as % of Total Expenditures
$988,353 $52,505,420 1.9%
To facilitate a thorough evaluation, the Evaluators conducted several primary research
and data collection activities, including interviews with program and implementer staff,
customer surveys, and site visits.
The Evaluators conducted participant surveys for each program using the collected self-
reported data to inform free ridership and spillover calculations for that program. The
results of these analyses informed our calculation of NTG values.
For all programs, the Evaluators performed telephone discussions with the primary OG&E
program staff and the primary implementation staff for each program. Specific primary
data collection activities are included in the table below.
Table 2-3 Summary of Data Collection Efforts
Program # of Site
Visits # of
Surveys # of
Interviews
HEEP 89 171 2
PE-NHC 20 0 8
WRAP 73 71 2
CEEP 69 244 6
2.6 High Impact Measures (HIMs)
This section outlines the High Impact Measures (HIMs) for each program and sector in
the OG&E portfolio of Demand Programs. A HIM is a measure that is responsible for
more than 5% of the sectors total savings.
2.6.1 Residential Programs
Four measures, LED bulbs, duct sealing, HVAC tune-up, and whole house new
construction projects comprise 84.0% of the residential sector’s energy savings (kWh).
Those measures, along with the percentage of savings and total energy (kWh) savings,
are shown in the figure below. The final measure, assessments that are performed in the
HEEP, had no measured savings.
OG&E Oklahoma Demand Program Portfolio Evaluation
29
Figure 2-1 Residential High Impact Measures
2.6.2 C&I Programs
Figure 2-2 shows the relative energy savings of measures installed through OG&E’s C&I
program. The overwhelming majority of program level ex ante savings were due to lighting
measures. High efficiency lighting and lighting controls accounted for 76.8% of the
program level savings. Within the high efficiency lighting measures, projects were
categorized within three measure categories, depending on the project type; new
construction, retrofit, and Midstream lighting. The contributions to the ex ante savings of
each of these are shown in Figure 2-3 below.
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
-
5,000,000
10,000,000
15,000,000
20,000,000
25,000,000
30,000,000
35,000,000
40,000,000
45,000,000
50,000,000
OG&E Oklahoma Demand Program Portfolio Evaluation
30
Figure 2-2 C&I Measures by End-Use
Retrofit lighting represented 63% of high efficiency lighting savings, with a total ex ante
savings of 45,572,081 kWh. The Midstream Lighting program channel contributed the
next highest amount with claimed savings of 20,143,968 kWh, 28% of high efficiency
lighting measure savings. New construction projects accounted for 5,581,790 kWh, 8%
of lighting savings. Finally, lighting controls contributed 1,142,436 kWh to the program ex
ante savings, accounting for 1.6% of lighting savings.
Figure 2-3 CEEP Lighting Measures
OG&E Oklahoma Demand Program Portfolio Evaluation
31
In addition to lighting, two other measure categories were above the threshold for HIMs.
Both custom and CEI projects contributed more than 5% to total program claimed
savings. The custom measure category includes all projects that used a custom
calculation approach to determine savings. These could either be lighting projects that
did not use deemed hours of use, or non-lighting projects that did not have TRM
algorithms. Overall, custom projects accounted for 6,271,000 kWh, 6.8% of program level
ex ante savings.
The final HIM was the new CEI program channel. This channel included behavioral
changes to commercial and industrial customers’ facility operations. The first cohort of
customers to participate in this program channel included seven customers, resulting in
a claimed savings of 8,239,906 kWh, accounting for 8.9% of program ex ante savings.
OG&E Oklahoma Demand Program Evaluation
32
3. Methodology
3.1 Introduction
This section details general impact evaluation methodologies by program-type as well as
data collection methods applied. This section will present full descriptions of:
Gross Savings Estimation;
Sampling Methodologies;
Free ridership determination;
Process Evaluation Methodologies; and
Data Collection Procedures.
3.2 Glossary of Terminology
As a first step to detailing the evaluation methodologies, the Evaluators provide a glossary
of terms to follow:
Deemed Savings – An estimate of an energy savings or energy demand savings outcome
(gross savings) for a single unit of an installed energy efficiency measure. This estimate (a) has been developed from data sources and analytical methods that are widely accepted for the measure and purpose and (b) is applicable to the situation being evaluated.
Ex ante Savings Estimate – Forecasted savings used for program and portfolio planning purposes (from the Latin for “beforehand”).
Ex post Evaluation Estimated Savings – Savings estimates reported by an evaluator after the energy impact evaluation has been completed (from the Latin for “from something done afterward”).
Ex Post Net Savings – When Ex Post Evaluation Estimated Savings are multiplied by the Net-to-Gross Ratio.
Free rider – A program participant who would have implemented the program measure or practice in the absence of the program. Free riders can be total, partial, or deferred.
Gross Realization Rate – The ratio of Ex Post Gross Savings and Ex Ante Savings.
Participant – A consumer who received a service offered through the subject efficiency
program in a given program year.
Net-to-Gross (NTG) – A factor representing net program savings divided by gross program savings that is applied to gross program impacts, converting them into net program load impacts after adjustments for free ridership and spillover. (1 – Free ridership % + Spillover %).
OG&E Oklahoma Demand Program Portfolio Evaluation
33
Spillover – Reductions in energy consumption and/or demand caused by the presence of the energy efficiency program that exceed the program-related gross savings of the participants. There can be participant and/or non-participant spillover rates depending on the rate at which participants (and non-participants) adopt energy efficiency measures or take other types of efficiency actions on their own (i.e., without an incentive being offered).
Stipulated Values – See “deemed savings.”
This glossary is drawn from several evaluation-related reference documents, such as the
2007 IPMVP, 2004 California Evaluation Framework, 2006 DOE EERE Guide for
Managing General Program Evaluation Studies and the Arkansas Technical Reference
Manual (TRM) version 5.0.
3.3 Overview of Methodology
The proposed methodology for the evaluation of the 2017 OG&E portfolio is intended to
provide:
Net impact results at the 90% confidence and +/-10% precision level; and
Program feedback and recommendations via process evaluation; and
In doing so, this evaluation will provide the verified net savings results, provide the
recommendations for program improvement, and ensure cost-effective use of ratepayer
funds. By leveraging experience and lessons learned from prior evaluations, the 2017
evaluation is streamlined to focus on areas in need of research and improvement.
3.3.1 Sampling
Sampling is necessary to evaluate savings for the OG&E portfolio insomuch as
verification of a census of program participants is typically cost-prohibitive. As per
evaluation best practices, samples are drawn to ensure 90% confidence at the +/- 10%
precision level. Programs are evaluated on one of three bases:
Census of all participants;
Simple random sample; or
Stratified random sample.
3.3.2 Census
A census of participant data was used for select programs or program channels where
such review is feasible. All program measures were evaluated.
OG&E Oklahoma Demand Program Portfolio Evaluation
34
3.3.3 Simple Random Sampling
For programs with relatively homogenous measures (largely in the residential portfolio),
the Evaluators conducted a simple random sample of participants. The sample size for
verification surveys is calculated to meet 90% confidence and 10% precision (90/10). The
sample size to meet 90/10 requirements is calculated based on the coefficient of variation
of savings for program participants. Coefficient of Variation (CV) is defined as:
𝑉 =𝑀𝑒𝑎𝑛𝑥
𝑆𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝐷𝑒𝑣𝑖𝑎𝑡𝑖𝑜𝑛𝑥
Where x is the average kWh savings per participant. Without data to use as a basis for a
higher value, it is typical to apply a CV of 0.5 in residential program evaluations.
The resulting sample size is estimated with the following:
𝑛0 = (1.645 ∗ 𝐶𝑉
𝑅𝑃)
2
Where:
1.645 = Z score for 90% confidence interval in a normal distribution
CV = Coefficient of Variation
RP = Required Precision, 10% in this evaluation
3.3.4 Stratified Random Sampling
For the OG&E C&I programs, Simple Random Sampling is not an effective sampling
methodology as the CV values observed in business programs are typically very high
because the distributions of savings are generally positively skewed. Often, a relatively
small number of projects account for a high percentage of the estimated savings for the
program.
To address this situation, the Evaluators use a sample design for selecting projects for
the M&V sample that considers such skewedness. With this approach, the Evaluators
select several sites with large savings for the sample with certainty and take a random
sample of the remaining sites. To improve the precision, non-certainty sites are selected
for the sample through systematic random sampling. That is, a random sample of sites
remaining after the certainty sites have been selected is selected by ordering them per
the magnitude of their savings and using systematic random sampling. Sampling
systematically from a list that is ordered according to the magnitude of savings ensures
that any sample selected will have some units with high savings, some with moderate
savings, and some with low savings. Samples cannot result that have concentrations of
sites with atypically high savings or atypically low savings.
OG&E Oklahoma Demand Program Portfolio Evaluation
35
3.3.5 Free ridership
In determining ex post net savings for the OG&E portfolio, the Evaluators provide
estimates of free ridership for individual programs. Free riders are program participants
that would have implemented the same energy efficiency measures at nearly the same
time absent the program. As per the Arkansas TRM version 6.0 guidelines, free riders are
defined as:
“…program participants who received an incentive but would have
installed the same efficiency measure on their own had the program not
been offered. This includes partial free riders, defined as customers who,
at some point, would have installed the measure anyway, but the
program persuaded them to install it sooner or customers who would
have installed the measure anyway but the program persuaded them to
install more efficient equipment and/or more equipment. For the
purposes of EM&V activities, participants who would have installed the
equipment within one year will be considered full free riders; whereas
participants who would have installed the equipment later than one year
will not be considered to be free riders (thus no partial free riders will be
allowed).”6
Given this definition, participants are defined as free riders through a binary scoring
mechanism, in being either 0% or 100% free riders.
3.3.6 Impact Evaluation Activities by Program
The Evaluators used established, industry-standard approaches to estimate energy
savings and demand reductions at the measure, program, and portfolio levels. The
Evaluators followed all applicable measure- and program-level guidelines and protocols
from the AR TRM 6.0.
To evaluate program impacts, ADM adjusted program-reported gross savings using the
results of our research, relying primarily on engineering desk reviews, AR TRM deemed
savings calculation, and on-site verification and metering for applicable programs. To
calculate deemed savings, the Evaluators verified the appropriateness of savings
algorithms and values in program tracking data as compared to guidelines in the AR TRM
6.0. Where sampling was used (for surveys and site visits), the Evaluators designed a
sampling plan to achieve a minimum precision of ±10% of the gross realized savings
estimate with 90% confidence.
6 Arkansas TRM V5.0, Pg. 49, found here: www.apscservices.info/EEInfo/TRM5.pdf
OG&E Oklahoma Demand Program Portfolio Evaluation
36
For each program and measure category, the Evaluators estimated energy savings and
demand reduction by applying a verified gross savings adjustment to program-reported
savings.
3.3.7 Net-to-Gross Approach by Program
The Evaluators estimated NTG for each program in the portfolio. Table 3-1 shows the
NTG approach the Evaluators followed for each program based on our assessment of
specific program needs and the availability of accurate, existing information. These data
collection and analysis activities are in compliance with one of the five accepted
approaches listed in the AR TRM 6.0.
Table 3-1 NTG Approach by Program
Program and Channel Approach
HEEP Residential Solutions: Home Assessment Self-reporting survey
HEEP Residential Solutions: Schools Outreach Stipulated net-to-gross ratio
HEEP HVAC Replacement and Tune-up Self-reporting survey
HEEP Consumer Products Lighting Purchase Survey of HEEP / Stipulated for Food Bank Distributions Participants
PE-NHC (New Homes Construction) Self-reporting survey
CEEP C&I Solutions
Self-reporting survey
CEEP Educational & Government Self-reporting survey
CEEP Small Business Solutions Self-reporting survey
The Evaluators completed three in-depth interviews with Oklahoma contractors who
helped deliver the Residential Solutions HEEP program track. The interviews targeted
those trade allies currently delivering the multifamily portion of the program, as they
contributed most significantly to program savings in 2017. CLEAResult provided this trade
ally contactor contact information, and the Evaluators considered trade ally project
volume in the year-end program tracking data when prioritizing interviews. Ultimately, the
thee contractors interviewed for this process evaluation ran businesses that represented
more than half of the PY2017 Residential Solutions track savings.
The team worked to complete in-depth interviews with contractors working within the
Residential Solutions program track between Monday, February 26 and Friday, March 2,
2018.
4.6.1.4 Process Evaluation Findings
This section details the findings from the process evaluation pertaining to program
delivery, program communications and marketing, participant energy efficiency
awareness and behaviors, and customer characteristics.
4.6.1.4.1 Program Communication and Marketing
OG&E and CLEAResult shared marketing responsibilities for the HEEP Residential
Solutions track in PY2017. Participants most frequently reported (38 percent) hearing
about this program through a utility bill message, which matched the most frequently
provided answer in the PY2016 process evaluation, as well. The utility’s website (16
percent) and word of mouth (14 percent) were also frequently mentioned sources of
program information. Many respondents also indicated they heard about the program
from “other” sources; “other” popular options were “Home and Garden show” and
“Retailer” – suggesting that participants may be getting information about OG&E energy
efficiency programs from multiple sources. A summary of the participant responses
appears in Table 4-60.
OG&E Oklahoma Demand Program Portfolio Evaluation
86
Table 4-60 How learned of program
Category Percent
Utility bill message 38%
Utility website 16%
Word of mouth from friends, relatives, or others 14%
Contractor 13%
OG&E program staff 10%
Information that came in the mail 7%
Email 5%
*Home and Garden Show 5%
*Retailer 4%
TV ad 4%
Other website 3%
*Previous OG&E program experience 2%
Newspaper or magazine article/ad 1%
Radio ad 1%
Other 5%
Total N=93
Source: Question a1
Don’t know and refused responses are excluded Note: Totals may not sum to 100 percent as respondents could select more than one answer
*Indicates category computed during analysis
We asked OK HEEP Residential Solutions survey respondents why they decided to participate in the program. Thirty-nine percent attributed their participation to wanting to save energy, followed by another 38 percent indicating they were driven by the desire to reduce their monthly utility bill. Among those who answered “other”, “Opportunity to get new equipment and/or have service done” (21 percent) was a popular response.
OG&E Oklahoma Demand Program Portfolio Evaluation
87
Table 4-61 Reasons for program participation
Category Percent
Save energy 39%
To reduce my monthly utility bill 38%
The HEEP program paid for some or all of the improvements 31%
*Opportunity to get new equipment / have service done. 21%
*Out of interest / to see what I could learn 7%
It is the right thing to do 6%
Contractor recommendation 4%
Help save the environment 3%
Improve comfort of home 2%
*Previous OG&E program experience 2%
Utility recommendation or information 1%
Recommendation from friend, relative, or neighbor 1%
Other 2%
Total N=99
Source: Question a2 Don’t know and refused responses are excluded Note: Totals may not sum to 100 percent as respondents could select more than one answer
*Indicates category computed during analysis
When respondents who provided multiple reasons for their program participation were asked to isolate their main reason for program participation, nearly half (43 percent) indicated it was to reduce their monthly utility bill.
We also asked Residential Solutions trade allies how they heard about the program. Two
of the three trade allies mentioned CLEAResult as the source, either directly or indirectly
through their distributor. One mentioned having relatives that worked for OG&E for many
years as their source of program awareness.
All three trade allies have been working with the program only in the past year or year
and a half. When asked why they or their company decided to get involved in the
programs, the reasons cited by the three trade allies were:
Interesting work—always been in the energy field
Encouraged by an associate and to “make money”
To fill in downtime when not doing other electrical and plumbing projects
OG&E Oklahoma Demand Program Portfolio Evaluation
88
While two trade allies interviewed said they prefer email, and the third trade ally said email
is good, they all prefer one-on-one contact to get program information. Two of the trade
allies were satisfied with the communications they receive about the program. The third
trade ally expressed frustration with the tone of the emails and telephone calls with a
CLEAResult staff person.
4.6.1.5 Program Delivery
The PY2017 process evaluation of the OK HEEP Residential Solutions track captured the
basic program delivery steps within both the single family and multifamily models. This
section highlights our understanding about how the program was delivered to single
family and multifamily buildings within this program year.
4.6.1.5.1 Single Family
The 2017 version of the HEEP Residential Solutions for single family participants differs
from last year’s program structure. Last year, the Residential Solutions program
experience generally began through the home owner taking an online home audit, rating
the energy efficiency of their home, and then getting an in-home audit experience. This
year, the single family program process was re-envisioned, moving away from the online
home audit experience and allowing more flexibility for a potential customer to sign up for
a home audit more directly – either online or through contacting the customer contact
center at OG&E.
CLEAResult was the implementer of this program portion this year. Participants in the
HEEP Residential Solutions program track received the same general program
components as were available last year: an in-home audit, direct install measures such
as LED light bulbs and smart strips, and a home audit report that may include
recommendations for additional energy efficiency improvements. Some program rebates
were available for customers who wanted to install recommended energy efficiency
equipment.
4.6.1.5.2 Multifamily
The process for delivering the multifamily portion of the Residential Solutions program
track involved a more program driven, direct-to-customer approach. CLEAResult and in
some cases, trade allies, reached out directly to property management teams in OG&E
territory in Oklahoma to drive multifamily enrollments. If a multifamily customer contact
agreed to participate in the program with either a CLEAResult program representative or
through a program-affiliated trade ally, CLEAResult would set up a time to visit each of
their properties, order necessary measures, and return with a contractor to complete the
work.
OG&E Oklahoma Demand Program Portfolio Evaluation
89
4.6.1.6 The Home Energy Assessment Experience
The Evaluators asked single family participants about various aspects of their home
assessment experience. The first was how they requested a home assessment through
this program track. Most respondents (61 percent) reported calling the Customer Service
center to request their assessment, while nearly one-third (30 percent) reported they
requested their assessment through the web site. The remaining respondents confirmed
they used some “other” sign-up avenue. Eighty-three percent of respondents indicated
they had NOT planned on having a home assessment before learning about the program.
Regardless of how they requested an assessment, customers had an easy time
communicating with the program and scheduling their home assessment. Among those
who called the Customer Service center, 98 percent indicated their experience was ‘very
easy’ or ‘easy’, while the remaining customers rated their experience as a “neutral” one.
Similarly, 95 percent of respondents who used the web site for assessment scheduling
rated their experience “very easy” or “easy”.
Next, we wanted to understand what aspects of the home assessment experience that
participants recalled. Nearly all (96 percent) of the survey respondents recalled that the
home assessor provided them with a home energy report which outlined custom energy
efficiency improvement recommendations specific to their home. Among those that
recalled the report, 31 percent felt the report was very helpful, while another 44 percent
rated it helpful. Sixteen percent of the respondents described the helpfulness of the report
as “neutral”. The nine percent of respondents who did not find the energy report helpful
attributed their frustration to the report not giving them any new or additional information.
Nearly all the respondents (96 percent) confirmed that the energy assessor discussed
improvement recommendations with them, and the energy savings they would realize if
the improvements were implemented. When we asked respondents if they had
implemented their home assessment recommendations, most respondents (72 percent)
indicated they had completed some, but not all, of the recommendations, while 14 percent
reported implementing all of them. Among the recommendations respondents had not yet
implemented, the most frequently mentioned items were adding insulation to their home
(n=14), replacing windows or doors (n=10), or replacing an HVAC or furnace unit (n=7).
A majority of respondents (54 percent) who did not implement some or all of the
recommendations indicated cost was their major barrier, while another 23 percent10 of
respondents indicated that time to implement a project was their largest barrier.
10 This pool of respondents includes the 14 percent who chose the survey response “time”, and another 8 percent
who answered “other” but expanded their “other” answer to say that time was a challenge in some way.
OG&E Oklahoma Demand Program Portfolio Evaluation
90
Table 4-62 When you had your home energy assessment, did the assessor. . .
Category
provide a home energy report with
recommendations?
discuss the potential energy savings that
could be achieved by implementing the
recommendations?
Yes 96% 96%
No 4% 4%
Total 100% 100%
N=71 N=68
Source: Questions HE7a and HE7b
Don’t know and refused responses are excluded
*Note: Assessment questions asked only of participants who were sampled for assessment as a measure.
4.6.1.7 Measure Installation
We asked Residential Solutions participants whether or not they recalled their assessor
installing direct install measures as part of their program experience. Nearly all
respondents (96 percent) recalled an assessor installing direct install measures as part
of their program participation. Most of them (88 percent) confirmed those measures were
still installed in their home. Among respondents reporting they no longer had their
measures installed, four respondents reported removing the smart power strips, and four
respondents indicated they had replaced program-provided LED light bulbs.
Respondents who gave a reason for removing their measures indicated they were either
unsatisfied with the configuration (smart strips) or the light quality (light bulbs).
Further, we asked participants to confirm what other measures they received through
their HEEP Residential Solutions program track participation. Most (86 percent) did not
recall receiving additional program measures past the program-provided direct install
items. Among those who did report additional measures, air sealing was the most
frequently recalled measure received through the program, with 29 percent of respondent
indicating that choice.
4.6.1.8 Contractor Perspective on Program Influence on Marketing, Sales, and
Equipment Recommendations
The trade allies were asked to rate the importance of the program, including the rebates
and information, in influencing their marketing and sales of equipment or measures during
OG&E Oklahoma Demand Program Portfolio Evaluation
91
2017. The rating was on a scale of 0 to 10 where 0 is “not at all important” and 10 is
“extremely important.”
Two of the three trade allies rated program importance a 10. One noted they would not
be installing energy efficiency measures at all without the program. The other trade ally
commented that, without the Program, property managers would not care about saving
money when their tenants are paying the energy bill.
The third trade ally rated the importance of the OG&E program a 7 despite saying that
their business does not install any LEDs or showerheads outside of the OG&E program.
All three replied that they would not be installing low-flow showerheads and LEDs without
the program.
4.6.1.9 Trade Allies and the Sales Impacts of the Residential Solutions Program
The trade allies were asked about impacts of the program on project volume. They were
asked specifically whether the number of multifamily projects had increased significantly,
somewhat, a little, hardly at all, or not at all. All three trade allies said the volume of these
types of multifamily projects had increased significantly because of the Residential
Solutions program. One commented that their business had grown steadily over the last
few years.
The trade allies were asked whether they expected the number of multifamily Residential
Solutions projects to increase, decrease, or stay the same in the next 12 months. The
responses were mixed. One trade ally said the number of projects would increase in that
they had three apartment complexes last year and already had six apartment complexes
signed up this year. The second trade ally thought the number of projects would stay the
same. The third trade ally said the number would decrease due to the communication
issues they experienced with the CLEAResult representative in PY2017.
4.6.1.10 Program Satisfaction
The Evaluators asked OK HEEP Residential Solutions program track participants to rate
their satisfaction with various program components using a scale of 1 to 5, where 1 is
"very dissatisfied" and 5 is "very satisfied" (Table 4). Participants’ mean overall program
satisfaction score was 4.5 with the HEEP Residential Solutions track. Their satisfaction
scores on nearly all program components were consistently rated 4 or higher; the only
exception was the participant mean score on monthly utility bill savings, which averaged
a 3.7 rating.
OG&E Oklahoma Demand Program Portfolio Evaluation
92
Table 4-63 HEEP Residential Solutions: Satisfaction by Program Component
Interactions you had with OG&E staff Mean 4.7
Respondents (n) 86
The effort required for the application process
Mean 4.7
Respondents (n) 97
Scheduling the home energy assessment
Mean 4.6
Respondents (n) 93
The quality of contractor’s work Mean 4.6
Respondents (n) 97
Interactions you had with the contractor
Mean 4.6
Respondents (n) 98
The performance of the equipment installed or the energy efficiency improvements that were made
Mean 4.5
Respondents (n) 98
Usefulness of the information provided by the home energy assessment
Mean 4.5
Respondents (n) 95
The wait-time to receive the services Mean 4.3
Respondents (n) 98
Improvement in home comfort Mean 4.0
Respondents (n) 97
The savings on your monthly utility bills Mean 3.7
The trade allies were asked to rate their satisfaction with OG&E Residential Solutions
on a scale of 0 to 10, where 0 means “very dissatisfied” and 10 means “extremely
satisfied.” The ratings were mixed for the three trade allies. One gave a satisfaction
rating of 10 saying it was an excellent program. The trade ally started their company to
do these types of projects and was 100 percent residential—mostly multifamily work.
Another trade ally gave the program a rating of 7 primarily due to some inconsistencies
in the turnaround time for payments. They are a small one-person plumbing business
OG&E Oklahoma Demand Program Portfolio Evaluation
93
who said 40 percent of their work is residential and 60 percent is commercial. A third
trade ally rated the program a 3 strictly due to their dissatisfaction with CLEAResult staff
communications.
The Evaluators asked trade allies to rate various aspects of the 2017 Residential
Solutions Program using a scale of 0 to 10 where 0 is “very dissatisfied” and 10 is “very
satisfied.” Below are the satisfaction scores and supporting information for each of the
topics we explored with the trade allies from the RESOL program:
The Level of Communications with CLEAResult Program Staff: Two of the trade allies were very happy with their level of communications with CLEAResult staff, rating their satisfaction a 9 and a 10. The third trade ally gave a rating of 1 based on their dissatisfaction with CLEAResult communication.
The Amount of Program Paperwork: The amount of program paperwork was not a problem for two of the three trade allies who rated the amount of paperwork a 7 to 8. The third trade ally thought the amount of paperwork for multifamily projects was somewhat burdensome, giving the paperwork a rating of 3 in terms of their satisfaction.
The Actual Program Measures and/or Rebated Equipment Offered through the HEEP Program: Those interviewed were satisfied with the types of measures and equipment offered. Two rated their equipment satisfaction a 10 while the third gave an 8-9 rating.
The Timeliness and Amount of Rebate Payments: The trade allies were asked to rate their satisfaction with the timeliness of the rebate payments. One trade ally gave rebate payment timeliness an 8, saying they had some problems last year but the timeliness had improved. The second trade ally gave a rating of 5 to 6 saying there was inconsistency in the turnaround time and that a few payments took much longer than expected. The third trade ally gave a very low satisfaction rating of 1 with rebate timeliness because it took over 60 days for a $100,000 rebate to be paid on a multifamily project. The three trade allies were all very satisfied with available program incentives, including LEDs and showerheads as direct install measures through the program. One commented that it was “free equipment.”
HEEP Program Training: All three of the trade allies said they either attended the program kickoff or had some type of technical training on the program, and their satisfaction ratings were high with two 10’s and one 8.
Trade Ally Suggestions for Program Improvement: After rating the various aspects of the Residential Solutions program, the trade allies were asked how the program design or operations could be improved. Despite some concerns mentioned about rebate
OG&E Oklahoma Demand Program Portfolio Evaluation
94
turnaround and communication, none of them had any additional concrete suggestions on how to improve the program.
4.6.1.12 Program Influence on Energy Saving Knowledge and Behavior
To assess the customer education portion of the HEEP Residential Solutions track, our
survey asked single family customers to assess their familiarity with various aspects of
energy efficiency before the program participation. First, we asked customers how familiar
they were with energy efficiency improvement benefits before participating in the
program. A majority of the HEEP RESOL participants indicated they were “very familiar”
(21 percent) or “somewhat familiar” (47 percent) with the benefits of energy efficiency
improvements. Twenty-six percent of respondents categorized their energy efficiency
familiarity levels as “very unfamiliar” or “somewhat unfamiliar”.
Table 4-64 Familiarity with benefits of energy efficiency improvements prior to assessment
Category Percent
Very familiar 21%
Somewhat familiar 47%
Neither familiar nor unfamiliar 5%
Somewhat unfamiliar 13%
Very unfamiliar 13%
Total 100%
N=98
Source: Question e1 Note: Totals may not sum to 100 percent due to rounding
HEEP Residential Solutions participants were more familiar with the benefits of energy
saving activities prior to the assessment, with 69 percent of them indicating they were
“very familiar”. Another nearly one-quarter (24 percent) reported they were “somewhat
familiar” with energy saving activity benefits prior to the assessment.
OG&E Oklahoma Demand Program Portfolio Evaluation
95
Table 4-65 Familiarity with benefits of energy saving activities prior to assessment
Category Percent
Very familiar 69%
Somewhat familiar 24%
Neither familiar nor unfamiliar 1%
Somewhat unfamiliar 3%
Very unfamiliar 3%
Total 100%
N=98
Source: Question e2 Note: Totals may not sum to 100 percent due to rounding
Most HEEP Residential Solutions participants (85 percent) indicated they had performed
energy saving activities prior to their assessment. Respondents could offer multiple
examples of energy saving activities. Over half mentioned the activity was “turn off lights
when not in the room” (56 percent), while another 46 percent confirmed they adjusted
their heating system settings. Roughly one-third (32 percent) reported they washed
clothes in cold water.
We then asked participants whether program participation had increased their energy
efficiency knowledge or changed their actions. Focusing first on knowledge, Table 4-66
highlights that over one-quarter of survey respondents (28 percent) report they are “much
more” knowledgeable about energy efficiency than they were before program
participation, while another 43 percent indicate they are “somewhat” more knowledgeable
because of their program experience.
OG&E Oklahoma Demand Program Portfolio Evaluation
96
Table 4-66 As a result of program, how much more knowledgeable about energy efficiency?
Category Percent
Much more knowledgeable than before participating 28%
Somewhat more knowledgeable 43%
Slightly more knowledgeable 21%
Not more knowledgeable 7%
Total 100%
N=99
Source: Question e5 Note: Totals may not sum to 100 percent due to rounding
Nearly two-thirds of our survey respondents (64 percent) reported that the program has
changed their actual energy savings behaviors. In addition, nearly 20 percent of the HEEP
Residential Solutions track respondents indicated that have participated in other energy
efficiency programs. Six respondents mentioned participating in the HVAC track of the
HEEP program, while another eight reported participating in OG&E’s Smart Hours
program.
4.6.1.13 General Respondent Characterization
Table 4-67 summarizes basic home information among our OK HEEP Residential
Solutions respondents. Nearly all (95 percent) HEEP Residential Solution participants we
surveyed were homeowners, and 94 percent of the respondents described their home as
a single family home. Other respondents indicated their home was a town house or
condominium, a duplex, or “other”. Twenty-three percent of this program track’s
respondents are the only people that live in their home, while another 33 percent of
respondents say they have two people in their residence. This program track serves
Do you rent or own the home we’ve been discussing today?
Rent Column N % 5%
Own Column N % 95%
What is the main fuel used to heat your home?
Electricity Column N % 24%
Natural gas Column N % 73%
Other Column N % 3%
Total Unweighted Count 99
What is the main fuel used to heat your water?
Electricity Column N % 20%
Natural gas Column N % 75%
Bottled gas propane Column N % 3%
Other Column N % 2%
Total Unweighted Count 99
Including yourself, how many people lived in your household last year?
1 person Column N % 23%
2 people Column N % 33%
3 people Column N % 27%
4 people Column N % 8%
5 people Column N % 5%
6 people Column N % 2%
7 or more people Column N % 1%
Total Unweighted Count 99
Including all money earned from wages, salaries, tips, commissions, workers´ compensation, unemployment insurance, child support, or other sources, about how much was your total annual household income before taxes in 2017?
Less than $10,000 Column N % 0%
$10,000 to less than $20,000 Column N % 7%
$20,000 to less than $30,000 Column N % 8%
$30,000 to less than $40,000 Column N % 4%
$40,000 to less than $50,000 Column N % 10%
$50,000 to less than $75,000 Column N % 21%
$75,000 to less than $100,000 Column N % 20%
$100,000 to less than $150,000 Column N % 18%
$150,000 to less than $200,000 Column N % 8%
$200,000 or more Column N % 5%
Total Unweighted Count 77
Source: Question dem1 dem3 dem3h dem4 dem5
Don’t know and refused responses are excluded Note: Totals may not sum to 100 percent due to rounding
OG&E Oklahoma Demand Program Portfolio Evaluation
98
4.6.2 HVAC Replacement and Tune-up Channel
We present the methodology used for the process-related data collection activities the
Evaluators performed in association with the OK HEEP HVAC Tune-Up and Replacement
track evaluation within this section. These activities included program staff interviews, a
survey of participating customers, and in-depth interviews with participating contractors
who helped deliver the program.
4.6.2.1 Program Staff Interviews
The Evaluators completed an in-depth interview with the OK HEEP program manager
and an additional interview with CLEAResult team members who implement the HVAC-
related program track. The Evaluators used these program staff interviews to identify
program updates or changes experienced in PY2017, explore staff roles and
responsibilities, and review overall program delivery processes for this past year.
4.6.2.2 Participating Customer Surveys
The Evaluators completed telephone surveys with OK HEEP HVAC Tune-Up and
Replacement track program participants through their in-house survey lab. The surveys
collected respondent feedback on program communication and offerings – especially, in
this case, their recollection about their air conditioning tune-up experience. The surveys
also measured changes in participant energy efficiency awareness and behaviors due to
program participation, participant satisfaction, and collected key household
characteristics and demographic information. The Evaluators received and reviewed
HEEP program tracking data from CLEAResult. These data included participating
customer contact information and project information.
We surveyed 72 Oklahoma HEEP HVAC Tune-Up and Replacement program track
participants, selected from a random sample of 245 participants. Our sampling strategy
was designed to achieve an overall 90/10 level of precision at the program level. The final
sample distribution and response rate for this survey can be found in Appendix A.
The Evaluators mailed an advance letter to sampled participants on Wednesday, January
24, 2018 explaining the upcoming survey and asking for their cooperation. The Evaluators
began survey fielding on Thursday, January 25, 2018. Participant data collection ended
The Evaluators completed five in-depth interviews with Oklahoma contractors who helped
deliver the HEEP HVAC Tune-Up and Replacement program track. Trade ally contactor
contact information, along with their project volume within the program, was available to
the team for review and consideration within the program participant tracking data
OG&E Oklahoma Demand Program Portfolio Evaluation
99
delivered by CLEAResult at the end of the program year. These data drove trade ally
interview selection. We prioritized trade ally selection for interviews based on their
program participation volume, opting to offer an interview opportunity to those who
worked most with the program.
The interviewer asked businesses about participation in the program and their experience
with the program on what worked well or improvements the trade allies might recommend
to OG&E. The team worked to complete in-depth interviews with HVAC Tune-Up and
Replacement contractors working within this program track between Monday, February
26 and Friday, March 2, 2018. Four of the five trade allies indicated most of their work
was residential. One company did residential single-family and multifamily, and
commercial projects but could not provide a breakdown by customer type. Most of the
program work for these interviewed contractors over the course of PY2017 were HVAC
tune-ups.
4.6.2.4 Process Evaluation findings
This section details the findings from the process evaluation pertaining to program
delivery, program communications and marketing, participant energy efficiency
awareness and behaviors, and customer characteristics.
4.6.2.5 Program communication and marketing
Program Communication and Marketing: Participant Perspective: CLEAResult is
responsible for HEEP HVAC Tune-Up and Replacement track program marketing,
although they market the program in cooperation with OG&E. As was the case last year,
tune-ups, in particular, are often promoted in-home during a Residential Solutions team
home energy audit. The HVAC Replacement portion of the program is offered and/or
mainly promoted through program-participating contractors.
Participants most commonly named a “utility bill message” (34 percent) as the way they
learned about the program. The utility’s website (21 percent) or information that came in
the mail (18 percent) were the next most frequently mentioned sources of program
information. A summary of the participant responses appears in Table 4-60.
OG&E Oklahoma Demand Program Portfolio Evaluation
100
Table 4-68: How Customer learned of program
Category Percent
Utility bill message 34%
Utility website 21%
Information that came in the mail 18%
Word of mouth from friends, relatives, or others
18%
Contractor 9%
Email 6%
TV ad 5%
Home/Garden show 2%
Retailer 2%
Previous experience with an OG&E program 2%
Newspaper or magazine article / ad 2%
Other website 0%
OG&E program staff 0%
Other 5%
Respondents (n) 67
Source: Question A1 Don't know and refused responses are excluded Note: May not total 100 percent as respondents could select more than one answer *Indicates category computed during analysis
We asked OK HEEP HVAC Tune-Up and Replacement survey respondents why they
opted to participate in the program; respondents could choose more than one response.
Forty-three percent reported that they participated in the program because “the program
was paying for some or all of the improvements they received”, while another 40 percent
of respondents said they wanted to “lower their monthly utility bill”. Other reasons
mentioned by at least 20 percent or more of the respondents included: “the opportunity
to get new equipment/have service done” or ‘Save energy”.
We asked participants who provided multiple reasons for participating what drove their
main reason for participating. “Reducing their monthly utility bill” was the most frequently
mentioned response.
OG&E Oklahoma Demand Program Portfolio Evaluation
101
Table 4-69 Reasons for program participation
Category Percent
The program paid for some or all of the improvements
43%
To reduce my monthly utility bill 40%
*Opportunity to get new equipment/have service done
28%
Save energy 21%
Help save the environment 6%
It is the right thing to do 6%
*Improve comfort of home 4%
Recommendation from a friend, relative, or neighbor 3%
*Out of interest / see what I could learn 3%
*Previous experiences with an OG&E program 3%
Utility recommendation or information 3%
Contractor recommendation 1%
Other 8%
Respondents (n) 67
Source: Question A2 Don't know and refused responses are excluded Note: May not total 100 percent as respondents could select more than one answer *Indicates category computed during analysis
Program Communication and Marketing: Contractor Perspective: When asked how
they heard about the HVAC Tune-up program, two of the five trade allies interviewed
indicated that CLEAResult was the source, either directly or indirectly through their
distributor. Two other trade allies were familiar with the program from working at another
company, while one heard about the OG&E program from a friend.
Two of the trade allies said they had been involved in the OG&E HVAC Tune-up program
for one or two years, while the others noted having been involved in OG&E programs
overall for three to seven years.
When asked why they or their company decided to get involved in the programs, the
reasons cited by the five trade allies were:
This is a new company and a good way to reach customers
They are a fairly new business and thought it would be helpful to their business
Encouraged by an associate and to “make money”
Was already working with OG&E, but under a different company
Because of past familiarity with OG&E programs
OG&E Oklahoma Demand Program Portfolio Evaluation
102
The trade allies interviewed said they prefer email or one-on-one training or “shadowing”
to get receive program information and training. Several noted that they contact
CLEAResult directly if they have any questions. All were very happy with the
communication they receive from CLEAResult.
One mentioned the annual program kickoff as being very useful and another mentioned
program materials were helpful to them and to their customers. Another trade ally
commented that consumers do not have effective communication from OG&E or
CLEAResult about the program.
None of the trade allies had any suggestions on needed improvements to
communications with the trade allies about the program.
4.6.2.6 Program Delivery
The PY2017 process evaluation of the OK HEEP HVAC Tune-Up and Replacement
focused on two main delivery components: a) the customer tune-up visit and b) the
contractor experience in delivering the program.
The AC Tune-Up Visit: Nearly all (Ninety-two percent) of HVAC Tune-Up and
Replacement participants recalled having a tune-up with the program, and we asked
these customers who confirmed they recalled the tune-up specifically about their program
experience. Nearly two-thirds (65 percent) did not have regular HVAC tune-ups prior to
participating in this program, while 35 percent indicated they did have HVAC tune-ups on
a regular basis. Among those who reported having tune-ups “regularly”, fifty percent
confirmed they had a tune-up annually, while another one-quarter (25 percent) confirmed
they had a tune-up every other year. Around the data edges, 15 percent of the regular
tune-up group said they serviced their HVAC twice per year, while the final 10 percent
had HVAC tune-ups done every three to five years.
We asked program respondents if their contractor explained their tune-up process to
them, and specifically, if they explained the difference between a standard HVAC tune-
up and the tune-up offered by the OG&E HEEP program. Just over one-third of
participants (36 percent) confirmed they received an explanation of this difference.
Program participants had HVAC units of varying age. Nearly one-third (32 percent) of
respondents indicated they had a unit that was 5 years old or less. The average overall
age of air conditioning units among respondents asked was 12 years old.
Contractor Perspective: Our team asked the five trade allies to comment on what
percentage of customers already know about the HVAC Tune-up program before the
trade ally tells them about it. Only two of the five trade allies thought most of their
customers were not very aware of the program. One trade ally felt that about 30 percent
of the customers knew about the program in advance.
OG&E Oklahoma Demand Program Portfolio Evaluation
103
Two trade allies said that most of their customers knew about the HVAC Tune-up
program. These trade allies thought customers heard media advertising, got pamphlets
or email blasts, or were repeat customers who had tune-ups in the past.
When asked why customers who qualify may not want to participate in the program, two
of the trade allies said there was no reason for customers not to participate. The other
three trade allies all mentioned that the customer may be skeptical of the program or lack
knowledge of the benefits of program participation.
The trade allies were also asked what they thought are the main benefits customers
receive by participating in the HVAC Tune-up program track. In addition to the energy
cost savings and the incentives, three of the trade allies mentioned the knowledge
customers receive on how and why to maintain their systems from a trained technician.
One trade ally shows customers how to clean their system and where and how to change
out filters.
4.6.2.7 Participant Satisfaction
The Evaluators asked OK HEEP HVAC Tune-Up and Replacement program track
participants to rate their satisfaction with various program components using a scale of 1
to 5, where 1 is "very dissatisfied" and 5 is "very satisfied" (Table 3). Participants report
being highly satisfied with the program track overall, through a mean program satisfaction
score of 4.4. Their satisfaction scores on seven of eight program components were
consistently above 4; the exception was the participant mean score on monthly utility bill
savings, which averaged a 3.7 rating.
OG&E Oklahoma Demand Program Portfolio Evaluation
104
Table 4-70 HEEP HVAC Tune-Up and Replacement: Satisfaction by Program
Component
The quality of contractor’s work Mean 4.6
Respondents (n) 72
Interactions you had with the contractor
Mean 4.6
Respondents (n) 72
Interactions you had with OG&E staff
Mean 4.5
Respondents (n) 62
The effort required for the application process
Mean 4.5
Respondents (n) 68
The performance of the equipment installed or the energy efficiency improvements that were made
The trade allies were asked to rate their satisfaction with the OG&E HVAC program track
on a scale of 0 to 10, where 0 means “very dissatisfied” and 10 means “extremely
satisfied.” The ratings were generally high with three trade allies giving the program a
rating of 10, one rated the program a 9, and one rated the program a 7.
Those who rated the program a 9 or a 10 felt the program was very good. One liked that
the program was always changing and looked forward to working on the Duct Sealing
component. One gave high marks to the CLEAResult staff, and there were no complaints
from any of the trade allies about program staff.
The one trade ally said they gave the program a 7 rating primarily instead of a higher
rating because of the current incentive structure for HVAC tune-ups. The trade ally
explained that the $175 is paid the same amount even if you need to be there all day to
OG&E Oklahoma Demand Program Portfolio Evaluation
105
do the HVAC Tune-up. The trade ally wanted the incentive structured like it was in a
previous program version, where allies got paid by task such as $50 for the coil clean and
$50 to clean the blower.
The trade allies were then asked to rate aspects of the 2017 HEEP Program using a scale
of 0 to 10 where 0 is “very dissatisfied” and 10 is “very satisfied.” Here is a summary of
their responses by program category:
The Level of Communications with CLEAResult Program Staff: The trade allies were
happy with their level of communications with CLEAResult staff. Three of the five trade
allies rated their level of communications with CLEAResult staff a 10. The other two gave
ratings of 7 and 8 but did not seem to have any problems with their communication with
CLEAResult.
The Amount of Program Paperwork: The amount of program paperwork did not seem
to be an issue for most. Four of the trade allies gave the amount of paperwork a 10 for
HVAC tune-up. Only one trade ally gave the paperwork a low rating of 5. The trade ally
felt there was some redundancy and commented they had to put in customer information
every time they go out, even if the customer information is already in the system. The
trade ally also noted that sometimes it is easier to fill out data later, but then it looks like
they did not spend enough time with the customer, if they do the work and then go back
to the truck to fill out the information.
Program Measures and/or Rebated Equipment Offered through the Program: Those
interviewed were happy with the types of measures and equipment offered. Two trade
allies gave the program measures a 10. The others primarily did HVAC tune-ups for
OG&E programs and were not sure how to answer the question.
The Timeliness of Rebate Payments to Customers: The trade allies were happy with
the timeliness of the rebate payments. When asked to rate the timeliness of rebate
payments to customers, four of the five trade allies gave ratings of 9 or 10. One trade ally
was a new business and did not have enough experience to give the timeliness of rebate
payments a rating.
The Rebate Amount: Three of the five trade allies were happy with the incentive amounts
with ratings of one 10 and two 8’s. The other two trade allies gave the rebate amount a
low rating of 4 and a 5. They both felt that the set rebate of $175 was too low in many
cases for the time needed and the work required to do a thorough HVAC tune-up on some
systems. One of the two trade allies felt the previous HVAC Tune-up program incentive
structure of providing rebates by tune-up task was more reasonable.
The OG&E Energy Efficiency Website: Only two of the five trade allies used the website
and they gave ratings of 8 and 10. The other three did not have enough experience, if at
all, in using the website so they did not provide a rating.
OG&E Oklahoma Demand Program Portfolio Evaluation
106
HEEP Program Training: All five of the trade allies said they had technical training on
the program, and their satisfaction ratings were high with four 10’s and one 8.
Trade Ally Suggestions for Program Improvement: After rating the various aspects of
the HVAC Tune-up program, the trade allies were asked how the program design or
operations could be improved. Four of the five had no suggestions at this point.
One trade ally, who had given the program paperwork a low rating, made one suggestion
to improve tracking of the status of various steps, particularly when they do Pre-cleaning
and then the rest of the tune-up later. The trade ally suggested using a checklist to show
key tasks such as: Scheduled, Not Scheduled, Pre-Cleaned Completed, and use check
the blocks for each step instead of notes fields. The trade ally said they are often working
on two different systems for the incentive amount, and it seemed like a lot of work to track
some information for the program system if they are also tracking the same information
on their side as well.
4.6.2.9 Program Influence - Energy Saving Knowledge and Behavior
Our survey asked OG&E customers to assess their familiarity with: a) energy efficiency
improvements, and b) energy efficiency activities before and since participating in the
program. First, we asked customers how familiar they were with energy efficiency
improvement benefits before participating in the program. More than half of all
respondents – or 57 percent of these HEEP participants – indicated they were “somewhat
familiar” or “very familiar” with these benefits. Conversely, fourteen percent confirmed
they were “very unfamiliar” while another 13 percent reported they were “somewhat
unfamiliar” about the energy efficiency improvement benefits the OG&E energy efficiency
programs provide.
Table 4-71 Familiarity with benefits of energy efficiency improvements prior to program
participation
Category Percent
Very familiar 15%
Somewhat familiar 42%
Neither familiar nor unfamiliar 17%
Somewhat unfamiliar 13%
Very unfamiliar 14%
Total 100%
N=72
Source: Question e1 Note: Totals may not sum to 100 percent due to rounding Don’t know and refused responses excluded
OG&E Oklahoma Demand Program Portfolio Evaluation
107
More than four out of every five HEEP HVAC Tune-Up and Replacement participants
were more familiar with the benefits of energy saving activities prior to their program
experience, with 72 percent of them indicating they were “very familiar”. Another 24
percent identified themselves as “somewhat familiar” with energy saving activity benefits
prior to their program participation.
Table 4-72 Familiarity with benefits of energy saving activities prior to program
participation
Category Percent
Very familiar 72%
Somewhat familiar 24%
Neither familiar nor unfamiliar 3%
Somewhat unfamiliar 1%
Very unfamiliar 0%
Total 100%
N=72
Most (85 percent) of HEEP HVAC Tune-Up and Replacement participants indicated they
had performed energy saving activities prior to their experience with the program.
Respondents could offer multiple examples of energy saving activities. The most
frequently mentioned activity was “turn off lights when not in the room” (41 percent), which
was followed closely by “adjust heating system settings” (37 percent).
We then asked participants whether program participation had increased their energy
efficiency knowledge or changed their actions. Focusing first on knowledge, Table 4-73
shows a majority of respondents felt they picked up some additional energy efficiency
knowledge as a result of their program participation. Eighteen percent of survey
respondents indicate they are “much more” knowledgeable about energy efficiency than
they were before participating, while another 43 percent report being “somewhat” more
knowledgeable as a result of their program experience.
OG&E Oklahoma Demand Program Portfolio Evaluation
108
Table 4-73 As a result of program, how much more knowledgeable about energy
efficiency?
Category Percent
Much more knowledgeable than before participating 18%
Somewhat more knowledgeable 43%
Slightly more knowledgeable 18%
Not more knowledgeable 21%
Total 100%
N=72
Source: Question e5 Note: Totals may not sum to 100 percent due to rounding Don’t know and refused responses excluded
More than half of our HEEP HVAC Tune-Up and Replacement survey respondents (56
percent) reported that the program has changed their actual energy savings behaviors.
Only ten percent of the HEEP HVAC Tune-Up and Replacement track respondents
indicated that have participated in other energy efficiency programs. Six respondents
mentioned the OG&E’s Smart Hours program by name or described it. Other responses
varied widely, with respondents often measures or other prescriptive measures they may
have received from a program, without recall of a describing direct install program name.
4.6.2.10 Program Influence - Trade Ally Marketing, Sales, and Recommendations
The trade allies were asked to rate the importance of the program, including the rebates
and information, in influencing the marketing and sales of equipment or measures during
2017. The rating was on a scale of 0 to 10 where 0 is “not at all important” and 10 is
“extremely important.”
Two of the five trade allies rated program importance high as a 9 and a 10 without further
comment. Two of the five trade allies gave a rating of 7 although both said that they would
not do the OG&E program type of tune-ups if the program were not available.
One trade ally rated the importance of the program as a 3 in influencing their marketing
and sales of tune-ups. The trade ally explained that HVAC tune-ups for OG&E is about
30 percent of their business and is not the "low hanging fruit." They rely on their own
customer leads to get projects. The program gets them introduced to new people but said
it is hard for them to quantify the impact. The same trade ally said they always market
HVAC tune-ups and have always been a proponent of tune-ups, so the program does not
influence them a lot.
The trade allies were also asked if the availability of the program incentives influence the
type, quantity, or efficiency level of items they recommended to customers. Three of the
OG&E Oklahoma Demand Program Portfolio Evaluation
109
five said yes and two said no. One would not market the HVAC tune-ups at all and the
other said they would only do “regular” tune-ups if the program were not available.
The trade ally who gave program importance a 7 said the program does not influence
what they recommend, because they always recommend the full-blown HVAC tune-ups
similar to OG&Es. The same trade ally said they sometimes did not want to mention the
OG&E program because if additional work such as coil cleaning is involved, it digs into
their profits if they do the work using the OG&E HVAC Tune-up program. The same trade
ally went on to say they like to do what is good for their customer though. Their company
offers 3 types of services: (1) Full-blown tune-up similar to OG&E; (2) another tune-up
that is less extensive than OG&E but competes with what most Local and National
companies do; and (3) a $39 Cleaning.
4.6.2.11 Sales Impacts of the HVAC Tune-up Program
The trade allies were asked about impacts of the program on project volume. They were
asked specifically whether the number of projects had increased significantly, somewhat,
a little, hardly at all, or not at all.
Only one trade ally said the volume of projects had increased significantly because of the
HVAC Tune-up program while commenting that their business had grown steadily over
the last few years. Two trade allies said the number of projects had increased “somewhat”
or “a little” while two trade allies said their projects had increased “hardly at all” because
of the HVAC Tune-up program. One trade ally said their business was just getting started
so the program had only had a little impact.
The trade allies said all projects that qualify do apply for the incentive. They were then
asked whether they expected the number of HVAC tune-up projects to increase,
decrease, or stay the same in the next 12 months. Four of the five trade allies said the
number of HVAC tune-up projects would increase. One said they thought the number of
projects would stay the same but thought their business may do more Duct Sealing work.
4.6.2.12 General respondent characterization
Table 4-74 summarizes basic home information among our OK HEEP HVAC Tune-Up
and Replacement respondents. Nearly all (97 percent) HEEP HVAC Tune-Up and
Replacement participants we surveyed were homeowners, and one-hundred percent of
these respondents indicated they lived in single-family homes. Twenty-three percent of
this program track’s respondents are the only people that live in their home, while 38
percent of respondents say they have two people in their residence. Income among
respondents within this program track is well-distributed.
OG&E Oklahoma Demand Program Portfolio Evaluation
110
Table 4-74 HEEP HVAC Tune-Up and Replacement Participant Characteristics
Do you rent or own the home we’ve been discussing today?
Rent Column N % 3%
Own Column N % 97%
What is the main fuel used to heat your home?
Electricity Column N % 24%
Natural gas Column N % 74%
Other Column N % 2%
Total Unweighted Count 72
What is the main fuel used to heat your water?
Electricity Column N % 17%
Natural gas Column N % 82%
Bottled gas propane Column N % 1%
Total Unweighted Count 72
Including yourself, how many people lived in your household last year?
1 person Column N % 23%
2 people Column N % 38%
3 people Column N % 16%
4 people Column N % 13%
5-6 people Column N % 10%
7 or more people Column N % 0
Total Unweighted Count 69
Including all money earned from wages, salaries, tips, commissions, workers´ compensation, unemployment insurance, child support, or other sources, about how much was your total annual household income before taxes in 2017?
Less than $10,000 Column N % 3%
$10,000 to less than $20,000 Column N % 8%
$20,000 to less than $30,000 Column N % 8%
$30,000 to less than $40,000 Column N % 8%
$40,000 to less than $50,000 Column N % 13%
$50,000 to less than $75,000 Column N % 27%
$75,000 to less than $100,000 Column N % 15%
$100,000 to less than $150,000 Column N % 10%
$150,000 to less than $200,000 Column N % 7%
$200,000 or more Column N % 0%
Total Unweighted Count 60
Source: Question dem1 dem3 dem3h dem4 dem5 Note: Totals may not sum to 100 percent due to rounding
4.6.3 Consumer Products Channel
This section presents the methodology used for the process-related data collection
activities the Evaluators performed within the OK HEEP Consumer Products track. These
activities included program staff interviews, and the collection of data through a short
panel of lighting-related questions within the participant HEEP survey. This survey was
administered to OG&E customers who participated in the Residential Solutions or HVAC
Tune-Up and Replacement tracks, respectively.
OG&E Oklahoma Demand Program Portfolio Evaluation
111
4.6.3.1 Program Staff Interviews
The Evaluators completed an in-depth interview with the OK HEEP program manager
and an additional interview with CLEAResult team members who worked to implement
the Consumer Products program track in PY2017. The Evaluators used these program
staff interviews to identify program updates or changes experienced in PY2017, while
also exploring staff roles and responsibilities, program communications and marketing,
and the overall program delivery processes in place during PY2017.
4.6.3.2 Participating Customer Surveys
Telephone surveys were completed with participants within the OK HEEP HVAC Tune-
Up and Replacement track and the Residential Solutions track, respectively. The surveys
were primarily designed to collect respondent feedback on program communication and
offerings, measure changes in participant energy efficiency awareness and behaviors due
to program participation, participant satisfaction, and collected key household
characteristics and demographic information. The team also used this data collection
opportunity to ask a series of lighting-specific questions to OG&E customers. These
questions collected information on their awareness of both CFL and LED light bulbs, and
further surveyed them on their lighting purchase history and decision making processes,
and measured their product pricing sensitivity.
We ultimately surveyed 72 Oklahoma HEEP HVAC Tune-Up and Replacement program
track participants and 99 HEEP Residential Solutions track participants. The lighting
question panel was asked of all survey respondents in the survey. The survey began
fielding on Thursday, January 25, 2018 and ran through February 5, 2018.
4.6.3.3 Program Delivery, Communication, and Marketing
Communication was the key program delivery component of the Upstream Consumer
Products track within HEEP. Internally, both OG&E and CLEAResult reported being
satisfied with the level of communications and reporting that has been established and
continues between their offices. Externally, CLEAResult works to market the program to
retailers and manufacturers to bring them on-board to the program, and they worked this
year to widen the program reach to additional retailers. Further, CLEAResult then
partners with enrolled retailers, distributors, and / or manufacturers to a) educate them on
key program offerings, and b) give them tools to help promote the program. CLEAResult
works to provide information to enrollees of the program through annual training which
highlights key benefits and features of energy efficient lighting products. CLEAResult also
brings in industry experts to these training, to connect retailers with resources with a deep
range of knowledge about energy efficiency lighting. They also engage in monthly field
training to regularly engage and educate retailers, answer questions, and provide
refreshed promotion materials throughout the program year.
OG&E Oklahoma Demand Program Portfolio Evaluation
112
Promotion materials that CLEAResult provides to retailers are a key tool in program
promotion. They provide a wide range of Point-of-Purchase (POP) materials, such as
posters, stickers, and pallet and other special displays. Field representatives working
with the program also hold in-store promotional events to further engage customers and
garner program and product attention.
4.6.3.4 OG&E Customer Lighting Product Awareness
HEEP Participant Awareness of Energy Efficiency Lighting: We opted to ask
customers about their awareness of different type of light bulbs available in the
marketplace. Consumer awareness of energy efficiency lighting was high overall. Just
over three-fourths of OG&E customers surveyed (76 percent) confirmed that they had
heard of CFL’s before our survey. We also asked respondents who had not received
LEDs through the program if they had heard of LEDs. All (100 percent) of Residential
Solutions and HVAC Tune-Up and Replacement respondents confirmed they had heard
of LEDs before our survey. We went further by asking respondents if they felt they could
confidently identify CFL and LED light bulbs if they were placed in front of them. Table
4-75 highlights that nearly every nine out of ten program participants felt they could, and
details answers by program participant type.
Table 4-75 Could you correctly identify a typical incandescent light bulb, a CFL, and a LED light bulb if placed in front of you, outside of their packaging?
Category Residential
Solutions Track HVAC Tune-Up and Replacement Track
Yes 91% 87%
No 9% 13%
Total 100% 100%
N=92 N=68
Source: Question L3
Don’t know and refused responses are excluded Note: Totals may not sum to 100 percent due to rounding
HEEP Participant Energy Efficiency Lighting Purchase Information: Most of our
remaining lighting analysis included only respondents who felt they could identify energy
efficiency lighting without its packaging, or in other words, had a good base of knowledge
about energy efficiency lighting products. Among that customer subset, sixty-nine percent
of Residential Solutions track participants had purchased LEDs, while HVAC Tune-Up
and Replacement participants were slightly less likely (61 percent) to have purchased
energy efficient LEDs.
OG&E Oklahoma Demand Program Portfolio Evaluation
113
Table 4-76 OG&E program participants who have purchased LEDs in the past six months
Category Residential
Solutions Track
HVAC Tune-Up and Replacement
Track Yes 69% 61%
No 31% 39%
Total 100% 100%
N=83 N=57
Source: Question L4 Don’t know and refused responses are excluded Note: Totals may not sum to 100 percent due to rounding
We asked HEEP participants who had purchased LEDs in the past 6 months for some
context about what the new LEDs were replacing in their home. Table 4-77 summarizes
their responses and highlights that roughly one in four OG&E customers are still replacing
traditional incandescent light bulbs. Nearly half of the participants we surveyed indicate
they are replacing a mixture of bulb types with LEDs.
Table 4-77 Type of bulbs LEDs replaced
Category Residential
Solutions Track HVAC Tune-Up and Replacement Track
Traditional incandescent light bulbs 30% 23%
CFLs 19% 19%
Existing LED light bulbs 2% 7%
A mixture of these bulbs 48% 45%
Some other type of existing bulb 0% 6%*
Total 100% 100%
N=54 N=31
Source: Question L8 Note: Totals may not sum to 100 percent due to rounding *One respondent indicated their LEDs replaced Halogen bulbs; another responded “all of the above”, while a final respondent indicated that their bulbs didn’t replace anything, but rather, were for new lamps.
While getting a handle on what the HEEP program participants had purchased, we also
asked them to name the retailer where they had purchased their LEDs. Table 4-78 lists
how HEEP program participants most frequently answered this question. This year,
Lowe’s surpassed The Home Depot was most frequently mentioned by Residential
Solutions and HVAC Tune-Up and Replacement tracks customers, respectively. Big box
discount retailers, such as Sam’s Club and Wal-Mart, also were popular answers among
participants in each track.
OG&E Oklahoma Demand Program Portfolio Evaluation
114
Table 4-78 Name of retailer where LED bulbs were purchased in past six months
Category Residential
Solutions Track HVAC Tune-Up and Replacement Track
Lowe’s 54% 62%
The Home Depot 53% 44%
Walmart Supercenter 44% 29%
Sam's Club 18% 24%
Ace Hardware 25% 12%
Target 5% 12%
True Value 2% 3%
None of these retailers 2% 3%
Batteries Plus 5% 8%
Totals N=57 N=34
Source: Question L5
Don’t know and refused responses are excluded Note: Totals may sum to more than 100 percent, as participants can choose more than one answer
When making a new energy efficiency lighting purchase, roughly half of the HVAC
respondents reported that bulb life was the leading deciding factor when selecting a bulb
for purchase, while 39 percent of Residential Solutions track participants also identified
this as their leading decision factor. “Energy efficiency” was a verbatim survey provided
second-most often among respondents overall, which suggests that participation in
OG&E energy efficiency programs is likely influencing its consumer buying decisions.
Price remains still near the top of mind for most respondents, as 25 percent of the
Residential Solutions track participants mentioned it, while 19 percent of HVAC Tune-Up
and Replacement track participants reported it was a deciding factor.
OG&E Oklahoma Demand Program Portfolio Evaluation
115
Table 4-79 Deciding factors when selecting bulbs for purchase
Category Residential
Solutions Track HVAC Tune-Up and Replacement Track
Bulb life 39% 50%
*Energy efficiency 34% 28%
Price 25% 19%
Lumens or brightness 16% 19%
Lighting facts/Energy Facts Label 18% 13%
*Cool burning 11% 13%
Wattage 11% 6%
ENERGY STAR label 5% 16%
Color appearance 5% 6%
*Availability of bulbs 2% 6%
Watt equivalency 4% 0%
Shape 2% 3%
Dimming 2% 0%
Something else (specify) 20% 3%
Totals N=56 N=32
Source: Question L7 Don’t know and refused responses are excluded Note: Totals may not sum to 100 percent as respondents could select more than one answer *Indicates category computed during analysis
Price Impact on the Participant Energy Efficiency Lighting Purchase Decision: Our
remaining questions about energy efficiency lighting offered through the HEEP Consumer
Products track focused on the impact of price on participants’ energy efficient lighting
decisions. We asked participants how likely they would have been to make the same LED
purchases if each bulb had cost an additional $2.50. More than two-thirds (67 percent) of
Residential Solutions customers indicated they "definitely" or "probably" would have still
purchased LEDs even if the LEDs cost more than $2.50 per bulb. HVAC Replacement
and Tune-Up respondents indicated they were just slightly less likely (60 percent
“definitely” or “probably” would) to buy the same LEDs if they cost more.
OG&E Oklahoma Demand Program Portfolio Evaluation
116
Table 4-80 Likelihood of purchasing LEDs if they cost more than $2.50 per bulb
Category Residential
Solutions Track HVAC Tune-Up and Replacement Track
Definitely would have still purchased LEDs 40% 33%
Probably would have still purchased LEDs 27% 27%
Not sure if you would have purchased LEDs 15% 21%
Probably would not have still purchased LEDs 16% 6%
Definitely would not have still purchased LEDs 2% 12%
Total 100% 100%
N=55 N=33
Source: Question L10 Don’t know and refused responses are excluded Note: Totals may not sum to 100 percent due to rounding
Despite their documented acceptance of LED bulbs through data in this section, less than
half of the OG&E HEEP respondents we surveyed were aware that OG&E discounted
LED bulbs in 2017. While nearly 40 percent of the Residential Solutions track participants
are aware that OG&E discounted LED bulb prices in 2017 through participating retailers,
while far fewer (15 percent) of HVAC Tune-Up and Replacement track participants are
aware of these same discounts.
Table 4-81 Aware OG&E discounted certain LED bulbs in 2017
Category Residential
Solutions Track HVAC Tune-Up and Replacement Track
Yes 39% 15%
No 61% 85%
Total 100% 100%
N=54 N=33
Source: Question L12 Don’t know and refused responses are excluded Note: Totals may not sum to 100 percent due to rounding
OG&E HEEP Residential Solutions customers have had more experience with buying
CFLs (71 percent) than LEDs (61 percent). The opposite is true among HVAC Tune Up
and Replacement participants, who more frequently answered “yes” to buying LEDs (66
percent) prior to their program participation in 2017, compared to roughly half of them (54
percent) buying CFLs.
Table 4-82 Bulb Types Purchased Prior to 2017
Category Residential Solutions
Track
HVAC Tune Up and Replacement
Track
Residential Solutions
Track
HVAC Tune-Up and Replacement
Track
OG&E Oklahoma Demand Program Portfolio Evaluation
117
CFLs prior to
2017 CFLs prior to
2017 LEDs prior
to 2017 LEDs prior to
2017
Yes 71% 54% 61% 66%
No 29% 46% 39% 34%
Total 100% 100% 100% 100%
N=90 N=68 N=98 N=70
Source: Question L15, L16 Don’t know and refused responses are excluded Note: Totals may not sum to 100 percent due to rounding
4.7 Planned Program Changes
The HEEP will remain in OG&E’s Demand Program portfolio in PY2017, with no
significant changes.
4.8 Conclusions & Program Recommendations
4.8.1 Conclusions
The key conclusions from the PY2017 evaluation of the HEEP are as follows:
High Participant Satisfaction: Results from the participant survey effort
indicate that customers highly value the services offered by HEEP, and recall
very positive experiences with the assessment and measure installation
process. Very few participants expressed dissatisfaction with any program
elements, and several participants provided open-ended commentary that
praised the program for the services it provides and the professionalism of its
trade ally staff. Satisfaction scores on nearly all program components were
consistently rated 4 out of 5 or higher, for both trade allies and participants
Demonstrated Education Effects: Feedback from the participant survey
suggests that the program is increasing customer knowledge of energy
efficiency equipment and energy efficiency behaviors that can be employed to
conserve energy and lower utility bills. Some customers have learned about
other utility offerings through the HEEP, potentially leading to additional energy
savings.
Smooth Enrollment Process: Regardless of how they requested an
assessment to begin participating in the HEEP, customers had an easy time
communicating with the program and scheduling their home assessment.
Among those who called the Customer Service center, 93 percent indicated
their experience was ‘very easy’, while the remaining customers rated their
experience as “easy”.
OG&E Oklahoma Demand Program Portfolio Evaluation
118
Adequate Database Quality: Except for the ex ante savings issues described
below, the Evaluators found the ex ante savings values within the database to
be accurate for most measures. Additionally, CLEAResult was very consistent
in responding to data requests and correcting errors when necessary.
Effective Trade Ally Portal: An elevated level of satisfaction was reported by
trade allies in relation to the portal that CLEAResult utilizes to share information
with trade allies.
4.8.2 Recommendations
The HEEP was very successful in PY2017. The Evaluators identified few specific,
systematic or persistent issues with program operation and design. As the utilities plan to
continue offering similar services and maintaining their current operational structure under
the HEEP; consideration of the following recommendations may be useful moving
forward:
Resolve possible program database issues (Schools Outreach): For
PY2017, RAP developed deemed savings estimates for the measures that are
implemented through the Schools Outreach channel, including LEDs, aerators,
and a showerhead. These deemed savings were based on algorithms and
inputs contained in the Arkansas TRM, updated for Oklahoma weather. After
reviewing and providing input on these documents, the Evaluators determined
them to be appropriate sources for calculating savings for PY2017, with one
exception. The ex ante gross savings for LEDs, in tracking data, were dividing
the lifetime savings by the EUL of the measure. RAP should consider modifying
their database so that ex ante and ex post gross savings are more closely
aligned.
Resolve possible program database issues (Consumer Products): An
issue was discovered with the baselines being assigned to some reflector bulbs
where a 40W baseline was being assigned in cases that should have a 65W
baseline. CLEAResult has indicated that this will be corrected for PY2018.
Continue to assess and potentially add additional energy efficient
measures to program offerings to increase customer satisfaction and
assist customers in realizing additional energy savings. Residential
Solutions customers continue to indicate that they often do not implement all
recommendations made by the program. Program staff should consider
offering incentives for measures and projects often recommended through the
program, and follow up with customers with marketing efforts to encourage
them to make additional program-recommended energy efficiency upgrades.
OG&E Oklahoma Demand Program Portfolio Evaluation
119
Continue to focus on building positive contractor relationships within the
program’s delivery mechanisms through strong training opportunities
and clear, ongoing communications. Last year, none of the Residential
Solutions contractors interviewed within this evaluation could recall any
program training however, this year, all three Residential Solutions contractors
interviewed for this evaluation recalled the training and were satisfied with it.
This year, two of three contractors we interviewed were satisfied with
CLEAResult communications, while one thought communications could be
improved. Training sessions and contractor communications create the
opportunity for OG&E and CLEAResult to build a solid relationship foundation
with the trade allies, and educate and better inform them of their program
offerings and expectations. Not only does this inspire confidence among the
trade allies, but it increases the possibility that the program participants have a
positive program experience within their current program, and increases the
possibility that trade allies will continue to support, market, and work within the
program.
Offer residential customers more information about other available OG&E
program programs and tracks. While OG&E participants within this program
are largely satisfied with it, only 10% of respondents within this track confirmed
they had participated in another OG&E program. Consider offering more
educational materials about OG&E program opportunities while onsite within
the HVAC Tune-up or Replacement appointment, and follow-up after the
appointment to cross-market additional energy efficiency programming.
Consider additional promotion of the program track’s HVAC replacement
options to Residential OG&E customers to increase participant
awareness of the program option. HVAC tune-ups are still a substantial part
of this program track’s savings, while contractors indicate they are doing few
actual replacements. The program stands to harvest more savings from the
replacement part of the program with increased awareness of it among its
customer population.
Continue to communicate regularly with Contractors about program
changes and expectations to maintain high levels of program
satisfaction. As was the case in our PY16 evaluation, HVAC contractors
working to support this program in PY17 are highly satisfied with program
communications, and they found the contractor training offered by CLEAResult
this year to be very helpful. CLEAResult should continue to maintain their
currently levels of communication frequency and detail to sustain their high
levels of contractor program satisfaction.
OG&E Oklahoma Demand Program Portfolio Evaluation
120
5. Positive Energy – New Home Construction (PE-NHC)
5.1 Evaluation Findings
The table below presents the ex ante and ex post energy (kWh) and demand (kW) savings
for the PE-NHC program. This program is divided into four strata.
The evaluator’s User Defined Reference Home (UDRH) evaluation approach uses
REMrate building simulation models to verify ex post savings. The ex post savings are
calculated for 49 homes. Using the ex post savings per house, realization rates for the
four strata are established. The program summary of energy and demand savings is
shown below.
Table 5-1 Energy Savings Summary for PE-NHC in PY2017
𝐷𝐿𝑝𝑟𝑒 = Pre-improvement duct leakage at 25 Pa (ft3/min)
𝐷𝐿𝑝𝑜𝑠𝑡 = Post-improvement duct leakage at 25 Pa (ft3/min)
13 Derived from the EnergyGauge software ceiling fan profiles. 14 Residential light logging study by Cadmus - Entergy Arkansas, Inc. 2013 EM&V Evaluation Report.
𝑊𝑏𝑎𝑠𝑒 = Based on wattage equivalent of the lumen output of the purchased LED
omni-directional lamp and the program year purchased/installed; for omni-
directional LEDs, use the following base and post case wattages
𝑊𝑝𝑜𝑠𝑡 = Wattage of LED purchased/installed. Use above table for post-wattages
𝐻𝑜𝑢𝑟𝑠 = Average hours of use per year = 961 hours/year (assuming 2.63
hours/day and 365.25 days/year operation)18
𝐼𝑆𝑅 = In Service Rate, or percentage of rebate units that get installed, to account
for units purchased but not immediately installed
𝐼𝐸𝐹𝐸 = Interactive Effects Factor to account for cooling energy savings and heating
energy penalties.
The required site-specific inputs as stated in the OKDS are as follows:
The quantity and lumen output of post fixtures;
Whether or not the retrofits were time of sale or direct install (this defines the in-
service rate);
Whether or not the bulbs were installed indoors or outdoors; and
The heating type of the residence.
17 “Memorandum: Average Pre-sealing and Post-sealing Duct Leakage Rates”. EnerNOC, 2012 18 ADM Associates, Inc., “Residential Lighting Hours of Use (HOU) for the Oklahoma Demand Programs in 2016”.
Prepared for Oklahoma Gas & Electric. September 2016.
OG&E Oklahoma Demand Program Portfolio Evaluation
147
For example, if 24 LEDs each with light output of 1,049 lumens were directly installed to
replace 24 incandescent lamps in a residence with gas heating, the residence would have
an annual gross savings of approximately 873.8 kWh.
The evaluation team completed 18 in-depth interviews with OK contractors in the Large
C&I, SBDI, and HVAC Tune-up Program channels. Five of these 18 contractors provided
information on two program channels and one other provided information on all three
program channels. In addition to the contractor in-depth interviews, the evaluation team
completed interviews with 13 lighting distributors working under the Midstream Lighting
Program in 2017.
Trade ally contact and lighting distributor contact information, along with their project
volume within the program, was available to the team for review and consideration within
the program participant tracking data delivered by CLEAResult upon completion of the
PY17 program year. These data drove the trade ally interview selection. We prioritized
trade ally selection for interviews based on their program participation volume, opting to
OG&E Oklahoma Demand Program Portfolio Evaluation
201
offer an interview opportunity to those who worked most with the Large C&I and HVAC
tune-up program channels. Given the small number of SBDI contractors and Midstream
lighting distributors, the evaluation team attempted to complete as many interviews as
possible with these two groups.
The in-depth interviews with contractors were completed between January 8 and January
28, 2018. The interviews with lighting distributers were completed between January 10
and January 20, 2018.
7.8.4 Process Evaluation Findings
This section details the findings from the process evaluation pertaining to program
communications and marketing, program delivery, program satisfaction, and customer
characteristics.
7.8.4.1 Program Communications and Marketing
CLEAResult was responsible for a majority of the OK CEEP program marketing in 2017,
although OG&E program and education managers, and account representatives also
played a role. The OK C&I programs are marketed via the internet, billboards, radio ads,
point of sale materials, and one-on-one outreach. Efforts varied by program channel. For
example, SAGE marketing was done mostly via direct outreach. Both Large C&I and
SBDI were more contractor-driven programs, while Midstream distributors promoted
program incentives directly to customers (end use customers and contractors to which
they sell) by word-of-mouth and program marketing materials provided to them (counter
mats, in-store displays) by CLEAResult. CLEAResult also maintains a toll-free number
for customers to call. The main exception to program marketing delivered by CLEAResult
is the OG&E program website. OG&E uses its website to promote their programs, and
CLEAResult works with OG&E to incorporate any program materials / messaging onto
their website. In 2017, CLEAResult updated the C&I webpage to make it easier for
customers to understand the various program offerings by OG&E. Finally, OG&E and
CLEAResult both took opportunities to present at nonprofits, social / civic activities or
meetings, trade shows and seminars throughout PY17.
The wide variety of program marketing tactics employed under the CEEP program
umbrella were apparent within the participant data we collected for this evaluation. Fifty
percent of participants reported hearing about this program directly from a contractor or
vendor, and another 19 percent confirmed that they heard about the program through
word of mouth – a colleague, other business, etc. Fifteen percent cited a Business Energy
Advisor or Program Representative and 14 percent cited an account representative.
Between six to seven percent each cited a conference/trade show/expo, a mailing from
OG&E, the OG&E website, and previous experience with an OG&E program.
OG&E Oklahoma Demand Program Portfolio Evaluation
202
Table 7-28 How learned of program
Category Large C&I SBDI SAGE
HVAC Tune
Up CEI Total
Contractor / Vendor 41% 58% 58% 56% 0% 50%
Colleague/Another business 16% 33% 32% 15% 0% 19%
Business Energy Advisor or Program Representative
20% 0% 11% 15% 33% 15%
Account representative 16% 0% 11% 13% 67% 14%
Conference/Trade Show/Expo 2% 17% 16% 5% 33% 7%
Mail from OG&E 4% 17% 0% 9% 0% 6%
OG&E website 14% 0% 5% 2% 0% 6%
*Previous experience with an OG&E program
10% 8% 11% 2% 0% 6%
Other 6% 25% 0% 0% 0% 4%
Email from OG&E 0% 0% 5% 4% 0% 2%
Radio / Print Advertising 0% 0% 0% 2% 0% 1%
*Internet search 2% 0% 0% 0% 0% 1%
Total (N) N=51 N=12 N=19 N=55 N=3 N=140
Source: Question A1 Note: May not total 100 percent as respondents could select more than one answer
*Indicates category created during analysis
Account representatives were the most frequently cited source of information for CEI
participants, while Large C&I, SBDI, SAGE, and HVAC Tune-up participants were most
likely to cite a contractor/vendor as the source of their program information. This was a
significant increase among HVAC Tune-up participants from FY16, where only 19 percent
of HVAC Tune-up participants reported hearing of the program from a contractor/vendor.
A summary of the CEEP participant responses across program channels appear in Table
7-28.
Respondents generally reported having received enough information about the program
when they first heard about it. Ninety-three percent of CEEP respondents overall
confirmed they received enough information about the program through their
communication channel; this is a slight increase from PY16 (88 percent).
HVAC Tune-up respondents were more likely to report that they felt they had enough
information about the program in their initial contact in PY17 (95 percent) than they did in
PY16 (81 percent). This may be because HVAC Tune-up respondents were more likely
to hear about the program directly from a contractor or vendor in PY17, and less likely to
OG&E Oklahoma Demand Program Portfolio Evaluation
203
learn about the program through OG&E direct mail or some other information source as
they did in PY16.
Table 7-29 Method learned of program provided enough information
Category Large C&I SBDI SAGE
HVAC Tune Up CEI Total
Yes 94% 92% 89% 95% 100% 93%
No 6% 8% 11% 5% 0% 7%
Total (N) N=49 N=12 N=19 N=55 N=3 N=138
Source: Question A2
When we asked respondents what kind of additional information they would have liked to receive about the program, answers varied. However, clear themes about more information – especially about more information on the project details and actual project savings – were repeatedly mentioned by respondents. As one respondent noted:
“There was almost no information on the website, so I had to be routed through a third-party vendor (CLEAResult) that was managing their program. It would be nice if it just laid out how the program worked and the forms needed.”
Program participants were asked if there are other ways that they would prefer OG&E
contact them about their CEEP programs. Overall, 59 percent of participants could not
think of another preferred method. Among those preferring another method, SBDI and
SAGE participants were most likely to do so (58 and 65 percent, respectively). Email or
a direct call from the contractor or OG&E were the most preferred methods among those
having another preferred method.
7.8.4.2 Trade Ally and Lighting Distributor Communications and Marketing
Trade allies were asked about how they became aware of the programs, and their
preferences for receiving program information. When asked how they heard about the
CEEP programs, about one-half of the trade allies said they heard about CEEP from
OG&E or CLEAResult staff. The rest learned about the program from a former employer,
word-of-mouth from industry sources, advertising, or by doing their own research.
Eleven of the 18 trade allies said they have been involved in the program for 2 to 3 years
or less. Four of the trade allies said their company had a long-time relationship with OG&E
programs dating back as far as 10 years ago. The rest of the trade allies did not know for
sure how long their company had been involved with the programs.
When asked why they or their company decided to get involved in the programs, reasons
cited by trade allies included:
OG&E Oklahoma Demand Program Portfolio Evaluation
204
The programs fit well with their company philosophy and the types of products and services offered to their customers
To make money or expand their customer base To help customers save money and energy To get their business name out there—recognition for newer businesses The incentives were a good deal.
Most trade allies (15 of 18) interviewed prefer to receive program information from OG&E
and/or CLEAResult by email. The remaining three trade allies prefer personal contact
including webinars and one-on-one training with CLEAResult where they can ask
questions. A little over half, 10 of the 18 trade allies, felt that the communication they had
received about the program was adequate. Two of the trade allies did not answer or were
unsure. One was very familiar with the programs and did not need much communication,
and one did not need much communication since they only had one project.
Midstream Lighting distributors were asked how they first heard about the incentives
available through the Midstream Lighting Program, and why they chose to participate in
the program. Of the 12 distributors who could answer this question, they mentioned
various sources of information and sometimes more than one source. Four mentioned
learning about the incentives through other OG&E programs they had worked on, and
four mentioned from a colleague at work. Three reported hearing about the incentives
from CLEAResult, one mentioned OG&E staff, and one heard about the incentives from
a customer.
Enhanced sales (7 of 13 distributors), enhanced customer service (4), saving customers
money (3), and energy savings (3) were the primary reasons given for getting involved
with the Midstream Lighting program. As one distributor added: “It is my job to offer
solutions to our customers”. Other reasons for participation mentioned by one distributor
each included keeping up with the competition, and because customers were asking
whether they were a participating distributor.
Lighting distributors were asked about the OG&E marketing materials they provide to
customers and customer awareness of OG&E’s discounts. Midstream flyers (6) and
instant rebate sheets (4) were the materials most often mentioned.
7.8.4.3 Program Influence on Trade Allies and Lighting Distributors
Trade allies were asked to rate the importance of the program, including the rebates and
information, in influencing their marketing and sales of equipment or measures during
2017. The rating was on a scale of 0 to 10 where 0 is “not at all important” and 10 is
“extremely important.”
OG&E Oklahoma Demand Program Portfolio Evaluation
205
Eight of the 18 trade allies rated the importance of program influence at 8 to 10 for at least
one program channel. Five trade allies gave program influence ratings of 5 to 7. Two of
the 18 trade allies could not provide a rating.
One Large C&I HVAC trade ally gave the program influence a 2 or 3 because they had
only had one project (although they recognize that the program helped to sell the
geothermal heating and cooling project). Another HVAC Tune-up trade ally gave a rating
of 3 to 4 and commented that when they started the program they expected a 9 or 10 on
the program influence. The trade ally further commented that because they were doing
one very large school project, they were too busy to take on any additional work with the
technical staff they had available. In addition, the trade ally said they would have done
more to aggressively pursue projects using the program if they had had enough staff.
The trade allies were also asked if the availability of the program incentives influenced
the type, quantity, or efficient level of items they recommended to customers. Only one
of the trade allies said they would make different recommendations if the CEEP program
was not available. One trade ally, an HVAC business, clarified that they would sell the
equipment anyway, but the rebate increases the number of units they would sell.
All Midstream lighting distributors reported using the discounts as a marketing tool.
Therefore, all customers are aware that they are receiving discounted lighting, and all
customers are informed that OG&E is the source of this discount.
Twelve of 13 lighting distributors expected that participating in the program would
increase their sales of efficient lighting, and 11 of these 12 felt their expectations were
met. As one distributor commented: “We can definitely tell the difference in our sales
since we’ve been participating.” Another distributor noted that: “It seemed like we were
hearing about it, other companies were doing it, and we were losing sales. We lost some
customers that first year when we only had a small allotment, but most of our customers
have come back since they’ve given us larger allocations. . . it totally benefitted our
customers and us.” The one distributor who said expectations were not met does not deal
with screw-in bulbs.
The only distributor who did not expect their sales to increase said “No, I did not expect it
to have the impact that it did. It was a pleasant surprise!”
In addition to increasing sales of efficient lighting, 10 of 12 distributors felt their
participation in the program has helped to expand their customer base by bringing in new
contractors and small businesses.
7.8.4.4 Program Delivery
The PY17 OK CEEP programs had oversight through an OG&E program manager and
CLEAResult implementation staff. CLEAResult worked directly with customers and
OG&E Oklahoma Demand Program Portfolio Evaluation
206
contractors in the field to assess project sites, make project recommendations, and
process program paperwork and rebate payments. CLEAResult also worked with trade
allies and Midstream distributors to assist in marketing the program channels, creating
and distributing marketing collateral for their use, and offering training as needed within
each channel – especially during the project year kickoff and wrap-up periods.
CLEAResult provided regular progress reports to OG&E for all CEEP program channels
that reported progress towards program goals, assessed active projects, and provided
details about the upcoming project pipeline. Both OG&E and CLEAResult also reported
regularly communicating with each other through email, by phone, and occasionally, in
person.
OG&E CEEP customers within the Large C&I, SBDI, or SAGE channels typically initiated
program participation by making an inquiry on the OG&E website or talking with an OG&E
or CLEAResult representative. After that initial contact, a participant generally takes these
steps to participate in these CEEP channel offerings:
1) After contacting the program, the customer signs a Participation Agreement or
Customer Proposal with OG&E which describes program commitments required
of the participant and agrees to the conditions and processes set forth for the
program. CLEAResult then contacts participants to provide details on program
participation, benefits, and requirements, and to begin the program process.
2) A customer (typically an owner or a facilities manager within this process) has their
facility assessed by CLEAResult. The assessment helps identify energy efficiency
improvement options, and the customer receives these recommendations for
improvements. In some cases – such as the SBDI channel – nonresidential
customers may receive direct install measures through the programs.
3) The customer also receives an estimate of potential energy efficiency savings
worked up through a program calculator and incentive amount estimates to help
them calculate net project costs (post-incentives).
4) If a contractor is not already involved, a customer may select a contractor to help
implement their energy efficient project.
5) If a contractor is selected to work with a customer, CLEAResult stays in touch with
the customer to monitor work progress and troubleshoot any problems or
challenges that may arise.
6) After project work is complete, CLEAResult performs a project post-inspection and
if applicable, pays the customer their incentive.
For the HVAC tune-up program channel, customers must provide necessary account
information to determine participation and eligibility. They then choose a Trade Ally from
the approved Contractor list to install eligible measures, and allow CLEAResult access to
verify completed A/C Tune-ups and installed measures. Trade Allies provide a customer
with the discount at the time of the service, and they are reimbursed for these discounts
after they have submitted completed documentation.
OG&E Oklahoma Demand Program Portfolio Evaluation
207
The CEI program track is designed to meet the needs of OG&E’s top customers.
CLEAResult engineers go to the customer’s facilities and conduct an audit focusing on
low cost / no-cost behavioral changes. The audits also bring capital projects to customers’
attentions as needed, which has led some customers to participate in other C&I program
channels. Participants are expected to meet jointly with the cohort of other participants
up to six times per year.
CLEAResult interacted with contractors working with the program in PY17 to provide them
with program tools such as promotional materials and energy savings calculators to
assess and recommend program-incented equipment to customers. CLEAResult offered
these contractors informal program training on the calculators, through meeting them
directly in the field, or in some cases, via webinar. CLEAResult additionally interacted
with Midstream distributors working within the lighting channel in PY17 by sending field
representatives out to distribution sites and storefronts. They provided distributors with
program marketing collateral, and an overview about how CLEAResult would support
them within their program experience.
Participating Customer Delivery Experience
We began understanding the program participant delivery experience by asking program
participants if they worked with an OK contractor to complete their CEEP project. Fifty-
eight percent of overall respondents confirmed working with a contractor or vendor, while
20 percent indicated they relied on their internal staff. Eighteen percent reported they
used a combination of internal resources and outside contractors. All SBDI participants
(100 percent) said they worked with a contractor/vendor, likely due to small businesses
typically having far fewer staff resources than larger commercial customers. CEI
participants relied heavily on internal staff.
OG&E Oklahoma Demand Program Portfolio Evaluation
208
Table 7-30 Worked with contractor / vendor / internal staff to implement project
Category Large C&I SBDI SAGE
HVAC Tune Up CEI Total
Worked with a contractor / vendor
63% 100% 56% 48% 33% 58%
Internal staff at company
11% 0% 11% 32% 67% 19%
Both the contractor and internal staff
23% 0% 33% 12% 0% 18%
Neither 3% 0% 0% 8% 0% 4%
Total (N) N=35 N=5 N=9 N=25 N=3 N=77
Source: Question A4 Note: Totals may not sum to 100 percent due to rounding
We then asked survey respondents why they decided to apply to the program(s) in 2017.
The rebate or reduced cost of equipment due to the rebate was cited most often as the
reason for applying to the program (62 percent). Another 28 percent cited the energy
savings they would get through the program, and 12 percent said they wanted to improve
the performance of their building or equipment. Other reasons included wanting a lower
energy bill (10 percent) and wanting to upgrade equipment (9 percent).
Application paperwork
Most survey respondents (76 percent) could recall the program application or other
paperwork processes. HVAC Tune-up respondents were least likely to recall this (64
percent). We asked respondents who could recall the process who submitted their
program paperwork, and we allowed them to select more than one answer. Among those
who could recall the process, over half of the respondents (56 percent overall) confirmed
that their contractor or vendor submitted program paperwork on their behalf. SBDI and
SAGE respondents were most likely to indicate their contractor or vendor completed their
program paperwork (78 and 65 percent, respectively). Forty-four percent of respondents
also confirmed they played a role in completing the program paperwork, and 27 percent
said an OG&E or CLEAResult representative completed the paperwork.
OG&E Oklahoma Demand Program Portfolio Evaluation
209
Table 7-31 Parties involved in submitting program application or paperwork
Category Large C&I SBDI SAGE
HVAC Tune Up CEI Total
Contractor / Vendor 56% 78% 65% 52% 0% 56%
Respondent 46% 56% 71% 24% 33% 44%
An OG&E or CLEAResult representative
32% 11% 47% 18% 0% 27%
Someone else at my company
20% 0% 18% 24% 100% 21%
Other 2% 0% 0% 3% 0% 2%
Total (N) N=41 N=9 N=17 N=33 N=3 N=103
Source: Question A8_O Note: Totals may not sum to 100 percent as respondents could select more than one answer
Regardless of how the program paperwork got done, OK CEEP program participants
were highly satisfied with the process. We asked these respondents to rate their
satisfaction with the paperwork process using a scale of 0 to 10, where 0 is "very
dissatisfied" and 10 is "very satisfied". Eighty-nine percent of participants rated the
paperwork process with an 8, a 9, or a 10. The mean overall paperwork satisfaction score
across all program channels was 8.9. When looking at respondent data in specific
channels, SAGE (9.7), HVAC Tune-up (8.9), and Large C&I (8.7) participants had the
highest paperwork satisfaction.
7.8.4.5 Measure Installation
We asked the program participants about their experiences with up to two of their installed
program measures (as applicable). We began this section by asking them to identify any
barriers they may have encountered while purchasing or installing program measures
(this excludes audits and tune-ups). Most respondents did not encounter any barriers with
either their first (92 percent) or second (95 percent) program measure. However,
respondents who participated in the SBDI program were slightly more likely to encounter
a barrier (17 percent with MEASURE 1, no barriers with MEASURE 2) than those in the
other program tracks. When SBDI respondents were asked to provide more information
about any barriers they may have faced, respondents contributed their challenges to the
length of time to complete the project and some failed lights.
OG&E Oklahoma Demand Program Portfolio Evaluation
210
Table 7-32 Did you experience barriers purchasing or installing program measures?
Category Large C&I
SBDI SAGE Total
Yes 6% 17% 10% 8%
No 94% 83% 90% 92%
Total (N) N=54 N=12 N=20 N=86
Source: Question M1R1
All respondents (100 percent for MEASURE 1, 100 percent for MEASURE 2) who
participated in the various PY17 CEEP program channels confirmed their measures were
still installed today. All respondents also confirmed the measures were in working order.
7.8.4.6 Program Satisfaction
Program Participant Satisfaction
The evaluation team asked program participants to rate their overall program satisfaction
using a scale of 0 to 10, where 0 is "very dissatisfied" and 10 is "very satisfied".
Participants’ mean overall program satisfaction score averaged across all program tracks
was 8.3, which is lower than the 9.1 satisfaction score given in PY16. SAGE and CEI
participants rated their overall satisfaction the highest with a mean score of 9.3, while
HVAC Tune-up customers rated their satisfaction the lowest (7.5).
The evaluation team also asked participants to rate their satisfaction with the measures
they had installed through the program, using a scale of 0 to 10, where 0 is "very
dissatisfied" and 10 is "very satisfied". Participants were highly satisfied (96 to 100
percent for MEASURE 1 and MEASURE 2 respectively) with the equipment they installed
through CEEP. The mean overall measure satisfaction score (all tracks) across the two
measures we discussed with them was 9.3 and 9.6 (for MEASURE 1 and MEASURE 2
respectively).
To further assess participant satisfaction with the OK CEEP program channels, we asked
respondents to rate their satisfaction with various CEEP program components. We
presented a 0 to 10 scale to respondents to use when scoring the program components,
where 0 is “very dissatisfied” and 10 is “very satisfied”. Table 7-33 displays the mean total
value score of each program component among CEEP respondents overall. Among all
participants, the highest satisfaction ratings were for the incentive amount compared to
the total project cost (8.5), the technical assistance from a contractor or vendor (8.4), and
the energy efficient measures that OG&E provides through the program (8.4).
OG&E Oklahoma Demand Program Portfolio Evaluation
211
Satisfaction with program components varied somewhat by program channel. CEI and
SAGE participants were the most satisfied with the various program components. HVAC
Tune-up customers were least satisfied with the communication from program
representatives (7.2), the technical assistance from a contractor/vendor (7.5), the
materials they received describing the program requirements and benefits (7.6), and the
technical assistance from OG&E or CLEAResult (7.7).
Table 7-33 Satisfaction with various program components?
OK CEEP Program Component
Program Track
Large C&I
SBDI SAGE HVAC Tune
Up CEI Total
The incentive amount compared to your total project cost
Mean 7.9 8.7 8.6 8.9 9.3 8.5
Technical assistance from your contractor or vendor Mean 8.9 8.5 9.4 7.5 9.0 8.4
The energy efficient measures that OG&E provides through the program
Mean 8.5 8.4 9.1 8.0 9.0 8.4
The amount of time it took between applying for the program and the work being completed
Mean 8.1 6.6 9.0 8.1 9.0 8.1
Technical assistance from OG&E or CLEAResult program representatives
Mean 7.9 8.5 9.1 7.7 9.3 8.1
Materials describing the program requirements and benefits
Mean 7.8 8.6 9.1 7.6 9.0 8.0
The amount of time it took to receive the incentive Mean 7.5 7.6 8.7 8.2 8.7 7.9
Communication from program representatives Mean 7.6 7.0 8.9 7.2 9.3 7.6
Another way to measure program satisfaction is to understand whether a program
participant has recommended the program to other OG&E customers. When we asked
CEEP respondents if they had recommended the OK CEEP programs offerings to others,
53 percent of overall participants had done so. SAGE participants were most likely to
have recommended the program to someone (95 percent) and HVAC Tune-up customers
were least likely to have recommended the program (42 percent).
OG&E Oklahoma Demand Program Portfolio Evaluation
212
Among those 47 percent who had not yet recommended the program, we asked them if
they would recommend the program to others if provided the opportunity. Eighty-two
percent of the respondents in the different program channels confirmed they would
recommend the program if the opportunity presented itself. HVAC Tune-up participants
were least likely to say they would recommend the program to others (69 percent).
Table 7-34 highlights what respondents said when asked what they would change about
the CEEP program channels if given the opportunity. Sixty-one percent indicated they
would not change anything. Both SAGE and CEI participants were much more likely to
suggest some type of change (65 and 50 percent respectively).
The second most popular answer to this question was “other”. These responses were
categorized during the analysis phase. Participants most frequently provided “other”
examples relating to improving program communications and having better
advertisement (10 percent). Other popular responses in the “other” category was having
better contractors and having funds available for rebates year-round (4 percent each). Six
percent said they would improve the program application process.
Table 7-34 What would you change about the program?
Large C&I SBDI SAGE
HVAC Tune
Up CEI Total Would not change anything 62% 64% 35% 69% 50% 61%
*Improve program communications/advertisement
6% 18% 15% 11% 0% 10%
Other 8% 9% 0% 9% 50% 8%
Improve the program application / paperwork process
11% 0% 5% 2% 0% 6%
*Better contractors 0% 0% 5% 9% 0% 4%
*Have funds available year-round 2% 9% 20% 0% 0% 4%
*Provide more information about the program
6% 0% 10% 0% 0% 4%
Improve initial processing time 6% 0% 5% 0% 0% 3%
Increase incentive amount 6% 0% 5% 0% 0% 3%
Improve incentive payment speed 4% 0% 0% 0% 0% 1%
Total (N) N=53 N=11 N=20 N=55 N=2 N=141
Source: Questions SAT1C01 SAT1C02 SAT1C03 SAT1C04 SAT1C05 SAT1C06 Note: Totals may not sum to 100 percent as respondents could select more than one answer *Indicates response category computed during analysis
Finally, respondents were asked to assess their overall satisfaction with the programs.
We presented a 0 to 10 scale to respondents to use when scoring the program
components, where 0 is “very dissatisfied” and 10 is “very satisfied”. Respondents rated
their overall satisfaction as an 8.5. SBDI and CEI participants recorded the highest
satisfaction with OG&E (9.0 and 9.3 respectively).
OG&E Oklahoma Demand Program Portfolio Evaluation
213
Trade Ally and Lighting Distributor Satisfaction
Trade allies were asked to rate their satisfaction with the OG&E program channel(s) they
worked with. Over two-thirds (69 percent) of the trade ally ratings given for the Large
C&I, SBDI, and HVAC Tune-up program channels were an 8 through 10 on a scale of 0
to 10, where 0 means “very dissatisfied” and 10 means “extremely satisfied.”
The average rating of CEEP program satisfaction for the nine Large C&I Lighting
businesses was 8.6, and five of the nine Large C&I trade allies gave CEEP a 9 or 10
rating. The lowest rating was a 7 rating by two Large C&I businesses. Those two
businesses commented that turnaround time on approvals took too long. The SBDI
program was rated a 9 or a 10 by four of the five SBDI businesses that were interviewed.
One gave a 7 rating but said the rating would be higher if the payments were made on
a timelier basis.
The HVAC Tune-up Program was rated an 8, 9 or a 10 by four of the eight business.
Three of the HVAC Tune-up businesses rated the program lower with a 4 or a 6. Two of
the three cited problems with approval and payment turnaround. One HVAC Tune-up
business that gave the program a low rating also noted that the program, with some
tweaking, could be a great program if it were not patterned after residential.
One HVAC Tune-up trade ally indicated the program was not financially viable for them
due to pre-cleaning requirements and having to remove the coil for cleaning without
adequate compensation to cover the labor costs. One Large C&I HVAC business also
did HVAC tune-ups and expressed similar financial concerns in not getting involved in
the program.
Trade allies were also asked to rate various aspects of the 2017 CEEP Program using a
scale of 0 to 10 where 0 is “very dissatisfied” and 10 is “very satisfied.” In general, most
trade allies were satisfied with the different program components.
The Level of Communications with CLEAResult Program Staff
Thirteen of the 18 trade allies rated satisfaction with the level of communications as an 8
through 10. There were three businesses who gave ratings of 7, but one said they just
did not have a lot of contact having only done one Large C&I HVAC project. Another
Large C&I lighting trade ally felt communication with CLEAResult was somewhat delayed
while the third who rated it a 7 said they needed more CLEAResult staff so that site visits
for SAGE and SBDI were not so delayed.
One SBDI trade ally rated communications a 4 indicating it took four to five days to get
responses to their emails. One of the eight HVAC Tune-up businesses gave a rating of
6, because they felt the performance had dropped off in the past year.
The Amount of Program Paperwork
OG&E Oklahoma Demand Program Portfolio Evaluation
214
Eleven of the 18 trade allies rated their satisfaction with the amount of paperwork required
for the program as 8 through 10. Of those who rated the paperwork lower (5 through 7),
four were Large C&I trade allies. Two of the HVAC Tune-up trade allies had issues with
data reporting, which is done electronically. One HVAC Tune-up trade ally said it took
much longer to get projects through the system in the second year of the program than in
the first year of the program. The other HVAC Tune-up trade ally commented that data
requirements are high and how the data gets reported in the system is difficult to match
up with invoicing.
The Actual Program Measures and/or Rebated Equipment Offered through the
CEEP Program
Most of the trade allies were happy with the types of measures and equipment offered.
Large C&I Lighting and SBDI trade allies gave ratings of 8 to 10. HVAC Tune-up trade
allies often did not provide ratings although only one suggested offering rebates for 14
SEER units. This HVAC business felt that their customers would be willing to change out
20-year old units, instead of repairing them, if they did not have to go to 16 SEER to get
the rebate.
The Timeliness of Rebate Payments to Customers
Thirteen of the 18 trade allies interviewed gave satisfaction ratings of 8 to 10 on the
timeliness of the rebate payments to customers for at least one of their program channels.
Three trade allies gave lower ratings on the timeliness of rebate payments for Large C&I
at 6 or 7. Two of the five SBDI trade allies gave rebate timeliness a 3 and a 5. One of
these trade allies said it took eight weeks to get rebate checks, and that is a major problem
for them since they are a small business.
Two of the eight HVAC Tune-up trade allies gave 5 and 7 satisfaction ratings for
timeliness of rebate payments. One felt there were more delays in the second year of the
program in getting paid after approvals. The other HVAC Tune-up business said there
was a delay when they were told addresses were incorrect, although they were correct.
The Rebate Amount
Eleven of the 18 trade allies rated their satisfaction with the rebate amount at 8 to 10.
Another trade ally did not give a rating but thought that just getting a rebate is great. Two
HVAC trade allies gave the rebates a lower satisfaction rating of 6 for HVAC Tune-ups.
They explained that for large HVAC systems, the rebate amount did not cover the cost of
the free materials that they were required to provide, and for the greater number of labor
hours needed to do the tune-ups. Another Large C&I HVAC trade ally did not provide a
rating for HVAC Tune-up, but commented that the rebates did not cover their costs of
doing a tune-up on a 200-ton unit.
OG&E Oklahoma Demand Program Portfolio Evaluation
215
The OG&E Energy Efficiency Website
A majority of the trade allies (11 of 18) said they do not use the OG&E Energy Efficiency
Website. Those who do use it, typically gave it a 7 or 8 rating. One HVAC business rated
the website a 2 in terms of satisfaction level, commenting that most of the content was
for lighting businesses.
CEEP Program Training
Most of the trade allies (16 of 18) have attended training sessions that typically include
the annual program kickoff. Some also mentioned webinars, technical and software
training, and one-on-one training with CLEAResult.
Of the 16 trade allies who attended training, 12 ranked their level of satisfaction as 8 to
10. All three trade allies who gave low ratings for their satisfaction with the training were
HVAC trade allies (one gave a 2 rating and two gave a 6 rating). The trade ally who rated
the training a 2 worked on both Large C&I HVAC and HVAC Tune-up projects and
commented that the annual training focused primarily on lighting with very little time on
HVAC. Another HVAC Tune-up trade ally who gave a 6-rating had two training sessions
with program representatives and felt that one was clearly a better trainer than the other.
Lighting Distributor Satisfaction
Lighting distributors rated their overall satisfaction with the program on a 5-point scale—
very satisfied, somewhat satisfied, neither satisfied or dissatisfied, somewhat
dissatisfied, and very dissatisfied. Ten of 12 lighting distributors were very satisfied with
the program overall, while two were somewhat satisfied.
Lighting distributors were also asked to rate aspects of the 2017 Midstream program
using a scale of very satisfied, somewhat satisfied, neither satisfied nor dissatisfied,
somewhat dissatisfied, or very dissatisfied. Program aspects rated by the distributors
included the enrollment process, sales tracking process, incentive processing, sales that
the incentives have generated, program managers and other staff involved with the
program, and their overall level of satisfaction. Generally, distributors were satisfied with
the program.
Enrollment
Nine of 11 distributors were very satisfied with the process of discussing the program with
a program representative and signing their firm up as a program lamp distributor. The
remaining distributors were somewhat satisfied with this process.
Sales tracking
OG&E Oklahoma Demand Program Portfolio Evaluation
216
Seven of 10 distributors who responded to this question were satisfied with the process
used to track the sales of program discounted products by their firm, with the remaining
three being somewhat satisfied.
Incentive processing
Distributor satisfaction with incentive processing varied. Six of 11 distributors were very
satisfied with the incentive processing process, while three were somewhat satisfied.
One distributor said they were neither satisfied nor dissatisfied, and another said they
were very dissatisfied. The distributor who was very dissatisfied said that 2017 was an
invoice nightmare—some invoices were lost and payments were taking 45-60 days.
Another distributor noted that their first payment took 5 months to receive, but after that
payment was fine.
Sales that the program incentives have generated
Ten of 12 distributors were very satisfied with the sales they have experienced because
of the program, one was somewhat satisfied, and one was neither satisfied nor
dissatisfied.
Program managers and other staff involved with the program
Ten of 12 distributors were very satisfied with the program managers and other staff,
while two were somewhat satisfied.
In general, satisfaction with the Midstream Lighting program is high, and distributors feel
the program, incentives, and marketing materials have increased their sales. Ten
distributors said that program staff provided them with training. Seven of these 10 were
satisfied with the training. Those who weren’t completely satisfied mentioned that
sometimes the training seemed disorganized, or that staff could be a little quicker to
respond to requests (e.g., if a zip code is qualified).
Distributors are generally happy with the amount of support the program provides to them.
Five distributors added that they are provided with immediate support whenever they ask.
The only request for additional support was the need for program flyers and adding a list
of authorized distributors on the website.
Finally, 11 of 12 distributors felt the discount offered by OG&E was sufficient to induce
customers to buy efficient products. The 12th distributor felt the discount offered was
sufficient in about one-half of the cases.
7.8.4.7 Participant Characterization
Table 7-35 summarizes basic information collected about the participating respondents
and their facilities. The OK CEEP program channels served OG&E business customers
OG&E Oklahoma Demand Program Portfolio Evaluation
217
within a variety of sectors and facilities, such as retail, schools, industrial & manufacturing
facilities, offices, and warehouse/distribution centers. As expected, the type of customers
served within each program channel varied.
The mean number of employees in these facilities also varied greatly – spanning from 21
within SBDI facilities to 542 in Large C&I facilities.
Two-thirds (66 percent) of overall respondents indicated they owned and occupied the
building they were doing business in. CEI track participants were most likely (100 percent)
to both own and occupy their building, and HVAC Tune-up participants were least likely
(54 percent) to own and occupy their building. We asked those who did not both own and
occupy their building if they at least paid the electric bill at their facility. Ninety-two percent
of overall respondents confirmed they paid the bill if they did not own their building. All
SAGE and SBDI customers answering this question indicated they paid the utility bill at
their facility.
OG&E Oklahoma Demand Program Portfolio Evaluation
218
Table 7-35 CEEP Participant Characteristics
Survey Query
Response Category
Large C&I
SBDI SAGE HVAC Tune
Up CEI Total
Business activity that accounts for most of the floor space covered by your OG&E utility bill
Retail 29% 50% 0% 29% 0% 26%
Industrial / Manufacturing
18% 8% 5% 13% 67% 14%
School K-12 2% 0% 70% 9% 0% 14%
Office 16% 8% 5% 14% 33% 14%
Warehouse or distribution center
13% 17% 0% 4% 0% 8%
*Other service 2% 0% 0% 9% 0% 4%
*Auto repair 2% 8% 0% 5% 0% 3%
Religious worship 2% 0% 0% 5% 0% 3%
Other healthcare 4% 0% 0% 4% 0% 3%
Public assembly 2% 8% 5% 0% 0% 2%
Restaurant 0% 0% 0% 5% 0% 2%
Lodging 4% 0% 5% 0% 0% 2%
Institution/government
2% 0% 5% 2% 0% 2%
*Parking 4% 0% 0% 0% 0% 1%
College / university 0% 0% 5% 2% 0% 1%
Grocery 2% 0% 0% 0% 0% 1%
Total (N) N=55 N=12 N=20 N=56 N=3 N=146
Best description of company's ownership of this facility
Your company owns and occupies this facility
66% 67% 95% 54% 100% 66%
Your company rents this facility from someone else
13% 25% 0% 40% 0% 23%
Your company owns this facility but it is rented to someone else
21% 8% 5% 5% 0% 11%
Total (N) N=53 N=12 N=20 N=57 N=3 N=145
Does the company pay the electric bill at this facility?
Yes 82% 100% 100% 96% N/A 92%
No 18% 0% 0% 4% N/A 8%
Total (N) N=17 N=4 N=1 N=26 N=0 N=44
Source: Questions FIRM1 FIRM2 FIRM3 Note: Totals may not sum to 100 percent due to rounding
OG&E Oklahoma Demand Program Portfolio Evaluation
219
Table 7-36 provides information on how budget and energy efficiency decisions are made
within the organizations of these OK CEEP respondents. Eight of every 10 (82 percent)
CEEP respondents confirmed their company's budget decisions are made locally, 7
percent are made nationally, and 6 percent are made regionally. Two-thirds of
respondents indicated their company requires that an energy efficiency purchase meet a
certain return on investment criteria (24 percent) or simple payback threshold (41 percent)
to be purchased and installed, while 35 percent indicated they had no such requirement.
Table 7-36 How CEEP Participants Make Energy Efficiency Project Decisions
Survey Query Response Category
Large C&I
SBDI SAGE HVAC Tune
Up CEI Total
Are your company's budget decisions made locally, regionally, nationally, worldwide, or something else?
Locally 67% 100% 90% 93% 67% 82%
Nationally 17% 0% 0% 2% 0% 7%
Regionally 7% 0% 5% 6% 0% 6%
Other 7% 0% 5% 0% 33% 4%
Worldwide 2% 0% 0% 0% 0% 1%
Total (N) N=54 N=11 N=20 N=54 N=3 N=142
Does your company require that an energy efficiency investment meet certain return on investment or simple payback thresholds in order to be purchased and installed?
Yes, specific ROI
29% 18% 11% 22% 67% 24%
Yes, simple payback
42% 55% 58% 33% 0% 41%
No 29% 27% 32% 45% 33% 35%
Total (N) N=52 N=11 N=19 N=49 N=3 N=134
Source: Questions FIRM5 FIRM6 Note: Totals may not sum to 100 percent due to rounding
OG&E Oklahoma Demand Program Portfolio Evaluation
220
7.9 Program Recommendations
Based on the findings from the 2017 evaluation of the CEEP program, the evaluation
team has developed the following recommendations:
Improve Quality Control on Project Documentation and Program Database:
Through the evaluation of sampled sites, ADM found several projects with project
documentation that did not match the claimed savings in the program database.
With these projects, it is often unclear how the final claimed program savings
(savings included in the program database) were determined. For determining
GRRs for these projects, the Evaluators referred to both kWh and kW savings
included in the program database and not in the project documentation (savings
calculators, etc.). It is recommended that the process for tracking and reporting ex
ante savings be reviewed to ensure the program database is being populated with
the correct kWh and kW savings for each project.
Provide Calculators for Direct Install Projects: None of the project
documentation requested for SBDI projects included a calculator or sufficient data
to determine how ex ante savings are calculated. Most often, the documentation
provided for these projects included an invoice from the contractor and a statement
of work which include fixture quantities and installation locations. However, there
is no calculator that shows fixture wattages or annual operating hours (or facility
type) used in the calculations. Providing a calculator would allow ADM to identify
differences between ex ante and ex post savings as well as allow a full review of
ex ante calculation approaches. If no calculators are available for SBDI projects,
the program tracking database should be updated to include all inputs required to
allow the Evaluators to recreate ex ante savings. The data fields required would
include quantities of lamps/fixtures installed, quantities removed, baseline and
installed lamp/fixture wattage, facility type, annual hours of use, etc.
Initiate a Pre-Construction Review Process: ADM recommends a pre-
construction review process be designed and implemented for large or custom
projects. The pre-construction review should be designed to allow both
implementer and evaluator access to project documentation and ex ante savings
calculations prior to projects being completed and incentives being paid. The
purpose of the review process would be to identify any potential M&V related
issues, determine data collection requirements, and establish project timelines
prior to funding being reserved for customers. This proposed process can help
minimize uncertainty and risk associated with large or custom projects and may be
especially useful with the planned program changes in PY2017.
OG&E Oklahoma Demand Program Portfolio Evaluation
221
Update Midstream Lighting ex ante measure wattages: ADM recommends that
the average watts per measure category figures be updated annually, using the
previous year’s data. Currently, the average efficient measure wattage is based on
a weighted average using years of sales data prior to the implementation of the
program. The PY2017 ex post analysis utilized program year sales data to
determine ex post measure wattages. It is recommended that the ex ante wattages
be updated annually to more accurately reflect market conditions within the
program territory.
Improve Midstream Lighting tracking database: During PY2017, two separate
datasets had to be reviewed to determine ex post savings for this program channel.
Inconsistencies between the two datasets significantly increased the effort
required to determine ex post savings and limited the level of evaluation that could
be completed. For example, the lack of a common data field between the two data
sets prevents the Evaluators from determining whether deemed parameters are
being used correctly in ex ante savings calculations and also prevents a direct
comparison of savings for each lamp type. It is recommended that the process for
updating and managing these two datasets be reviewed. If it is deemed necessary
to continue using two datasets, it is recommended that improvements be made to
ensure lamp type designations, invoice number, PRJ number, ex ante savings,
etc. are included in both datasets.
Continue to Track Midstream Lighting End Customer: It is recommended that
the implementation contractor continue to track the end customer for all Midstream
lighting projects. Tracking the end customer allows for a better evaluation of market
conditions and allows for a more accurate determination of HOUs and CFs for this
program channel. It is recommended that this information continue to be collected
and that deemed HOUs and CFs be updated after two to three years of data is
collected.
Based on the findings from the process evaluation, the Evaluators pose the following
recommendations for program design and implementation.
Ensure that vendors and contractors are educated on the program details and expected savings since this is the most common source of awareness among participating customers. Update the website to contain adequate information on program participation details and potential savings to provide additional program support to customers who are not dependent on contractors. Clear themes emerged from the customer survey about the need for more information – especially about more information on the project details and actual project savings. As one respondent noted: “There was almost no information on the website, so I had to be routed through a third-party vendor
OG&E Oklahoma Demand Program Portfolio Evaluation
222
(CLEAResult) that was managing their program. It would be nice if it just laid out how the program worked and the forms needed.”
Improve turnaround time for communications with trade allies. Some of the
trade allies and lighting distributors were unhappy with the level and/or the timing
of communication. Contractors and distributors are key actors in selling the
program and ensuring participation runs smoothly. Having to wait for
communications on a specific customer’s eligibility or project approvals is
counterproductive.
Program staff should identify ways to maintain or increase program
satisfaction ratings. Participants’ overall program satisfaction score across all
program tracks was 8.3, which is lower than the 9.1 satisfaction score given in
PY16. Satisfaction with the length of time it took to receive the rebate and program
communications received the lowest satisfaction scores and should be an area to
focus on improving. HVAC Tune-up customers were the least satisfied with the
program (rating of 7.5) and gave the lowest satisfaction to program
communications.
Improving communication and increasing program satisfaction will increase
the likelihood that participants recommend the program to others. SAGE
participants were most likely to have recommended the program to someone (95
percent) and HVAC Tune-up customers were least likely to have recommended
the program (42 percent). Among those who had not yet recommended the
program, we asked them if they would recommend the program to others if
provided the opportunity. Eighty-two percent of the respondents in the different
program channels confirmed they would recommend the program if the opportunity
presented itself. HVAC Tune-up participants were least likely to say they would
recommend the program to others (69 percent).
Consider improving HVAC-specific training. Three of the trade allies who gave
low ratings for their satisfaction with the training were HVAC trade allies (one gave
a 2 rating and two gave a 6 rating). The trade ally who rated the training a 2 worked
on both Large C&I HVAC and HVAC Tune-up projects and commented that the
annual training focused primarily on lighting with very little time on HVAC. Another
HVAC Tune-up trade ally who gave a 6-rating had two training sessions with
program representatives and felt that one was clearly a better trainer than the
other.
OG&E Oklahoma Demand Program Portfolio Evaluation
223
Appendix A. Portfolio Cost-effectiveness
A.1 Overview
ADM estimated the cost-effectiveness for the overall Demand Program portfolio and
programs, based on 2017 costs and savings estimates provided by OG&E and their third-
party implementers. This appendix provides the cost-effectiveness results, as well as a
brief overview of the approach taken by the Evaluators.
The portfolio, and programs, pass all the cost-effectiveness tests except the RIM test.
The table below presents the cost effectiveness results for the PY2017 portfolio.
Table A-1 Cost Effectiveness Results
Program TRC UCT RIM PCT SCT
HEEP 3.37 3.34 0.63 8.65 4.95
PE-NHC 2.48 4.04 0.61 2.67 3.48
WRAP 2.83 2.83 0.83 3.77 4.30
CEEP 1.58 3.42 0.69 2.62 2.21
IVVC 0.52 0.52 0.47 1.11 0.73
Energy Education Res 0.00 0.00 0.00 0.00 0.00
Energy Education C&I 0.00 0.00 0.00 0.00 0.00
Planning 0.00 0.00 0.00 0.00 0.00
Regulatory 0.00 0.00 0.00 0.00 0.00
R&D 0.00 0.00 0.00 0.00 0.00
Total 1.65 2.32 0.65 2.85 2.37
A.2 Approach
The California Standard Practice Model was used as a guideline for the calculations,
along with guidance from the Arkansas TRM version 6.0. The cost effectiveness analysis
methods which were used in this analysis are among the set of standard methods used
in this industry and include the Utility Cost Test (UCT), Total Resource Cost Test (TRC),
Ratepayer Impact Measure Test (RIM), and Participant Cost Test (PCT). All tests weigh
monetized benefits against costs. These monetized amounts are presented as Net
Present Value (NPV) evaluated over the lifespan of the measure. The benefits and costs
differ for each test based on the perspective of the test. The definitions below are taken
from the California Standard Practice Manual.
The Total Resource Cost Test (TRC) measures the net costs of a demand-side
management program as a resource option based on the total costs of the
program, including both the participants' and the utility's costs.
The Utility Cost Test (UCT) measures the net costs of a demand-side
management program as a resource option based on the costs incurred by the
OG&E Oklahoma Demand Program Portfolio Evaluation
224
program administrator (including incentive costs) and excluding any net costs
incurred by the participant. The benefits are similar to the TRC benefits. Costs are
defined more narrowly.
The Participants Cost Test (PCT) is the measure of the quantifiable benefits and
costs to the customer due to participation in a program. Since many customers do
not base their decision to participate in a program entirely on quantifiable variables,
this test cannot be a complete measure of the benefits and costs of a program to
a customer.
The Ratepayer Impact Measure Test (RIM) test measures what happens to
customer bills or rates due to changes in utility revenues and operating costs
caused by the program. Rates will go down if the change in revenues from the
program is greater than the change in utility costs. Conversely, rates or bills will go
up if revenues collected after program implementation is less than the total costs
incurred by the utility in implementing the program. This test indicates the direction
and magnitude of the expected change in customer bills or rate levels.
A common misperception is that there is a single best perspective for evaluation of
cost-effectiveness. Each test is useful and accurate, but the results of each test are
intended to answer a different set of questions. The questions to be addressed by