Energy Savings Assistance (ESA) Program Energy Education Research Report Public Workshop October 17, 2013.

Energy Savings Assistance (ESA) ProgramEnergy Education Research Report

Public Workshop

October 17, 2013

Agenda

2

Time Topic Lead

11:00 am Welcome / Introductions / Purpose Carol Edwards

11:15 am Objectives and Research Methods Steve Westberg

11:45 am Key Findings Steve Westberg

12:30 pm Lunch Break All

1:30 pm Recommendations Steve Westberg

2:15 pm Summary Wrap-Up / Next stepsSteve WestbergValerie Richardson

3:00 pm Adjourn All

Key Project Team Members

3

Consultant Team Member

Responsibilities Company

Steve WestbergProject Manager, HINER Lead (Contractor Research, Customer Research)

HINER

Valerie RichardsonAssistant Project Manager, KEMA Lead(Materials and Literature Review)

KEMA

Stephanie Yang Materials and Literature Review KEMA

Paul CaraccioloField Interviews (Customer In-Home Interviews)

HINER

Luke ThelenSurvey Programming and Quantitative Analysis (Contractor Internet Survey, Customer Telephone Survey)

HINER

Study Team MemberCarol Edwards Project Administrator / Oversight for SCE SCE

Mary O’Drain Project Oversight for PG&E PG&E

Brenda Gettig Project Oversight for SCG & SDG&E Sempra

Syreeta Gibbs Project Oversight for Energy Division CPUC

Project Objectives

Key issues addressed by the research:

• Identify best practices and potential Improvements related to HOW Energy Education is delivered (e.g., format, time, etc.)

• Identify best practices and potential Improvements related to WHAT materials and content are provided (e.g., relevance, value, gaps)

Issues Not Addressed:

• Energy savings estimation

4

5

Project Components: Data Sources

Four data components for the study:

Phase Component Purpose

1 Materials and Literature Review

(1) Guidelines, training, and materials provided to contractors (and customers). (2) IOU programs and technologies, and Out-Of-Area materials or practices that could be leveraged/adopted.

2 Contractor Research

(1) Understand current practices, knowledge, etc. of educators. (2) Solicit ideas for improvement from those closest to the education.

3aCustomer In-Home Interviews and Focus Groups

Understand and explore: (1) range of educational experiences, (2) retention of content, (3) adoption of energy efficient behaviors, and (4) unmet needs.

3b Customer Telephone Survey

(1) Measure the prevalence of experiences, knowledge, behaviors and needs across the population of ESA participants.

6

Meeting Objectives

Objective 1: HOW energy education is delivered

Component Research Questions

Materials and Literature Review

What training has been provided to contractors? How do IOU’s assess or monitor performance? What other methods of education delivery could be employed?

Contractor Research

How is education delivered? What differences in delivery exist? How do customers respond? What can interfere with effective delivery?

Customer In-Home Interviews and Focus Groups

How has education been provided? What methods stand out? What prompts you to put learning into practice? What new methods of delivery have potential? What is missing or lacking?

Customer Telephone Survey

How many or what percent of customers … based on qualitative issues

7

Meeting Objectives

Objective 2: WHAT materials are provided

Component Research Questions

Materials and Literature Review

What materials are provided to contractors/customers? What is the specific content? What new content could be added?

Contractor Research

What resonates with customers? What do customers ignore? What do customers ask about that is not included?

Customer In-Home Interviews and Focus Groups

What content is most useful? What content does not seem to apply? What issues within the home prevent adoption? What potential new content appeals? What is missing?

Customer Telephone Survey

How many or what percent of customers … based on qualitative issues

8

Phase 1: Materials and Literature Review

Purpose: • Document what contractors currently provide regarding Energy

Education• Provide a resource for the project team to identify potential new

content, delivery methods, or additional resources

Method: • Review included:

• Program documentation for training and delivery practices • Educational materials provided customers• Contractor implementation and supervisory practices• Third party studies• Interviews with IOU managers• Ride-alongs with experienced assessors

9

Phase 2: Contractor Research

Purpose: • Document, along with the Materials and Literature Review:

• What contractors currently provide regarding energy education • The range of differences between contractors• Barriers to effective education • Best practices• Ideas for improvements from the contractors’ perspective

Approach Overview: • What training or education do you provide in homes• What are the most and least effective topics and materials• What barriers or problems that interfere with the training in the home

do you encounter (e.g., language, householder availability and interest, topics relevant to the householder)

• What type of training did you receive prior to making field visits• What was missing or lacking in your training or materials• Etc.

10

Phase 2: Contractor Interviews

Sampling:

• There are approximately 400 assessors statewide in about 80 contracting agencies

Contractor Data Source

Number of Completed Interviews

Total PG&ESCE-Only

SCG-Only

SCE & SCG SDG&E

In-Depth Interviews 12 3 2 2 2 3

Internet Survey 171 70 18 34 46 7

11

Phase 3: Customer Research Qualitative

In-Home Interviews Purpose: • Determine what motivates customers to participate• Determine what customers have retained and put into action• Compare contractor-provided information to what customers said

about the education they received to identify retention gaps• Determine what aspects of delivery are most effective• Identify gaps between what customers need and what they received• Identify additional opportunities for new topics, delivery methods, and

resources

Focus Groups Purpose: • Discuss motivations, needs, and unmet needs• Elicit reactions to current program content and materials, and to

potential new content and materials

12

Phase 3: Customer Research Qualitative

Sampling: • In-Home interviews conducted in 10 geographically dispersed clusters

(representing different climate zones) among recent (<3 months) ESA participants

• Focus groups locations also selected for geographic differences (Fresno, Orange, San Diego) among recent participants and CARE non-ESA participants

Customer Data Source Language

Number of Completed Interviews & Focus Groups

Total PG&ESCE & SCG SDG&E

In-Home Interviews

English 24 12 9 3

Spanish 6 - 3 3

Focus GroupsEnglish 5 1 2 2

Spanish 1 1 - -

13

Phase 3: Customer Research Quantitative

Purpose:• “Validate” and quantify the qualitative findings with a telephone

survey among a representative sample of recent ESA participants

Survey Topics: • Screening for person most involved/present during assessment

visit• Measurement of attitudes, motivations, and barriers to reducing

energy use• Energy education: how much time spent, information &

materials received, etc. • New behaviors as a result of education & beliefs about success• Evaluation of assessor with energy education• Interest in new content, programs, tools, etc.• Demographics

14

Phase 3: Customer Research Quantitative

Sampling: • 505 telephone interviews among recent (<3 months) ESA participants:

• Margin of error = 4.4% at 95% confidence• Sample provided by IOUs from participant databases

• “Total” results (representing statewide ESA participants) were weighted to match the proportion of treated homes in each IOU service territory from 2012

Customer Data Source Language

Number of Completed Interviews

Total PG&E SCE SCG SDG&E

Telephone Interviews (Unweighted)

English 411 162 83 84 82Spanish 94 23 24 23 24

Total 505 185 107 107 106Telephone Interviews (Weighted)

  505 197 63 205 39

15

Findings: Delivery of Energy Education

Assessor recruitment, selection, and retention processes have been effective

• Key characteristics are out-going personality and desire to help others

• Most are well educated (college degrees), have very high job satisfaction, and feel good about their work

• Tenure is very stable at 3.8 years (average)

• Compensation (paid for completed, treated homes) and canvassing rewards them to work well independently

Assessors are a valuable resource for the program – they can take on whatever might be asked of them

16

Findings: Delivery of Energy Education

Assessor performance delivering energy education has been excellent overall but a small minority of customer experiences and/or assessors need improvement

• Customer evaluations are very high (% 8-10 ratings)• 88% said assessors are “courteous and polite”

• 81% for “ability to clearly communicate”

• 80% for “interest and ability to answer questions”

• 78% for “knowledge of material and subject matter”

• 78% for “sensitivity or awareness of specific needs”

• Could benefit from greater knowledge of energy efficient practices (tips) and awareness of household differences (e.g., children, seniors)

• About one in ten customers not satisfied with the energy education received (or not received)

17

Findings: Delivery of Energy Education

Assessor training prepares assessors well but there are areas for improvement so all assessors provide similarly high quality energy education

• Initial training assessors recalled receiving: • 83% classroom training (by IOU, except SDG&E)

• 83% training on materials (by IOU, except SDG&E)

• 63% role playing (by IOU)

• 60% ride-along training

• Differences between IOUs (time spent, materials used, etc.)

• Additional (refresher) training:• 60% said they received it (34% in-office training on new materials,

20% refresher class, 13% additional field training)

• Assessors stated they want more training – more information (e.g., tips), more customer interaction or role playing

18

Findings: Delivery of Energy Education

Language barrier problems are minimal

• Over two-thirds of assessors are bilingual or multilingual• 63% speak Spanish, 13% speak another language (Chinese, Korean,

Tagalog, Punjabi, Vietnamese, etc.)

• English-only assessors estimated 3% of visits were “language barrier,” multi-lingual estimated 1%

• Contractors schedule visits to match assessor language skills with customer language preference

• Just over half of all visits are conducted in English, about one-third are conducted in Spanish, and <2% are conducted in another language

• Communication problems can also exist between assessors (or other program personnel) who are not fluent in English and English-speaking customers

19

Findings: Delivery of Energy Education

In-Home delivery methods are on target but not providing education until after qualification on measures has been determined is likely to reduce energy education effectiveness

• Customers reported an average of about 25 minutes of education, assessors believe this is the right amount

• Energy education most frequently provided during the walkthrough – most effective time to “show and tell,” not lecture, and customer is there anyway

• Education also provided throughout the visit - initially as “icebreaker,” and after walkthrough with guidebook for review

Best approach – educate throughout the visit

• Guidebook and materials typically provided after walkthrough (77% of the time), opportunity to review

• Some write in the book (helps customer remember)

20

Findings: Delivery of Energy Education

(continued) Practice of not providing education until after walkthrough

(when qualification may be determined) not ideal from an educational effectiveness perspective• But, doing so results in non-compensated education for contractors who

provide this to households that do not qualify for measures (SCE is most affected because of more “electric only” assessment visits)

• 41% said they at least “sometimes” provide the guidebook to non-qualifying households

• Assessors want to avoid customer disappointment

• Customers want and appreciate the materials

• Time and effort already invested by assessor and customer

• Customer is also a very ready and willing student

21

Findings: Delivery of Energy Education

Customer retention of information is a problem for some

• One in three customers said they could not recall any specific energy saving information from the visit

• Half recalled receiving information about other programs/ resources, and going to their utility’s website for more info

• One in three recalled information about how to read their bill

• Some had trouble remembering where they put their guidebook

• Seniors in particular had more trouble

• Customer and assessor interest in reminder aids was relatively high

• Key information: how much it costs to run appliances or electronics for a period of time, specific actions that can be taken to reduce energy use

22

Findings: Delivery of Energy Education

Households with multiple adults and/or children in the home face challenges with gaining cooperation, so more could be done

• About 80% of households in program have more than one person, half of these have children

• None of the content focuses on multiple person households

• Assessors currently do try to engage others but are only partially successful (other members are not home during the appointment, materials are not geared to children or roommate situations)

• Customers try to pass on information as well but limited by the materials and ideas provided to them

23

Findings: Content and Materials

The guidebooks are key tools but all have room for improvement

• Assessors rely on its content to remind them what to include in the energy education (SDG&E assessors are trained exclusively from it)

• 86% of customers said they got one, and 94% who got one said they reviewed it. 78% said they reviewed it later on their own or with someone else in the household.

• Each IOU has its own guidebook (SCE and SCG share), with different look & feel, content, organization, use of graphics, etc.

• Guidebooks do not include all content from P&P manual• All topics except GHG are relevant to most customers

• GHG is not a motivation to most, but energy reduction no matter what the motivation provides GHG reduction benefits

24

Findings: Content and Materials

Additional materials could be developed that (1) provide appliance cost of use information (similar to the “energy wheel”), (2) help overcome problems with forgetting, and (3) help multi-person households

• Customers nearly unanimous in desire for ‘cost of use’ information

• PG&E’s “energy wheel” provides this as do two of the guidebooks, but this information should be more “front and center” in the guidebooks and as a separate tool

• Materials (and education implementation processes) currently provide few ‘reminders’

• Guidebooks get reviewed with the assessor after the walkthrough, and after the visit, but otherwise are stored out of site

• Customers want follow-up, and reacted favorably to leave-behind ideas that would be simple, visible reminders

25

Findings: Content and Materials

Educational content is relatively comprehensive but information that customers consider new is more memorable

• Energy saving information and tips provided in energy education cover nearly all main energy using appliances and other items in the home

• About one-third of “tips” were considered new• New is more memorable (e.g., unplug when not in use, keep

refrigerator full)

• But what’s new to one customer may be known to another, so need lots of tips

• Some “known” tips are also beneficial for the purposes of reminding customers about things they can do

As noted previously, customers appreciate “reminders”

26

Recommendations

1. Standardize more of the training across IOUs Consistent & rigorous training for new assessors overseen by the

IOU Formal classroom training to inform as many energy saving practices

(“tips”) as are known across the IOUs (customers tend to prefer tips focused on major items – AC, laundry, lighting, etc.)

Classroom role playing to cover a wide variety of situations (household size, age of household members, etc.)

Field training including observation and then performance under supervision

Write in the guidebooks

Consistent & rigorous refresher training also provided by the IOU Establish some general annual standards

Provide a large number of energy saving practices

Consistent and rigorous standards for field training provided by contracting organizations

27

Recommendations

2. Provide Follow Up Problems of (1) customers feeling visit was not complete,

and (2) customers tend to forget Consider more consistent follow up surveys with

participants (some currently receive this but not all) What did you learn? What do you plan to put into practice? What, if anything, was not completed?

Consider periodic communications Quarterly “newsletter” with new and reminder tips, seasonal

information (prepare for summer/winter), new programs, MyAccount/MyEnergy, etc. Mail or email

Text or twitter or voice message “opt-in” messaging (enroll during visit)

28

Recommendations

3. Consider Modified and Additional Education Materials Revise the resource guidebooks to be more user friendly

and inviting Hourly energy costs for appliances and other common and

uncommon items Lots of well organized and user friendly tips CFL and LED “buying guide” pages Charts and graphs Prioritize with ‘action oriented” information in front, reference

information (phone numbers, programs, safety) in back Color to make key points stand out Title that reflects saving energy

29

Recommendations

3. Consider Modified and Additional Education Materials Provide reminder tools

Magnets (for major appliances) with reminder tips “Energy Wheel” or something similar “Checklist” of energy saving tips Calendar with season-oriented “tip of the month”

30

Recommendations

4. Consider More Customized Information for Customers Top 5 list of tips “customized” for the household Age-appropriate materials for children Teach the “Energy Bank” Information “modules” for households with other adults (senior

parents, roommates) and/or children

31

Recommendations

5. Provide Energy Education Throughout the Visit Allows for more interaction, show-and-tell Teach this in training

6. Revise the Protocol of Not Providing Education Until After Qualification on Measures Supports #5 above Provide the resource guidebook to households not

qualified on measures (3MM rule)

32

Next Steps and Key Dates

Comments on Draft Report Due October 23Final Report Posted October 31, 2013

http://www/energydataweb.com/cuc/search.aspx