LIHEAP PERFORMANCE MEASURES€¦ · received its first Federal Program Assessment Rating Tool (PART) assessment and the score of “results not demonstrated.” Current LIHEAP reports

LIHEAP PERFORMANCE MEASURES Grantee Webinar

September 18, 2013

Webinar Objectives Grantees should leave with a better understanding of:

• What are OCS expectations and commitments regarding implementation of LIHEAP Performance Measures?

• Why are new Reporting Requirements and Performance Measures being proposed?

• What are the benefits of Performance Measurement to the LIHEAP program?

• What are the Reporting Requirements and the best practices for fulfilling those requirements?

Lauren Christopher-OCS

• Timeline for Implementation

• Training and Technical Assistance Resources

• What are “Developmental” Performance Measures?

• OMB Clearance Process—What Happens Next?

LIHEAP Performance Measure Implementation

• Timeline:

FFY 2014: OMB Review/Approval , Reporting Forms, Measures, T/TA

FFY 2015: Data Collection by Grantees

FFY 2016: Data Reporting by Grantees

• Training and Technical Assistance Resources: OCS is committed to helping grantees with additional data collection and reporting associated with LIHEAP Performance Measurement.

• “Developmental” Performance Measures

• Grantee Capacity Questionnaire

OMB Clearance Process

• 60 Day Notice and Comment Period: June 7 – August 7

• OCS is currently reviewing public comment and putting

together responses (themes to be addressed later in presentation).

• Next Steps: Revised OMB Package, including 30 Day Notice

and Public Comment Period

Taura Edwards, PMIWG

• Need for LIHEAP Performance Measurement—In 2003, LIHEAP received its first Federal Program Assessment Rating Tool (PART) assessment and the score of “results not demonstrated.”

Current LIHEAP reports reflect who grantees serve and how funding is spent, but do not directly demonstrate the impact of LIHEAP on individuals and households.

• Performance Measures Work Group—In 2008, a group of state coordinators began working on developing LIHEAP Performance measures. In 2009, they recommended a set of “tiered” performance measures to HHS.

PMIWG Process

• Logic Model Review – The PMIWG (with input from NEADA members) examined each type of LIHEAP benefit and documented how each benefit addresses the needs of clients.

• Performance Measures Development– For each type of benefit, the PMIWG identified ways to measure program inputs, outputs, outcomes, and impacts.

• Implementation – The PMIWG decided that Performance Measurement should involve both individual grantees and the overall program.

• Grantees – Considered what would be required for a grantee to collect the required information and report on program performance.

• Program – Considered how all of the information from grantees can be combined into a single performance measurement framework.

PMIWG Recommendations

• Tiered System – For each type of LIHEAP benefit, adopt a tiered performance measurement system.

• Tier I – Inputs and Outputs

• Tier II – Short-Term Outcomes

• Tier III – Long-Term Outcomes

• Tier IV – Program Impacts

• Grantee Role– Establish a common set of data elements and performance measures. Over time, the data collection process will move from a focus on Outputs (Tier I) to Outcomes (Tiers II and III).

• Federal Role –OCS should conduct additional research to develop the Program Impact measures. (Tier IV)

PMIWG Recommendations

• OCS – Should mandate data collection of key program performance data so that there is common data across grantees.

• Proposed Measures– Should focus on energy burden reduction and prevention of loss of home energy service as the two most important outcomes of the program.

• Energy Burden – Measure energy burden reduction.

• Restoration of Service – Document the number of homes where service is restored through fuel assistance or equipment.

• Prevention of Service Loss – Encourage programs and clients to move from service restoration to prevention of loss of service.

• Technical Support – OCS should furnish technical support to increase the capacity of grantees to collect and report data.

Performance Measure Benefits

• Increase transparency and accountability

• Communicate success (“Telling our Story”)

• Document how limited resources are targeted

• Support grantee Performance Measurement

• Enhance peer exchange by using common data elements

• Lead to better policy at grantee and Federal level

David Carroll, APPRISE

• Review of Original Measures and Form

• Public Comment Themes

• Rethinking Approach

• Benefit of, data required, and strategies for implementing:

a. Home Energy Burden Measure

b. Restoration of Home Energy Measure

c. Prevention of Home Energy Loss Measure

Original Measures

• Burden - The average reduction in home energy burden for households receiving LIHEAP fuel assistance

• Required Data - Energy Expenditures and Consumption, Supplemental Heating Fuel and Air Conditioning Equipment

• Service Restoration - The percent of unduplicated households

where benefits restored home energy.

• Required Data – Utility Service Terminations, Ran Out of Delivered Fuels, Repair or Replace Equipment

• Service Loss Prevention - The percent of unduplicated households

where benefits prevented loss of home energy.

• Required Data – Utility Overdue Notice, Will Run Out of Delivered Fuels, Repair or Replace Equipment Prior to Loss of Service

Public Comment Themes

• Relevance and Usefulness of Data: Questions about why the measures are relevant to assessment of program outcomes. Assertions that the program has operated effectively without these data.

• Estimated Burden: Concerns for burden on sub-grantees, grantees, small energy vendors, and large energy vendors. Specific concern that resources devoted to data collection and reporting would detract from level of benefits and timely service.

• Data Quality: Questions about whether good quality data could be collected for a mobile population that uses multiple energy sources and vendors.

Relevance and Usefulness of Data

• Relevance and Usefulness of Data: Program needs these data to improve performance.

• Energy Burden – OCS analysis shows that high burden households

have greatest energy needs, but that program does not allocate higher benefits to high burden households.

• Service Restoration – One key outcome of the program is the restoration of service. But, programs cannot report comprehensive information on restoration consistently across grantees.

• Service Loss Prevention – Grantees should get credit for moving clients from restoration after loss of service to prevention of loss of service.

Rethinking Approach • Performance Measures – Are there performance measures that better

represent the concepts that the PMIWG was concerned about?

• Benefit Targeting Index • Burden Reduction Targeting Index

• Sampling – Does OCS need to specify sampling procedures to reduce

the burden associated with data collection?

• Delivered Fuel Vendors • Coops and Municipals

• Optional Data – Can OCS make certain data optional so that we can

learn from grantees have that have the capacity to collect the data, but give the other grantees time to develop that capacity?

• Consumption Data • Supplemental Heating Fuel Data • Air Conditioning Equipment Data

Home Energy Burden • Proposed Measures

• Benefit Targeting Index – Do you give the highest benefits to the high burden households?

• Burden Reduction Targeting Index – Are the benefits adequate to deliver the same burden reduction to high burden households as to low and moderate burden households.

• Purpose of Measures

• OCS research has shown that high burden households have a higher rate of Energy Insecurity including: service disruptions, other financial problems, and health and safety problems. Targeting benefits to these households is expected to increase the impact of LIHEAP on overall client health and safety.

• Required Data – For All and by Main Heating Fuel / For All and High Burden

• Average Income

• Average Benefit

• Main Heating Fuel Bill

• Electricity Bill

• Indexes – Automatically Calculated on Form

• How are states already collecting this information?

• Number of States – About one-third (17) of grantees are currently collecting these data.

• Time of Application – Some states collect at time of application to use in benefit determination

• End of Year – Some states collect at end of year to assess program performance.

Restoration of Home Energy Services

• Proposed Measure

• The unduplicated count of households where LIHEAP restored home energy services to the client.

• Purpose of Measure

• One core purpose of LIHEAP is to ensure that low-income households have access to necessary home energy services. By restoring services to clients who do not currently have access to energy service, the program is eliminating a significant risk to the health and safety of clients.

• Required Data

• Number of clients where utility service was restored

• Number of clients where a fuel delivery was made to a home that was out of fuel

• Number of clients where broken heating or cooling equipment was fixed

• How are states already collecting this information?

• Number of States - It is assumed that most states collect these data for purposes of assessing the risk to clients at the time of application.

• Reporting of Data – It is expected that some states do not record these data in their databases, but merely use the data to establish the priority of service for a client.

• Consistency of Data – It is expected that states do not have consistent definitions for the required data.

Prevention of Loss of Home Energy Services

• Proposed Measure

• The unduplicated count of households where LIHEAP prevented the loss of home energy services.

• Purpose of Measure

• By preventing the loss of service to at risk clients, the program can eliminate the costs of service restoration (e.g., reconnection charges) and can minimize client health and safety risks.

• Required Data

• Number of clients where utility service termination was prevented

• Number of clients where a fuel delivery prevent a loss of service

• Number of clients where heating or cooling equipment was repaired or replaced prior to failure

• How are states already collecting this information?

• Number of States - It is assumed that few states currently collect these data in the required format.

• Reporting of Data – It is expected that most states do not record these data in their databases, but may use the data to establish the priority of service for a client.

• Consistency of Data – It is expected that states do not have consistent definitions for the required data.

Strategies to Address Burden

• Estimated Burden: Burden can be managed as demonstrated by grantees already collecting data. [Note: Focus on Burden]

• End of Year Data Collection – One data request to vendors for all clients minimizes grantee and vendor burden.

• Electronic Data Transfer – Automated processing of information request can minimize burden to vendors with those capabilities.

• Sampling – For vendors without data processing capabilities, selection of a sample of vendors will limit grantee burden and selection of a sample of vendor clients will limit vendor burden.

• On-Line Reporting – Best practices have shown that on-line reporting systems reduce vendor burden and minimize grantee burden.

• Sub-Grantees - In most states, there is no sub-grantee burden. In states with sub-grantee managed programs, reports to grantee will be needed.

Strategies to Address Data Quality

• Quality of Data: Good quality data can be developed for most LIHEAP recipients. Not having data for some clients is not a reason to not collect data. [Focus on Energy Burden]

• Standardized Procedures – OCS will work with grantees to develop standardized procedures that ensure the consistency of data received from vendors.

• Mobility – About 5 percent of low-income owners and 25 percent of low-income renters move each year. Vendors will only have partial year data for those clients. However, part-year data can be annualized through standardized procedures.

• Multiple Energy Vendors – There are a number of situations where clients may use multiple vendors each year for both utility fuels and delivered fuels. These generally affect a small proportion of clients and, over time, procedures can be developed to improve the quality of data for these clients.

• Multiple Energy Sources – Some clients use multiple sources of energy for home heating. For example, a household may use a gas furnace, electric space heaters in individual rooms, and a wood stove in the family room. That is why the data reporting form asks for information on how many clients use supplemental sources of heating.

Conclusion • Why Needed? – The Administration and Congress expects the

program to be able to furnished detailed information on who gets services and what impact the services have on the lives of clients.

• Why Now? – Ten years ago only 2 states were collecting energy expenditure data. Today 17 states are collecting energy expenditure data. The enhancement of information systems is facilitating the collection and analysis of these data.

• With What Support? – OCS has made a commitment to furnish technical assistance and peer exchange to help every grantee increase their capacity to collect and report these data.

Question and Answer Session

• To ask a question, webinar participants should press 01 on their keypad. When it is your turn, your line will become unmuted and the system will prompt you to speak. Questions will be addressed in the order they are received.

• If your question is answered, or you wish to withdraw your name from the

queue, please press 02.

**

Due to time constraint, some questions may not be addressed. For additional

information after the webinar, please contact:

Yuliya Rzad, Program Analyst Office of Community Services

202.401.9289 [email protected]