Healthy Futures Performance Measures Session. Session Overview Combination of presentation and interactive components Time at the end of the session for.

Healthy FuturesPerformance Measures Session

Session Overview

• Combination of presentation and interactive components

• Time at the end of the session for Q&A• Use notecards on table to write questions

Objectives

• Strengthen understanding of the three elements integral to a strong performance measurement plan:– Data Collection

– Data Management

– Data Reporting

• Experience planning for the data collection process

Agenda

Activity Duration

Review Data Quality and Data Collection 30 minutes

Activity 45 minutes

Review Data Management and Reporting 10 minutes

Additional resources and Q&A 5 minutes

How does my program demonstrate impact?

What evidence do I need to collect to show I achieved my selected outcome?

Community Need

Demonstrated by facts presented in application

Service Activity

What service, how often and for how long the volunteers provide

Anticipated Output

Number of individuals who receive service

Anticipated Outcome

Number of individuals who experience change

6

Amount of service provided (i.e. people served).

Reflect the changes or benefits that occur.

How does my program demonstrate impact?

Outputs

Outcomes

Work Plan Refresher

• The Service Activity Description should explain what the Senior Corps volunteers are doing in a way that shows how they will achieve an output and outcome.

• It should include:• who the beneficiaries are;• what the volunteers will be doing with the beneficiaries; • how often volunteers will provide the service;• for how long, and; • where the service will take place.

Work Plan Refresher

• 50 Senior volunteers will provide respite services to 150 primary family caregivers of older, homebound adults. They will serve a minimum of 8 hours per week for at least 12 weeks. They will serve each client a minimum of 2 hours per week for 12 weeks. Service activities may include companionship/friendly visiting and reminiscing, light meal preparation, letter writing/correspondence, going for short walks, and reading stories together.

Work Plan Refresher

• The Instrument Description should:• give the name of the instrument;• briefly describe who will collect the data; • from whom the data will be collected, and;• when it will be collected.

Work Plan Refresher

• Station supervisors will conduct the Respite Care Survey with caregivers every 6 months at the middle and end of the program year.

Five Data Quality Elements

A Standard Process to Consider Data

Validity Completeness Consistency

Accuracy Verifiability

Validity: Data Relevance• Ask yourself:

– Is the data relevant?

– Are your reported items consistent with the approved goals of the current grant?

– Are you measuring what you intended to measure?

ExampleH14 Outcome: Number of caregivers of homebound OR older adults and individuals with disabilities who reported having increased social ties/perceived social support.

Data Instrument: Survey

Questions in Survey:• Did you have a long wait for a

volunteer? • Are you happy with the service?

Data Completeness

• Ask yourself: – Does the reported data

contain enough information to represent performance measure activities?

– Did the reported data come from all stations and/or a random sampling of volunteers/ service recipients?

ExampleH14 Outcome: Number of caregivers of homebound OR older adults and individuals with disabilities who reported having increased social ties/perceived social support.

Data Instrument: Survey

Survey Distribution: Jane provides the survey to 7 of 12 stations because she is running behind on her schedule

Consistency: Standard Protocols and Procedures

• Ask yourself: – Do you clearly document data

collection methods?

– Do you use the same data collection method for all sites?

– Do you train data collectors?

– Have you taken steps to eliminate bias?

ExampleH14 Outcome: Number of caregivers of homebound OR older adults and individuals with disabilities who reported having increased social ties/perceived social support.

Data Instrument: Survey

Survey Distribution: Jane provides the same survey to all stations at the same time

Accuracy: Data Free From Error

• Ask yourself: – Do you have a plan to

collect and review data?

– Do you have documentation on your data collection plan?

– Does the data vary significantly in unexpected ways?

ExampleData Collection Approach: Collect data from all 12 stations

Data Received: 150 surveys

Data Check: Jane’s colleague reviews the data to check her calculations, as per her organization's Data Collection Handbook

Verifiability: Data Collected According to Plan

• Ask yourself: – Did the grantee double

check that the data collected followed the plans and procedures?

– Did the grantee use controls throughout data collection?

– Does the grantee regularly review and update its data quality control procedures?

Example

Situation: Jane submits her PPR and leaves for vacation; her colleagues cannot validate the data she input into it.

Doing Data Collection: Identifying Your Source

• List pros and cons of possible data sources

• Identify a preferred data source

• Consider back-ups

Doing Data Collection: Considering Methods

• Survey

• Interviews

• Observation

• Standardized tests

• Tracking sheets

• Focus groups

• Diaries or journals

• Secondary data

Method Requirements

Sample questions are provided for the required survey

Doing Data Collection: Creating a Schedule

• Identifies who will collect data, using which instrument, and when

– Include stakeholders in planning

– Include time for collecting, analyzing, and reporting data

• Share with team to keep everyone informed

Doing Data Collection: Selecting a Final Measurement Instrument

• For CNCS priorities and performance measures, look for instruments by program– Go to http://

www.nationalservice.gov/resources/performance-measurement/senior-corps

– Programs and projects can also look anywhere they like to find instruments:• Use Internet search engines• Leverage experience and known best practices in your field• Talk to others within you professional network to find out what they

are using

Doing Data Collection: Selecting a Final Measurement Instrument

• You may have a pre-existing tool or want to modify a resource found on the Knowledge Network

• Consider: • How else might you use this data? • What may other stakeholders want?• How will this affect your data collection and

instrument design?• Keep in mind the 5 data quality elements when

developing or modifying a measurement instrument.

Learning Activity

Data Management

• Now what? • Consider:

– Potential usage

– Access

– Storage and maintenance

Data Management

• Data collection procedures and guidelines

• Source documents for data (e.g. sign-in sheets, activity logs, attendance logs, client tracking database, tracking forms, surveys, etc.)

• Formalize data sharing agreements in MOUs

Data Reporting

Data Reporting

Q&A

Thank you

Stay current by visiting the Knowledge Network! http://www.nationalservice.gov/resources/performance-measurement/senior-corps