Using Data To Build, Grow, And Sustain Your Afterschool System In Partnership with May 13, 2015 @aypf_tweets @afterschool4all.

Using Data To Build, Grow, And Sustain Your Afterschool

System

In Partnership with

May 13, 2015

@aypf_tweets@afterschool4all

Webinar Technical Support

• GoToWebinar Technical Assistance: 1-800-263-6317

• To submit live questions, please use the “Questions” box on the control panel

• A recording of the webinar and other resources will be available at www.aypf.org

@afterschool4all

Role of Data in Building Afterschool Systems:  Needs Assessment Mila Yochum, Director, Allegheny Partners for Out-of-School Time

(APOST) Audience Q&A Role of Data in Scaling Afterschool System:  Use of Quality

Improvement Tools for Accountability Charles Smith, Executive Director, David P. Weikart Center for Youth

Program Quality Audience Q&A Role of Data in Sustaining Afterschool System:  Communicating

Outcomes Lisa Caruthers, Director, Center for Afterschool, Summer and

Expanded Learning (CASE) for Kids Audience Q&A Closing Remarks: Using Data in Afterschool Systems-Building

Jen Rinehart, Vice President, Research & Policy, Afterschool Alliance

Agenda

@afterschool4all

Mila Yochum, Director, Allegheny Partners for Out-of-

School Time (APOST)

Youth in Allegheny County will realize their fullest potential by participating in high quality out-of-school time opportunities.

APOST focuses on improving quality of and access to out-of-school time opportunities through committed collaborative and innovative efforts of community organizations, government, foundations, schools and other partners.

About Allegheny Partners for Out-of-School Time (APOST) Strategies

Quality Campaign Professional Development

of Youth Practitioner Marketing, Communication

and Outreach Policy for Funding

Sustainability Partnerships, Convening

and community based system-building

The Need for Data

http://www.afterschoolpgh.org/state-of-afterschool-reports/

The Funding Inventory

The State of OST Programs

5 6 7 8 9 10 11 12 13 14 15 16 17 18 190

100

200

300

400

500

600

700

526 541 539

586 590 594567 567

116 116 103 103 101 91

20

Frequency of OST Programs Serving Each Age-Group

Child Age (years)

Num

ber

of

Pro

gra

ms

The State of OST Programs

For every 1000 school-age children in a given zip code, there are 3 OST programs

The Funding Inventory

0 500 1000 1500 2000 2500 30000

10

20

30

40

50

60

70

Association Between Children in Poverty and Opportunity for OST Programs by Zip Code

Number of Children in Poverty Per Zip Code

Tota

l N

um

ber

of

OST P

rogra

ms

Per

Zip

Code

For every 1000 school-age children in poverty a given zip code, there are 19 OST programs

Marketing, Communication and Outreach Policy for Funding Sustainability Partnerships, Convening and community based

system-building

Using Information and Data

Mila Yochum Director APOST (Allegheny Partners for Out-of-School Time)Mila.yochum@uwac.org

Questions & Contact Information

Questions?

To submit live questions, please use the “Questions” box on the control panel

Charles Smith, Executive Director, David P. Weikart Center for Youth Program

Quality

QIS for Afterschool Services | May 13, 2015

Quality Improvement Systems (QIS) that Build & Grow Afterschool Services: Exemplary QIS in 21st CCLC

QIS for Afterschool Services | May 13, 2015

Necessaries for a QIS

• QIS elements

• Lower stakes approach

• Effective Performance Data

QIS for Afterschool Services | May 13, 2015

Service Production Map

Transfer Outcome

Staff Engagement

Quality Management

Practices

Quality Instructional

PracticesSkill-building

Youth Engagement

Network Leader

Engagement

Quality System Building

QIS for Afterschool Services | May 13, 2015

Staff Engagement

Quality Management

Practices

Quality Instructional

PracticesSkill-Building

Youth Engagement

Network Leader Engagement

Quality System Building

• Vertical Communication

• Horizontal Communication

• Job Satisfaction

• Staff Capacity• Continuous

Quality Improvement

• Youth Role in Governance

• Communication• Student Data• Community

Engagement

• Youth Engagement and Belonging

• Academic Planning• Homework

Completion• Instructional

Quality• School-Day

Content• Growth and

Mastery Focus

• California Outcomes Toolbox (Vandell et al)

• DESSA Mini (Devereaux/Apperson)

• Access• Targeting Academic

Risk• Accountability• Collaboration

• Collaboration• Job satisfaction

QIS for Afterschool Services | May 13, 2015

Item Level Description

Targeting Academic Risk

Proportion of students statement is true for: “1” = Almost none, “3” = About half, “5” = Almost all

• Targeted because scored below “proficient" on local or state assessments

• Targeted because did not receive a passing grade in preceding period

• Referred by a teacher for assistance in reading, mathematics or science

• Targeted because of status as an English Language Learner (ELL)

QIS for Afterschool Services | May 13, 2015

Telling Stories with Data

2010-2011

2011-2012

2012-2013

2013-2014

2

3

4

5

Targeting Academic RiskSchool Day ContentSchool Day Data

0 1 2 3 4 5 6 7 8 9 101112131415160

2

4

6

8

10

12

14

16

2013-14 Risk Index Score

Nu

mb

er

of

Sit

es

Growth & Contribution Profiles

QIS for Afterschool Services | May 13, 2015

In closing…

• Scale of the Leading Indicators systems

• Early findings

• Continuous improvement culture AND systems

QIS for Afterschool Services | May 13, 2015

Contact: For More Information:

• Charles Smith

charles@cypq.org

• www.cypq.org

• Arkansas 21st

CCLC 2012-13 Statewide Evaluation Report

• Oklahoma 21st

CCLC Statewide Evaluation Report

• Leading Indicators Performance Measurement System

Questions?

To submit live questions, please use the “Questions” box on the control panel

Lisa Caruthers, Director, Center for Afterschool, Summer and

Expanded Learning (CASE) for Kids Harris County Department of

Education

Webinar 5.13.2015

ENRICH Afterschool:A Collaborative Research Initiative

Lisa Thompson Caruthers, MSW, Ph.D.,Harris County Department of EducationDirector, Center for Afterschool, Summer and Expanded Learning (CASE) for Kids

Evaluating Out-of-School Time Needs,Resources andInitiatives in the Communities of Houston

Primary Goal:– Use research to drive change:

Phase I:Funding MapLogic ModelStakeholder Perception StudyHistorical Trend Analysis

Phase II:Return on Investment StudyBest Practice StudyImpact Study

Promotion

Research

Resource

70%

4%

5%

20%

2%

FederalStateLocal PrivateIn-Kind

Funds/Resources Used for Houston/Harris County After-School and Out-of-School Time Programming

1.4 M

837,714US Census/TEA

830,776 Estimated Harris County Youth Participating in Out-of-School

Time Programming*,**

214,118

30,000

44,461

*Public funding streams have different attendance requirements which determine how youth are counted as served. This final estimate is based on publically available or self-report average daily attendance from participating ERNICH After-School organizations.** US Census, 2011, American Community Survey

25% est.

served

5% est. served

3% est. served daily

Return on Investment FrameworkCost Analysis (Survey)

• Identify the current financial investment in after-school programs and the outputs it supports

• Collect program and cost data at an organizational level

Cost-Benefit Analysis (Literature Review)• Examine the benefits derived from after-school programs• Identify financial proxies

Return on Investment Framework (Apply Crosswalk)• Apply study data to proposed cost-benefit crosswalk to create formulas for

assessment

Crosswalk

A• Effect of After-

School Programming

B• Range of

Benefits

C• Overall

Formula

Effect of After-School Programming

Increased Retention RatesIncreases in AttendanceIncreased Graduation RatesDelinquency Prevention

Range of Benefits

School-based System SavingsIncreases in School-based EarningsIndividual EarningsReduction in societal costs

Preliminary Cost Analysis Results

An investment of $23.7 million provides support to 17 out-of-school time organizations in Harris County, Texas, employing 141 full-time and 465 part-time Houstonians, while providing activities and services to 41,367 participating youth grades prekindergarten through 12th grade.

Preliminary Findings

$4.58 return for every dollar spent on

afterschool programs

Evaluating the Out-of-School Time Needs,Resources, andInitiatives in the Communities of Houston

Modifications• Focusing only on necessary data elements• Data collection plan will capture higher

percent of funding for programs• Stronger relations with school-age childcare

community.

Evaluating the Out-of-School Time Needs,Resources, andInitiatives in the Communities of Houston

ENRICH Successes:• Robust Dissemination Plan• Community mobilization• City of Houston now invests over

$500,000 in after-school programming

Evaluating the Out-of-School Time Needs,Resources, andInitiatives in the Communities of Houston

Thank You!Lisa Thompson Caruthers, MSW, Ph.D.lcaruthers@hcde-texas.org713.696.1336

www.afterschoolzone.org@Afterschool4HOU

Questions?

To submit live questions, please use the “Questions” box on the control panel

Jen Rinehart, Vice President, Research &

Policy, Afterschool Alliance

44

WE CAN HELP TELL YOUR STORY

Better Together: A Resource Directory for Afterschool System

Builders

Available online at http://www.aypf.org/better-together-a-resource-directory

The Directory Includes:

• A comprehensive listing of over 50 organizations

• Resources categorized by key themes

• Easy access to relevant information, data, and briefs

Please fill out the survey upon exiting the webinar. Your feedback is important to us!

To learn more and access copies of the slides and event recording, please visit

www.aypf.org

www.afterschoolalliance.org

THANK YOU!

@afterschool4all