Using Data for Program Improvement

Using Data for Program Improvement

Christina KasprzakMay, 2011

Using data for improvement

Analyzing the DataInterpreting the Data

Planning for Improvement

• Evidence refers to the numbers, such as“89% of families reported ...”

• The numbers are not debatable

Analyzing the data – what is the evidence?

• How do you interpret the numbers?• What can you conclude from the

numbers?• Does evidence mean good news? Bad

news? News we can’t interpret?• To reach an inference, sometimes we

analyze data in other ways (ask for more evidence).

Interpreting the data – what do you mean?

• Given the inference from the numbers, what should be done?

• What are recommended actions?• What quality practices could help accomplish

this outcome?• Another role for stakeholders?

Planning for improvement- what actions will you take?

Looking at Family Survey Data

• Return rate• Representative of

population served

Ensuring quality of our data

What do I do with my family outcomes data?– Looking at outcomes for all families in

the program– Looking at outcomes by sub-groups– Looking at outcomes over time

Analyzing family outcomes survey data

• Overall means on indicator(s)

• Frequency distributions

Looking at family outcomes for all families

Overall family survey data

Overall family survey data

Palau: Frequency distributionTo what extent has your child’s program: • helped you know how to help your child learn? • encouraged you to be actively involved in your child’s education?• included you as a full partner in making decisions about your

child’s education?

Figure 3: 2009 Preschool Survey Results

System Characteristics

Service Characteristics

Family Characteristics

Child Characteristics

Looking at family outcomes by subgroups

… across the system?

•Do you have regions/programs that are reasonable subgroups?•What about classrooms, teams, providers?•Type of program?

Are there differences in outcomes…

… across service characteristics?

•Types of service•Intensity of services•Where services received•etc.

Are there differences in outcomes…

Breakdown for each FSM state

Guam Part B

What system or service variables make the most sense when analyzing YOUR data?

-Region, program, classroom, team, provider, type of program, etc.?

-Type of service, intensity of service,where service is received, etc.?

… across family characteristics?

•Race/ethnicity•Family income•Primary language•etc.

Are there differences in outcomes…

… across child characteristics?

•Race/ethnicity•Family income•Primary language•etc.

Are there differences in outcomes…

CNMI-C

Table 2. Percent of Families who Report that Early Intervention Services has Helped their Family in Measurement A, B, and C

Analysis by child’s disability

What family or child characteristics make the most sense when analyzing YOUR data?

-Family race/ethnicity, income, language, length of time in service, etc.?

-Child race/ethnicity, type of disability, length of time in service, etc.?

Do family outcomes change over time?

OSEP indicators 4a, 4b, 4c:Statewide results (2006-2008)

AS- PART C

Q16: Know their rights Indicator 4: Year-to-Year comparisons for question 16

Guam Part C- Item comparison over time

June 2004 & June 2005 Parent Survey Responses Related to Family Capacity

How are families doing on each item?

Family survey:Statewide response to questions 1-15

If the lowest rated items are:KNOWING YOUR RIGHTS AND ADVOCATING FOR YOUR CHILD

4. How much does your family know about the programs and services that are available?6. How familiar is your family with your rights?

HAVING SUPPORT SYSTEMS 10. How often does your family have someone your family trusts to listen and talk with when they need it?11. How often does your family have someone you can rely on for help when your family needs it?12. How often is your family able to do the things your family enjoys?

ACCESSING YOUR COMMUNITY 15. How often does your child participate community/social activities right now?

Then what kinds of improvements in practices would you want to implement?

AS- Part B items (not their chart)

Teachers and administration…1. seek out my input;2. Show sensitivity to the needs of my child; 3. Encourage me to participate in the decision-making

process;4. Respect my cultural heritage; 5. Help me to understand the Procedural Safeguards [the

rules in federal law that protect the rights of parents]; and

6. Show a willingness to learn more about my child’s needs.

AS- Part B items (fake data!)

More than 90% of the parents surveyed during FFY 2009 monitoring visits also indicated they would be able to:

1. locate policies and procedures related to special education and free appropriate public education;

2. identify RMI’s special education programs and services; 3. exercise rights as parents of a child with disabilities;4. participate in the education of their children; and5. assist and support the public education system.

MH- Report by items

Analyzing the DataInterpreting the Data

Planning for Improvement

Using data for improvement

Then… did your improvement activities make a difference?