Good Morning Chairman Cupp, RankingMember Sykes … · Good Morning Chairman Cupp, ... I’m sure you all ... don’t want to do more than one survey this school year, ...

Good Morning Chairman Cupp, Ranking Member Sykes and members of the Joint

Education Oversight Committee. Thank you for this opportunity to present to you an

update on the research that JEOC has undertaken since October 2016. I’m sure you all

recall that the committee approved eight research areas for JEOC. You also already know

that in February, the JEOC staff increased from one to three. Since February, we have been

able to work on multiple projects simultaneously with greater efficiency. Today, I will

discuss the three projects that are currently active.

1

The Active Research areas for JEOC are: Early Learning, Assessments and Career Technical

Education. I have been working on these projects since October, but much greater progress

has been made in the last few months, since our new staff members have joined me.

The other five research areas are still on our radar, but we are not spending as much time

on these right now, in order to make faster progress on the three areas I will present to you

today.

2

The first area we are working on is Early Learning. The proposed research plan indicated

that JEOC would look at what other states are investing in Early Learning and determine

how Ohio compares to other states and the nation. This work would be done based on

three papers that came out in the summer of 2016 which included information about early

learning investments in all 50 states. Additionally, the work would include recent Ohio data

and analysis of which investments in Ohio have been successful.

3

We have written up a scope of work for this project now, which you have in front of you or

on the iPads. In this document we have identified the scope of the project in which we are

engaged. We are studying investment in early childhood education efforts receiving either

state or federal funding in Ohio.

4

The objectives of this research are to:

Compile accurate data about the licensed childcare programs in Ohio.

Determine the amount of funding for licensed childcare programs that has a state and/or

federal government source.

Develop criteria for areas of need (where funds spent are not supporting, or not supporting

enough, the intended population) and areas of success (where funds spent are supporting

the intended population) in early childhood education in Ohio.

If possible, determine where future investments would increase support for the intended

population, based on prior programs and impacts. Depending on what is learned in the first

three objectives, the criteria for this may be based on number of children served, or

number of children served that achieved a high outcome on, say, the Kindergarten

Readiness Assessment.

5

Our hope is to gather together information about the early childhood programs in Ohio to

learn about how early childhood education is working in Ohio. As a state, Ohio has

provided targeted support for early childhood education. That is, the state of Ohio is not

offering universal preschool; rather, the state of Ohio has elected to support the children

most in need, as determined by socio-economic status (PFCC and ECE) and special need

(PSE). The state of Ohio currently funds, through its early childhood education grant, four

year olds who meet the financial requirements, and are in programs that feed into districts

with low KRA and low 3rd Grade OAA Reading scores. In prior years, ECE grants were also

available to three year olds. The change was made to reach a greater percentage of

qualifying four year olds. Knowing how the funds are distributed helps to paint a clearer

picture of which children are being served.

The long-term impact analysis is not yet being done at the state level. However, in some

areas in Ohio, there are organizations that are looking at long-term effects of early

childhood education. We will reach out to these organizations to learn what data they have

collected and what conclusions can be drawn from those data. This may not be

generalizable to the state, but it will be a peek into the future of what we could learn about

early education in Ohio in the future, when the Early Learning Assessment and the

Kindergarten Readiness Assessments are both in place.

6

This part of the research will depend on whether we can identify the providers and money

spent on children at those facilities for the providers and districts that have linked their

data. Again, this information may not be generalizable to the entire state.

7

We have laid out four phases for this project.

First we wanted to review the reports from last summer and get some foundational

information to frame the questions.

Then, we went to ODE and ODJFS to ask for some data to help clarify the Ohio part of the

picture.

Then, we went to Head Start to get their data to include in the analysis.

Finally, we will analyze all the data and look at outcomes and spending.

8

So far, we’ve already completed the first phase. We’ve read the papers and started to build

the framing for the report. We’ve also been working with ODE and ODJFS to get the data

we are looking for. This is going to take a while, though. We are learning how to ask the

right questions to get the data we are seeking and we know we are asking for information

that is not easy to compile, because it cuts across both agencies and Head Start.

We also have some data requests in to the National Head Start office. Again, this takes

time, but we are learning more and more about the systems and how they interact as we

progress with this research.

As I stated before, we are beginning to frame the brief, but need to wait for the data to

come in. We anticipate that this work can be completed by the end of this fiscal year.

9

Our second area of research that we are focusing on is Assessments. Originally, I discussed

getting a list of commercially available assessments used in schools and to get information

about the use of these assessments. We wanted to know both what and why.

10

As defined in the Scope of Work, JEOC will do a study to determine which districts are using

which commercial test instruments.

11

To accomplish this work, we have put together a list of commercially available assessments

including those that are included in law. There are many screeners and diagnostic

assessments required in law, for which the ODE has created a list of hundreds of

appropriate commercially available assessments.

We will get a sense of usage by asking whether all students take the test or whether the

test is used for teacher evaluation. By asking the number of days the test is administered,

we can get a sense of frequency of use, without having to make the survey too detailed to

be easy to complete. If the exact amount of time becomes important, we can get to that in

the second survey phase.

We will compare the number of days used for these tests to the number of days used for

statewide testing. This may be interesting as it should show how much time is spent on

testing in general, as well as how much of that time is required by law.

We will also collect technical documentation on the tests to help us understand what

information districts are getting from those tests that is unique and what information is

duplicative, perhaps.

12

We will report on how widely used the different tests are as well as what information is

being used from the tests for what purposes in districts. We can also distinguish between

things like screeners and things like SLOs, so we can understand the required testing

environment better.

Last but not least, we want to share the information provided by the vendors about what

information the test generates about students.

13

We also hope to be able to share this kind of detailed information about the assessments

that are most widely used.

14

For this project, the four phases start with a data request for student level data on the

assessments collected by EMIS. The request is for student level data so that we can do an

analysis of the functionality of the system. For example, to answer the question: Does

screening for giftedness in sixth grade after screening for giftedness in 2nd grade produce a

more accurate identification process? We could look at the data from a school with

screening in both grades. With student level data we would be able to see whether the

same students are identified both times, or whether new students are added to the

identification pool. Answering a question like this, tells us whether the system of testing is

functional or not.

Since we are learning, from talking with district officials, that each school system looks very

different, we feel that we need more information about the environment to know which

questions we need to ask. So, we would like to do a structured interview with 10 school

districts from around the state to help us frame the research questions for the second

survey.

At the same time that we are performing the structured interviews, we will deploy a short

survey to districts to gather usage information about the known assessments. This survey

will be short and will give us a high level view of the landscape of assessment in Ohio

schools.

15

Finally, with the high level view and the rich information from the interviews, we believe we

can create a second survey to be deployed in the fall that will give us the information we

need to understand why districts are using the assessments they have chosen. Following this

survey, we believe we will have the information needed to write the report.

15

So far we have put in the request for data from ODE. This is a large, complex data set, so it

will take some time to get the data back from ODE.

For our structured interviews, we would like to ask each JEOC member to identify a school

district in their part of the state for us to interview. We will then visit the school and

conduct the interview to collect the information. We want to get the same information

from each interview so that we can compare the different systems we encounter.

The first survey is constructed and has been piloted. This week, we are working with a

small number of schools to confirm that we have the right amount of information in the

survey and that the questions are easily answered by a district official familiar with the

local testing program.

The last phase will begin once the structured interviews are completed. We expect this

work to continue over the summer, since schools will soon be finished for the year. We

don’t want to do more than one survey this school year, so we will do the second survey in

the fall, when schools start up again. This work should be able to be completed in the fall,

perhaps by October 1, but we can give an update on the timeline once phase 4 is begun.

16

The last research area we are engaged in is Career Technical Education. This topic was put

on the list primarily to understand how students have access to Career Tech. The intent of

the original proposal was to create a report that was largely the state of CTE in Ohio.

17

In our Scope of Work we have defined the project as a study of access to Career Technical

Education for all students.

18

We have learned that, by law, all students must have access to CTE, but there are still

questions about how that might occur. For example, what is the difference between how a

student in rural Ohio accesses CTE and a student at Columbus Public Schools? Also, how

many students are in each program and pathway?

We want to look at funding for CTE by both the state and federal government to

understand the similarities and differences between CTE and traditional education. To this

end, we will also look at costs, as these differ significantly in both environments.

We want to know the number of programs available in each school district as a way of

comparing access to CTE. We also want to know which programs are full and which are not

to understand what choices students have.

19

At the highest level, we want to know what CTE programs students can access in each

district, how many students are enrolled at the CTC as well as how many students are

enrolled in each pathway at each CTC.

20

The analysis will look at capacity as well as access, since one informs the other. Even if a

student has access, if the program is at capacity, the student does not really have access.

We also want to look at how these CTCs are structuring their programs and whether there

are lessons that can be learned from CTC and applied to traditional high schools and vice

versa. This is important because access to CTE is also dependent on how the district and

CTC create that relationship (Comprehensive vs. JVSD vs. CTC).

21

We will visit several different types of CTE facilities to better understand how CTE works.

There are JVSDs, CTCs and Comprehensives.

We will request data from ODE and OACTE to determine the pathways available at each

location and the number of students enrolled in each location by pathway.

We will analyze the data to determine the capacity of the programs, then compile the

analysis into a report.

22

We have visited several Career Technical Center and Joint Vocational School Districts to

familiarize ourselves with the environment of CTE. We have been to the Mideast Career

Center, Scioto County Career Center, Fort Hayes Career Center, and Stebbins

Comprehensive High School.

OACTE has made a data request from ODE to get the number of students in each pathway

at each location. Some data has already been shared with JEOC on this.

The analysis and report writing will begin as the data are available to us. We expect to be

able to complete this project by the end of this fiscal year.

23

JEOC will be adding a new page to its website called “Resources”. This page will consist of

useful links to resources about education in Ohio. If members have suggestions of links

they would like to see, please send them to Haley and we will get them added to the new

page.

24

At this time I’d be happy to entertain any questions you may have about this work.

25