Recommendation Follow-up Report: Oregon Department of ... · The Oregon Department of Education (ODE) agreed with all 13 audit recommendations. Our follow-up work found ODE has fully

Secretary of State Bev Clarno Audits Division Director Kip Memmott

Oregon Department of Education

Recommendation Follow-up Report: The Oregon Department of Education Should Take

Further Steps to Help Districts and High Schools Increase Oregon’s Graduation Rate

April 2019

Report 2019-15

This page intentionally left blank

April 2019

Oregon Department of Education

Recommendation Follow-up Report: The Oregon Department of Education Should Take Further Steps to Help

Districts and High Schools Increase Oregon’s Graduation Rate Recommendation Follow-up Results The Oregon Department of Education (ODE) agreed with all 13 audit recommendations. Our follow-up work found ODE has fully implemented three and partially implemented four recommendations. While ODE has made some progress, significant work is needed to implement the remaining 10 recommendations. Implementing these recommendations will help districts and schools with their efforts to increase Oregon’s graduation rate.

Highlights from the Original Audit

ODE has prioritized improving four-year graduation rates in recent years. We found ODE could make further progress on graduation rates by helping districts and schools focus on specific student groups with historically lower graduation rates, such as students who transfer between districts, low-income students, and middle school students. The audit found that ODE can also better help districts and schools to use data to identify students at risk of not graduating, implement effective improvement tools, and communicate the importance of high school graduation to parents and the community.

Background

Despite recent graduation rate gains, one in five Oregon public high school students does not graduate on time. Our audit concentrated on schools with low and middle-range rates that had not improved as much as the state as a whole between the 2011-12 and 2015-16 school years. These represented 40% of high schools in 2015-16.

Purpose The purpose of the audit was to determine how ODE and school districts could increase four-year graduation rates in Oregon’s public high schools. The purpose of this follow-up report is to provide a status on the auditee’s efforts to implement the audit recommendations.

Key Findings from the Original Audit

1. More than 70% of students who did not graduate on time are low income. Further, students who changed districts during high school, more than a quarter of all high school students, had graduation rates roughly 30% lower than students who did not.

2. Schools with mid-range graduation rates (67% to 85%) receive limited improvement support from ODE, though most non-graduates attend these schools.

3. The Legislature and ODE has not emphasized middle school performance or student transitions from middle school to high school, though students who struggle in middle school are already at risk of not graduating.

4. ODE does not track student grades or specific credits attained, data the agency could use to help more students graduate. ODE should also improve its communication and help districts and schools communicate the importance of graduation to parents and the community.

Oregon Secretary of State | Report Number 2019-15 | April 2019 | Page 1

Introduction

The purpose of this report is to follow up on the recommendations we made to ODE as included in audit report 2017-29, “The Oregon Department of Education Should Take Further Steps to Help Districts and High Schools Increase Oregon’s Graduation Rate.”

The Oregon Audits Division conducts follow-up procedures for each of our performance audits. This process helps assess the impact of our audit work, promotes accountability and transparency within state government, and ensures audit recommendations are implemented and related risks mitigated to the greatest extent possible.

We use a standard set of procedures for these engagements that includes gathering evidence and assessing the efforts of the auditee to implement our recommendations; concluding and reporting on those efforts; and employing a rigorous quality assurance process to ensure our conclusions are accurate. We determine implementation status based on an assessment of evidence rather than self-reported information. This follow-up is not an audit, but a status check on the agency’s actions.

To ensure the timeliness of this effort, the division asks all auditees to provide a timeframe for implementing the recommendations in our audit reports. We use this timeframe to schedule and execute our follow-up procedures.

Our follow-up procedures evaluate the status of each recommendation and assign it one of the following categories:

Implemented/Resolved: The auditee has fully implemented the recommendation or otherwise taken the appropriate action to resolve the issue identified by the audit.

Partially implemented: The auditee has begun taking action on the recommendation, but has not fully implemented it. In some cases, this simply means the auditee needs more time to fully implement the recommendation. However, it may also mean the auditee believes it has taken sufficient action to address the issue and does not plan to pursue further action on that recommendation.

Not implemented: The auditee has taken no action on the recommendation. This could mean the auditee still plans to implement the recommendation and simply has not yet taken action; it could also mean the auditee has declined to take the action identified by the recommendation and may pursue other action, or the auditee disagreed with the initial recommendation.

The status of each recommendation and full results of our follow-up work are detailed in the following pages.


Recommendation Implementation Status

Recommendation Auditee Action Status

1. Research and recommend effective approaches to districts and schools on ways to better serve students who change districts and schools.

While ODE has not made progress on this recommendation, management stated there were efforts underway that could assist in better serving this vulnerable population. For example, ODE management stated it is creating a new Career Technical Education plan for the state that addresses multiple entry and exit points for students.

Not implemented

2. Analyze and report performance for students who transfer between school districts and between schools during high school, and share this information with districts and schools.

ODE staff analyzed trends in student mobility and graduation data, and published the results online in a report entitled Student Mobility in Graduation in June 2018. Additionally, ODE provided districts with metrics such as graduation rates and absenteeism for their mobile students for districts to use in developing their High School Success plans.1 According to ODE management, many district leaders were either unaware or never looked in depth at the mobile student population, and this opened up opportunities to talk about district supports.

Implemented

1 High School Success is a fund established by from Measure 98, which was passed by Oregon the voters in 2016. The measure provides direct funding to school districts to increase high school graduation rates. These funds can be used for Career and Technical Education, college-level courses while in high school, and dropout prevention programs. Districts receive funds based on the number of students they have.


3. Include schools with mid-range graduation rates while engaging in efforts to improve school performance as identified in ODE’s approved ESSA plan.2

ODE began a pilot program for its new school improvement program under the Every Student Succeeds Act (ESSA) in March 2018.3 The pilot project included eight school districts and 28 high schools, 12 of which had mid-range graduation rates for the 2017-18 school year. While the pilot has been completed, ODE management stated that increasing the threshold to include schools whose graduation rates range from 67% to 78% for further support would require significant fiscal and personnel resources. For 2019-21, ODE is requesting a policy option package to provide better comprehensive and targeted support to districts and schools.

Partially implemented

4. Identify strategies to better support economically disadvantaged students, such as reducing food insecurity, increasing access to medical services and increasing access to school counselors.

ODE has published strategies for improving graduation rates on a dedicated graduation improvement website. These strategies include alternative breakfast service models and school-based health centers, both of which address areas of particular concern for economically disadvantaged students.

Implemented

5. Recommend state initiatives and performance measures to the Legislature that focus on middle schools.

While ODE management stated they proposed initiatives to the Governor for Pre-Kindergarten through grade 12, and the Governor’s Recommended Budget for 2019-21 does include several initiatives that impact middle school students, none focus specifically on middle schools. ODE has also not changed its key performance measures since the release of the initial audit.

Not implemented

2 The original audit defined high schools with mid-range graduation rates as those whose four-year graduation rates ranged from 67% to 85%. 3 The Every Student Succeeds Act (ESSA) provides federal funds to states to support students in poverty and other historically underserved groups. It also requires states to have an accountability system that meets certain requirements. ESSA gives states some flexibility in designing accountability systems and in identifying and supporting schools and districts that need improvement. Federal officials approved Oregon’s most recent ESSA plan in August 2017.


6. Support coordination between middle and high schools that includes guidance on collaboration, curriculum alignment, and the importance of middle school for keeping students on track to graduate.

No progress has been made with this recommendation. ODE management stated the agency does not have the resources to provide support and guidance for programs that coordinate student transitions between middle and high school. According to ODE management, almost half of districts that received High School Success funds are allocating some of those funds to middle school initiatives. As of May 2018, those districts project spending about 6% of their funds on programs in middle school.

Not implemented

7. Collect and analyze student grade and credit attainment data in middle and high school grades and share the results of this analysis with districts and schools.

No progress has been made with this recommendation. ODE management stated implementing such a data collection would likely require funding from the Legislature. A bill to collect high school credit attainment did not pass out of the 2017 legislative session.

Not implemented

8. Provide guidance and support to districts and schools to help them better use their student data.

ODE has been working on an approach to an early warning data system for districts and schools to use. ODE and the Chief Education Office worked with several stakeholders on a plan for an Early Indicator and Intervention System (EIIS) that would use student data to provide supports that help students stay on-track for graduation.4 ODE has yet to provide support to districts within existing systems on effectively using their own student data.

Partially Implemented

4 An Early Indicator and Invention System (EIIS) uses individual student data to generate and report indicators of on-track status for graduation, including attendance, behavior, and course performance. This tool is used to identify struggling students earlier in their school career, provide appropriate interventions to students, and regularly examine and address on-track patterns among groups of students.


9. Evaluate how schools are using the required student education plan and profile, and provide guidance on how schools can use them effectively to improve graduation, including expectations at each grade level.

No progress has been made on this recommendation. ODE management stated the agency asked many districts how they used the Education Plan and Profile (EPP) as part of its ongoing High School Success efforts but did not have the resources to evaluate those responses. Also, ODE has not offered any further guidance on how EPPs can be used effectively to improve graduation.

Not implemented

10. Provide guidance to districts and schools on how to effectively solicit student feedback and gauge school climate.

ODE facilitated the creation of the Safe and Effective Schools advisory committee, which issued a report in June 2018 that included the importance of student voice and engaging youth in key decisions at their schools and educational communities. ODE management stated this report was used as the basis for proposed legislation for a pilot program to develop inclusive schools and a policy option package in the 2019-21 Governor’s Budget. While these actions can help with increasing student voices in the decision-making process, ODE has yet to provide specific guidance or tools that address effectively soliciting student feedback and gauging school climate.


11. Provide better guidance and feedback on how districts are using Continuous Improvement Plans, how those plans could satisfy multiple planning requirements, and how they can be better used to enhance district improvement efforts in areas such as graduation.

ODE requires all districts to develop Continuous Improvement Plans (CIPs) for the 2019-20 school year under the new ESSA plan. While ODE has provided a framework for the new CIP under design and development, there have been no changes in the guidance and feedback provided to districts on their CIPs.

Not implemented


12. Prioritize its current efforts to improve communication and coordination within the agency, and develop a process to ensure that these improvements are sustained. This would help ODE in strategically devoting its resources to improvement efforts including graduation.

Starting in 2018, ODE staff from different programs and units formed the Internal Graduation Team, a group that meets monthly with the goal of improving graduation rates by reducing duplication of efforts and other problems arising from lack of alignment between departments within the agency. ODE is also engaged in revisions to its strategic plan for 2019-2024 and expects that strong communication will be among the key outcomes.


13. Develop a statewide communication framework and share best practices that reinforce the importance of graduation to students, parents, and the community, and helps high schools and districts better engage families and community members.

ODE undertook initiatives in 2018 to engage stakeholders and communicate the importance of graduation. These include the Graduation Improvement website for publishing graduation rate improvement strategies, the Every Day Matters online attendance campaign, and a web-based communication network for teachers and school administrators to share best practices through the High School Success program.

Implemented


Conclusion

The audit included 13 recommendations to ODE regarding additional efforts it could take with districts and schools to increase on-time graduation rates, such as focusing on specific student groups and schools, better using student data, and effectively using improvement tools.

We found that ODE has implemented three and partially implementing four audit report recommendations. No substantive action has been taken to address the other six recommendations. Fully resolving the remaining recommendations would help in addressing challenges with student mobility, schools with mid-range graduation rates, coordination between middle and high schools, using student graduation and credit attainment data, and using improvement tools. Implementing these recommendations would also help improve the state’s overall graduation rate. While that rate has improved, Oregon’s rate of 76.7% ranked 49th nationally for 2016-17, the most recent comparable school year.

In response to audit recommendations, ODE has implemented three recommendations related to student mobility, strategies to support economically disadvantaged students, and engaging communities and sharing best practices to improve graduation rates. Specifically, ODE:

Conducted a data report on student mobility in graduation and included the mobile student population in student data it provided to districts for use in their High School Success plans;

Published strategies for improving graduation rates to better support economically disadvantaged students on its graduation improvement website; and

Undertook initiatives to engage stakeholders and communicate the importance of graduation through its graduation improvement website, the Every Day Matters attendance campaign, and creation of a network for the sharing of best practices through the High School Success program.

ODE has partially implemented four recommendations by providing some supports to high schools with mid-range graduation rates, working on an Early Indicator and Intervention System as a tool for districts and schools to help them better use student data, proposing legislation related to Safe and Effective Schools, and creating an internal graduation team to help with aligning efforts within the agency. ODE needs to do more work to fully resolve these recommendations and to address the remaining six recommendations that had no progress toward implementation.

Despite agreeing to the recommendations at the time of the audit, ODE management stated the lack of personnel and resources needed to make the improvements contributed to agency’s limited progress in implementing recommendations. Additionally, for the recommendations related to middle schools, ODE management stated that, while current and future efforts at the agency will impact middle school students, the delivery of support and guidance exclusively to middle schools is severely hampered by the limited resources available and the need to allocate those limited resources in a way that helps all K-12 students.

We sincerely appreciate the courtesies and cooperation extended by officials and employees of ODE during the course of this follow-up work.

Follow-up Report Team

William Garber, CGFM, MPA, Deputy Director

Andrew Love, CFE, Audit Manager

Karen Peterson, Principal Auditor

T. Cornforth, Staff Auditor

About the Secretary of State Audits Division

The Oregon Constitution provides that the Secretary of State shall be, by virtue of the office, Auditor of Public Accounts. The Audits Division performs this duty. The division reports to the elected Secretary of State and is independent of other agencies within the Executive, Legislative, and Judicial branches of Oregon government. The division has constitutional authority to audit all state officers, agencies, boards and commissions as well as administer municipal audit law.

This report is intended to promote the best possible management of public resources. Copies may be obtained from:

Oregon Audits Division 255 Capitol St NE, Suite 500 | Salem | OR | 97310

(503) 986-2255 sos.oregon.gov/audits

Report

Recommendation Follow-up Report: Oregon Department of ... · The Oregon Department of Education (ODE) agreed with all 13 audit recommendations. Our follow-up work found ODE has fully

Documents