Commissioner’s Performance-Based Accountability Task Force: A Proposal for a Multi-level System Deb Wiswell & Scott Marion January 29, 2010.

Commissioner’s Performance-Based Accountability Task Force: A Proposal

for a Multi-level System

Deb Wiswell & Scott MarionJanuary 29, 2010

Adequacy Accountability System

Commissioner's Performance Based Accountability System Task Force. 1/29/10

Two approaches for demonstrating that the school is providing the opportunity for an adequate education Input System—Based on a subset of the school

approval standards Performance System—what this committee is

required to design

A school has to demonstrate that it is providing an opportunity for an adequate education by meeting the requirements of at least one of the systems, but not both.

A Multi-Level Performance Accountability System for NH


Level One A very limited set of common (across the state)

indicators and metrics Level Two

Locally determined goals, targets, and indicators

Level One Overview: State System


A very limited set of common indicators and metrics

Applied consistently across all schools in the state

Focused on unarguable outcomes, e.g., NECAP, graduation rate, postsecondary assessments, attendance

Level One: A K-8 Sketch Inclusion Factors Status Measures Growth Measures Gap Analyses


Level One: A High School Sketch Inclusion Factors Status Measures Gap Analyses Postsecondary Indicators


Level One: State System Indicators


All schools Test participation NECAP index scores (status)—reading, writing,

math, science NECAP status gaps—reading, writing, math, science Attendance/truancy

Additional indicators for K-8 Schools NECAP growth percentiles (growth)—reading, math NECAP growth gaps—reading, math

Additional indicators for High Schools Graduation rate Postsecondary assessments (AP, SAT, ACT)?

Inclusion: Common Indicators for K-8 & HS Test Participation—required level of

participation (e.g., 95%) to meet performance requirements Weighted average across all state tests given in the

school School must average 95% participation to meet

adequacy standard Attendance/Truancy—percent of students

absent fewer than 15 days Actual percentage will get folded into the overall

adequacy determination Committee and other stakeholders, informed by

data, will have to determine what is “good enough”Commissioner's Performance Based Accountability System Task Force. 1/29/10

Level One: Achievement Factors (K-8 & HS) Status—a weighted (by # of tested students)

average of index scores in reading, mathematics, writing, and science

Status Gaps—effect size differences (standardized difference using NECAP scale scores) for reading and mathematics comparing students receiving FRL with those not receiving FRL What about science and writing? What about other groups (e.g. , SWD)


Level One (K-8): Growth Factors Student growth percentiles in reading and

mathematics Specific details being worked out by AYP task force

Growth gaps—are FRL students growing at an equally appropriate rate as non-FRL students? Specific details being worked out by AYP task force


Level One: Postsecondary Readiness (HS) Graduation rate—Using the new federally

required graduation rate Dropout rate—Using NH method for

calculating dropout Other postsecondary measures???


Level One: Putting it together Aggregate indicators within major classifications,

e.g., status, growth, postsecondary Determine “adequate” for each class of indicators This allows for more informative feedback

Aggregate these “adequate” determinations across major classifications Use a profile or related approach to determine overall

adequacy for Level One

Obviously, the devil is in the details, but once we get general agreement on the overall plan, we can begin filling in some of the details


Level Two: Locally-determined system


A very limited set (e.g., 2-5) of district/school-determined goals, targets, and indicators For example, “increase the % of students

achieving their NWEA growth targets to 90% by 2015”

The school results related to such goals and targets would count in the performance-based accountability system

Level Two: Establishing the Goals & Targets Schools will be required to identify at least two,

but no more than five (for these purposes) goals for which they want to be held accountable Goals must be measureable with identifiable

measurement approaches Goals could (should) stretch over several years, but

yearly measureable targets must be established Committee could require that the goals could be:

Academic only A mix of academic, social, emotional, and physical, but

requiring at least one academic Any goals that the school chooses that can be measured


Level Two: Justifying the Goals The school must provide a rationale for selecting

each individual as well as the set of goals This rationale should be constructed in terms of a

“theory of action,” i.e., a logical flow that describes how focusing (and measuring) this particular goal and associated targets will lead to the ultimate goal of improving student achievement

For example…. Goal: Increase the number/percent of students taking

and passing (earning a 3) AP exams Target: In 2010, we will increase by 5% the percentage

of seniors who have taken at least one AP course with no drop in the percentage of students earning a 3 or better


For example (continued)… Rationale: Our school team has determined that

one way to ensure the postsecondary preparation of students leaving our school is to increase the rigor of our HS courses. The AP program provides a vehicle for accomplishing and measuring our goals

Theory of action: The focus on AP, will lead our school to ensure that our

prerequisite course are aligned with AP expectations. This, in turn, will provide students with more preparation and

interest in the particular subjects to increase their likelihood of enrolling in AP.

Counselors ensure that essentially all students are steered toward this academic pathway

The district will support this goal by creating an “AP fund” to pay for tests for students unable to pay


Approving the goals The school will be required to have the goals

approved by the district leadership and the board

The school will be required to publish the goals, targets, indicators, and results

NH DOE will have to approve the goals (or just the process by which the goals were established?) and the plans for determining targets and measures


Measuring the goals and targets Setting the yearly targets

Are the targets ambitious enough? Was the target-setting process appropriate and

inclusive? Measures

Are the measures/indicators appropriate for the purpose?

Are the measures of appropriate technical quality? E.g., is the measurement error larger than the yearly

target?


Support and Resources NH DOE will have to:

Provide guidelines on appropriate process and outcome indicators

Approve the goals and indicators process (but not necessarily the specific goals)

Approve the use of the specific measures Evaluate the results for each year

We will have to design a process for simplifying this evaluation

Integrate the results of levels one and two

All without any additional resources???


Considerations


Level One—can meet SB 180 requirements Level Two

Should it be required or left optional? Theory of action supports requirement “unfunded mandate” argues for optional If optional, it all rests on Level One

Combining evaluations from Level One and Two? Relatively equal weight? Level two as extra credit or tie-breaker?

Level Two can be a powerful vehicle and incentive for having schools engage in systematic school improvement efforts

Other considerations, concerns, questions, etc?

Commissioner’s Performance-Based Accountability Task Force: A Proposal for a Multi-level System Deb Wiswell & Scott Marion January 29, 2010.

Documents

level onea

state indicators

mathadditional indicators

state tests

designa school

mathnecap growth gapsreading

tested students

metricslevel twolocally