1 Approaches to Implementing the 2% Cap for Adequate Yearly Progress NCES Summer Data Conference Washington, DC July 2008.

1

Approaches to Implementing the 2% Cap for Adequate Yearly Progress

NCES Summer Data ConferenceWashington, DCJuly 2008

2

Nancy Stevens [email protected] of Assessment, Accountability, and Data QualityTexas Education Agency

Li-Chin Wu [email protected] of Performance ReportingTexas Education Agency

http://www.tea.state.tx.us/ayp/2008

3

Texas Before NCLBState Developed Alternative Assessment instructional level rather than enrolled grade level ARD committee set level and student performance standard about 7-8% of students

Locally Determined Alternate Assessments locally developed or selected tests fewer than 1% of students

4

Texas After NCLB All students included in state assessment program

Texas Assessment of Knowledge and Skills (TAKS) Performance of all students evaluated against grade-

level achievement standards Federal cap limit on use of proficient results based

on alternate or modified achievement standards in AYP performance measures: 1% alternate achievement standards (TAKS-Alt) 2% modified achievement standards (TAKS-M)

5

Texas After NCLB (cont.) Student performance will be a greater factor

than the caps The caps will apply to a very small number of

all students tested 1% and 2% caps represent very high

standards

6

Goals for 2% Cap Students: promote appropriate assessment decisions for

students with disabilities

Statute: meet statutory requirements and intent

Validity: minimize unintended consequences

Equity: distribute “exceeders” and “keepers” across campuses fairly

Simplicity: understandable easy to replicate by school districts

Resources: staff and time

7

Incorporating School District Input During Policy Development Process

State solicits feedback on options State selects approach that more closely reflects local

decisions Before AYP Determinations

Districts set campus caps or prioritize campuses During AYP Determinations

Districts identify individual students whose proficient test scores are retained if cap 2% cap exceeded

8

Overall Design 1% cap

Did not have to be same approach as 2% cap Fewer options considered

By random assignment By disability category

9

Overall Design for 2% Cap One district-wide pool

Rules for selecting students from district pool Separate pools for each campus

Rules for assigning campus caps or ranking campuses

Rules for selecting students from campus pools

10

Campus Pools Option 1: Campus Cap Determine cap for each campus based on current

and/or historical proportion of district students: receiving special education services tested on alternate assessments proficient on alternate assessments

Rules for selecting “keepers/exceeders” if campus exceeds cap

Rules for allocating extra “spaces” if campus does not use all allowed under campus cap

11

Campus Pools Option 1:Campus Cap (cont.)

Pros Reflects local policy decisions Potentially rewards campuses that historically

and appropriately serve high number of students with disabilities

12

Campus Pools Option 1:Campus Cap (cont.)

Cons Potentially rewards campuses that over-

identify students for alternate assessment May encourage concentrating programs on

specific campuses or discourage mainstreaming in order to maintain campus cap

13

Campus Pools Option 1:Campus Cap (cont.)

Cons Slight variations in testing from year to year

may result in changes to campus cap

May be difficult to implement

Could result in the district missing AYP

14

Campus Pools Option 2: Campus Ranking Rank campuses Select proficient scores from highest ranked

campus first, going down the list until district cap limit is reached

Rules for selecting students from campus pool

15

Campus Pools Option 2:Campus Ranking (cont.)

Example: Strategic Campus Ranking Rank campuses strategically:

Highest stage identification for SIP Title I missed AYP in prior year Title I campus

16

Campus Pools Option 2:Campus Ranking (cont.)

Example: Strategic Campus Ranking (cont.)

Pros Balance perceived inequities in AYP

interventions (Title I vs. non-Title I campuses)

Simple to understand

17

Campus Pools Option 2:Campus Ranking (cont.)

Example: Strategic Campus Ranking (cont.)

Cons Reward campuses with performance problems Not consistent with intent of NCLB May not help top-ranked campuses Could result in the district missing AYP

18

Selection of Students for 2% Cap By Random Assignment By Test Score By Grade Level By Maximum Benefit

19

Selection of Students (cont.) District pool or separate campus pools

Significantly different outcomes Single or combined selection criteria

First or primary sort is greater factor in determining outcomes

Final unique sort as a tie-breaker

20

Selection of StudentsBy Random Assignment Students randomly selected up to the cap limit District or campus pools Does not need tie-breaker Can be used as final tie-breaker with other

methods

21

Selection of StudentsBy Random Assignment (cont.)

Pros Simple to understand Simple to implement for most districts Impartial over time No unintended policy consequences (cannot

be manipulated)

22

Selection of StudentsBy Random Assignment (cont.)

Cons Cannot be replicated by districts May not appear to be fair in any one year Does not provide any incentive – disconnect

between campus behavior and outcomes

23

Selection of Students By Test Performance Students sorted from lowest to highest test

score and “keepers” selected up to the cap limit

District or campus pools Can be used in conjunction with other criteria Needs a final tie-breaker

24

Selection of Students By Test Performance (cont.)Pros Encourages testing higher performing

students on the regular test Simple to understand Can be replicated by districts Simple to implement Most similar to method used in Texas with

SDAA/LDAA

25

Selection of Students By Test Performance (cont.)

Cons If implemented at the district level:

may be perceived as punitive toward campuses with strong instructional programs

may not result in fair distribution of “keepers” and “exceeders” across campuses

26

Selection of StudentsBy Grade Level Students sorted from highest to lowest grade

and “keepers” selected up to the cap limit District-level approach Needs to be used with at least one more

criteria

27

Selection of StudentsBy Grade Level (cont.)Pros Provides strong incentive for elementary schools to

focus instruction on maintaining grade-level proficiency and testing on regular grade-level assessment

Rewards high schools that have successfully accelerated instruction so that students previously instructed and tested below grade level are meeting grade-level modified academic achievement standards

28

Selection of StudentsBy Grade Level (cont.)Pros High schools, which are overrepresented among

campuses not meeting AYP, are least adversely affected by the cap

Simple to understand Can be replicated by districts Simple to implement

29

Selection of StudentsBy Grade Level (cont.)Cons Has appearance of being unfair to elementary

schools AYP results for elementary schools may be

adversely affected disproportionately Positive instructional incentives may be short-term May have unintended consequences long-term

30

Selection of StudentsBy Maximum Benefit Select proficient results from each campus

that will result in the maximum benefit for the campus Select number and type of students (student

groups) needed for the campus to meet AYP Campus-level approach Criteria for each campus based on need

31

Selection of StudentsBy Maximum Benefit (cont.)

Pros Potentially minimizes the number of

campuses that miss AYP solely due to selection criteria for the 2% cap

Uses state data processing capacity to select students that districts and campuses would likely select if 2% cap implemented locally

32

Selection of StudentsBy Maximum Benefit (cont.)

Cons Students included in the 2% cap will be

selected from student groups that do not meet the AYP standards

Selection based on campus need could result in the district missing AYP

33

Texas AYPCombination Method Campus Ranking

By campus type High School Combined Elementary/Secondary School Middle/Junior High School Elementary School

By grade (high to low) By percent special education (high to low)

34

Texas AYPCombination Method (cont.) Campus Ranking

Based on fall enrollment data District opportunity to modify campus ranking

35

Texas AYPCombination Method (cont.) Student Selection in 3 Stages

First by maximum benefit for campus (campus pool) From highest to lowest ranked campus Select students needed for campus to meet AYP Skip campuses that already meet AYP or will not

meet AYP for subject

36

Texas AYPCombination Method (cont.) Student Selection in 3 Stages (cont.)

Second by maximum benefit for district(district pool)

Third by random selection (district pool)

37

Texas AYPCombination Method (cont.) Pros

Campus ranking by grade level has many of the advantages of selecting students by grade level Provides incentive for elementary schools to focus

instruction on maintaining grade-level proficiency Rewards high schools that have successfully

accelerated instruction

38

Texas AYPCombination Method (cont.) Pros

District input before AYP determinations does not interfere with processing timelines Supports local policy decisions on selection of

appropriate tests for students with disabilities

39

Texas AYPCombination Method (cont.) Pros

Student selection uses state data processing capacity to provide maximum benefit to campuses in implementing 2% cap

Second selection for maximum benefit to district removes potential disadvantages of processes that focus on campuses

40

Texas AYPCombination Method (cont.) Cons

Benefits of ranking campuses by grade level may be short-term

District input into campus ranking resource intensive for little gain and potentially negates benefits of ranking by grade level

Disadvantages of selection by maximum benefit – students disproportionately selected from student groups that do not meet AYP

41

Example District - 2% Cap Scenario F: Campus 1 meets AYP *** Campus 2 missed AYP *** *** Campus 3 missed AYP *** Campus 4 meets AYP *** District missed AYP ***

Seven Student Groups: A - All B - African American H - Hispanic W - White E - Economically Disadvantaged S - Special Education L - LEP