ANDREW PARR, WASHINGTON STATE BOARD OF EDUCATION GREG LOBDELL, CENTER FOR EDUCATIONAL EFFECTIVENESS NATIONAL CONFERENCE ON STUDENT ASSESSMENT JUNE 24, 2015 Achievement of Former-ELL Students and their Impact on the Washington Achievement Index Washington State Board of Education
57
Embed
ANDREW PARR, WASHINGTON STATE BOARD OF EDUCATION GREG LOBDELL, CENTER FOR EDUCATIONAL EFFECTIVENESS NATIONAL CONFERENCE ON STUDENT ASSESSMENT JUNE 24,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
A N D R E W PA R R , WA S H I N G T O N S TAT E B O A R D O F E D U C AT I O N
G R E G L O B D E L L , C E N T E R F O R E D U C AT I O N A L E F F E C T I V E N E S S
N AT I O N A L C O N F E R E N C E O N S T U D E N T A S S E S S M E N T
J U N E 2 4 , 2 0 1 5
Achievement of Former-ELL Students and their Impact on the Washington Achievement Index
Washington State Board of Education
Washington State Board of Education
Presentation Roadmap
Former ELL subgroup performance
Impact of the Former ELL subgroup on the Index
Preliminary findings from new data
English Language Learners (ELLs) and Former ELLs are the fastest growing subgroup in Washington - we will continue this research to
inform all who are working with these students.
Washington State Board of Education
Key Questions
Joint effort between the SBE and Greg Lobdell at the Center for Educational Effectiveness
1. What does the academic performance of the Former ELL subgroup look like through the Index?
2. How does the Former ELL subgroup impact the Index school ratings?
3. Are the earlier results being confirmed by the more recent analyses?
Washington State Board of Education
Achievement Index as a Data Source
Building is the unit of analysisContinuously enrolled students onlyPerformance Indicators:
Proficiency: reading, math, writing, science percent of students meeting standard
Median Student Growth Percentiles: reading and math Graduation Rate: 5-year Adjusted Cohort method
Washington State Board of Education
Washington Comprehensive Assessment Program
Through 2013-14 Grades 3-8 Measures of Student Progress in reading, math, science,
and writing Grade 10 high School Proficiency Exams in reading and writing End of Course assessments (Math 1, Math 2, & Biology) 200 to 600 point scale – 400 cut-score = Meets Standard
Beginning in 2014-15 Smarter Balanced Assessments Grades 3-8, ELA , math, and science Grades 5 & 8 Grade 10 HS SBAC ELA as a graduation requirement Grade 11 ELA, math, and Biology EOC
Washington State Board of Education
Achievement Index
School Index rating: proficiency and growth for elementary and middle schools Proficiency, growth, and CCR for high schools
Washington State Board of Education
Achievement Index
Washington State Board of Education 8
AI – Growth Measures
Washington State Board of Education
9
Washington State Board of Education
Disaggregated Subgroup Performance
Washington State Board of Education
DATA FROM THE INDEX
Former ELL Students
Washington State Board of Education
Former ELL Group Outperforms the ALL Group
60.2
%
63.9
%
63.8
%
20.7
%
23.2
%
26.9
%
61.9
%
67.5
%
71.4
%
0%10%20%30%40%50%60%70%80%90%
100%
2011 2012 2013 2011 2012 2013 2011 2012 2013
All Students ELL Students Former ELL Students
Math Proficiency: All Grades
Washington State Board of Education
Former ELL Math for Elementary Schools
Former ELL subgroup outperforms the All Students group
62.2
%
64.8
%
64.2
%
24.9
%
26.2
%
29.2
%
71.0
%
72.2
%
75.8
%
0%10%20%30%40%50%60%70%80%90%
100%
2011 2012 2013 2011 2012 2013 2011 2012 2013
All Students ELL Students Former ELL Students
Elementary Math
Washington State Board of Education
Former ELL Math for Middle Schools
Similar performance between the Former ELL group and the All Students group
55.6
%
58.6
%
59.1
%
12.1
%
14.9
%
16.7
%
51.8
%
55.9
%
58.1
%
0%10%20%30%40%50%60%70%80%90%
100%
2011 2012 2013 2011 2012 2013 2011 2012 2013
All Students ELL Students Former ELL Students
Middle School Math
Washington State Board of Education
Former ELL Math for High Schools
The Former ELL group underperforms compared to the All Students groups
66.8
% 75.4
%
79.1
%
21.1
%
19.0
%
37.0
%
55.3
% 70.5
%
76.9
%
0%10%20%30%40%50%60%70%80%90%
100%
2011 2012 2013 2011 2012 2013 2011 2012 2013
All Students ELL Students Former ELL Students
High School Math
Washington State Board of Education
Comparison Summary
-60%
-40%
-20%
0%
20%
40%
60%
Grad
e 3
Grad
e 4
Grad
e 5
Grad
e 6
Grad
e 7
Grad
e 8
Grad
e 10
Grad
e 3
Grad
e 4
Grad
e 5
Grad
e 6
Grad
e 7
Grad
e 8
Grad
e 10
Reading Math
3-Year Proficiency Gap: Former-ELL compared to ALL
Washington State Board of Education
Reading SGP Comparison
Elementary Middle High30
40
50
60
70
50.3 49.8 49.3
52.8 53.1
47.3
Reading3-Year Median Growth Percentile
All Students
Former ELLs
School Level
Me
dia
n S
GP
Washington State Board of Education
Math SGP Comparison
Elementary Middle High30
40
50
60
70
52.050.1
47.5
55.953.4
48.0
Math3-Year Median Growth Percentile
All Students
Former ELLs
School Level
Me
dia
n S
GP
High proficiency and high growth
Moderate proficiency and
high growth
Low proficiency and low growth
Washington State Board of Education
5-Year Graduation Rate
Former ELL Schools All Schools0
20
40
60
80
100
5-Year Graduation Rate
All Students
Former ELLs
Per
cen
t G
rad
uat
ing
Washington State Board of Education
IMPACT TO THE ACHIEVEMENT INDEX
Former ELL Students
Washington State Board of Education
How Many Schools Were Impacted?
Most schools benefited when the Former ELL group is included in the Index calculations.
Schools School Ratings
Year Rated Former ELL
No Change
Increased Decreased
2011 1817 553 8 513 32
2012 1861 753 6 717 30
2013 1886 819 9 770 37
Comp AI 1801 866 4 826 24
Washington State Board of Education
How Were the School Ratings Impacted?
A small average gain when the Former ELL group is used in the Index calculations.
2011 2012 2013 Comp AI
Average 1 2 3 4-0.25
0.25
0.75
1.25
1.75
Impact to the Index Ratings
Ind
ex
Ra
tin
g
Washington State Board of Education
How Were the Indicators Impacted?
Modest average rating point gains for all indicators when the Former ELL group in included.
Proficiency Growth Graduation
Average 1 2 3
-0.75
-0.25
0.25
0.75
1.25
1.75
Impact to the Indicators (2013 AI)
Ind
ica
tor
Ra
tin
g
Washington State Board of Education
Impact to Indicators by School Level
Elementary Schools Middle Schools
Proficiency Growth
Average 1 2
-0.75
-0.25
0.25
0.75
1.25
1.75
Impact to the Indicators (2013)
Ind
icat
or
Rat
ing
Proficiency Growth
Average 1 2
-0.75
-0.25
0.25
0.75
1.25
1.75
Impact to the Indicators (2013)
Ind
icat
or
Rat
ing
Washington State Board of Education
New Disaggregation Capacity
Historically, in the achievement records each student was ELL=Yes or No
In 2014 we have, for each student: Never-ELL ELL Transitioned within 2 years Transition more than 2 year ago
You probably have a pretty good guess as to how the groups’ performance compare?
3rd Grade Reading
ALL
Curre
nt E
LL
Never
ELL
Form
er E
LL ≤
2
Form
er E
LL >
20
10
20
30
40
50
60
70
80
90
100
Roadmap Districts3rd Grade Reading
Pe
rce
nt
Me
eti
ng
Sta
nd
ard
6th Grade Reading
ALL
Curre
nt E
LL
Never
ELL
Form
er E
LL ≤
2
Form
er E
LL >
20
10
20
30
40
50
60
70
80
90
100
Roadmap Districts6th Grade Reading
Pe
rce
nt
Me
eti
ng
Sta
nd
ard
10th Grade Reading
ALL
Curre
nt E
LL
Never
ELL
Form
er E
LL ≤
2
Form
er E
LL >
20
10
20
30
40
50
60
70
80
90
100
Roadmap Districts10th Grade Reading
Pe
rce
nt
Me
eti
ng
Sta
nd
ard
Washington State Board of Education
New Data Received in Late May 2015
2012-13 School Year Assessment Records Student Demographics SGP Records School Enrollment Records Special Programs Records WELPA Records ELL Status File
Data to still acquire Language level at entry TBIP program type
Student-levelPre-SBAC Field Test
Eliminates potential biasing from SBAC Field Test
Voluntary participation 1/3 of students/schools
Washington State Board of Education
Years in Bilingual Education
The most common home languages in Washington
On average: Asian students spend
fewer years in Bilingual Education
Hispanic/Latinos are among the highest on this measure
Former ELL=Yes, SWD=No, FRL=Yes, & Home Language=Spanish
Reading SS Math SS
Math EOC1 SS
Reading % Proficient
Math % Proficient
Not Long-Term ELL 414.9 421.3 414.2 82.4% 73.5%
Long-Term ELL 399.9 391.4 385.1 63.5% 52.0%
Reading SS Math SS
Math EOC1 SS
Reading % Proficient
Math % Proficient
Not Long-Term ELL 409.6 408.5 400.0 76.2% 64.3%
Long-Term ELL 398.8 388.5 381.8 61.1% 48.5%
Washington State Board of Education
Former ELL – Never ELL Scaled Score Performance Gap
In reading, the Former ELL group outperforms the Never ELL group up to the 5th grade and underperforms after that. See the higher performance in all grades for math.
Gra
de 3
Gra
de 4
Gra
de 5
Gra
de 6
Gra
de 7
Gra
de 8
Gra
de 1
0
Gra
de 3
Gra
de 4
Gra
de 5
Gra
de 6
Gra
de 7
Gra
de 8
Gra
de 1
0
-10
-5
0
5
10
15
20
2013 Scaled Score Performance Gap (MSP/HSPE) Former ELL vs. Never ELL
Scal
ed S
core
Poi
nts
Reading Math
Washington State Board of Education
Former ELL – Never ELLProficiency Performance Gap
At all grade levels, the proficiency rate in math for the Former ELL group exceeds the rate for the Never ELL group. See the lower proficiency rates in reading beginning in the 8th grade.
Gra
de 3
Gra
de 4
Gra
de 5
Gra
de 6
Gra
de 7
Gra
de 8
Gra
de 1
0
Gra
de 3
Gra
de 4
Gra
de 5
Gra
de 6
Gra
de 7
Gra
de 8
Gra
de 1
0
-5%
0%
5%
10%
15%
20%
2013 Proficiency Performance Gap (MSP/HSPE)Former vs. Never ELL
Perc
enta
ge P
oint
s
Reading Math
Washington State Board of Education
Scaled Score Gap Proficiency Gap
Former ELL – Never ELL Math EOC 1 Performance Gaps
Gra
de 7
Gra
de 8
Gra
de 9
Gra
de 1
0
-4.0
-2.0
0.0
2.0
4.0
6.0
8.0
2013 Scaled Score Gap Math EOC 1Former ELL vs. Never ELL
Scal
ed S
core
Poi
nts
Gra
de 7
Gra
de 8
Gra
de 9
Gra
de 1
0
-4%
-2%
0%
2%
4%
6%
2013 Proficiency Gap Math EOC 1Former ELL vs. Never ELL
Perc
enta
ge P
oint
s
Washington State Board of Education
FELL 0 FELL 1 FELL 2 FELL 3 All FELL Never ELL350
375
400
425
450
3rd Grade Reading and Math Scaled Score AveragesFormer ELL vs. Never ELL
Read SS Math SS
ELL Status
Scal
ed S
core
3rd Grade Performance
Controlling for SWD (No) and FRL (Yes) status
All Former ELL groups perform as well or better than the Never ELL group
FELL 0 FELL 1 FELL 2 FELL 3 All FELL Never ELL0%
20%
40%
60%
80%
100%
3rd Grade Reading and Math Proficiency AveragesFormer ELL vs. Never ELL
READ_PRO MATH_PRO
ELL Status
Perc
ent M
eetin
g St
anda
rd
Washington State Board of Education
4th Grade Performance
The Former ELL groups outperform the Never ELL group in Math
All Former ELL groups perform as well or better than the Never ELL group
FELL 0 FELL 1 FELL 2 FELL 3 FELL 4 All FELL Never ELL375
400
425
450
475
4th Grade Reading and Math Scaled Score AveragesFormer ELL vs. Never ELL
Read SS Math SS
ELL Status
Scal
ed S
core
FELL 0 FELL 1 FELL 2 FELL 3 FELL 4 All FELL Never ELL
0%
20%
40%
60%
80%
100%
4th Grade Reading and Math Proficiency AveragesFormer ELL vs. Never ELL
READ_PRO MATH_PRO
ELL Status
Perc
ent M
eetin
g St
anda
rd
Washington State Board of Education
5th Grade Performance
Three years after reclassifying, Former ELL students outperform the Never ELL group.
Controlling for SWD (No) and FRL (Yes) status
FELL 0 FELL 1 FELL 2 FELL 3 FELL 4 FELL 5 All FELL Never ELL
350
375
400
425
450
5th Grade Reading and Math Scaled Score AveragesFormer ELL vs. Never ELL
Read SS Math SS
ELL Status
Scal
ed S
core
FELL 0 FELL 1 FELL 2 FELL 3 FELL 4 FELL 5 All FELL Never ELL
0%
20%
40%
60%
80%
100%
5th Grade Reading and Math Proficiency AveragesFormer ELL vs. Never ELL
READ_PRO MATH_PRO
ELL Status
Perc
ent M
eetin
g St
anda
rd
Washington State Board of Education
6th Grade Performance
Controlling for SWD (No) and FRL (Yes) status
In the third year after reclassification, Former ELL student groups outperform the Never ELL student group.
FELL 0 FELL 1 FELL 2 FELL 3 FELL 4 FELL 5 FELL 6 All FELL Never ELL
350
375
400
425
450
6th Grade Reading and Math Scaled Score AveragesFormer ELL vs. Never ELL
Allows Former ELL students to be included in the ELL student group for up to two years after reclassification.
The underlying assumption is that Former ELLs for two years are expected to perform similarly to but higher than Never ELL students (which is not necessarily true in all cases)
Our work shows: many Former ELL student groups do not perform similarly to Never ELL students for many
years and some never reach the levels of the Never ELL group
Washington is looking for new ELL accountability measures
How do your states include Former ELL students in accountability?
Washington State Board of Education
TBIP Task Force
Transitional Bilingual Instructional Program (TBIP) accountability
Just finished Phase I work with recommendations forthcoming
Expecting recommendations to enhance district- and school-level TBIP effectiveness Predictive analytics for ELLs progressing toward reclassification Monitoring ELL and Former ELL Learning Assistance Program (LAP)
Washington State Board of Education
Policy Implications
Schools with reportable Former ELL groups are demographically different and those differences are reflected in student outcome
measures.
Do you think current funding and staffing policy effectively addresses these differences?
Do you think current PD policy is adequate to address the learning needs of this diverse group?
Do you think that expanded learning opportunities would benefit this diverse group?
Other thoughts?
Washington State Board of Education
Ongoing Research
Academic performance disaggregated deeper by Primary language Years in (and out of) TBIP program TBIP model Socioeconomic status Other student characteristics (mobility, migrant, etc.)
Potential or Potential ImpactsTBIP accountabilityWashington Achievement AwardsGap reduction work Index revisions