Purpose of Title III Improvement Webinars

How to Interpret and Use Standards of Learning (SOL) and ACCESS for ELLs® Data to Make

Instructional Decisions for English Learners

Virginia Department of EducationJanuary 31, 2012

Robert Fugate, LEP Assessment SpecialistJudy Radford, ESL Coordinator

2

Purpose of Title III Improvement WebinarsThese webinars will address• compliance with federal and state provisions for

the education of English learners (ELs);• understanding and using assessment data to

inform instruction for ELs;• VDOE resources available for reading and

mathematics instruction; and • developing and implementing a continuous

improvement model for Title III.

3

Purpose of WebinarUnderstanding the Elementary and Secondary Education Act of 1965 (ESEA) requirements for assessment of ELs is critical for compliant administrations of the Standards of Learning and ACCESS for ELLs tests to ELs. Additionally, understanding how to analyze assessment data can inform instruction and identify areas in need of improvement.

4

Standards of Learning (SOL) Assessments

Participation Requirements

5

Title I of the Elementary and Secondary Education Act (ESEA)

Academic Assessments

Section 1111(b)(3)(C)(v)(I)(II) and (III)

Federal Law

6

In general, all English learners (ELs) are expected to participate in SOL assessments, including reading, mathematics, science, history/social science, and writing.

Assessment ParticipationAssessment Participation

7

Certain exemptions on SOL assessments are available to ELs. Exemptions must be documented in the LEP Student Assessment Participation Plan and conveyed to the parent(s) or guardian(s). The table on the next slide displays available exemptions on SOL assessments.

Note: Available does not mean required.

Exemptions

8

Exemptions

9

Content Area Available Exemptions on SOL Assessments

Reading

LEP students who have attended school in the United States for less than 12 months may receive a one-time exemption for the SOL Reading test in grades 3 through 8. This guideline also applies to students arriving from Puerto Rico. No exemption is available for the End-of-Course Reading test.

Mathematics LEP students may NOT be exempted from the SOL Mathematics tests.

Science

Under the requirements of NCLB, all students, including LEP students, are required to participate in Science tests once at the elementary school level, once at the middle school level, and once at the high school level. However, LEP students in Virginia may be exempt from the grade 3 SOL Science test but must take the grade 5 SOL Science test. No other exemptions for the SOL Science tests are available to LEP students.

History/Social Science and

Writing

Under the Regulations Establishing Standards for Accrediting Public Schools in Virginia, 8 VAC 20-131-30.G, LEP students in grades 3 through 8 may exercise a one-time exemption from the SOL History and Social Science test and from the SOL Writing test (grade 5 or 8 only). No other exemptions for the SOL History and Social Science and Writing tests are available to LEP students.

Limited English Proficient Students: Guidelines for Participation in the Virginia Assessment Program, page 3

Reading Assessments available to ELs

10

• Grades 3 through 8• End-of-Course• Virginia Grade Level Alternative (VGLA)

Assessment• Grades 3-5: ACCESS for ELLs Overall Score

(Composite) of 1.0 to 3.5• Grades 6-8: ACCESS for ELLs Overall Score

(Composite) of 1.0 to 3.3

• Grades 3 through 8• End-of-Course

• Algebra I• Geometry• Algebra II

• Plain English Mathematics and Algebra I• Grades 3-5: ACCESS for ELLs Overall Score

(Composite) of 1.0 to 3.5• Grades 6-8: ACCESS for ELLs Overall Score

(Composite) of 1.0 to 3.3• Grades 9-12: ACCESS for ELLs Overall Score

(Composite) of 1.0 to 3.3

Mathematics Assessments

11

• Grades 3, 5, and 8• End-of-Course

• Earth Science• Biology• Chemistry

Science Assessments

12

• Grade 3 • Content Specific History

• Virginia Studies• US History I• US History II• Civics and Economics

• End-of-Course• Virginia & US History• World History I• World History II• World Geography

History & Social Science Assessments

13

• Grades 5 and 8• End-of-Course

Writing Assessments

14

AccommodationsAccommodations

15

Limited English Proficient Students: Guidelines for Participation in the Virginia Assessment Program, page 10

Limited English Proficient Students: Guidelines for Participation in the Virginia Assessment Program

http://www.doe.virginia.gov/testing/participation/index.shtml

Resource Document

16

SOL Test Examiner’s Manuals

http://www.doe.virginia.gov/testing/index.shtml

Resource Document

17

Test and Standards of Learning Resources

Test and Standards of Learning Resources

Test and Standards of Learning Resources

Test and Standards of Learning Resources

• Complete local test examiner training• Read the SOL Test Examiner’s Manual• Become familiar with test security guidelines• Become familiar with Appendix D, Selection of

Testing Accommodations for ELs• Become familiar with each EL’s LEP Student

Assessment Participation Plan• Become familiar with each EL’s testing

accommodations• Become familiar with the online test

environment

SOL Test Administration Best Practices

22

Standards of Learning Tests Score Reports

23

SOL Score ReportsSOL Score Reports

24

SOL Score ReportsSOL Score Reports

25

1. Test Name2. Reporting Categories3. Number Correct4. Number Possible5. Performance Results (Scaled Score and Performance Level)6. Performance Level Description

1

2

3 4 5 6

Non-Writing Test Sample Student Report

26

Writing TestSample Student Report

1. Test Name2. Reporting Categories3. Multiple Choice Column (MC) (Number Correct and Number Possible)4. Short Paper Column (SP) (Area, Points Earned, and Points Possible)5. Total Score (MC+SP) (Total Score and Total Score Possible)6. Overall Test Summary (Scaled Score and Performance Level)

2

543

1

6

Writing Test Sample Student Report

27

• SOL assessments scaled scores for reading and mathematics tests, grades 3-8• Pass/Advanced: 500 to 600• Pass/Proficient: 400 to 499• Fail: 0 to 399

http://www.doe.virginia.gov/testing/scoring/index.shtml

Analysis and Dissemination of SOL Test Scores to Teachers

28

• SOL assessments scaled scores for EOC reading and mathematics tests• Pass/Advanced: 500 to 600• Pass/Proficient: 400 to 499• Fail: 0 to 399

• Course specific performance descriptors

http://www.doe.virginia.gov/testing/scoring/index.shtml

Analysis and Dissemination of SOL Test Scores to Teachers

29

Virginia Grade Level Alternative (VGLA) Reading Assessment

30

VGLA Resources

31

VGLA Score Reports

32

1. Content Area2. Reporting Categories3. Points Earned4. Points Possible5. Performance Results (VGLA Score and Performance Level)6. Performance Level Description

Virginia Grade Level Assessment (VGLA) Score Ranges and Proficiency Levels Established by the Board of Education are available at http://www.doe.virginia.gov/testing/alternative_assessments/vgla_va_grade_level_alt/score_ranges.pdf .

5

2

4 63

1

VGLA Reading Sample Score Report

33

Analysis and Dissemination of VGLA Scores Ranges and Proficiency Levels

34

http://www.doe.virginia.gov/testing/alternative_assessments/vgla_va_grade_level_alt/score_ranges.pdf

School, School Division, and State Report Cards• Percentage of Students Passing/Tested/Not

Tested• Assessment-Based Other Academic

Indicators• AYP Objectives• Assessment Results at each Proficiency Level

by Subgroup

https://p1pe.doe.virginia.gov/reportcard/

School Report Cards

35

English Language Proficiency Assessment

Participation Requirements

36

Titles I and III of ESEAEnglish Language Proficiency Assessment

• Section 1111(b)(7); and • Sections 3113(b)(3)(D) and 3116(b)(3)(C)

Federal LawFederal Law

37

The ACCESS for ELLs test is the annual English language proficiency (ELP) assessment for ELs in kindergarten through grade 12.

Note: Formerly LEP students do not participate in English language proficiency testing.

ACCESS for ELLs®

38

The Alternate ACCESS for ELLs test is the annual ELP assessment for ELs in grades 1 though 12 with significant cognitive disabilities that prevent participation in the ACCESS for ELLs test.

Alternate ACCESS for ELLs™

39

This checklist is the annual ELP assessment is for ELs in kindergarten with significant cognitive disabilities that prevent participation in the ACCESS for ELLs test. (No Alternate ACCESS for ELLs test form is available for kindergarten.)

Virginia English Language Proficiency Checklist for LEP Students in

Kindergarten with Significant Cognitive Disabilities

40

Virginia English Language Proficiency Checklist for LEP Students in

Kindergarten through Grade 12 with Hearing and Visual Impairments

41

This checklist is the annual ELP assessment for ELs in kindergarten through grade 12 with hearing and visual impairments that prevent participation in the ACCESS for ELLs test.

• Online training available on the WIDA Web site (www.wida.us)

• Locally provided training

Test Examiner Training

42

ACCESS for ELLs® Score Reports

43

Proficiency level (1.0 through 6.0) for Accountability Purposes

Proficiency level (K1.0 through K6.0) for Instructional Purposes

Kindergarten Teacher Report

44

The Teacher Report contains data for

individual students.

Teacher Report

45

Language Domain Scale Scores and Proficiency Levels

Teacher Report

46

Composite Scale Scores and Proficiency Levels

Teacher Report

47

Student’s performance in Comprehension: Listening and Reading by WIDA English Language Proficiency Standards

Teacher Report

48

Student’s performance in Speaking Tasks by WIDA English Language Proficiency Standards

Teacher Report

49

Student’s performance in Writing Tasks by WIDA English Language Proficiency Standards

Teacher Report

50

Teacher Score Report• Scale scores• Confidence band• Proficiency levels• Raw scoresScore interpretation webinar at

www.wida.us

Analysis and Dissemination of ACCESS for ELLs Test Scores to Teachers

51

• Complete online test examiner training at the WIDA Web site and local training

• Read the Test Administrator’s Manual, Sections 1, 4, 5 and 6

• Become familiar with test administration times for each section of the test

• Become familiar with the Test Administrator’s Script

• Become familiar with test security guidelines• Become familiar with each EL’s tier placement• Become familiar with each EL’s testing

accommodations

ACCESS for ELLs Test Administration Best Practices

52

Standards of Learning (SOL) and

ACCESS for ELLs Data Analysis

53

• Guidelines for Determining K-12 World Class Instructional Design and Assessment (WIDA) English Language Proficiency (ELP) Levels

• Annual Measurable Achievement Objective (AMAO) 1 (progress) and AMAO 2 (proficiency) Calculations

A Refresher…

54

Guidelines for Determining K-12 WIDA ELP Levels

(Instructional Levels)WIDA ELP Levels ACCESS for ELLs Scores

Level 1 Composite Score of 1.0 through 1.9

Level 2 Composite Score of 2.0 through 2.9

Level 3 Composite Score of 3.0 through 3.9

Level 4 Composite Score of 4.0 through 4.9

Level 5 Composite Score of 5.0 through 6.0 and a Literacy Score less than 5.0

55

Guidelines for Determining K-12 WIDA ELP Levels

Formerly LEP (Monitored Levels)WIDA ELP Levels ACCESS for ELLs Scores

Level 6 Year 1(Formerly LEP)

For kindergarten students: Accountability Proficiency Score;Composite Score of 5.0 or above; and Literacy Score of 5.0 or above.

For students in Grades 1-12:Tier C; Composite Score of 5.0 or above; and Literacy Score 5.0 or above.

Level 6 Year 2(Formerly LEP)

*Note: Level 6, Year 1, and Level 6, Year 2, Formerly LEP students do not take the annual ELP assessment and are only included in the calculation for AMAO 3 (Adequate Yearly Progress) for reading/language arts and mathematics.

For kindergarten students: Accountability Proficiency Score;Composite Score of 5.0 or above; and Literacy Score of 5.0 or above.

For students in Grades 1-12: Tier C;Composite Score of 5.0 or above; and Literacy Score 5.0 or above.

AMAO 1 (progress) and AMAO 2 (proficiency) Calculations

The calculations for AMAO 1 (progress) and AMAO 2 (proficiency) will be determined based on results from the statewide administration of Assessing Comprehension and Communication in English State-to-State for English Language Learners (ACCESS for ELLs).

57

Students Included in the Calculations for AMAO 1 and AMAO 2

• AMAO 1 includes all ELLs that have two ACCESS for ELLs scores.

• AMAO 2 includes all ELLs that are reported in the 2011 Spring ACCESS for ELLs Test Results.

58

AMAO 1 (progress)• AMAO 1 (progress) is defined as those

students who meet one of the following criteria:

• Earning a composite proficiency score of 6 on the 2011 administration of the ACCESS for ELLs.

• Showing an increase in composite scale score that meets or exceeds targets by level according to the table in the next slide.

59

AMAO 1 (progress)Initial WIDA Level

2009-2010Minimum composite

scale score change to have demonstrated

progressLevel 1 29

Level 2 20

Level 3 14

Level 4 10

Level 5 3

60

AMAO 2 (proficiency)• AMAO 2 (proficiency) is defined as those

students who meet the following criteria:• For kindergarten students:

• Accountability Proficiency Level on ACCESS for ELLs; • Composite Score of 5.0 or above; and • Literacy Score of 5.0 or above.

• For students in Grades 1 - 12: • Tier C on ACCESS for ELLs;

• Composite Score of 5.0 or above; and • Literacy Score 5.0 or above.

61

Ask Yourselves…

• Have you analyzed your SOL data at the division, school, and student level?

• Have you analyzed your ACCESS for ELLs data at the division, school, and student level?

62

AMAO Division Level Data Analysis Template

Located on the ESL Web Site under Administrative Resources

http://www.doe.virginia.gov/instruction/esl/standards_resources/index.shtml

63

• Consolidate AMAO data by division and school level.

• Analyze AMAO data after retrieved from AYP Detail and Spring ACCESS for ELLs Test Results reports.

• Modify as needed by school division.

AMAO Division Level Template may be used to…

64

Where are the SOL and ACCESS for ELLs® data located?

• SOL and ACCESS for ELLs data are located in the Single Sign-on for Web Systems (SSWS)

• SOL data are located in the Adequate Yearly Progress (AYP) Application – AYP Detail Report

• ACCESS for ELLs data are located in the Title III (ESL) Application - Spring ACCESS for ELLs Test Results Report

65

66

• Log in to the Single Sign-on for Web Systems (SSWS)

• Click on Adequate Yearly Progress (AYP) application

• Click on Reports located on the right side of page

• Click on the AYP Detail Report

Steps for AccessingSOL Data

Sample AYP Detail ReportDivision Level

3 4 5 6 7 1 2 8

68

• Log in to the Single Sign-on for Web Systems (SSWS)

• Click on Title III (ESL) Application

• Click on Reports located on the right side of page

• Click on Spring ACCESS for ELLs Test Results

Steps for Accessing ACCESS for ELLs Data

- 69

-

Sample Spring ACCESS for ELLs

Test Results

1 2

Sample AMAO 1 (Progress) DataSchool

Division: Total EL Enrollment Total Number Of ELs TestedTotal Number of ELs

Meeting AMAO 1 (Progress)

AMAO 1 Percentage Rate Met AMAO 1 Target

2010 2011 2010 2011 2010 2011 2010 20112010 (65%)

2011 (66%)

Division Level

549 526 549 526 420 415 77 79 Yes YesSchool Level

Elementary

School 1 88 95 88 95 67 70 76 74 Yes Yes

School 2 88 86 88 86 67 72 76 84 Yes Yes

School 3 89 99 89 99 70 75 79 76 Yes Yes

Middle

School 1 89 44 89 44 70 30 79 68 Yes Yes

School 2 59 50 59 50 51 50 86 100 Yes Yes

School 3 61 68 61 68 38 55 62 81 No Yes

High

School 1 18 29 18 29 11 18 61 62 No No

School 2 36 30 36 30 27 25 75 83 Yes YesSchool 3 21 25 21 25 19 20 90 80 Yes Yes

1 2

School Division:

Total EL Enrollment Total Number Of ELs Tested

Total Number of ELs Meeting AMAO 2

(Proficiency)AMAO 2 Percentage Rate Met AMAO 2 Target (Yes

or No)

2010 2011 2010 2011 2010 2011 2010 2011 2010 (16%) 2011 (17%) Division

Level 170 185 170 185 38 37 22 20 Yes Yes

School Level

Elementary

School 1 30 38 30 38 8 4 27 11 Yes No

School 2 25 22 25 22 5 2 20 9 Yes No

School 3 21 20 21 20 6 4 29 20 Yes Yes

Middle

School 1 15 17 15 17 3 5 20 29 Yes Yes

School 2 18 20 18 20 5 3 28 15 Yes No

School 3 29 22 29 22 3 5 10 23 No Yes

High

School 1 15 18 15 18 4 2 27 11 Yes No

School 2 8 15 8 15 2 4 25 27 Yes YesSchool 3 9 13 9 13 2 8 22 62 Yes Yes

Sample AMAO 2 (Proficiency) Data1 2

Sample AMAO1 (Progress) Data (2011-2012 results based upon the spring 2010 ACCESS data)

School name Total number of students tested

Total number of students met progress

AMAO 1 (progress) percentage rate

Met Target (66%)

Elementary Schools

School 1 48 35 73% YesSchool 2 28 20 71% YesSchool 3 49 25 51% NoSchool 4 29 22 76% Yes

Middle SchoolsSchool 1 23 10 43% NoSchool 2 35 15 43% No

High SchoolsSchool 1 39 35 90% Yes

School 2 26 18 69% Yes

Division AMAO 1 (progress) Results

277 180 65% No

What is reflected by the data at the middle school?

How could the elementary school data be used to determine strengths and weaknesses at this level?

How could elementary and high school data be used to develop interventions for the middle school?How could

data be used to determine the effectiveness of the English Program at all levels?

How can ACCESS for ELLs test scores be used to improve instruction?

• Review language domains and standards’ scores for strengths and areas in need of improvement

• Develop instructional strategies to address areas in need of improvement

• Work cooperatively with content area teachers to provide instruction to address areas in need of improvement

73

The Teacher Report contains individual data

for one student.

Sample Teacher Report

74

How can SOL test scores be used to improve instruction?• Review reporting categories’ scores for

strengths and areas in need of improvement

• Develop instructional strategies to address areas in need of improvement

• Work cooperatively with content area teachers to provide instruction to address areas in need of improvement.

75

1. Test Name2. Reporting Categories3. Number Correct4. Number Possible5. Performance Results (Scaled Score and Performance Level)6. Performance Level Description

1

2

3 4 5 6

Non-Writing Test Sample Student Report

76

A Few Best Practices for Analyzing Data

1. Determine why and what data needs to be analyzed;

2. Identify the most appropriate and knowledgeable participants to be a part of the data team;

3. Know where the data is located;4. Take steps to minimize the time in collecting and

analyzing the data; 5. Take advantage of what is working; and6. Be prepared to make recommendations for “next

steps” once data results have been analyzed.

77

Questions?

78

Contact InformationRobert FugateLEP Assessment SpecialistVirginia Department of Educationstudent_assessment@doe.virginia.gov(804) 225-2102

Judy RadfordESL CoordinatorVirginia Department of EducationJudy.Radford@doe.virginia.gov(804) 786-1962

79