ELPA21 2018 Score Interpretation Guide 1 Iowa Department of Education Disclaimer: The contents of this document were developed under a grant from the U.S. Department of Education. However, those contents do not necessarily represent the policy of the U.S. Department of Education and you should not assume endorsement by the Federal Government Score Interpretation Guide 2017-2018 Summative Assessment Grades K-12
30
Embed
Iowa Department of Education - Iowa ELPA21 Portal · 2018-03-13 · ELPA21 2018 Score Interpretation Guide 1 Iowa Department of Education Guide Disclaimer: The contents of this document
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
ELPA21 2018 Score Interpretation Guide
1
Iowa Department of Education
Disclaimer: The contents of this document were developed under a grant from the U.S. Department of Education. However, those contents do not
necessarily represent the policy of the U.S. Department of Education and you should not assume endorsement by the Federal Government
Overall Score Evaluating program outcomes, computing accountability metrics, calculating growth
Reporting Scores
The Domain Scores
A numeric three-digit scale score describes performance on each of the four domains of listening,
reading, speaking and writing. A measure of variability, termed the standard error of measurement
(SEM), describes the variability around each numeric score. Each score is classified into one of five
performance levels, where each level corresponds to a text descriptor stating what students in each
level know and can do. The cut scores defining each level are listed in Appendix A, with the
corresponding descriptors (ALDs) documented in Appendix C.
Although performance on a single domain cannot determine English language proficiency, the
question “what level on each domain is sufficient?” begs to be addressed. Corresponding to the
policy definition, level 4 on each domain represents the English language knowledge, skills and
abilities that are required to interact with and engage in grade-level content instruction at the same
level as non-ELs and is referred to as the “assessment target” for each domain. Once the assessment
target is met on all non-exempt domains (e.g., a student scores a four on each domain, or “4444” )
ELPA21 recommends the student be eligible for reclassification.
The Profiles
The ELP Standards maintain that proficiency can be achieved in multiple ways, and may look
different for individual students. Students develop skill in each domain at different rates and may
exhibit some skills of a domain at higher levels and still struggle with other skills at a lower level.
As such, ELPA21 recognizes the possibility of other profiles of skills across the domains of
listening, reading, speaking and writing that may describe proficiency in addition to the “4444”
profile.
Profiles may be expressed as 1) four numbers representing the level of proficiency on each of the
four domains, or as 2) rules that summarize a common pattern for sets of profiles. As an example,
hypothetical scores could be expressed as a profile of “4343” or by the rule “no domain score falls
below Level 3”. Profiles are based on the domain scores of proficient students identified during the
ELPA21 2018 Score Interpretation Guide
6
ELPA21 contrasting group study, were refined and vetted by a panel of member state EL experts,
and were refined and vetted again during standard setting.
Relying on student profiles instead of an overall composite score as is traditionally done benefits
educators in a couple ways. A profile provides more instructional information about students who
may have the same overall score, but differ in skills and needs. Proficiency is a function of the
domains, not of the overall scores with domains combined. It also highlights the relationship
between the domains in a way that an overall score would not.
Profiles are used to differentiate “Proficient” students from those who are “Progressing” or
“Emerging”. Table 2 describes how the different profiles are expressed and used to determine
proficiency on ELPA21.
Table 2. Profiles of Proficiency
Rules Profiles English Proficiency Determination
A profile of 4s and 5s meets assessment targets and indicates overall proficiency
5555 5554 5544 5444 4444 555E 554E 544E 444E
Proficient
A profile with one or more domain scores above Level 2 that does not meet the requirements to be Proficient
Any other combination: At least 1 domain >=3 AND At least 1 domain <=3
Progressing
A profile of 1s and 2s indicates an “Emerging” level of proficiency.
2222 2221 2211 2111 1111 222E 221E 211E 111E
Emerging
Note: Order of domains is irrelevant on the table. “E” indicates an exempt test.
The English Proficiency Determination
Using the profiles, different combinations of skills and abilities across the domains are deemed as
“Proficient”, “Progressing” or “Emerging” (see Table 9). The English Proficiency Determination
identifies ELs whose language skills enable full participation in grade-level academic contexts.
ELPA21 2018 Score Interpretation Guide
7
Table 3: Policy Definition for the English Proficiency Determination
Proficient Students are Proficient when they attain a level or English language skill necessary to independently produce, interpret, collaborate on, and succeed in grade-level content-related academic tasks in English. This is indicated on ELPA21 by attaining a profile of Level 4 or higher in all domains. Once Proficient on ElPA21, students can be considered for reclassification.
Progressing Students are progressing when, with support they approach a level of English language skill necessary to produce, interpret, and collaborate, on grade-level content-related academic tasks in English. This is indicated on ELPA21 by attaining a profile with one or more domain scores above Level 2 that does not meet the requirements to be Proficient. Students scoring Progressing on ELPA21 are eligible for ongoing program support.
Emerging Students are emerging when they have not yet attained a level of English language skill necessary to produce, interpret, and collaborate on grade-level content- related academic tasks in English. This is indicated on ELPA21 by attaining a profile of Levels 1 and 2 in all four domains. Students scoring Emerging on ELPA21 are eligible for ongoing program support.
Note: Each definition consists of three elements: a learning expectation (first sentence), an operational definition
(second sentence), and a policy impact statement (third sentence).
Defining proficiency as profiles of skills rather than as an average or a sum across those skills
allows for richer, more nuanced and flexible distinctions between proficient and not proficient
students. For example, a profile of “4131” is more useful and provides educators with more
information than does an overall composite score of 360. The profile, while not diagnostic, does
show clear strength and weakness.
Note that summative assessments are developed by grade-band, but scores are reported at grade-
level, and as a result, different profiles may indicate proficiency in different grades.
Because of this, students at different grades within the same grade-band who earn identical scores
may fall into different achievement levels and receive different English proficiency determinations.
This is because the expectation (e.g., cut score, or standard for proficiency) increases for each
grade. For example, a student receiving a 570 in listening, a 585 in reading, a 600 in speaking and a
590 in writing may be Progressing in 8th grade (a profile of 4343), but Proficient in 6th grade
(profile of 5454).
The Growth Indicators ELPA21 provides two cross-domain measures of language ability, 1) the Overall Score and 2) the
Comprehension Score. They are not composites in the traditional sense, as weighted aggregates of
separate domain scores, but is the total test scale score based on all items administered. The Overall
Score is computed from a single score estimation for each student including items from all four
domains. The Comprehension Score is computed from a single score estimation for each student
ELPA21 2018 Score Interpretation Guide
8
using all reading and listening items. Because these scores are computed differently than the
domain scores, they are reported on a different reporting scale. Both are reported as a four-digit
number.
The Growth Indicators are designed for program evaluation, (i.e., reviewing school or district
growth or progress over time). Because this, they are not mapped to performance levels and do not
have associated cut scores. They can be useful for identifying growth year-over-year, but do not
provide detailed information about a student’s ELP. The domain profiles and English proficiency
determinations are the best information about individual student performance.
Currently, the Overall Score and Comprehension Score can be compared across years for tests
within a grade band, but not from one grade band to another. ELPA21 is working to develop
crosswalks to facilitate comparison of performance across adjacent grades in different grade bands.
(e.g., to compare a student’s 5th grade performance with her 6th grade performance).
Since these scores are based on the same items that are grouped for the domain scores, the Growth
Indicators and domain scores will be closely related. However, since the proficiency judgments are
not based on the Overall or Comprehension Scores, there may be cases where students with the
same Overall score will have different profiles and therefore possibly different English proficiency
determinations.
Score Report Interpretation To make ELPA21 results accessible to all audiences, graphics, numbers, and text descriptions are
utilized to represent information.
General Score Interpretation Guidance ELPA21 offers unique information to students, parents, educators, administrators, and policy-
makers. When used appropriately, this information describes what ELLs know and can do in terms
of the grade-level language skills required to engage with the content that is taught according to
rigorous academic standards.
ELPA21, like all tests, has limitations. No single test can measure all aspects of a student’s
language use, and no test can measure this perfectly. ELPA21 scores are provided with a measure
of error. Summative assessment scores should be viewed as one indicator among multiple sources
of evidence (such as classroom-based tests, course grades, teacher observations, and samples of
student work) when interpreting and making decisions about a student’s English language
proficiency.
When used as designed, ELPA21 provides useful information. However, like any other test, it may
have unintended consequences if used outside the specific purposes and populations for which it
was designed and validated.
ELPA21 used valid psychometric processes to ensure that scores from different test forms describe
the same level of performance. For example, score from a 5th grade student scoring just above
ELPA21 2018 Score Interpretation Guide
9
proficient on test form A and a 5th student scoring just above proficient on form B would represent
the same performance, and these scores are comparable. When aggregated, these scores can also
describe school- or district-level changes in proficiency and can measure gaps in achievement
among different groups of students.
Scores within a single grade band are comparable. Those across grade band are not.
Appropriate uses of ELPA21 Summative Scores6
ELPA21 scores represent a snapshot of student proficiency at one point in time. The test was
designed so that scores may be used to:
1. Measure English language proficiency progress from year-to-year, allowing teachers and
parents to monitor annual progress towards proficiency.
2. Inform ELL reclassification decisions.
3. Inform state accountability calculation and reporting.
4. Inform evaluations of program effectiveness for various subgroups of students and adjust
educational programming and resources as needed.
5. Help school officials identify the types of professional development and/or support that
teachers need.
LPA21 scores should not be used to:
● Provide a measure of all components of ELP. The assessments are developed to assess ten
standards, in four domains, with items assessing specific skills and abilities in each.
● Evaluate ELP without recognizing that some error in measurement is inherent in all test
scores.
● Provide a single data point that alone is sufficient for making decisions. Test scores should
be interpreted within the context of the student and teacher, and the academic and social
environment.
● Describe why an individual student scored as he/she did.
● Provide a grade-level assessment of student proficiency. Scores cannot be used to suggest an
8th grader is literate at a 4th grade level, but do determine if a student is proficient in his or
her grade-level.
● Evaluate teacher performance or effectiveness.
Score Report Interpretive Guidance by Audience
Interpreting Results (Students and Families):
ELPA21 results are one measure of your child’s progress towards English language proficiency.
Like all tests, ELPA21 scores should be interpreted along with other information, like classroom
tests, assignments, teacher observations, content area test scores, and grades. Your student’s scores,
performance levels, and English Proficiency Determination describe his or her English proficiency
and how that compares to the level of proficiency needed for success in school.
6 ELPA21 Theory of Action.
ELPA21 2018 Score Interpretation Guide
10
Interpreting Results (Educators):
These results are one measure of a student’s progress towards English language proficiency and
provide specific but limited information. Like all measures of student proficiency, ELPA21 scores
should be interpreted along with information like classroom tests, assignments, teacher
observations, content area test scores, and grades.
ELPA21 scores are most useful when used collaboratively by English as a second language
(ESL)/English language development (ELD) teachers and content area teachers. Explicit
recognition that language acquisition takes place across the content areas benefits ELs’ academic
experiences. Ensuring that English language development does not interfere with access to grade-
appropriate content and practices requires a shift in perception from “What language skills does the
student have?,” to “What is the student able to do with language in the content areas?” Knowing
what students are able to do with the academic language necessary to engage in grade-level content
instruction is useful to guide instructional practice and to identify necessary supports. Scores by
domain also identify student strengths and weaknesses, inform reclassification decisions, discern
patterns of performance in classrooms or groups of students, and identify opportunities for
instructional collaboration.
The ELP Standards, and ELPA21 assessments, assume simultaneous development of language and
content-area knowledge, skills, and abilities. Because ELs have the same potential as their non-EL
peers they do not need to wait until their ELP is sufficiently developed to participate in content area
instruction and assessment.7 ELPA21 scores summarize the extent to which ELs language skills
enable (or hinder) an EL’s participation in grade-level mathematics, English language arts, and
science content courses. As language proficiency increases, grade-level content learning is able to
increase. Other classroom and statewide summative tests describe student ability in specific content
areas like mathematics, science, reading, and writing while ELPA21 scores describe how an EL is
able to use the language necessary for success in content area courses.
Interpreting Results (Administrators):
These results provide one measure of student progress towards English language proficiency.
ELPA21 scores may be used for staff and resource planning, curriculum selection, professional
development, program evaluation, informing reclassification decisions, and provide one measure of
accountability.
ELPA21 summative assessments align with the ELP Standards, and the ELP Standards correspond
to rigorous state standards by describing the academic language skills necessary to access and meet
rigorous college- and career-ready standards. Explicit recognition that language acquisition takes
place across the content areas may also encourage collaboration among educators and benefits ELs’
learning experiences.
A determination of proficient on ELPA21 indicates that a student has demonstrated the language
skills required by the content area expectations described by the CCSS and NGSS (including state
versions of the NGSS).
7 The ELP Standards.
ELPA21 2018 Score Interpretation Guide
11
Administrators should consider the following when looking at ELPA21 reports over time. First,
ELL status is temporary and changes as a direct result of high-quality language instruction. As a
result, more successful ELLs exit ELL programs as they reach required levels of ELP (and often,
academic performance also). A natural consequence of this is that faster-progressing, higher-
attaining students exit ELL status sooner than slower-progressing students. Secondly, because
students who reach proficiency no longer participate in ELPA21, the growth of ELLs longitudinally
will be underestimated (by ELPA21 or any test) because scores represent the progress only for
students who have not yet met exit criteria.
The ELPA21 consortium recommends that multiple criteria be used, with some redundancy, when
making high stakes decisions.8
8 See Abedi, 2008; Tippeconnic & Faircloth, 2002
ELPA21 2018 Score Interpretation Guide
12
References
Abedi, J. (2008). Classification System for English Language Learners: Issues and
Recommendations. Educational Measurement, Issues and Practice. 17-31.
Council of Chief State School Officers. (2014). English Language Proficiency (ELP) Standards
with Correspondences to K–12 English Language Arts (ELA), Mathematics, and Science Practices,
K–12 ELA Standards, and 6-12 Literacy Standards. Washington, DC: CCSSO.
Council of Chief State School Officers. (2013). Operational Best Practices for Statewide Large-
Scale Assessment Programs. Washington, DC: CCSSO.
English Language Proficiency Assessment for the 21st Century (2014). Theory of Action.
Tippeconnic, J. W., III, & Faircloth, S. C. (2002). Using culturally and linguistically appropriate
assessments to ensure that American Indian and Alaska Native students receive the special
education programs and services they need (EDO-RC-02-8). Charleston, WV: ERIC Clearinghouse
on Rural Education and Small Schools.
U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and
Program Studies Service, National Evaluation of Title III Implementation
Supplemental Report: Exploring Approaches to Setting English Language Proficiency Performance
Criteria and Monitoring English Learner Progress, Washington, DC 2012.
ELPA21 2018 Score Interpretation Guide
13
Appendix A Iowa Department of Education
ELPA21 Performance Standards (Cut Scores)
Level 1 Level 2 Level 3 Level 4 Level 5
Description Beginning Early
Intermediate Intermediate Early Advanced Advanced
Kindergarten
Domain Level 1 Level 2 Level 3 Level 4 Level 5
Listening 466 or below 467-506 507-612 613-644 645 or above
Reading 472 or below 473-513 514-591 592-626 627 or above
Speaking 486 or below 487-534 535-597 598-624 625 or above
Writing 496 or below 497-561 562-650 651-672 673 or above
Grade 1
Domain Level 1 Level 2 Level 3 Level 4 Level 5
Listening 434 or below 435-466 467-548 549-593 594 or above
Reading 478 or below 479-514 515-583 584-628 629 or above
Speaking 527 or below 528-576 577-592 593-618 619 or above
Writing 497 or below 498-547 548-612 613-640 641 or above
Grade 2
Domain Level 1 Level 2 Level 3 Level 4 Level 5
Listening 407 or below 408-437 438-511 512-563 564 or above
Reading 456 or below 457-488 489-554 555-594 595 or above
Speaking 489 or below 490-528 529-554 555-587 588 or above
Writing 451 or below 452-492 493-554 555-590 591 or above
ELPA21 2018 Score Interpretation Guide
14
Grade 3
Domain Level 1 Level 2 Level 3 Level 4 Level 5
Listening 408 or below 409-447 448-535 536-597 598 or above
Reading 494 or below 495-540 541-609 610-643 644 or above
Speaking 499 or below 500-537 538-571 572-611 612 or above
Writing 497 or below 498-541 542-602 603-635 636 or above
Grade 4
Domain Level 1 Level 2 Level 3 Level 4 Level 5
Listening 397 or below 398-430 431-491 492-562 563 or above
Reading 452 or below 453-487 488-549 550-593 594 or above
Speaking 461 or below 462-505 506-543 544-583 584 or above
Writing 436 or below 437-480 481-567 568-599 600 or above
Grade 5
Domain Level 1 Level 2 Level 3 Level 4 Level 5
Listening 412 or below 413-454 455-497 498-580 581 or above
Reading 467 or below 468-510 511-587 588-626 627 or above
Speaking 482 or below 483-525 526-572 573-606 607 or above
Writing 437 or below 438-485 486-597 598-627 628 or above
Grade 6
Domain Level 1 Level 2 Level 3 Level 4 Level 5
Listening 409 or below 410-439 440-497 498-564 565 or above
Reading 460 or below 461-495 496-564 565-603 604 or above
Speaking 464 or below 465-510 511-561 562-594 595 or above
Writing 424 or below 425-471 472-563 564-593 594 or above
ELPA21 2018 Score Interpretation Guide
15
Grade 7
Domain Level 1 Level 2 Level 3 Level 4 Level 5
Listening 429 or below 430-472 473-552 553-596 597 or above
Reading 485 or below 486-533 534-608 609-641 642 or above
Speaking 474 or below 475-526 527-581 582-610 611 or above
Writing 473 or below 474-519 520-596 597-624 625 or above
Grade 8
Domain Level 1 Level 2 Level 3 Level 4 Level 5
Listening 431 or below 432-477 478-564 565-612 613 or above
Reading 493 or below 494-546 547-639 640-668 669 or above
Speaking 475 or below 476-527 528-589 590-618 619 or above
Writing 483 or below 484-532 533-618 619-646 647 or above
Grade Band 9-12
Domain Level 1 Level 2 Level 3 Level 4 Level 5
Listening 450 or below 451-490 491-570 571-612 613 or above
Reading 487 or below 488-538 539-630 631-661 662 or above
Speaking 480 or below 481-535 536-592 593-618 619 or above
Writing 484 or below 485-532 533-614 615-640 641 or above
ELPA21 2018 Score Interpretation Guide
16
Appendix B ELPA21 Proficiency Descriptors
Emerging
● ELPA21 Official Students are Emerging when they have not yet attained a level of English
language skill necessary to produce, interpret, and collaborate on grade-level content‐
related academic tasks in English. This is indicated on ELPA21 by attaining a profile of Levels
1 and 2 in all four domains. Students scoring Emerging on ELPA21 are eligible for ongoing
program support.
● “Family friendly” version A student at the Emerging level does not yet have the ability to
produce grade-level academic content in the English language. For the ELPA21 annual
assessment, this means the student scores either Level 1 or Level 2 in each of the four
domains of reading, writing, listening, and speaking.
Progressing
● ELPA21 Official Students are Progressing when, with support, they approach a level of
English language skill necessary to produce, interpret, and collaborate, on grade-level
content‐related academic tasks in English. This is indicated on ELPA21 by attaining a profile
with one or more domain scores above Level 2 that does not meet the requirements to be
Proficient. Students scoring Progressing on ELPA21 are eligible for ongoing program
support.
● “Family friendly” version A student at the Progressing level is approaching the ability to
produce grade-level academic content in the English language with support. For the
ELPA21 annual assessment, this means the student scores above a Level 2 on one or more
domains, but does not yet meet the requirements to be at the Proficient level on the four
domains of reading, writing, listening, and speaking.
ELPA21 2018 Score Interpretation Guide
17
Proficient
● ELPA21 Official Students are Proficient when they attain a level of English language skill
necessary to independently produce, interpret, collaborate on, and succeed in grade-level
content‐related academic tasks in English. This is indicated on ELPA21 by attaining a profile
of Level 4 or higher in all domains. Once Proficient on ELPA21, students can be considered
for reclassification.
● “Family friendly” version A student at the Proficient level can produce grade-level
academic content in the English language. For the ELPA21 annual assessment, this means
the student scores either Level 4 or Level 5 on each of the four domains of reading, writing,
listening, and speaking.
Adapted from the Oregon Department of Education with all permissions granted 11/9/16
recognizing and understanding the meaning of basic words, phrases, and expressions to make meaning of text; responding to basic questions; gaining basic information.
identifying and understanding common words, phrases, and expressions to make meaning of text; identifying main idea; responding to basic questions.
identifying main idea and some key details; responding to questions; understanding the meaning of content vocabulary and less common words and expressions to make meaning of text; beginning to understand how word structures affect meaning.
determining the main idea and a number of key details that support the main idea; answering more complex questions; understanding the meaning of some academic and content vocabulary and less common words and expressions to make meaning of text; understanding how word structures affect meaning.
determining the main idea and explain how key details support the main idea; answering more complex questions; understanding the meaning of some academic and content vocabulary and less common words and expressions to make meaning of text; understanding how word structures affect meaning.
iting
Level 1 Beginning
Writing Level 2
Early
Intermediate
Writing
Level 3 Intermediate
Writing
Level 4 Early Advanced
Writing
Level 5 Advanced
2nd
Score Range: 451 or below
3rd
Score Range: 497 or below
2nd
Score Range: 452-492
3rd
Score Range: 498-541
2nd
Score Range: 493-554
3rd
Score Range: 542-602
2nd
Score Range: 555-590
3rd
Score Range:603-635
2nd
Score Range: 591 or above
3rd
Score Range:636 or above
When writing, the student at
Level 1 is working on:
When writing, the student at
Level 2 is working on:
When writing, the student at Level 3 is
working on:
When writing, the student at Level 4 is
working on:
When writing, the student at Level 5 is
working on:
creating words by filling in a missing letter; using a small number of frequently occurring nouns and verbs; answering basic questions and producing basic information using isolated words or phrases.
creating words by filling in a missing letter; composing words, phrases, and simple sentences related to prompt; recognizing and using frequently occurring parts of speech; choosing basic words to communicate meaning.
creating words by filling in a missing letter; composing simple and compound sentences with clear meaning that relate to a prompt; recognizing and using parts of speech; developing control of verb tenses; expanding word choice to communicate meaning.
composing simple, compound, and some complex sentences with a clear meaning that relate to a prompt; recognizing and using parts of speech; increasing use of complex grammar structures; expanding word choice to communicate meaning.
composing compound and complex sentences with a clear meaning that relate to a prompt; recognizing and using parts of speech; accurately using complex grammar structures; expanding word choice to communicate meaning.