NH ALPs Science Scoring Training July 2012. Training Outline Science Learning Progressions Levels of Scoring Clarifications Scoring – Forms, Template,

NH ALPs Science Scoring Training

July 2012

Training OutlineScience Learning ProgressionsLevels of ScoringClarificationsScoring – Forms, Template,

EvidenceVideoQuestion & Answer Period

Required Content Areas Gr. 4: Earth-Space Science & Life Science

Gr. 8: Earth-Space Science & Physical Science

Gr. 11: Life Science & Physical Science

Each Entry Requires

ONE Content Standard

And

TWO Process Skill Standards

Science Learning Progressions

How To Read The Progressions

Content StandardsChallenge LevelSub-LevelColumn

Process Skill Standards

STUDENT AS AUTHORTHE CRITICAL ROLE OF ACCESS

TO LEARNING, COMMUNICATION & PERFORMANCE

What does “independent performance” mean for an alternately assessed student?

Directive prompts violate the learning construct that we are trying to measure. No credit can be earned when directive prompts are used.Supportive prompts allow students to show personal authorship of standards-based tasks. Students can

earn credit when supportive prompts are used because the constructs being tested are honored and

demonstrated.

Two Levels of Score Review

1. Task Level ReviewContent Fidelity

2. Item Level Review

Task Level ReviewTask Level Review determines

whether or not the student performance sample can advance to the Item Level Score Review, based on sufficiency of match to the required academic content standards (Content Fidelity). If a student performance work sample is found not to sufficiently match the required content, the sample does not pass task review.

TASK LEVEL REVIEWDuring this review, a scorer must

ask the following two (2) questions:

1. Is the student attempting to apply knowledge or skills that are clearly within the content area?

AND2. Is the type of skills and

knowledge clearly related to the standards of the progression?

Task Level Review continued…

If the answer to both of these questions is YES, then sufficient evidence of content knowledge exists and scoring may continue to the Item Level Review.

If the answer to either or both of these questions is NO, then sufficient evidence of content knowledge does not exist and scoring may not continue. If student performance does not meet this criterion, then the task sample cannot continue to the Item Level Review and the applicable comment code must be documented.

Task Level Review Comment Codes1. Student method of communication could not be

determined.

Content Assessment Cover Sheet information was

not submitted and Summary Description of Student Access to Learning and Performance was not submitted.

2. Challenge Level and/or Sub-Level was not identified.

3. Performance task evidence provided, as shown by student work sample, does not match content progression.

(i.e.: Required Reading text type was not submitted, required Writing genre not submitted, required Mathematics progression not submitted, or required Science domain not submitted).

4. Copy or summary of text was not submitted for Response to Text writing sample.

5. The Science column could not be determined (last letter in the Science Content ALPs code).

Item Level Review

An item is referred to as a scorable unit within

a content standard. There may be one or multiple scorable units within a content standard. Each scorable unit is

considered to be worth one (1) point.

ITEM LEVEL REVIEW

1. Is the performance criteria met as stated in the standard (or in the clarifications)?

AND2. Is the student clearly the author of this academic performance?

If the answer to both of these questions is YES, then one (1) point is credited.

If the answer either or both of these questions is NO, then no point is credited.

If student did not earn credit for any scorable units within the chosen standard then it is considered Not Demonstrated (ND) and applicable comment code must be documented.

Item Level Comment Codes

6. Quality of video impacted score (i.e., sound quality, camera angle, clarity)

7. Video exceeded maximum allowable length. Score review ended at maximum time.

8. Student did not demonstrate standard(s), as written, in identified Progression or sub-progression.

9. Directive prompts prevented observation of student authorship.

10. Directive prompts impacted observation of student authorship.

11. Science Process Skill was not indicated.12. Challenge level of Science Process Skill did not

match Science content challenge level in this entry.13. Science Process Skill was duplicated; only 1 Process

Skill qualified for score review.

Question 1Is the Performance Criterion Met?

Science success at the item level is defined as one clear demonstration of a science scorable unit - as written that also includes clear evidence of student authorship. “One clear demonstration” means that student performance must meet the scoring criterion defined by the scorable unit(s) with the

standard- as written.

Question 2 Is the student clearly the author of this academic performance?

Credit or non-credit of the scorable unit is also determined by the demonstration of student authorship. There must be compelling

evidence that the student is clearly the author, as

demonstrated in his/her performance of the scorable units (or

item responses) within the standard.

The following questions may serve as guidance to a scorer

judgment in making the determination regarding the presence or absence of student authorship:

Does evidence of student access to learning, communication, and performance show student engaging in the task as a willful agent who is effectively communicating his/her decision-making to indicate personal choice and authorship of performance of this scorable unit/item?

Did the teacher provide supportive prompts and not directive prompts? (See Training Manual)

Is there a biasing response context?

Student as Author

Student As Author continued…

Is the student able to communicate choices effectively?

If a student has a severe motor impairment and some responses appear to be a result of the clear willful intention of the student but some responses seem possibly accidental or random, is evidence of student response control sufficiently clear to make authorship compelling? (Requires scorer judgment)

Terminology Used in Science Progressions

Understanding the specificity of the Science

scorable units as they are written.

The words “And” or “&” when placed between SUs of a standard

requires the student to demonstrate one SU OR the other SU,

demonstration of both is NOT necessary to receive credit for one or

the other. However, when the word “and” is used to connect

elements INSIDE a single scorable unit of a standard, then both

elements linked by the word “and” ARE required, (i.e. “student must

match, produce, and compare”) requires demonstration of all elements

in the group connected by the word ‘and’ are required before credit can

be earned for this scorable unit. Demonstration of only one is not sufficient

for credit when a scorable unit is written this way.

Terminology continued…

“Or” - when used to link a group of elements within a scorable unit, requires the student to demonstrate either one or the other element that is connected by the word ‘or’ (minimum of one is required).

“Describe” means using an effective communication method, student gives an account of someone or something, including relevant characteristics, qualities, or events. Demonstration of the Process Skills will show full breadth of student’s knowledge, allowing scorer to give credit for all scorable units.

Sample Standard Earth-Space Science (E3D) “ Describe the Sun as a

star. (1 SU) (S:ESS2:4:1.2)”

Terminology continued…

 “Explain” means to define; expound; to offer reasons for or a cause of; justify and make an idea clear using your own words or through use of a valid communication system. Demonstration of the Process Skills will show full breadth of student’s knowledge, allowing scorer to give credit for all scorable units. Examples include describing causes, effects or relationships; or identifying steps in a process.

Specific Examples found in Science standards include: “Explain what happens when heat is applied to an ice cube;” or “Explain how falling water can be used to produce electricity;”

or “Explain the stages (steps) an open field goes through over

time to change into a forest.”

Sample StandardEarth-Space Science (E2D) “Explain that people should not look directly at the Sun because it is dangerous and may cause injury to the eye (1 SU) (S:ESS2:2:4.3) ” 

Terminology continued…

“Sort or Classify” at Emergent or Beginning Challenge Levels means defined and /or structured templates are provided by teacher. The term “Sort” is the preferred term for earlier challenge levels.

“Sort or Classify” at Transitional, Intermediate, and Advanced Challenge Levels requires the student to demonstrate more independent sorting and classifying behavior - with more student initiated categorizations beginning at Transitional Challenge Level. The term “classify” is usually the preferred term at higher challenge levels. (See also ‘Open Sort’ below.)

“Open Sort” requires the student to determine sorting category independently, working at a higher challenge level, without teacher template sample. Student independence in sorting/classifying increases as challenge levels increase. Open sort is consistent with higher challenge levels.

Terminology continued…

Words that mean a minimum of at least two (2) performance instances are required: “all”, “most”, “many”, or other words that use the plural form(s).

“Pairs” is plural and means more than one pair, requiring minimum of two (2) pairs, (vs. ‘Pair’ = 1 pair.)

“Variety” means two (2) or more.

Commas (,) when used to link a group of elements, are treated as ‘OR’ statements – if there are only commas and no words connecting the elements in the group, (i.e. “student must match, produce, compare”). This statement, as written, uses only commas and therefore, requires performance of only one (1) of these elements.

Terminology continued…

“Recognizing” (a) requires naming or clearly choosing the correct example from an array of options, Or, (b) demonstrates understanding of specified concept through proper usage.

“Distinguish” means to differentiate between (See wording of standard for specific distinction intended.) Student must demonstrate that he/she knows the difference between two things, indicating; that which is an example of something, as compared to something else that is not.

“Precise” means the language used is highly

specific in a way that is appropriate to the content being written about. Precise terms for measurement might mean to specify units of inches, feet or meters; etc.

ClarificationsGeneral Scoring Clarifications for Science

Content-Process Skills Connection Rule: Student must make the connection between Science Content Progression and Process Skills Progression by using the process skills to explore and/or perform the content standard identified. Credit will not be given for demonstration of Process Skills performed in isolation, without the connection to content.

More Than 2 Process Skills Rule: Only the first two Process Skills demonstrated will be scored.

Student Authorship Rule for Sorting: Student will be credited with authorship of “Sorting” even if teacher provides a defined/structured sorting template and other relevant evidence of student authorship is present.

Clarifications continued…

Cross-Over Standards, defined: Cross-over standards are duplicate standards that span across more than one Learning Progression; (i.e. same standard is found in both Earth-Space Science, and Physical Science Progressions.) Although the standards replicate the text language, the content will guide scoring approach. Student performance must always be demonstrated in relation to specified content progression being sampled in order to receive credit.

Sample StandardThis exact standard, (S:ESS1:2:7.1) “ Recognize that

water can be a liquid or a solid; and explain that it can be made to change from one state to the other, but the amount (mass) of water always remains the same in either state. (1 SU)” is found in two Science Content Progressions, as follows:

Clarifications continued…

Earth-Space Science Content Progression: Beginning Challenge Level (Earth Materials Sub-Progression) (B3B) Student

performance must demonstrate understanding of standard as it applies to the Earth-Space Science content progression (i.e.: earth science student work sample will have to clearly connect with earth materials- water found in air, found in dirt, streams, etc.) (S:ESS1:2:7.1)

And in:  Physical Science Content Progression:

Beginning Challenge Level (Properties of Matter Sub-Progression) (B1B) Student

performance must demonstrate understanding of standard as it applies to the Physical Science content progression (i.e.: physical science student work sample will have to clearly connect with properties of matter more broadly than just earth materials - water found in synthetics, manufactured, or other materials, etc.) (S:ESS1:2:7.1)

Clarifications continued…

Cross-Over Standard Scoring Rule:

Student work sample must connect in a distinctly identifiable way to the stated content standard to be eligible for credit. There are two possibilities:

If teacher stimulus provides same exact task activity for both Earth-Space Science and Physical Science Entries, the duplicated task that does not distinctly connect to Earth-Space Science or to Physical Science will not be scored.

If teacher stimulus provides same general activity for both Earth-Space Science and Physical Science Entries, BUT the student does demonstrate a distinct and specific connection to the identified Earth-Space Content AND also specifically to the identified Physical Science Content, then student work sample can receive credit for both Entries.

Clarifications continued…

Clarifications for Specific Science Progressions

Life Science

Death and Dying: Due to a possible bias concern related to death and dying, credit will be given even if student does not demonstrate a full understanding or provide an explanation of

“die” or “death” words as related to this progression. Knowledge and understanding of life cycle as a whole will receive credit.

For Example:Life Science E1E Recognize that living things have a life cycle, during

which they are born, grow, and die S:LS1:2:3.2 N (1 SU)

B2E (1) Recognize that living organisms have life cycles, which include birth, growth and development, reproduction, and death; and (2)explain how these life cycles vary for different organisms S:LS1:4:3.2 N (2 SUs)

Clarifications continued…

Process Skills

Emergent Challenge Level EPS8 “(1) Sort and classify objects,

materials and events based on one or more attributes; and (2)

explain the methods used for sorting (2 SUs)” (S:SPS1:2:1.5)

Credit will be given if student demonstrates sort and classify object materials

OR Events based on one or more attributes. They do not

have to demonstrate objects, materials and events (as identified by the word “and” placed inside this first scorable unit). They must only demonstrate sorting and classifying of one of these elements.

The second scorable unit (2) in this standard stands on its own and must be met as written to be credited.

Scoring Sample

Scoring Materials

Student Score FormStep by Step Scoring Procedures

for ScienceScoring Comment CodesTraining Manual:

Learning Progressions Glossary

Scoring Samples

Questions