Academic Rigor: Where Are We Now Looking at Student Reading Achievement and Increasing Rigor Using Grade 8 NAEP Item Maps and Percentile Graphs Prepared.

Academic Rigor: Where Are We Now

Looking at Student Reading Achievement and Increasing Rigor

Using Grade 8 NAEP Item Maps and Percentile Graphs

Prepared by Jeanne Foy

Alaska State NAEP Coordinator

NAEP: the Common Measurement of Student Achievement among States

The National Assessment of Educational Progress is called the “gold standard of assessment”

Created in 1969; the No Child Left Behind Act (NCLB) mandated state participation in NAEP reading and math every other year

Alaska has NAEP data for 2003, 2005, 2007, 2009, and 2011

Purpose of This Presentation

NAEP reading item maps and percentile graphs can help to frame the question of where Alaska is now in terms of student achievement and what would need to be done to raise student achievement.

Alaska is revising reading, writing, and math standards against the background of a national push to raise the rigor of standards.

Alaska’s Revised Standards Many states have adopted the Common

Core standards for English language arts and mathematics. Alaska has developed proposed standards to equal the Common Core standards in rigor: they will be posted at http://www.eed.state.ak.us/regs/

The next slide shows the rigor intended to be reached by the new standards

CC Standards Designed to Be Internationally Competitive The standards have been informed by the best

available evidence and the highest state standards across the country and globe . . .

These standards are designed to ensure that students graduating from high school are prepared to go to college or enter the workforce and that parents, teachers, and students have a clear understanding of what is expected of them.

The standards are benchmarked to international standards to guarantee that our students are competitive in the emerging global marketplace.

http://www.corestandards.org/frequently-asked-questions

NAEP Resources Set Stage for Discussions about Higher Rigor

NAEP percentile box graphs show current academic achievement gaps by ethnic groups

NAEP item maps show what students are achieving now

A Disclaimer! The issues of increasing academic

performance and closing gaps are complicated

This presentation is designed to show current student achievement in Alaska and serve as a springboard for discussion in considering the challenges of increasing academic rigor and closing achievement gaps among ethnic groups

How Percentiles Represent Student Achievmeent

A percentile is a value on a scale of one hundred that indicates the percent of a distribution that is below it.

For example, if a student scores at the 84th percentile on a test, he scored higher than 84% of the students who took the test

Understanding Box Plots

Box plots allow data to be explored by showing the median value of a set of data and the 25th and 75th percentiles

The middle line in the box shows the median value, the reading score at which half the students performed below and half the students performed above

Students who scored at the 75th percentile performed better than 75% of students who took the NAEP; students who scored at the 25th percentile performed better than 25% of students who took the NAEP

The range of the distribution is shown on the box plot by showing a minimum and maximum value (the 10th and 90th percentiles for the NAEP box plots)

Alaska Gr. 8 Reading Percentile Box Plots for 2011 NAEP

Note where the median (the dot in the middle of the box) is for each group in relation to the other groups

Note where the other data points are in relation to the other groups

What are your observations? What questions or issues do

your observations raise?

NAEP Item Maps Illustrate What a Reading Score Represents

The item maps have descriptions of questions students who achieved certain scores on the NAEP are likely to have correctly answered

Item maps combined with percentile box charts help to illustrate the knowledge and skills demonstrated by students performing at different scale scores

For instance, the item or question description at 268 on the scale represents the types of items grade 8 students who scored a 268 on NAEP answered correctly; item descriptions in blue represent items that are available to the public

Looking at the actual question for score 268 helps show what students can do in a concrete way.

The score of 268 is just below the median score of 276 for White students. White students scoring at the 50th percentile would probably have answered this question correctly. The question is shown on the next slide.

268 Recognize specific information in expository event—Locate/recall

Question:According to “Home on the Range*,” some scientists think that moving African animals to the Great Plains would help improve the area’sA. resistance to alien speciesB. economyC. biodiversityD. research facilities

Nationally, 68% of students chose C (correct answer). In Alaska, 57% chose C, 26% chose B, 13% chose A.

*The two reading passages referred to in this presentation are nonfiction. The proposed English/Language Arts standards emphasize the importance of reading nonfiction.

Each NAEP Question Has a Cognitive Target There are three targets--locate/recall,

integrate/interpret, and critique/valuate. Locate/recall description:

Identify textually explicit information and make simple inferences within and across texts, such as:

Definitions Facts Supporting details

What Skills Are Required to Answer the Question What is the text complexity of the passage

for eighth grade students? What skills do students demonstrate when

they answer the question correctly? Why would students choose the wrong

answer? Consider the percentage who chose the wrong

answers. What might lead students to choose those

answers?

Consider Another Locate/Recall Question

The question at 239 is close to the median score for 237 for American Indian/Alaska Native students. The question is on the next slide.

239 Recognize causal relationship between facts in article—locate/recall

QuestionAccording to “Alien Invasion,” Australian farmers imported cane toads into Australia in order toA help save their native beetle populationsB eat the sugarcane that was taking over their other cropsC stop an alien species of beetle imported from HawaiiD eat beetles that were destroying sugarcane crops

Nationally, 84% chose D (correct answer). In Alaska 87% of students chose D and 9% chose C.

How Do the Two Locate/Recall Questions Differ

What makes one question easier than the other? What skills are needed for students to answer the more difficult question?

What Does Item at Upper End of Item Map Show

The question at 340 is located above the 90th percentile for any ethnic group. The question is shown on the next slide.

340 Compare two texts of different genres to provide similarity and difference—integrate/interpret Question

Describe a similarity and a difference between the way the two articles* approach the subject of invasive species. Support your answer with references to both of the articles.

Note: the score of 340 is for students who scored at the extensive level according to the scoring guide, which is presented in the following slides

*”Alien Invasion” and “Home on the Range”

Integrate/Interpret

Make complex inferences within and across texts to describe problem and solution, cause and effect: Compare or connect ideas, problems, or

situations. Determine unstated assumptions in an

argument. Describe how an author uses literary

devices and text features.

Item: Compare two texts of different genres to provide similarity and difference

Consider text complexity of reading passages.

How difficult do you think this item is?

How do you think grade 8 students should perform on this item?

Scoring Guide for ExtensiveResponses at this level describe a similarity and a difference between the way the two articles approach the subject of invasive species and support the answer using a reference to a harm or a benefit discussed in the articles.•  Both articles talk about invasive species being a problem with the environment. The Home on the Range article differs in the fact that it supports putting species in a different environment to make a more diverse animal population and prevent extinction.Or, responses describe a similarity and a difference by referring to author's craft or the structure of both articles.   •  The similarity is they both give some examples and give you at least one side of the story. The difference is that Alien Invasion only gives one side of the story but leaves out the other side. Home on the Range gives you both sides of the story and supports both sides. So Home on the Range is neutral, but Alien Invasion is not neutral.

Scoring Guide for Essentiala) Responses at this level describe a similarity or a difference between the way the two articles approach the subject of invasive species and support the answer using reference to a harm or a benefit discussed in the articles.•  The difference is in Alien Invasion it talks about getting rid of species and Home on the Range talks about saving African animals.•  The articles are similar because they both say why alien species are bad. The first article tells about the cane toad in Australia and the damage it caused.ORb) Responses describe a similarity or a difference by referring to author's craft or the structure of both articles.•  Alien Invasion uses multiple examples of invasive species to make a point. In Home on the Range the author presents both sides of the issue.

Scoring Guide for Partiala) Responses at this level describe a similarity and/or a difference between the way the two articles approach the subject of invasive species but do not refer to a harm or a benefit.•  The articles are the same because they both say alien species can be bad. •  Both stories talk about animals moving from one place to another. •  They both talk about cane toads.•  In Alien Invasion they talk about toads and zebra mussels. In Home on the Range they talk about camels and lions.ORb) Responses describe a similarity or a difference by referring to author's craft or the structure of one article.•  The second article presents both sides of the alien species debate. It describes pros and cons of moving animals from Africa.

Scoring Guide for UnsatisfactoryResponses at this level provide irrelevant details or personal opinions or draw incorrect conclusions or may simply repeat the question.

•  Snakehead fish kill native fish and many species of frogs.•  I think the articles are trying to get people's attention. •  Both of the articles are against invasive species.•  They are both about invasive species.

What Skills Do Students Need to Demonstrative Extensive Comprehension?

Example of Extensive Comprehension

Another Example of Extensive Comprehension

An Example of Essential Comprehension

What Does Partial Comprehension Reveal?

Example of Partial Comprehension

What Is Going On with Unsatisfactory Comprehension?

Example of Unsatisfactory Comprehension

What Does Data Suggest about Student Skills?

In Alaska, 18% got extensive, 19% got essential, 34% got partial, and 22% got unsatisfactory.

National performance data

Questions to Consider

What does data show about the current achievement gaps?

What are the challenges for increasing academic rigor and decreasing student achievement gaps?

Additional Resources The Alaska-created “NAEP Assessment Toolbox

for Teachers” show how NAEP questions can be used instructionally

http://www.eed.state.ak.us/tls/assessment/naep.html

NAEP questions for a variety of subjects can be found at

http://nces.ed.gov/nationsreportcard/itmrlsx/default.aspx