vasshandouts

A Division-wide Approach to Using Balanced Assessments

Jared A. Cotton, Ed.D.Superintendent

Henry County Public Schools

Chris JacksonCollege & Work Readiness Assessment (CWRA)

May 7, 2012

• “Our primary focus is on teaching and assessing those skills our students need to thrive as 21st century learners, workers, and citizens.”

• Outcomes for student success: academic proficiencycreative thinking critical thinking problem solvingeffective communication

Compass to 2015

Strategic Objective #2

VBCPS will develop and implement a balanced assessment system that accurately reflects student demonstration and mastery of VBCPS outcomes for student success.

A balanced assessment system is . . .

Activity – Think, Pair, Share

What does the term balanced assessment system mean to you?

A balanced assessment system is . . .

• an organized system of different assessments that are purposefully selected to improve learning during instruction and measure core subject mastery and 21st century skills.

• comprised of traditional assessments, performance tasks, and other types of assessment (formative and summative) used at the classroom, school, and division levels.

Balanced Assessment (continued)

Balanced assessment has two dimensions:1.Using a variety of assessment types

appropriately matched to our varied learning goals; and

2.Recognizing that there are varied purposesfor assessment (diagnostic, formative and summative) and that all assessments should ultimately be used to improve learning, not simply evaluate it.

Jay McTighe, 2010

An assessment analogy . . .

Multiple-choice test

Commonassessment

Digitalportfolio

Performance task

Strategic Objective #2

• Develop and/or adopt high-quality assessments that measure Outcomes for Student Success.

• Assist with providing professional development related to assessment literacy.

• Monitor assessment quality for local assessments.

• Report on student progress.

How do we measure VBCPS Outcomes for Student Success?

AcademicallyProficient

SOL

Stanford 10

AP

Local Assessments

SAT

ACT

Effective Communicators and Collaborators

Globally aware, independent, responsible…

Critical and creative thinkers, innovators, and problem solvers

“We value what we measure rather than measure what we value.”

“Key Indicators of Student Success”

“Key Indicators of Student Success”

School Data Portfolios

School Data Portfolios

The VBCPS Continuum for 21st

Century Skills

FAQ for the VBCPS Continuum

Local Assessments – Where we were …

• Measuring learning objectives with multiple-choice tests administered divisionwide.

The past local assessment program . . .

• included up to four annual tests per subject in each tested grade level or course.

• was designed to measure Virginia Beach Objectives (VBOs) and the Standards of Learning (SOLs).

Multiple-Choice Tests: Advantages• Tests can be administered to large groups at low cost.• A wide range of content and learning objectives from

factual to evaluative can be assessed in a short time.• Related concepts or theories can be compared. • Multiple-choice tests are easily graded by hand, with a

scoring template, or by using a scantron machine or scoring software with answer sheets.

• Online multiple-choice tests can be scored immediately.

Note of Caution

• Each student has at least a 25% chance of correctly answering a multiple-choice item that has four response choices.

• Do not rely on a single item to ascertain student understanding, especially on a diagnostic assessment.

Multiple-Choice Tests: Disadvantages

• Learning may be overestimated due to students’use of the elimination process to select correct answers.

• “Testwiseness” and reading ability may bias results.

• Items tend to assess lower-order thinking skills.

• Students are limited to given response choices.

Multiple-Choice Tests: Disadvantages• A student’s ability to organize and express ideas is

not being assessed.

• Tests generally do not provide enough feedback for the teacher to correct errors in understanding.

• Test security can become an issue.

Revised Local Assessment Program

Purpose: Evaluate 21st century skills andacademic proficiency using a variety of assessments.

• Includes a limited number of mandatory, divisionwide multiple-choice tests per subject or course each year.

• Encourages teachers to create common assessments using an online item bank in SchoolNet.

Revised Local Assessment Program

• Contains performance tasks at every grade level and an integrated district performance task for grades 4, 7, and 12.

• Includes digital portfolios that students will begin developing in the primary grades and present in grades 5, 8, and 12 (starting at select schools).

Revised Local Assessment Program

Digital portfolios . . .

• are a purposeful collection of student work that exhibits, in electronic format, the student's efforts and progress. A digital portfolio should include:

– student participation in selecting contents.– criteria for selection.– criteria for judging merits.– evidence of a student's self-reflection.

Adapted from Paulson, Paulson, & Meyer, "What Makes a Portfolio a Portfolio?”Educational Leadership, February 1991

Modifications to Norm Reference Testing

• Stanford 10 Online—Grade 4

• Explore—Grade 8

• College and Work Readiness Assessment (CWRA)—Grade 12

Divisionwide Performance Tasks• Present realistic problems and assess the

ability of students to use given information to develop reasonable solutions.

• Require students to think critically and evaluate the information in order to address the problems successfully.

• Focus on skills that are essential for work and college in the 21st century.

Adapted from Benjamin, Chun, & Shavelson, Holistic Tests in a Sub-Score World: The Diagnostic Logic of the Collegiate Learning Assessment, CLA White Paper, 2007

Adapted from Benjamin, Chun, & Shavelson, Holistic Tests in a Sub-Score World: The Diagnostic Logic of the Collegiate Learning Assessment, CLA White Paper, 2007

“The College and Work Readiness Assessment (CWRA) presents realistic problems that require students to analyze complex materials varying in reliability and accuracy, and to constructwritten responses that demonstrate their abilities to think critically, reasonanalytically, solve problems, and communicate clearly and cogently.”

stop counting

#CWRA | @cla_beat

“Bad…”

We ask our students,

quite simply, to count fish.

The need for adifferent assessment

Institutions are equipped toimprove higher-order skills

when they connectteaching, learning, and assessment

through authentic, performance-based practices.

(a) William Shakespeare(b) John Updike

(c) Ernest Hemingway(d) None of the above

Rewards content regurgitationFails to gauge transferability

Contentknowledge

Contentskills

Generalknowledge

Generalskills

Contentknowledge

Contentskills

Generalknowledge

Generalskills

The provision of transferrable skills

is as important as the provision of

content.

The top ten in-demand jobs in 2010 did not exist in 2004. We

are currently preparing students for jobs that don’t yet exist,

using technologies that have not yet been invented in order

to solve problems that we don’t even know are problems yet.

SO WHAT?

SO WHAT?

Academically adriftLimited learning on college campuses

Proportion of employers who say colleges should place MORE emphasis than they do on selected

learning outcomes

•The ability to effectively communicate orally and in writing

•Critical thinking and analytic reasoning skills

•The ability to apply knowledge and skills to real-world settings

•The ability to analyze and solve complex problems

89%

81%

79%

75%

AAC&U | 2008 | aacu.org/leap/public_opinion_research.cfm

What is the [cwra]?

bias

small sample

quantitative reasoning

appropriate comparison

group

correlation vs. causation

incorrect (improper?) use of data

ability to filter

metrics

aligning teaching, learning &

assessment:faculty

development

What is authentic assessment?

How does effective teaching support that (and vice versa)?

What are the higher-order skills we value? Why?

How are those skills demonstrated in student responses?

What are the components of a Performance Task?

How do we effectively build them?

Working | Sharing | Feedback

VBCPS Integrated Performance Tasks…

• use the CWRA as a model.• are developed by

classroom teachers and specialists.

• are reviewed, field-tested, and revised extensively.

• yield scores for critical thinking, problem solving, and written communication.

VBCPS Integrated Performance Tasks…• are administered twice a

year to students in grades 4 and 7.

• can be used for diagnostic and summative purposes.

• will provide data for Compass to 2015 Key Indicators.

• serve as a catalyst for changing instruction.

Skills measured by the IPT• Critical Thinking: Analyze and evaluate

information and ideas to determine appropriate actions or develop a point of view.

– Student analyzes the information to determine its relevance, quality, and depth.

• CT1: Judges the quality of information in the IPT documents

– Student considers the strengths and weaknesses of the two given options.

• CT2: Sees the need for additional information not included in the IPT documents

Skills measured by the IPT• Problem Solving: Anticipate and identify problems

and challenges to develop solutions that effectively address them.– Student evaluates the information and uses it to defend

his/her decision.• PS: Makes a decision and gives reasons for the decision

• Written Communication: Articulate ideas and information clearly and appropriately for the given context, medium, and audience.– Student uses information and ideas to create a clear,

organized response.• WC: Presents information and ideas that are clear, organized,

detailed, and written for the intended audience

Skills Measured by the IPT

Critical Thinking•Critical Thinking 1 – Is the information in the IPT booklet is correct and believable?

•Critical Thinking 2 – What important information is not in the IPT booklet?

Skills Measured by the IPTProblem Solving

•Problem Solving – Student makes a choice and gives reasons for the choice.

Written Communication•Written Communication – Student presents information and ideas that are clear, organized, detailed, and written for the intended audience.

Introducing the IPT to Students

Grade 4 Video:•http://www.youtube.com/watch?v=HZV3mIBXnKw

Grade 7 Video:•http://www.youtube.com/watch?v=9ICealf0-2w

Grade 7 IPT Document

Grade 4 IPT Document

IPT ImplementationCreating situations and documents

Field testing

Administering across the division

Scoring student responses

Interpreting and using results

IPT Development Process

Task creation: December 2011

Field tests: January-February 2012

Final revisions: February 2012

Printing: March 2012

IPT Administration Process

Training school test coordinators

Delivering materials to schools

Loading prompts on Schoolnet

Monitoring during testing window

Picking up materials from schools

IPT Scoring Process

Fall 2011 IPT: Decentralized scoring

Spring 2012 IPT: Centralized scoring

Fall 2012: Interpreting results

Change at the Classroom Level• Encourage teachers “rethink” assessment.• Focus on “Assessment Practices that Support

Learning.”• Develop resources to assist teachers with

implementation of key assessment practices.• Provide professional development in developing

common assessments, performance tasks and rubrics, and using protocols to review assessment results.

Rethinking Classroom Assessment

• Why do we assess? • How might assessment promote learning

as well as measure it? • How do we raise student achievement on

high-stakes assessments without "practicing for the test"?

• What is "falling through the cracks" of conventional testing?

Jay McTighe, 2010

Rethinking Classroom Assessment

• How can thinking like an assessor focus curriculum planning and instruction?

• Why should we assess before teaching? • What kinds of assessments will motivate

learners to do their best? • What types of feedback most benefits

students and teachers?

Jay McTighe, 2010

School-Based Vanguard Leadership Team

• Chaired by the principal• Comprised of key school-based staff

members who will support this initiative• Designed to build capacity within each

school• Provided additional training throughout the

year• Work with the Vanguard Support Team to

provide on-site professional development

Professional Development

• Members of the School-Based Vanguard Leadership Team received additional training

• On-site training was provided by the Vanguard Support Team and the School-Based Vanguard Leadership Team

• All PD was provided on site during the school year

• Additional professional development was offered during collaborative meetings throughout the school day

Professional Development(continued)

• Examples of professional development opportunities in order of complexity*– Learning Walks– Peer Observation– Collaborative Action Research– Lesson Study

*These will be implemented based on staff readiness

Assessment Practices that Support Learning

• Pre-assessment• On-going assessment• Descriptive feedback• Assessment variety• Authentic performance tasks• Development of rubrics• Models/examples for students• Self-assessment/goal-setting• Teacher collaboration

Protocols

chris jackson212.217.0845

[email protected]@cla_beat

collegiatelearningassessment.org/cwrapresentations

jared cotton276.634.4711

[email protected]

www.henry.k12.va.us