Calibrated Peer Review TM

Calibrated Peer ReviewTM

Orville L. Chapman,

Arlene A. Russell, and Michael A. Fiore

University of California, Los Angeles

CPR Design

CPR design is based on peer review of scientific proposals and manuscripts.

CPR permits only competent, calibrated peer review of student documents

CPR ensures careful reading for content. CPR makes students conscious of style issues. CPR enables learning by writing.

Motivation for Creating CPR

Science Literacy Constructivist Learning Critical Thinking

Science Literacy

Literacy

Discourse based on comprehension, understanding, organization, and logic.

Construction of Knowledge

Integration of new knowledge Organization of knowledge Critical evaluation of new

information

Critical Thinking

Rigorous thinking Testing ideas and data against

existing intellectual models Rejecting intellectual models Creating new intellectual models

Bloom’s Taxonomy of the Cognitive Domain

Evaluation--highest level Synthesis Analysis Application Comprehension Knowledge--lowest level

***

**

CPR

What Is CPR?

CPR is a network learning tool. CPR is accessed through an Internet browser. CPR is platform independent. CPR enables writing assignments in any class,

regardless of size or subject. CPR works even with the largest classes.

CPR enables shared writing assignments among institutions over the Internet.

Two factors Make CPR Special: CPR is discipline independent. Art,

anthropology, English, social science, history, economics, business, law, chemistry, physics, life science, and mathematics are using it. CPR can serve any curriculum.

CPR is level independent. It serves grades 6-12, colleges and universities, graduate and professional schools.

What Does CPR Comprise?

Student interface Instructor interface Assignment authoring tool Reporting tool Assignment management system Assignment libraries

How Does CPR Work? The instructor gives the student a writing

assignment. The student submits his or her document to CPR

electronically. CPR then gives the student three calibration

documents for review. When the student has completed the calibration,

CPR provides a detailed report on the student’s performance.

How Does CPR Work?

After the student has successfully completed the calibration, CPR provides three peer documents for review.

CPR then gives the student his or her document for self-review.

Finally, CPR provides a detailed report of the peer review and the self-review.

The Review Process

The review process is the same for calibration, peer review, and self-review.

The student answers content questions for each document.

The student answers style questions for each document.

The student assigns a score to each document on a 1 to 10 basis.

Student Reports

summary of the calibration results

Instructor Reports

CPR provides a detailed report on each individual student’s work.

CPR delivers a detailed summary report for the class. CPR reports are broken down among calibration,

peer reviews, and self-review. Students are scored on their writing and their

reviewing.

Assignment Libraries

Molecular Science Library CSUN Business School Library Libraries enable frequent writing

assignments with minimal instructor time.

Creating CPR Assignments

Identify important course topics. Locate appropriate sources for student learning. Compose appropriate guiding questions. Write the calibration documents--one should be

an exemplar. Create carefully crafted content and style

questions. Have peers review your assignment.

Creating CPR Assignments:

Start by reading “Workshop Manual and Writing CPR Assignments,” by Arlene Russell.

To obtain a copy of the manual or to get advice on creating CPR assignments, contact Arlene Russell: russell@chem. ucla.edu.

Collaborative Library Building

Among institutions (NM CC’s; Other CC’s) Within an institution (CSUN Business School) Within a project (MSC consortium) Within a school cluster (Narbonne-San Pedro

School Cluster, ~24,000 students). Publishing Companies Consortia and individual partnerships

If You Create CPR Assignments,

You will change the way you teach. You will sharpen the focus on what you want

your students to learn. You will become more effective. You will be forced to ponder issues that you

are currently ignoring. You will be forced to define your assessment

strategy first.

How We Use CPR

Create a scientific database. Design an exploration of that database. Give the students guiding questions that lead them to

ask the questions that really matter. Assign a writing task. Create carefully thought out content and style questions. Have colleagues criticize the content and edit the

assignment.

A Useful Guide

John C. Bean, “Engaging Ideas: The Professor’s Guide to Integrating Writing, Critical Thinking, and Active Learning in the Classroom,” Jossey-Bass, San Francisco, 1996

What Does CPR Achieve?

Students learn science by writing. They construct meaning.

Students develop their writing skills. Students learn to evaluate peer writing. Students learn abstracting, reading for content,

and self-evaluation. Students think more deeply about important

aspects of what they are studying.

Using CPR, Students Learn

Observation through description. Comprehension and understanding through

analysis and clear, precise expression. Evaluation through critical assessment of peer

work. Synthesis and argument through organization

and logical expression.

CPR Enables

Development of an instructor’s own writing assignments, at a cost of about 6-8 hours per assignment using the authoring tool.

Collaborative assignment development over the net.

Significant student writing with little instructor effort (about 5 minutes per assignment), using assignment libraries.

Cooperative distance-learning courses

An AP Chemistry Course, Arlene Russell A one-year AP course, which bears credit,

for rural high school students is being distributed as distance learning.

This project is funded by the Office of the President of the University of California.

This course is designed around use of learning objects from the Molecular Science Curriculum project and CPR.

Does CPR Work?

A study from CSUN Business School: Two economics classes, each about 40

students, were selected for the study. Both classes took the same examinations. On each examination, two questions were

taken from material not covered by CPR assignments.

Does CPR Work?

On each examination, two questions covered material that the test class had studied as CPR assignments and the control class had learned in lecture and discussion sections.

On the essay questions in which neither class had access to CPR assignments, the two classes were indistinguishable.

Does CPR Work?

On essay questions in which one class had used traditional methods and the other class used CPR, the CPR class scored 60% better than the traditional class.

These results were confirmed in a second test of the same design.

What Makes CPR Special?

CPR is discipline independent. CPR is level independent. CPR has been used by more than

10,000 students.

CPR Funding

The National Science Foundation through the initiative for reform in chemical education

The Howard Hughes Medical Institute, for applications in life science

If you have questions about CPR,

Contact Orville L. Chapman, [email protected], or Arlene A. Russell, [email protected]. You may access the slidesand text for this presentation at http://www.molsci.ucla.edu/chapman/cpr.htm.