Steve Tanimoto, Professor, Dept. of Computer Science & Engineering Steve Kerr, Professor, Dept. of Curriculum and Instruction University of Washington,

REAssess: Resources for Educational AssessmentSteve Tanimoto, Professor, Dept. of Computer Science & EngineeringSteve Kerr, Professor, Dept. of Curriculum and InstructionUniversity of Washington, Seattle

Jamie Cromack, Microsoft ResearchLaura Bofferding, Stanford University

Turadg Aleahmad, Carnegie-Mellon Univ.James Wen, Positive Motion

Tyler Robison, University of Washington, Seattle Brad Goring, University of Washington, SeattleEliana Medina, University of Washington, SeattleYen-Ling Lee, University of Washington, SeattleMikhail Manyak, University of Washington, Seattle

Outline

Origins of the REAssess project

Goals of the REAssess project

Assessment Framework

What is REAssess?

REAssess Demonstration

Issues & Feedback

Origins of the REAssess Project

Wanted to determine impact of funded projects on studentsCreated a document-based “assessment toolkit” for use on a small scale (Arizona State University)Recognized the value of deploying a web-based site to support all STEM facultyIdentified team of PIs to head the project (University of Washington)

REAssess the future of assessment and learning in science, technology, math and engineering with searchable annotated resources, tools and templates focused on the STEM disciplines

Goals of the REAssess Project

Engage the science, technology, engineering and math teaching (STEM) communities in deeper thinking about educational assessment

Offer useful, annotated, well-organized, targeted resources for educational assessment

Provide a clearinghouse for educational assessment-related activities (search, contribute, communicate, learn)

Host assessment-related services (future)

Assessment Framework

Traditional assessment landscape“Measure whether they can recall facts and do procedures”

Standardized tests

Short-answer tests

Essay tests

Summative privileged over formative

Provide valuable informationBut are only part of the picture

Growing concerns over traditional assessment approaches

How People Learn (NRC/NAP 1999)National Action Plan for … STEM Education (NSF, 2007)ABET 2008-09 Criteria for Accrediting Computing ProgramsCommission on … Higher Education (Miller-Malandra, 2006)

Quality of US higher education systemThe Post-American World (Zakaria, 2008)

Assessment Framework

Contemporary Assessment Approaches“Assess in order to see how much they understand and can do”Authentic assessment (“in context”)

Real world settings, simulations, portfolios

Group learningProjects, presentations, role-plays

Identifying components of competenceAdaptive expertise and self assessment

Continuous unobtrusive assessmentRegular formative feedback to learners

And to improve instruction!

Assessment Framework

What is REAssess?

Search

Contribute

Communicate

Learn

What is REAssess?

SearchSimple Search by Keywords and DisciplineSearch by WizardSearch Multiple Choice QuestionsExplore Assessment Services

What is REAssess?

ContributeContribute Multiple Choice QuestionContribute Other Resources & Reviews

What is REAssess?

CommunicateShare News about AssessmentRead the Assessment BlogContribute to the Assessment Wiki

What is REAssess?

LearnCases in Effective AssessmentAbout Educational AssessmentFeatured ResourcesFAQ

Eliana Medina

REAssess demo: search

Tyler Robison

REAssess demo: multiple choice

questions

Turadg Aleahmad

REAssess illustration:

Data Fusion Assessment Engine

Issues & Feedback

Which of these REAssess Features is Most Important to You?1. Promote sharing and reuse

2. Reviews and ratings of assessment resources

3. Easily adaptable tools and templates

4. Attract and promote cutting-edge assessments

5. Assessment tutorials & information

6. Respect for authorship of materials

7. Options for peer communication / interaction

8. Free access to assessment materials Pro

mote sh

aring a

nd reuse

Reviews a

nd ratings

of asse

ssment r

...

Easily

adaptab

le tools

and te

mplates

Attract an

d promote

cutti

ng-edge

as...

Assessm

ent t

utoria

ls & in

formati

on

Respect

for a

uthorship of m

ateria

ls

Options f

or peer

communica

tion / in...

Free

acce

ss to asse

ssmen

t materia

ls

0% 0% 0% 0%0%0%0%0%

Under Which Circumstance Would You be Likely to Use the REAssess Multiple Choice Question Generator for Tests?

Only if

new question in

stanc..

Only if

“use

with

variation” i

...

Only w

ith lim

ited acc

ess by .

..

Only if

it is r

estrict

ed to use

...

Only fo

r practi

ce te

sts, n

ot f..

0% 0% 0%0%0%

1. Only if new question instances are automatically generated.

2. Only if “use with variation” is encouraged

3. Only with limited access by students

4. Only if it is restricted to use by faculty

5. Only for practice tests, not for real tests

How Would Student Access Change Your Level of Contribution of Materials?1. I would contribute much more

2. I would contribute more

3. No effect

4. I would contribute less

5. I would contribute much less

I would co

ntribute

much

more

I would co

ntribute

more

No effect

I would co

ntribute

less

I would co

ntribute

much

less

0% 0% 0%0%0%

What Would be a Realistic way to have Separate Access for Faculty & Students?1. Bona-fide faculty vetted through detailed application

process

2. Bona-fide faculty vetted through a return-email process

3. Access to resources increases with level of involvement with REAssess site

4. Other

Bona-fide fa

culty

vetted t..

.

Bona-fide fa

culty

vetted t..

.

Access

to re

source

s incre

as.. O

ther

0% 0%0%0%

Should REAssess Resources be Associated with Particular Textbooks or Publishers?

1.Yes2.No

Yes N

o

0%0%

Microsoft Research Faculty Summit 2008