Fair, Fish, Fowl, or Fraud? - Association of American ... PPT.pdf · Cornell University Standardized Testing in College Admissions Fair, Fish, Fowl, or Fraud? Robert J. Sternberg

Cornell University

Standardized Testing in College Admissions

Fair, Fish, Fowl, or Fraud?

Robert J. Sternberg

Cornell University

[email protected]

Cornell University

Cornell University

Problems with Standardized Tests

• Content

• Response format

• Proxy for SES

• Assume abilities are static

• Self-fulfilling prophecies

• Inertia

Cornell University

Main Messages

• Current tests are narrow and inequitable for certain groups

• We can do better

– Create better prediction of academic and nonacademic success

– Reduce racial/ethnic/SES differences

– Increase goodwill of applicants

Cornell University

Goal

• To measure equitably and with

high validity and reliability the

skills needed for

–Deep, reflective thinking

–Concerned, active citizenship

–Ethical leadership

Cornell University

Organization of Talk

• What is Successful Intelligence?

• Demonstration Projects

• HA*BIT: Holistic Assessment *

Broad Indicator of Talents

• Conclusions

Cornell University

Successful Intelligence

Students are SUCCESSFULLY INTELLIGENT when they have the skills to set and achieve their own goals in life according to their own standards within their life context, by:

1. Recognizing and capitalizing on strengths;

2. Recognizing and correcting or compensating for weaknesses.

Cornell University

Successful Intelligence

Capitalization and compensation enable

people to balance:

1. Adaptation to existing environments;

2. Shaping environments to improve

them;

3. Selecting new environments.

Cornell University

Successful Intelligence

• In order to succeed in school and life, you

need:

–Creative skills

–Analytical skills

–Practical skills

–Wisdom-based/ethical skills

Cornell University

Creative Skills

• create

• design

• invent

• imagine

• suppose

Cornell University

Analytical Skills

• analyze

• compare and contrast

• evaluate

• explain

• judge

• critique

Cornell University

Practical Skills

• use

• apply

• implement

• employ

• contextualize

Cornell University

Wisdom-Based Skills

• Seek a common good

• Balance one’s own, others’, and

larger interests over the long and

short terms

• Find an ethical solution

Cornell University

Cornell University

The Rainbow Project with the Rainbow Project Collaborators

• Develop a battery assessing

creative, analytical (SAT/ACT-like),

and practical intellectual skills that is

–Reliable and valid

–Practical

–Enjoyable to students

Cornell University

Movies

• The Unwanted Guest

• Jerry’s “Beauty-rest Sleeper”

• Letter of Recommendation

Cornell University

Item Types

Creative

• Multiple-choice items--verbal, quantitative, and figural

• Performance tasks—written stories, oral stories, cartoon captions

Cornell University

Written Stories

• “A Fifth Chance”

• “2983”

• “Beyond the Edge”

• “It’s Moving Backwards”

• “Not Enough Time”

Cornell University

Cornell University

Cornell University

Participants

• Total sample

–777 students from 13 colleges

and universities around the

country varying widely in

geographic area, race, ethnicity,

and ability level

Cornell University

Predicting GPA: SAT + Analytical

Step 1: SAT-Verbal, SAT-Math

Step 2: Analytic (STAT)

9.8 9.9

0

5

10

15

20

Step 1 Step 2

R s

qu

are

d (

%)

Cornell University

Predicting GPA: SAT + Practical

Step 1: SAT-Verbal, SAT-Math

Step 2: Practical (STAT + Separate Tasks)

9.8

12.9

0

5

10

15

20

Step 1 Step 2

R s

qu

are

d (

%)

Cornell University

Predicting GPA: SAT + Creative

Step 1: SAT-Verbal, SAT-Math

Step 2: Creative (STAT + Separate Tasks) 9.8

18.6

0

5

10

15

20

Step 1 Step 2

R s

qu

are

d (

%)

Cornell University

Predicting GPA: SAT + Analytic, Creative, Practical

Step 1: SAT-Verbal, SAT-Math

Step 2: All Rainbow Project Items

(STAT Analytic, Practical, Creative,

Practical Tasks,

Creative Tasks)

9.8

20.9

0

5

10

15

20

Step 1 Step 2

R s

qu

are

d (

%)

Cornell University

Predicting GPA: All measures (practical before creative)*

Step 1: SAT-M

SAT-V

HSGPA

Step 2: + Analytic

Step 3: + Practical

Step 4: + Creative

*Controlling for school quality in

dependent variable

15.6 15.2 15.9

24.8

0

5

10

15

20

25

30

Step

1

Step

2

Step

3

Step

4

R s

qu

are

d

Cornell University

Amount of Each Measure That Is Predicted

by Racial / Ethnic Differences (ω²)

0.09

0.04

0.00

0.02

0.01

0.02

0.03

0.01

0.02

0.01

0.03

0.00

0.05

0.10

Verb

al

Math

ST

AT

ST

AT

Written

Sto

ries

Ora

l

Sto

ries

Cart

oons

ST

AT

Movie

s

Colle

ge

Life

Com

mon

Sense

SAT Analytic Creative Practical

Pro

po

rtio

n e

xp

lain

ed

by

ra

ce

dif

fere

nc

es

Cornell University

Kaleidoscope With Lee Coffin and the Kaleidoscope Collaborators

• Insert creative, analytical, practical,

and wisdom-based essays as part

of the Tufts-specific admissions

application in order to broaden the

way we think about applicants

Cornell University

Types of Items

• Suppose the Nazis had won WW II.

What would the world be like today?

• Design a science experiment.

• Analyze a book you have read.

• How do you convince people of

ideas they do not initially like?

• How can you help improve the

world?

Cornell University

More Creative Story

• If the Trojans had heeded Laocoon’s

advice and thrown Odysseus’ wooden

horse into the sea, they would have

defeated the Greeks at Troy. Aeneas

would then never have had reason to flee

the city, and he would never have

ventured to Italy to found Rome. Without

Rome, neither the Roman Republic nor a

Roman Empire would have existed.

Cornell University

More Creative Story

• Concrete, the arch, plumbing, and the sauna

might never have been invented. The modern

implications of Rome never having existed are

indeed drastic. Lacking even concrete floors,

people would resort to sleeping in the mud,

and, without plumbing or saunas, they would

be perpetually filthy and, generally, quite chilly.

France could not have built the base of the

Eiffel Tower without arches, so tourists would

be unable to purchase miniature collectible

Towers in Parisian convenience stores.

Cornell University

Less Creative Story

• What if the ratification of the nineteenth

amendment did not pass and women were

never given the right to vote? What would life

for women, like me, be like in the United

States? For one thing, I probably would not be

writing this essay. If women were not given

their right to vote, I probably would stop going

to school after this year and it would be unlikely

that I would receive a college education.

Cornell University

Less Creative Story

Without suffrage, my career options would be

limited, if a career were a possibility at all. My

accepted practices would be limited to staying

home and taking care of the family. Rather than

being equals, women would be subservient to men.

I might not drive, I might not dress in the way in

which I choose to, and I might not be able to live

the way I want.

Cornell University

Data

• Kaleidoscope

– improves prediction of academic and

nonacademic success

– produces an academically stronger

class

– Greatly reduces ethnic and other group

differences

– Is enjoyable for and meaningful to

applicants

Cornell University

Data

• Number of applications rose

• Bottom third of old application pool

greatly diminished; many more top

applicants

• Average SATs rose slightly

• African-American applications up 25%,

acceptances up 30%

• Hispanic-American applications and

acceptances up 15%

Cornell University

Panorama With Kyle Wray and the Panorama Collaborators

• Project done at Oklahoma State

University (when I was provost and

senior vice president) to identify students

who would not be admitted by normal

criteria of ACT and/or high school GPA

• We admitted substantial number of

students with skills for success who

otherwise would have been rejected, and

who have succeeded at OSU

Cornell University

HA*BIT

• New generation of assessment that

measures creative, analytical, practical,

and wisdom-based skills

• Scored for each of these skills as well as

writing skills and passion for learning

Cornell University

OTHER PROJECTS

• Graduate-school admissions

• Business-school admissions

• Medical-school admissions

Cornell University

Conclusions

• Traditional abilities tests are narrow and

limited

• Our new measures can

– Broaden the range of skills tested for

educational purposes

– Increase predictive validity

– Decrease ethnic-group differences

– Increase customer satisfaction

Cornell University

Conclusions

• If you are interested in learning more, or

in collaborating in experimenting with

new admissions assessments for your

institution, please contact:

• Robert J. Sternberg

• [email protected]