Admissions Testing: Predicting College Success CRESST September 2002 Wayne J. Camara The College Board
Admissions Testing: Predicting College Success
CRESST September 2002
Wayne J. Camara
The College Board
Overview Presentation focuses on…
• Admission Models and Practices• Predictors/Criterion of College
Success• Validity and Subgroup Differences• SAT I and SAT II• Changes to SAT in 2005
Importance of Admisson Factors – 4 year Public
1
1.5
2
2.5
3
3.5
4
4.5
1979 1985 1992 2000
HSGPA/rank
Admissions test
HS Courses
Letters
Essays
Portfolio, Auditions
Interviews
Importance of Admisson Factors – 4 year Private
1
1.5
2
2.5
3
3.5
4
4.5
1979 1985 1992 2000
HSGPA/rank
Admissions test
HS Courses
Letters
Essays
Portfolio, Auditions
Interviews
Preference in Admissions Test (4-year institutions)
• 82% of all colleges require test in 99 vs 86% in 89-90
• For non-open colleges, (5%) 91% required test in 89-90 vs 87% in 99
• Fairtest errors – only 173 of the 390 institutions listed are truly SAT/ACT optional.
Annual Survey of Colleges (College Board)
0102030405060708090
89-
90
94-
95
97-
98
98-
99
Non-Open Colleges
1 test
Either
none
Predictor-Criterion Relationship: CriteriaCriterion - What outcome do you want to
measure? Is it measurable? • Predict college success:
• completion (persistence, graduation)• academic (FGPA, Cum GPA)
• Life success: • occupational (salary, job title, degrees)• skills (team work, creativity, leadership)• community service
• Benefit: • social / economic mobility/ growth
Average Predictability of Cumulative GPA By Year (Kunkel et. al. 2002)
0.650.61
0.720.68
0.65
0.76
0.680.64
0.75
0.670.64
0.74
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
HS VM VMH
Year 1
Year 2
Year 3
Year 4
Four Years of Course Grades by Combined SAT Score
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
700 andbelow
710-800 810-900 910-1000 1010-1100
1110-1200
1210-1300
1310-1400
1410-1500
1510-1600
A (3.90-4.30)
A- (3.60-3.89)
B+ (3.30-3.59)
B (2.90-3.29)
B- (2.60-2.89)
C+ (2.30-2.59)
C (1.90-2.29)
C- (1.60-1.89)
D (0.60-1.59)
F (below 0.59)
Graduation
0%
20%
40%
60%
80%
100%
700 700-
849
850-
999
1000-
1149
1150-
1299
> 1300
A
B
C+
SAT V + M (Astin et al, 1996)
365 institutions, 76,000 students
Added value beyond high school grades
SAT HSGPA Both IncrementAll .57 .61 .68 .08 (Bridgeman, et. Al., 2000)
Male .56 .58 .61 .07Female .62 .61 .71 .10Cumulative .36 .42 .52 .10 (Burton & Ramist 2002)
Persistence (all correlations are uncorrected)Non-minority.25 .34 -Minority .35 .44 -Graduation .29 .33 .36 .03
Added value beyond high school grades
• Increment of SAT is about .10 above HSGPA and greatest for African American students
• HS grades overpredict College grades for Hispanic (.19) and African American males (.22) most. They underpredict grades for white females (-.09)
NELS Proficiency Levels x Ethnicity and Income
0 20 40 60
Inc
om
e
Percent
Math 5
Math 4 or 5
Reading 3
Same Group Differences in HS Grades and Class Rank
MeanHigh School Grades
Mean High School Rank
Subgroup A B C BelowC
90th 80th 60th Below60th
African-American
18.9 53.2 26.8 1.1 11.9 18.8 28.7 40.6
Asian-American
47.5 42.7 7.0 0.0 27.8 24.9 25.7 21.7
Hispanic 30.0 53.4 16.1 0.5 16.5 21.5 28.5 33.5White 40.3 47.8 11.7 0.3 23.2 23.2 27.3 26.3
Differences between HS & College Grades
Subgroup HS GPA FGPA Difference
African American
3.18 2.14 1.04
Asian 3.58 2.80 .78
Hispanic 3.33 2.37 1.16
White 3.40 2.66 .74
Parental Ed and HSGPA
0
10
20
30
40
50
60
<HS
HS Graduate
Some College
College G
raduate
Some Grad. S
chool
A
B
C or below
Graduation
Six-year graduation rates:
• African Americans (38%)
• Hispanics (45%)• Asians (63%)• Whites (56%)
• SAT scores have only slightly less weight in predicting graduation than HS grades.
• Actual effects are under estimated as students transfer and attrite who are in good standing.
Impact of SAT - Test/Grade Discrepant Score Analysis
• 68% of students have SAT scores that are highly consistent with their pattern of grades.
• The remaining 32% of students have discrepant SAT/grades. Half with higher grades. Unadjusted correlation of FGPA with SAT/HSGPA
SAT =HSGPA
HSGPAHigher
SATHigher
SATV+M
.457 .379 .452
HSGPA .462 .356 .464
N = 48,410, 23 colleges
SAT I vs SAT II
FGPA Course Grade
SAT I .60 .59
HSGPA .63 .57
SAT II .62 .57
SAT I + HSGPA .71 .71
SAT I + II .63 .63
SAT II + HSGPA .72 .69
SAT I, II+ HSGPA .72 .73
42,985 students entering 38 colleges in 82 and 85. Corrected for shrinkage, restriction of range and crit. unrel.
SAT I vs SAT II• SAT II more selective test takers • Mean SAT I and II for CA Students taking both:
V+M Writing Math 1c Math 2c 3 SAT II’s white 1200 593 574 646 1764 Latino 1018 (182) 506 (87) 500 (74) 572 (74) 1630 (134) Black 1016 (184) 509 (84) 487 (87) 576 (70) 1503 (261) Asian A 1150 (50) 540 (53) 581 (-7) 656 (-10) 1730 (34)
• Correlations:• SAT I V (SAT II W .79) (SAT II Lit .83)• SAT I M (SAT II M1C .84) (SAT II M2C .79)
SAT I vs SAT IIStandardized Differences
SAT I SAT II Writing/ Math IC
SAT II Writing, Math and 3rd
Asian Amer .27 .50 / -.11 .12
African Amer .98 .84 / .93 .95
Latino .97 .84 / .78 .50
K/C/J Spanish Other 3rd test
Latino - .31 (4,440) .69 (3,896)
Asian Amer -.47 (3,581) - .27 (14,359)
K/C/J = Korean, Chinese, Japanese
SAT I and ACT
• SAT I 138 questions vs ACT 210 questions in similar time
• SAT two scores (V, M), ACT one score (C, and four subscores)
• Concordance Study by CB/ETS/ACT (1997)• SAT V+M and ACT Composite r=.92• SAT M and ACT M r =.89• SAT V and ACT Reading or English r=.83• SAT V and SAT M r = .66
• Correlations between SAT I and SAT II are lower than Correlation between SAT I and ACT
New SAT – Related to College Skills (March/2005)
• All students take same test – equivalent to current SAT I
• Three scores (200 – 800 scale) - Critical Reading, Math Reasoning, & Writing
• Writing includes a 25-minute essay and MC• Distributed reading and essay available on web
• Math more curriculum-related, retain open ended items (qc items removed)
• Critical Reading – more passages based on curriculum (analogies removed)