A Tale of Two Cities: Comprehensive Assessment of College Student Readiness Paul Gore, Ph.D. University of Utah [email protected].

A Tale of Two Cities: Comprehensive Assessment of College Student Readiness

Paul Gore, Ph.D.

University of Utah

[email protected]

Student Readiness

There is concern that college students are not prepared to meet the challenges of attaining a college education.

• Average six year graduation rate at two-year institutions = 34% (Swail, 2004)

• Average six-year graduation rate at four-year institutions = 53% (Carey, 2004)

• First to second year retention

• Traditional 73.4%; Selective 81.7%;Two year colleges 52%

Student Readiness

Student Readiness

Percent of Students Receiving Bachelor's Degree by Remedial Course Taking Behavior

58

27

17

0

10

20

30

40

50

60

70

No Remediation One or two MathCourses

Any RemedialReading

Per

cen

t

• Predicting student success– Primary focus on cognitive factors (ACT, GPA)– Pre-enrollment situational (SES, first generation)– Post-enrollment situational (Housing, hours

worked on campus, campus climate)– Non-cognitive/motivational (engagement,

achievement motivation, self-efficacy)

• Your institutions probably focuses on all 4

Student Readiness

Student Readiness

• In 2004, Robbins and his colleagues published a meta-analysis of non-cognitive prediction of student outcomes

• Their work included calculations of incremental validity

Student Readiness

Persistence and SuccessHigh School GPA

Non Cognitive Factors

Overview of the SRI

The Student Readiness Inventory (SRI) was designed to measure the content domains that surfaced in the Robbins meta-analysis

• It is a low stakes, self-report instrument• Has 108 items that are face-valid • Ten content domains exist as scales• Each scale measures a construct that allows for

intervention• The instrument contains two risk indices that

predict probability of success (performance and persistence)

Overview of the SRI

Overview of the SRI

*

*

*

**

*

*

1st Sem 2nd Sem 2nd Sem 1st Year GPA GPA Retention Retention

Academic Discipline 0.28 0.29 0.11 0.13

Academic Self-Confidence 0.17 0.19 0.06 0.09

Commitment to College 0.12 0.10 0.12 0.12

General Determination 0.12 0.12 0.06 0.06

Social Connection 0.05 0.04 0.10 0.09

ACT Composite Score 0.41 0.46 0.11 0.17

High School GPA 0.45 0.49 0.15 0.20

SES 0.16 0.19 0.07 0.10

Overview of the SRI

Institutional Application

• Programs designed to promote the transition of first-year students abound

• Over 75% of all colleges and universities have an FYE seminar course

• Many colleges and universities lack a systematic, integrated, and coordinated set of programs

• Effort was undertaken to bring together first-year campus programs around a common focus and to use data to drive coordination efforts


• Project developed by workgroup on first-year including advising, faculty, FYE, Native American and Multicultural Student Centers, Assessment, LAC, New Student programs

• Goal: proactive, intrusive interventions which could influence first year student success.


Requirements

• Data from assessment of student success tool must be usable by both student services/faculty and students

• Data must be presented in a user friendly, accessible format

• Data must be provided early enough for dissemination and use by both student services and students themselves

• Results must connect students to university resources

• Different areas of campus must be willing to provide programming to meet the needs identified by the students success tool


1. ID early and make direct contact with students who were determined to be at risk for either failure to persist at NAU or failure to remain in good academic standing

2. Use the information to match student needs with specific university resources and services

3. Embed the student profiles in FYE seminars to facilitate understanding of the connection between student needs and campus resources

4. Use composite indices of retention and academic success to build a customized retention prediction equation for NAU


1. Direct contact with students at risk

Composite indices in SRI used Identified group of reachable students based on resource

decisions. Interventions varied NAU used retention risk index and did not allocate their

resources to the lowest percentiles

2. Connect students to campus resources using scale scores

Direct contact via email or postcards encouraging students to access services or participate in events. SRI not explicitly referenced

Examples: Students with potential academic problems invited to LAC; Students with high scores on academic discipline and goal striving invited to meet with Res Life and New Student Programs to participate in leadership opportunities


3. Including student SRI profiles in lesson and instructional activities in FYE seminar courses

Traditional 3-credit FYE for special admit students One-credit FYE course for regularly admit students Scores used regularly as opportunities for reflection in class Students met one-on-one with instructor to develop student

success action plan



0 200 400 600 800 1000 1200 1400 1600 1800

Number of Studentslow med high very high

Academic self-confidence

Emotional control

Social activity

Social connection

Communication skills

Study skills

Commitment to college

Goal striving

General determination

Academic discipline

Official Freshman Cohort n=3171

SSS4%

LC4%

STAR5%

INVALID7%

GSSC SA LOW

9%

EPS Class5%

LAC6%

MSC7%NO SRI

11%

General GSSC21%

FYE Class15%


Resource Use by GPA

2.67

2.76

2.87

2.63

2.91

2.86

2.53

2.95

2.62

2.82

2.67

2.3

2.4

2.5

2.6

2.7

2.8

2.9

3

0-4 5-19 20+ 0 1 2+ 0 1+ 0 1-3 4+

GP

A

Recreation Resources Social Resources Academic Referrals Advising/Career Sessions


Resource Use by Retention

0.71

0.77 0.77

0.69

0.85

0.72

0.6

0.88

0.52

0.76

0.84

0.4

0.5

0.6

0.7

0.8

0.9

1

0-4 5-19 20+ 0 1 2+ 0 1+ 0 1-3 4+

Ret

enti

on

Recreation Resources Social Resources Academic Referrals Advising/Career Sessions


• Final Predictive Model of GPA (33% of variance)– Academic risk– Academic services– Academic services X Academic risk– Recreational resources– Social Resources– Academic referrals– Academic referrals X Academic risk– Advising/Career Services– Ethnicity– Gender


• Final Predictive Model of Retention (13% of variance)– Retention risk– Academic services– Social resources– Academic referrals– Advising/Career services

Institutional Initiative

• Establish baseline understanding of U of U students’ strengths and weaknesses

• Develop strategic data-use plan with academic and student support services

• Provide training as needed to support outreach and early warning efforts


• When compared to the national sample– 7.8% of students fall in the lowest quartile

on the Retention Risk Index (N=175)

– 9.2% of students fall in the lowest quartile on the Performance Risk Index (N=206)


AssessmentAdmissions Index/SRI

Pre

-Enr

ollm

ent

Data Processing Early Outreach

Firs

t S

emes

ter

AdvisingCollege Success

Class

Special PopulationServices Outreach

Early Warning SystemOther UniversitySupport Services

First-Year Seminars

Long

-Ter

m

Career ServicesLeadership/Mentors

Utilization/Engagement

Retention/Graduation

Program Refinement


• Push back– Students don’t like taking the instrument– Too much time out of orientation (20

minutes)– Not enough time available in advising– Tracking concerns

– Support exists at the administrative level but not the front line

A Tale of Two Cities: Comprehensive Assessment of College Student Readiness Paul Gore, Ph.D. University of Utah [email protected].

Documents

student readiness slide

year student success

student needs

sri slide

student profiles

student servicesfaculty

persistence slide

student readiness persistence