PROMOTING FIRST-YEAR STUDENT SUCCESS Using Assessment & Collaboration … · 2013. 5. 21. · PSY #1803 CHEM #1456 ENG #2400 BUS #1625 HIST #1461 COMM #1444 SOC #778 PHIL #509 LEARN

PROMOTING FIRST-YEAR STUDENT SUCCESS Using Assessment & Collaboration To Improve Student Retention

Through Early Identification & Intervention

23rd International Conference on the First-Year Experience

J Kremer, K Thedwall & K Wendeln June 2010 © KEAW Page # 1

©2010 KEAW

John F Kremer – Psychology Kate A Thedwall – Gateway

Kenneth EA Wendeln – Business

Promoting First-Year Student Success

Using Assessment & Collaboration To Improve Student Retention


23rd International Conference on the First-Year Experience 2

Today’s Agenda 1.   Introduction

IUPUI and ‘Gateway’ First-Year Initiatives

2.   Measuring Progress Retention and ‘DFW’ Trends

3.   Explaining & Motivating Students B104 Introductory Psychology

4.   Collaboration at IUPUI ‘Gateway’ Success Stories

5.   Questions



Forbes: Best Colleges in the Midwest

http://www.forbes.com/2009/11/18/best-colleges-midwest-thought-leaders-public-2009_slide_9.html


‘Gateway’ First-Year Initiatives

  10 Year Retention Initiative

  55 ‘Gateway’ Gen-Ed Courses

  22,600 Fall Semester Grades

  450 Instructors

  55 Course Coordinators

  Director and Advisory Board

Current Focus: Early Identification

and Active Intervention

40%

45%

50%

55%

60%

65%

70%

75%

80%

F96 F97 F98 F99 F00 F01 F02 F03 F04 F05 F06 F07 F08 F09

All Entering FTFT (F-T Beginners) Transfers P-T Beginners


15pp Improvement (+27%) In Fall-to-Fall Retention

IUPUI Entering Students Fall-to-Fall Retention Trends

Source: KEAW Analysis of IU Retention Reports

-90%

-80%

-70%

-60%

-50%

-40%

-30%

-20%

-10%

0%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

F96 F97 F98 F99 F00 F01 F02 F03 F04 F05 F06 F07 F08 F09

23rd International Conference on the First-Year Experience 6 6

Fall-to-Fall Retention for Full-Time Beginners Only

IUPUI FTFT Students Retention vs DFW Grades

Source: KEAW Analysis of IUPUI DFW Reports

DFW Grades for Fall Gateway Courses

FTFT Retention

FTFT DFW Grades

Cohort





-95%

-85%

-75%

-65%

-55%

-45%

-35%

-25%

-15%

-5%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

F96 F97 F98 F99 F00 F01 F02 F03 F04 F05 F06 F07 F08 F09

23rd International Conference on the First-Year Experience 7 7

Fall-to-Fall Retention for All Students Entering in Fall

IUPUI All Entering Students Retention vs DFW Grades


DFW Grades for Fall Gateway Courses

Retention All Entering

Students

All Gateway DFW Grades

0%

0% 5% 10% 15% 20% 25% 30% 35%


Fall 07

27.2% #22637

Fall 06

Fall 05

Fall 04

Fall 03

Fall 02

Semester

31.5% #23041

32.1% #23345

31.5% #24124

31.1% #24091

29.7% #24089

Gateway Courses DFW Trends - All Students


All Gateway DFW% Total # of Grades

Fall 08

29.2% #22376

23.6% #22099 Fall 09

9

0 5

10 15 20 25 30 35 40 45 50

MATH #4370

PSY #1831

CHEM #1246

BIOL #1077

Gateway #23225

BUS #1869

SOC #881 ENG #2552

HIST #1649

PHIL #620

COMM #1503

ANTH #567

LEARN COMM #2378

Fall 09 Fall 08 Fall 07 Fall 06 Fall 05 Fall 04 Fall 03 Fall 02

Gateway Courses DFW Trend by Department

# Grades Fall 02-09

Ranked by Average DFW%



Σ


0 5

10 15 20 25 30 35 40 45

MATH #4112

BIOL #1097

Gateway# 22099

PSY #1803

CHEM #1456

ENG #2400

HIST #1461 BUS #1625

COMM #1444

SOC #778

PHIL #509

LEARN COMM #2544

ANTH #504

Fall 09 All

Fall 09 FTFT

Fall 09 Other

Gateway Courses Fall 2009 DFW by Student Type

DFW% by Type

Σ

# Grades Fall 09 by Dept


Psychology B104 DFW Trends

Semester Fall

2002 Fall

2003 Fall

2004 Fall

2005 Fall

2006 Fall

2007 Fall

2008 Fall

2009

# Grades 1516 1692 1559 1463 1365 1222 1377 1434

DFW % 26.8 31.6 35.9 32.7 45.9 34.3 32.0 22.9


Can We ….. These Students?

…..Describe

…..Explain

…..Predict

…..Change

12 23rd International Conference on the First-Year Experience

Example: B104: Introductory Psychology





Possible Reasons for DFW Grades

  Prior to IUPUI o  High School GPA and SAT/ACT

  Personal Preparation, Attitudes and Perceptions o  Test Preparation Skills o  Perseverance in Studying o  Interest in Doing Homework o  Expect Success in Study Efforts

  Current Academic Behavior o  Number of Hours This Semester o  Homework Completion

  Current Non-Academic Behavior o  Hours Working o  Life Stress

13 23rd International Conference on the First-Year Experience 14

% of DFWs

Attend Class

Did HWK

Pass Tests

Non-attenders ? No No No

Non-compliers ? Yes No No

Low Performers

? Yes Yes No

Drop Outs ? Yes Yes Yes

What Group has the Lowest Percentage of DFW Students?


15

% of DFWs

Attend Class

Did HWK

Pass Tests

Non-attenders No No No

Non-compliers Yes No No

Low Performers

Yes Yes No

Drop Outs 10% Yes Yes Yes

What Group has the Lowest Percentage of DFW Students?


Explaining Dropouts (10%)

 Are Dropouts related to HS GPA or SAT/ACT?


Explaining Dropouts (10%)

  NOT related to HS GPA or SAT/ACT

  No quantitative data explained these students. Qualitative interviews suggested the following

  Severe Stress

  Working 40 hrs/wk

  Taking 6-15 hours

  Plus: One course takes 2-3 hrs/day; &/or

  Plus: Major life event

  Cannot Predict Dropouts 17 23rd International Conference on the First-Year Experience

Examples Dropouts (10%)

  Person A   Working 40 hrs/wk

  Another Course: 2 hrs/day

  Husband sent to jail

  Can’t afford internet connection

  Nothing related to the course

  Person B   Working 42 hrs/wk

  Brother committed suicide

  Sleeps 10-12 hrs/day: Migraine pain

  Nothing related to the course






Support for Stressed Students

Create Flexible Course Structure C: Tests available 10 hrs/day, 7 days/wk during 3-

week period

C: Count highest of 3 test scores

C: If miss a class, may substitute any of 30 other sections: JagTag

Actively Reach Out M: Contact & Understand

U: Advising?


C: Course Design

M: Course Mentoring

U: University Support & Policies

20

% of DFWs

Attend Class

Did HWK

Pass Tests


Non-compliers ? Yes No No

Low Performers

? Yes Yes No


What Group has the Highest Percentage of DFW Students?


21

% of DFWs

Attend Class

Did HWK

Pass Tests

Non-attenders No No No

Non-compliers 40% Yes No No

Low Performers

Yes Yes No


Describing Non-compliers


Explaining Non-compliers (40%)

 Are non-compliers related to HS GPA OR SAT/ACT?



  NOT related to HS GPA OR SAT/ACT

  Motivational factors explained ONLY 21% of homework completion

  Hours working

  Perseverance in Studying

  Perception of Quality of Homework

  Implies many factors differing remarkably across students



  Key Factor:

Homework in High School o  HS = 4 hrs/wk for ALL courses

o  Two separate surveys of 400 students each

o  Survey of 15 high school, social studies teachers

o  B104 & all SOS courses = 4 hrs/wk for each

o  15 hour load = 20 hrs/wk

o  College requires approximately 5X the amount of studying outside of class than high school






Predicting Non-compliers (40%)

  Homework predicts test scores (r=.4-.5)

  Homework in the 1st week of the semester predicted total homework completed for the semester(r2 = .62)

25 23rd International Conference on the First-Year Experience 26

% of DFWs

Attend Class

Did HWK

Pass Tests



Low Performers

? Yes Yes No


What Percentageof DFWs Were Non-attenders?


27

% of DFWs

Attend Class

Did HWK

Pass Tests

Non-attenders 25% No No No


Low Performers

? Yes Yes No


What Percentageof DFWs Were Non-attenders?


Explaining Non-attenders (25%)

 Are non-attenders related to HS GPA or SAT/ACT?


Explaining Non-attenders (25%)

  NOT related to HS GPA or SAT/ACT

  No quantitative data predicted these students.

  Unable to interview even with extreme effort

  Students are Unavailable and do not attend for Unknownreasons

29

WHO should be responsible for contacting chronic non-attenders?


10 Minute Student Survey

  100% of students on first day of class stated they were definitely or very likely to complete the course

  Yet, 25% do not complete the course

  Why?

  A few reasons?

  Many different reasons?

30

Beginning of the semester asked all students about completing the course






Follow-up Student Survey

  80 students who did not take all exams

  51 different reasons (listed are >5%)

  Personal reasons (job – 8%)

  Test Taking (nervous, poor test taker – 15%)

  Study behaviors (didn’t do HWK or didn’t prepare well/didn’t know material – 16%)

  Motivation (lazy, skip class – 16%)

31

Why would you NOT successfully complete the course?


Understanding Non-attenders to Motivate Them

C: Motivate on 1st day of class

M: Contact & Understand

U: Threat of Administrative Withdrawal

U: Revise Early Warning System

U: Who (advisors?) follows-up on early warning

U: Change Ws to WX (not official grade)?

32

Need Early & Active Interventions


C: Course Design

M: Course Mentoring


 6 weeks

 4 weeks

 2 weeks

 1 week

33

Predicting the Incompleters Non-attenders and Non-compliers

When can we predict the students who will not complete their work?


  Students who do not complete the class can be identified in first 3-4 weeks of the semester by:

  Attendance &

  HWK completion (r2 = .42)

  Students who do not complete the class can be identified in first week of the semester by:

  Attendance &

  HWK completion (r2 = .26)

34

Attendance & homework completion QUICKLY predict the Incompleters



  Students who do not complete the class can be identified in first 5 days of the semester by:

  attendance &

  Homework completion   54% successfully completed 2 points of homework

  13% successfully completed 1 point of homework

  7% completed 1 HWK but below criteria

  26% did not complete any homework

35

Attendance & homework completion QUICKLY predict the Incompleters



Developing Motivation Create Relevance in Course Content

C: Class pts HWK & timely finish

C: Interesting HWK with validity

C: Application to students’ lives

Early and Multiple Interventions M: Interview, identify obstacles, problem solve, set

goals, follow-up

M: Emphasize weeks 1-4

U: Administrative Withdrawal tied to homework completion, response to an early warning system?


C: Course Design

M: Course Mentoring






 Moderate changes in homework

  1st change recommended homework = 3

  Actual homework completion = 2.74

  Excluding online courses

37

Results for Incompleters I Non-attenders and Non-compliers

Did these changes in course structure, assistance, & policy increase homework completion?


 Homework completion dependent on faculty who are teaching the course

 2 faculty had the lowest 6 sections with a 2.2 homework rate

 3 faculty had 9 of the highest 11 sections with a 3.1 homework rate

38

Some notable exceptions:


Results for Incompleters II Non-attenders and Non-compliers

 Homework completion dependent on when class was offered

 Sections outside of 7:30 – 4:30 M-R had 6 of 9 lowest sections

  2 online (1.03)

  4 on-campus (2.25)

39



Results for Incompleters III Non-attenders and Non-compliers

Athlete section (1 of 28)

 At the top in attendance

 2nd highest in homework completion  The highest course completion rate  2nd highest ABC (pass) rate

 2nd lowest test scores 40



Results for Incompleters IV Non-attenders and Non-compliers

 Athletic Advisors   Immediate feedback

to advisors  Talk to athletes  Consequence

  Study hall   Talk to Coaches

 Coaches – If consequences  Former students

41

Why Athletes Better??


  University/State/National Policy

  Course Structure

  Instructor

  Student Assistants

  Advisors

  Support Group

  Former Students 42

Multiple Factors for Success


Many are Required

& Differ by Student





43

% of DFWs

Attend Class

Did HWK

Pass Tests



Low Performers

? Yes Yes No


What Percentageof DFWs Were the Low Performers?


% of DFWs

Attend Class

Did HWK

Pass Tests



Low Performers

25% Yes Yes No


Describing All DFW Students


Explaining Low Performers (25%)

 Are low performers related to HS GPA or SAT/ACT?



46

#32

#30

r= .35

r=-.16

SAT Predicts Total Points? (r =.20)

#23

#35

r= .36

r=-.10

HS GPA Predicts Total Points? (r =.26)

SAT & HS GPA predict the High Performers, But NOT the Low Performers & Incompleters


  Study Skill factors (primarily) determine 58% of test performance

o  Test Preparation Skills = Best Predictor

o  Expectancy for Study Success

o  Homework Points

o  Hours Working

  HS GPA or SAT/ACT = 32%

47

Cannot change HS GPA or SAT/ACT, But CAN change test prep skills



 28% said poor (20%), very poor (6%), or terrible (2%)

 An additional 24% said fair

48


How would you rate your test preparation skills?






 Read the book TWICE

  LESS than this

 MORE than this

49


How will you prepare for your first test?


 Read the book TWICE ~ 20%

  LESS than this ~40%

 MORE than this ~40%

50


How will you prepare for your first test?


Developing Study Skills Low-risk skill-building opportunities C: Develop study aids

Flashcards, memory outlines, practice tests, Q/A text boxes

C: Multiple test opportunities

C: Drop a test

C: Immediate Feedback

Provide a skill-building discipline M: Match study method with student interest, set

goals, follow-up

U: Nothing? 51 23rd International Conference on the First-Year Experience

C: Course Design

M: Course Mentoring


Pre-College GPA and SAT/ACT is limited in predicting B104 success:

  Predicted only 1 of the 4 groups of students: Low Performers on Tests

  Had little relationship to the DFW grades of 75% of the students in the course

  SAT and HS GPA have low r with total course points

52

Putting It All Together – I


Culture to Schools: Pass Them

  Very low failure rates in HS

Students Pass even though:   Low levels of HWK activity

  Low levels of HWK completion

  Low test preparation skills

Thus, LOW ACADEMIC SKILLS

53

Putting It All Together – II


Students Come to College with:

 High Expectations for:   Completing the course (100%)

  Course grades (95% As or Bs)

  Low Expectations for:   Completing homework

  Required test preparation skills

54

Putting It All Together – III






And many students have LOW PERSEVERANCE when facing academic obstacles

  35% said facing these academic obstacles would get to them and probably prevent them from doing their best

  They think they…

  Can’t do it

  Course is too difficult for them

55

Putting It All Together – IV


Changing Marginally Motivated Students Depends on

Type of Student   Dropouts - easy

  Low Performers – semester

  Non-compliers – more difficult

  Non-attenders – many semesters

  Most have chronic habits that require many semesters of integrated work

  4 courses, 2 years

56

Levels of Intervention C: Class, Course

&Curriculum

M: Mentoring & Support Programs

U: University Administrative Policies

  Must use all Areas   Areas must be

coordinated   Some areas more

important for some problems


A Silver Bullet Is NOT a Model for Success

57

In the Silver Bullet model, you search to find the single straight

forward solution that will have a dramatic impact on student learning.


✗58

For the Additive model, each improvement increases the

amount of student learning. 23rd International Conference on the First-Year Experience

+ Course Design

Faculty

"Mentoring

"Advising

"Coaching

Administrative

Policy

Additive Model Can Lead to Improvement

Multiplicative Model Recognizes ALL Key Factors

59

In the Necessary but not Sufficient or Multiplicative model,

improvement in student learning occurs only when ALL of the necessary steps are present and active.

23rd International Conference on the First-Year Experience 23rd International Conference on the First-Year Experience 60

IUPUI Meta-Course

Mentoring/Support & Policies Courses &

Curriculum

Improving Retention from the ‘Inside Out’

My Class Your Class

http://gateway.uc.iupui.edu

[email protected] [email protected] [email protected]

PROMOTING FIRST-YEAR STUDENT SUCCESS Using Assessment & Collaboration … · 2013. 5. 21. · PSY #1803 CHEM #1456 ENG #2400 BUS #1625 HIST #1461 COMM #1444 SOC #778 PHIL #509 LEARN

Documents