Affecting Student Persistence Via Institutional Levers A Report on the Pilot Study on Student Retention Don Hossler ([email protected])[email protected].

Affecting Student Affecting Student Persistence Via Persistence Via

Institutional LeversInstitutional Levers

A Report on the Pilot Study on A Report on the Pilot Study on Student RetentionStudent Retention

Don Hossler ([email protected])

Mary Ziskin ([email protected])Jacob P.K. Gross ([email protected])

Indiana University – Project on Academic Success

22

OverviewOverview

• What student retention efforts What student retention efforts seem to work (or not)?seem to work (or not)? • But why?But why?

• Where does this leave us?Where does this leave us?

• What are we learning from pilot What are we learning from pilot studies on student retention?studies on student retention?

33

A Sampling of Retention A Sampling of Retention InitiativesInitiatives

• Supplemental instructionSupplemental instruction

• Major/Career counseling programsMajor/Career counseling programs

• Learning communitiesLearning communities

• First-year experience seminarsFirst-year experience seminars

44

Supplemental InstructionSupplemental Instruction

• Participants more than twice as Participants more than twice as likely to persist year-to-year as likely to persist year-to-year as nonparticipantsnonparticipants

• Undergraduates were trained in Undergraduates were trained in cooperative learning strategies cooperative learning strategies & stuck to this approach& stuck to this approach

• They had a training manualThey had a training manual• In other words, the institution In other words, the institution

devoted time & resources to devoted time & resources to training – it continues to do sotraining – it continues to do so

55

Major/Career CounselingMajor/Career Counseling• Program for undeclared Program for undeclared

students at a church-students at a church-affiliated residential affiliated residential institutioninstitution• Participants 6 times more Participants 6 times more

likely to persist to degreelikely to persist to degree• Strong institutional focus Strong institutional focus

– campus has a director – campus has a director for this programfor this program

• Could success be linked Could success be linked to faith-based approach?to faith-based approach?

• Program is now required Program is now required & we are continuing & we are continuing study of the effectsstudy of the effects

• Program for undecided Program for undecided students at a students at a commuter campus run commuter campus run by career centerby career center• Participants were over 8 Participants were over 8

times more likely to times more likely to persist year-to-year as persist year-to-year as non participantsnon participants

• Participation was Participation was strongly encouraged by strongly encouraged by campuscampus

• Program connects Program connects students to staff students to staff advisors, faculty advisors, faculty mentors, and potential mentors, and potential employers employers

66

Learning Communities Learning Communities on a Commuter Campuson a Commuter Campus

• Had positive impact at a nonselective Had positive impact at a nonselective institution on first-to-second-semester institution on first-to-second-semester persistencepersistence

• No effects on year-to-year persistenceNo effects on year-to-year persistence

• We know little of how the program We know little of how the program was implemented and sustained over was implemented and sustained over timetime

• Has this been a success?Has this been a success?

77

First-Year Seminars First-Year Seminars at a Community Collegeat a Community College

• Program implemented at three separate Program implemented at three separate campuses within one regioncampuses within one region

• IPAS undertook program evaluation studies IPAS undertook program evaluation studies using both qualitative and quantitative methodsusing both qualitative and quantitative methods

• No evidence of program impact from these No evidence of program impact from these studies studies

• Little training of or coordination among faculty Little training of or coordination among faculty instructorsinstructors

• No additional institutional resources No additional institutional resources devoted to programdevoted to program

• Why would we expect this to work?Why would we expect this to work?

88

Some Caveats Some Caveats

• Student self-selection into programs Student self-selection into programs likely contributes to positive findingslikely contributes to positive findings

• The time commitments of The time commitments of participating campuses & their lack of participating campuses & their lack of experience with doing evaluation experience with doing evaluation research resulted in their not research resulted in their not collecting key data elements needed collecting key data elements needed for rigorous assessmentfor rigorous assessment

99

Where Does This Leave Us?Where Does This Leave Us?• We remain interested in better We remain interested in better

understanding how campuses understanding how campuses can intervene to positively can intervene to positively influence persistence.influence persistence.

• Because the commitments of Because the commitments of campuses to the way they campuses to the way they deliver programmatic deliver programmatic initiatives is so variable, we are initiatives is so variable, we are also interested in how also interested in how campuses organize themselves campuses organize themselves to address issues of student to address issues of student persistence. We think this persistence. We think this might be an important factor.might be an important factor.

1010

And…We Are Trying To Do And…We Are Trying To Do This!This!

• To understand how campuses To understand how campuses can adopt policies that enhance can adopt policies that enhance persistencepersistence

• We next report on two pilot study We next report on two pilot study efforts we have underway, both efforts we have underway, both funded by the College Board.funded by the College Board.

• One is a student survey that helps One is a student survey that helps campuses identify the policy levers campuses identify the policy levers they can use to improve they can use to improve persistence.persistence.

• The other survey looks at how The other survey looks at how institutions organize themselves to institutions organize themselves to enhance persistence & how this enhance persistence & how this might differentially affect might differentially affect persistence rates.persistence rates.

1111

College Board Pilot Study College Board Pilot Study on Student Retentionon Student Retention

• A survey of first-time, full-time, first-year A survey of first-time, full-time, first-year students at 8 four-year institutionsstudents at 8 four-year institutions

• Students surveyed at the end of their first Students surveyed at the end of their first year (spring 2006)year (spring 2006)• Web-based instrument, orWeb-based instrument, or• In-class paper-and-pencil administrationIn-class paper-and-pencil administration

• Response rates varied widely from under Response rates varied widely from under 10% to over 35%10% to over 35%

• Follow-up data collected from institutions Follow-up data collected from institutions to show enrollment in fall 2006to show enrollment in fall 2006

• Allows us to look at persistenceAllows us to look at persistence

1212

A few examples of survey A few examples of survey questionsquestions

• How certain are you that you have received useful How certain are you that you have received useful academic advising at this college?academic advising at this college?

• How often have you participated in classroom How often have you participated in classroom discussions… that included contributions from discussions… that included contributions from students with diverse backgrounds? students with diverse backgrounds?

• How certain are you that you've taken advantage of How certain are you that you've taken advantage of all federal and state financial aid programs you are all federal and state financial aid programs you are eligible for? eligible for?

• How often have you been unable to register for a How often have you been unable to register for a course that you needed…? course that you needed…?

• How many times per semester have you received How many times per semester have you received prompt feedback…from Instructors? prompt feedback…from Instructors?

• How often have you felt out of the loop with regard to How often have you felt out of the loop with regard to campus policies and procedures? campus policies and procedures?

1313

Participating CampusesParticipating Campuses

• Campuses includedCampuses included• 3 commuter campuses3 commuter campuses

• 2 small private liberal arts colleges2 small private liberal arts colleges

• 2 residential public universities2 residential public universities

• 1 public HBCU1 public HBCU

• Institutions in four statesInstitutions in four states

1414

Institution-Specific AnalysesInstitution-Specific Analyses• Descriptive informationDescriptive information

• Participation in student programsParticipation in student programs

• Classroom experiencesClassroom experiences

• Time diary itemsTime diary items• SatisfactionSatisfaction

• Inferential analysesInferential analyses• Explore factors associated with intent to persistExplore factors associated with intent to persist

• Merge data with SAT Questionnaire program Merge data with SAT Questionnaire program and fall 2006 enrollment data to explore and fall 2006 enrollment data to explore covariates of persistencecovariates of persistence

1515

Preliminary Results from Preliminary Results from Residential CampusesResidential Campuses

• A high proportion of students (87-92%) intend to persistA high proportion of students (87-92%) intend to persist• Factors capturing aspects of academic engagement emerge Factors capturing aspects of academic engagement emerge

on one campuson one campus• Campus 2: High Academic Engagement Campus 2: High Academic Engagement ((αα=.629 )=.629 )• Campus 2: Use of Campus 2: Use of Public Space for Learning (Public Space for Learning (αα=.607)=.607)

• Logistic regressions showed that a traditional Logistic regressions showed that a traditional persistence model enhanced the prediction of which persistence model enhanced the prediction of which students did not intend to persist students did not intend to persist

• Variables that contribute significantly to intent to Variables that contribute significantly to intent to persistpersist• Campus 1: development of friendship networks, class Campus 1: development of friendship networks, class

attendance, and positive perceptions about placement practicesattendance, and positive perceptions about placement practices• Campus 2: high combined SAT scoreCampus 2: high combined SAT score

• Variables that detract significantly from respondents’ Variables that detract significantly from respondents’ intent to persistintent to persist• Campus 2: distance of residence from campus, time spent Campus 2: distance of residence from campus, time spent

preparing for classpreparing for class

1616

Results from Commuter Results from Commuter CampusesCampuses

• A high proportion (84-93.5%) intend to persistA high proportion (84-93.5%) intend to persist

• Logistic regressions showed that a scaled-down Logistic regressions showed that a scaled-down traditional persistence model enhanced the traditional persistence model enhanced the prediction of which students did not intend to prediction of which students did not intend to persist persist • Campus 1: 71.4% correctly predictedCampus 1: 71.4% correctly predicted• Campus 2: 70.0% correctly predictedCampus 2: 70.0% correctly predicted

• However… However… • Overall variance explained by the models was relatively low Overall variance explained by the models was relatively low

(14.5-18.3%), though comparable to other research on (14.5-18.3%), though comparable to other research on persistencepersistence

• Academic engagement variables included in the models did Academic engagement variables included in the models did not show a significant effectnot show a significant effect

• Variables that contribute significantly to intent to Variables that contribute significantly to intent to persistpersist• Campus 1: development of friendship networksCampus 1: development of friendship networks• Campus 2: certainty of being able to pay for collegeCampus 2: certainty of being able to pay for college

1717

Factor AnalysisFactor AnalysisFactorFactor VariablesVariables AlphasAlphasPerception of Perception of opennessopenness

Ratings of social & academic Ratings of social & academic climate on campusclimate on campus

.70 .70 to .89to .89

Interaction w/ Interaction w/ studentsstudents

Support, advice, and academic Support, advice, and academic assistance from peersassistance from peers

.68 .68 to .87to .87

Interaction w/ Interaction w/ facultyfaculty

Support, advice, assistance, and Support, advice, assistance, and feedback from instructorsfeedback from instructors

.73 .73 to .85to .85

SatisfactionSatisfaction Satisfaction ratings on several Satisfaction ratings on several aspects of college experience: aspects of college experience: quality of teaching, support for quality of teaching, support for students, financial aid, overall students, financial aid, overall educational experienceeducational experience

.85 .85 to .90to .90

Certainty of Certainty of college choicecollege choice

Ratings of institutional quality, Ratings of institutional quality, advising practices, and own choice advising practices, and own choice of collegeof college

.70 .70 to .80to .80

1818

Cross-Case FindingsCross-Case Findings

• Differences across campuses are evidentDifferences across campuses are evident

• Robust factors emergeRobust factors emerge

• Early analyses show how policies, Early analyses show how policies, practices and environment play into practices and environment play into intent to persist, institution-by-institutionintent to persist, institution-by-institution

1919

Lessons Learned: Pilot Year-Lessons Learned: Pilot Year-11

• Small residential campuses have the highest Small residential campuses have the highest response rates, commuter campuses the lowest.response rates, commuter campuses the lowest.

• Paper-and-pencil administration on commuter Paper-and-pencil administration on commuter campusescampuses• Increased costIncreased cost• Much higher response rateMuch higher response rate• More complicated to administer, but may also be a good More complicated to administer, but may also be a good

indicator of…how serious campuses are about improving indicator of…how serious campuses are about improving student persistencestudent persistence

• Timing: We hope earlier administrations will Timing: We hope earlier administrations will improve response rates for residential campuses improve response rates for residential campuses that use a Web-based survey.that use a Web-based survey.

• Studying and improving student persistence is Studying and improving student persistence is difficult. It takes institutional commitment.difficult. It takes institutional commitment.

2020

Next StepsNext Steps

• We will merge fall enrollment records We will merge fall enrollment records to see who actually returned and then to see who actually returned and then re-run our current set of analyses.re-run our current set of analyses.

• Then we will also merge financial aid Then we will also merge financial aid data, and SAT questionnaire data to data, and SAT questionnaire data to see how these data affect our results.see how these data affect our results.

• We will readminister the survey in the We will readminister the survey in the winter of 2007.winter of 2007.

2121

Institutional Survey Sneak Institutional Survey Sneak PreviewPreview

• Survey of four-year institutions in California, Survey of four-year institutions in California, Georgia, Indiana, New York, and TexasGeorgia, Indiana, New York, and Texas

• Web-based administration, summer 2006Web-based administration, summer 2006• 32.8% response rate32.8% response rate

• Preliminary findingsPreliminary findings• 57.1 % of responding institutions have a 57.1 % of responding institutions have a

retention coordinatorretention coordinator• 97.3% of institutions analyze retention data 97.3% of institutions analyze retention data

annually annually • Annual analyses, broken out by Annual analyses, broken out by

race/ethnicity, 87.8%race/ethnicity, 87.8%• Annual analyses, broken out by major, 70.8%Annual analyses, broken out by major, 70.8%

2222

Institutional Survey Institutional Survey Preliminary Findings Preliminary Findings (Continued)(Continued)

• 43.5% report having semester-long 43.5% report having semester-long orientation programsorientation programs

• 60.5% reported average class size for first-60.5% reported average class size for first-year students at 30 or loweryear students at 30 or lower

• 82.9% require students to meet with 82.9% require students to meet with advisors each termadvisors each term

• 44.2% of retention coordinators rated the 44.2% of retention coordinators rated the availability of academic support at their availability of academic support at their institutions as higher than at similar institutions as higher than at similar institutionsinstitutions

2323

Final ThoughtsFinal Thoughts• Both surveys are Both surveys are works works

in progressin progress, but show , but show potentialpotential

• We will refine both We will refine both instruments and re-test instruments and re-test them early second them early second semestersemester

• We hope to shed light on We hope to shed light on how institutions organize how institutions organize themselves and what themselves and what they can do to enhance they can do to enhance student persistencestudent persistence

2424

Contact UsContact Us

Indiana UniversityIndiana University

Project on Academic SuccessProject on Academic Success

http://pas.indiana.eduhttp://pas.indiana.edu

Appended SlidesAppended Slides

2626

Data & Case DescriptivesData & Case Descriptives• A Public Residential UniversityA Public Residential University

• Public, residential, research-extensive Public, residential, research-extensive universityuniversity

• Approximately 9,000 undergraduatesApproximately 9,000 undergraduates• 92% White; next highest group: APA (2.5%)92% White; next highest group: APA (2.5%)• 49% expressed confidence that their 49% expressed confidence that their

families would be able to pay for collegefamilies would be able to pay for college• 87% reported an intent to persist87% reported an intent to persist

• Data for this institutionData for this institution• Response rate 22%Response rate 22%• Men underrepresented among respondents Men underrepresented among respondents

(46% of population; 27% of respondents)(46% of population; 27% of respondents)

2727

A Public Residential A Public Residential UniversityUniversity

2828

Findings in 4 casesFindings in 4 cases• Campus A (Nagelkerke=.245)Campus A (Nagelkerke=.245)

• (+) Friendships/Social network (+) Friendships/Social network (p<.01)(p<.01)• (+) Positive perceptions of English placement (+) Positive perceptions of English placement

(p<.01)(p<.01)• (-) Missing class (-) Missing class (p<.05)(p<.05)

• Campus B (Nagelkerke=.342)Campus B (Nagelkerke=.342)• (+) SAT (+) SAT (p<.05)(p<.05)• (-) Distance (-) Distance (p<.05)(p<.05)

• Campus C (Nagelkerke=.320)Campus C (Nagelkerke=.320)• (+) Certainty of major (+) Certainty of major (p<.05)(p<.05)• (-) Discussions with peers (-) Discussions with peers (p<.05(p<.05))

• Campus D (Nagelkerke=.209)Campus D (Nagelkerke=.209)• (+) Friendships/Social network (+) Friendships/Social network (p<.05)(p<.05)• (+) Feedback from instructors (+) Feedback from instructors (p<.05)(p<.05)

2929

Additional Survey Item Additional Survey Item ExamplesExamples

• Academic engagementAcademic engagement• During the current school year, how often have During the current school year, how often have

you met for discussion with other students?you met for discussion with other students?• During the current school year, how often have During the current school year, how often have

you used the library for research, reading, or you used the library for research, reading, or homework?homework?

• This college provides an open and welcoming This college provides an open and welcoming academic climate.academic climate.

• How many times per semester you have How many times per semester you have received support or encouragement from: received support or encouragement from: Instructors?Instructors?

3030

Additional Survey Item Additional Survey Item Examples Examples (continued)(continued)

• Use of Public Spaces for LearningUse of Public Spaces for Learning• On average, how many hours per week On average, how many hours per week

do you use campus computer facilities?do you use campus computer facilities?• How many times have you used a How many times have you used a

campus lounge to relax or study?campus lounge to relax or study?

3131

Additional Survey Item Additional Survey Item Examples Examples (continued)(continued)

• I am developing a social network and I am developing a social network and friends at this college.friends at this college.

• How many times have you been absent How many times have you been absent from courses?from courses?

• How certain are you that your placement How certain are you that your placement into English courses match your academic into English courses match your academic preparation?preparation?

• How certain are you that your placement How certain are you that your placement into math courses match your academic into math courses match your academic preparation?preparation?

3232

Additional Survey Item Additional Survey Item Examples Examples (continued)(continued)

• While you attend college, how many While you attend college, how many minutes away from campus do you minutes away from campus do you live? live?

• On average, how many hours per On average, how many hours per week do you spend preparing for week do you spend preparing for class (studying, reading, writing, class (studying, reading, writing, doing homework or lab work, doing homework or lab work, analyzing data, rehearsing, and other analyzing data, rehearsing, and other academic activities)?academic activities)?

3333

Additional Survey Item Additional Survey Item Examples Examples (continued)(continued)

• Ability to payAbility to pay• When considering financing your college When considering financing your college

education, how certain are you that you education, how certain are you that you and your family will have sufficient and your family will have sufficient funds to meet your expenses?funds to meet your expenses?

• How certain are you of your knowledge How certain are you of your knowledge about your financial aid options?about your financial aid options?

3434

Example of Survey Items Example of Survey Items (continued)(continued)

• Campus ClimateCampus Climate• How often have you noticed the How often have you noticed the

influence of multicultural and diverse influence of multicultural and diverse perspectives in campus artwork, perspectives in campus artwork, sculpture, or décor?sculpture, or décor?

• How often have you observed racist, How often have you observed racist, anti-gay/lesbian, or sexist behavior on anti-gay/lesbian, or sexist behavior on campus?campus?