CUNY Office of Policy Research Working Paper At-Risk At College · 2017-04-17 · CUNY Office of Policy Research Working Paper At-Risk At College: Achievement Gaps at CUNY Colin C.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
New York University, Institute for Education and Social Policy, Steinhardt and Wagner Schools
Abstract
Increasing access to postsecondary education for students who have traditionally been excluded from
participation is a goal that is gaining greater attention from foundations and policymakers in recent years.
Access without completion, however, is problematic and college completion rates are stubbornly low.
Indeed, while between 1992 and 2006, the national college-going rate increased from 54% to 62%, college
completion rates remained relatively flat. Furthermore, disadvantaged students completed college at a much
lower rate than their more advantaged peers. Our study aims to examine the educational pathways of
students who are successful in higher education despite their disadvantaged backgrounds.
Success in this paper is measured as graduation from a four–year City University of New York (CUNY)
college within six years. We define individual (dis)advantage as individual-level demographic factors
shown to increase (decrease) a student’s odds of graduating from college and pre-college and college (dis)advantage as factors at these stages that support (impede) college success. Conducting multivariate
regression analysis with fixed effects to account for unobserved college characteristics, we examine
whether early indicators of student success vary depending on students’ (dis)advantage profiles.
Our data are uniquely suited to answering questions about students’ higher education trajectories; CUNY
maintains a rich database that allows us to track 11 cohorts of students -- from the application process to
graduation -- and to identify several successful outcomes, one of which, six-year graduation, we focus on in
this paper. We improve on previous research by including students who entered college in the spring
semester in our analyses. Ultimately, our results may inform the development of an early warning system at
CUNY, and other systems that serve large numbers of disadvantaged students, to help target interventions
to students who are at highest risk of failure and drop out.
2
Introduction
Increasing access to postsecondary education for students who have traditionally been
underrepresented is a goal that is gaining greater attention from foundations and policymakers in
recent years. Access alone, however, will not be sufficient to close gaps in post-secondary
degree attainment if disadvantaged students complete college at lower rates than their more
advantaged peers. Trends in college completion rates over the last 15 years suggest concern
about college completion is warranted. While the national college-going rate increased from
54% to 62% between 1992 and 2006, college completion rates remained relatively flat.
Unfortunately, there is little research to guide education policymakers and leaders in colleges,
high schools, or governmental and non-governmental organizations who aim to increase student
success rates. This study explores the early behavioral markers associated with academic
success or failure to aid in the development of interventions that reduce drop out.
Using data on students attending urban four-year colleges at the City University of New
York (CUNY), the city’s public university system, we examine the educational pathways of
students who are successful in higher education despite of their disadvantaged backgrounds.
What distinguishes those who succeed from those who do not? How early can we effectively
identify students who are at high risk of failure? We first focus our analyses on two background
characteristics known to be related to the rates of degree attainment among college students --
race and gender -- to investigate whether risk factors differ along these dimensions of
demographic background. We then add factors that capture high school performance and
ultimately college enrollment, program participation, and performance. We improve on previous
research by including students who entered college in the spring semester in our analyses and by
using 11 cohorts of students.
3
What Factors Might Influence Success?
High School and College Factors
Students’ academic preparation in high school, including enrollment in a college-
preparatory curriculum and grade point average, clearly shape a student’s prospects for success
at college (Adelman, 2006; Bowen, Chingos, & McPherson 2009; Murtaugh, Burns, & Schuster,
1999). Adelman (2006) suggests that enrolling in a college preparatory curriculum is the most
important step on the path to a baccalaureate. Bowen, et al. (2009) suggest GPA matters because
it is a marker of the motivation and skills needed to succeed in college, such as good study habits
and time management.
Several aspects of first-year college performance and enrollment are also related to
degree completion. GPA and number of credits accumulated in the first year are important early
college predictors of persistence and ultimate baccalaureate completion (Adelman, 2006;
Herzog, 2005). Students who enroll immediately after graduating from high school, enroll full
time, and attend summer school are also more likely to earn a bachelor’s degree (Adelman, 2006;
that focus exclusively on fall entrants exclude important demographic groups. For example,
Hispanic and black students are less likely than white students to start their studies in the fall
term. Lower SES students are also less likely to enroll in the fall, as compared to students from a
higher SES background (Adelman, 2006). A preliminary analysis of CUNY data suggests that
students who begin their post-secondary education in the spring semester tend to be less
academically prepared than fall entrants. Including spring entrants in our sample is an important
step to remedying this oversight in previous research. We also investigate the impact spring
entrance has on the likelihood of student success.
Data Source and Sample
To address our research questions, we use a student-level dataset drawn from
administrative files maintained by CUNY’s Office of Institutional Research and Assessment
(OIRA). Our data are uniquely suited to answering questions about students’ higher education
trajectories. CUNY’s database allows us to track student progress -- from the application
process to graduation -- and to identify proximal and distal college outcomes such as credit
accumulation, grade point average (GPA), and graduation. We track 11 Baccalaureate cohorts
across six years – or 150% of on-time graduation – in order to capture the majority of CUNY
graduates, who do not graduate “on-time” (i.e., four years).
Our sample consists of first-time freshmen students who entered CUNY Baccalaureate
programs in the fall or spring of academic years 1999-2004. We follow these students through
the CUNY system for six years, even if they transfer within the system. If students leave the
CUNY system, however, we do not have data on them and they are recorded as not being
7
retained and not graduating.1 As mentioned previously, we include both fall and spring entrants
in our analytic sample. The sample excludes the students who attended high school overseas or
who entered CUNY with GEDs, because they are missing data on key variables.
Table 1 displays descriptive statistics for the fall 1999 and fall 2004 cohorts of
Baccalaureate students entering CUNY; the fall 2004 cohort is the last cohort for which we can
calculate 6-year graduation rates. The proportion of black and Hispanic students per cohort
decreased slightly, while the proportion of Asian students per cohort increased slightly.
Approximately half of the fall 2004 cohort received Pell grant aid, down four percentage points
from the fall of 1999. Women composed 58% of the Fall 2004 cohort, as compared to 62% in
fall 1999. The graduation rate increased 5 percentage points from 43% to 48% over the
observation period.
We also calculated descriptive statistics (not shown) for the 57,211 students in the entire
sample. The analytic sample has an overall six-year graduation rate of 45%. This rate varies
substantially across subgroups of students, however, with white and Asian students graduating at
rates of 53-54% while the black student graduation rate is 35% and that of Hispanic students is
36%. Female graduation rates are higher than males (49% versus 39%) as is true nationwide,
with some variation in the gender gap within race groups. Among white students, women
graduate at a 59% rate, considerably higher than the 45% graduation rate for men. Among black
students, the disparity is somewhat lower: women graduate at a 38% rate while the graduation
rate for black men is 28%, and Hispanic women graduate at a 41% rate which is considerably
1 To be more precise, we analyze systemwide graduation rates rather than institutional rates. That is to say, students
in each cohort are counted as retained or graduated as long as they remain enrolled in, or graduate from, the
university system, not just their original institution. Therefore, a student who is enrolled in year two at a school within the system other than the one in which s/he originally enrolled is counted as having been retained.
Additionally, four- and six-year graduation rates are cumulative and include students who graduated within the
university system before the end of that particular reporting period. Although for this project we cannot track
students who leave the CUNY system, other internal analyses by OIRA suggest that few obtain a BA degree within
six years from other non-CUNY institutions.
8
higher than the 28% rate among Hispanic men. The lower overall rates for black and Hispanic
men make them a particularly important subgroup upon which to focus early indicators and
intervention efforts.
Methods
We estimate a series of probit models with cohort year and college fixed effects to
identify factors that increase (decrease) the likelihood of graduating within six years. We then
split the sample by race and gender and explore whether – and how much – the predictors of
success differ across subgroups.
Our first model – labeled “Background” – includes only background (non-academic)
characteristics of students:
(1) G i = β0 + β1R i + β2S i + β3I i + β4SPi + α + γ + εi
where G is a dichotomous variable that takes a value of 1 if student i graduated within six years;
R represents indicators for the five non-overlapping racial/ethnic groups; S is the gender of the
student taking the value of 1 for female; I represents variables related to student income,
including their dependency status (for aid purposes), and receipt of Pell grant; α represents
cohort year fixed effects; γ represents college fixed effects; and ε is an error term with the usual
properties.
The second model – labeled “Pre-CUNY”-- adds high school performance variables and
first term financial aid:
(2) G i = β0 + β1R i + β2S i + β3I i + β4SP + β5C i + β6RGTi + β7D i + β8F i + α + γ + εi
9
where G, R, S, I, SP, α, γ, and ε are defined as above; C represents the College Admissions
Average (CAA), more commonly known as high school grade point average, and College
Preparatory Initiative (CPI) units2, both expressed as z scores; RGT represents the highest score
on the New York State high school math test (Regent’s math) attained by a student and the
student’s high school English test score (Regent’s English), both expressed as z scores. The
Regent’s exams are a series of high school exit exams required for graduation in the state of New
York; D represents “no delay after high school” (less than 15 months); and F represents total
financial aid award in a student’s first term, expressed as a z score.
The third and final model – labeled “Post-CUNY”– supplements the previous two by
adding non-academic characteristics and program participation indicators for students when they
enter CUNY as well as including CUNY intermediate academic outcomes of students. This third
model is specified as:
(3) G i = β0 + β1R i + β2S i + β3I i + β4SPi + β5C i + β6RGTi + β7D i β8F i + β9SEEKi + β10SMR i
+ β11FTi + β12GPA i + β13CREDi + α + γ + εi
where G, R, S, I, SP, C, RGT, D, F, α, γ, and ε are defined as above; SEEK takes a value of 1 if
a student participated in the SEEK program; SMR takes a value of 1 is the student took courses
in the summer following the first year at CUNY; FT represents full-time attendance in the first
semester; CUNY GPA represents a student’s cumulative GPA after the first year (z score); and
CRED is a student’s credits accumulated after the first year (again, expressed as a z score).
2 High school units (or CPI units), total and by subject, are calculated as the number of units taken in courses
designated as college-preparatory by CUNY’s University Application Processing Center (UAPC), which is charged with standardizing high school course credits and grade point averages within and across school districts. Generally,
a full year course is worth 1 unit; units are pro-rated for courses offered for less than a full-year (e.g., a one-semester
course is worth 0.5 units). A student’s overall CAA is calculated as the weighted average of grades received in all
high school courses designated as college preparatory by UAPC: English, Math, Social Sciences, Science, Foreign
Language, and Fine Arts.
10
As mentioned previously, all models include college fixed effects to distinguish the
different CUNY’s baccalaureate granting colleges. Finally, we include cohort fixed effects by
year.
Results
All Students
As shown in Table 2, where marginal values of probit estimates, evaluated at means are
displayed -- holding other variables constant -- black and Hispanic students are less likely to
graduate than other racial and ethnic groups, although the magnitude of this effect is muted as
controls for high school performance and early college factors are introduced. As expected,
female students are always more likely to graduate, with the magnitude of the effect becoming
more muted with additional variables. In the pre-CUNY model, which includes high school
performance variables, we find that CPI units and CAA are significant predictors of graduation.
CAA appears to be more strongly associated with graduation success than the number of college-
preparatory course units earned in high school because the z-score coefficient is twice as large.
This makes sense, because students need a minimum number of CPI units to graduate from high
school. In our final model, summer school attendance, first semester GPA, and first semester
credits earned all emerge as strong predictors of graduation, while spring entrance is associated
with substantially lower likelihood of graduation (the latter in all models).
Racial/Ethnic Subgroups
Table 3 displays results for students by racial/ethnic group. Examining the coefficients
for demographic characteristics we see that the importance of gender varies across racial/ethnic
groups. Women of all ethnic groups are more likely to graduate, but the gender effect is
substantially larger for Hispanic students and white students. White students who are financially
11
dependent on their parents graduate at higher rates than white students who are financially
independent, but dependency status does not significantly predict graduation for other
racial/ethnic groups. Pell receipt significantly increases the likelihood of graduation for all
racial/ethnic groups, especially Asians. Spring entry is negatively associated with graduation for
all racial/ethnic groups.
Two high school performance variables are significant and positive for all racial/ethnic
groups: the number of college preparatory units (CPI) taken by a student in high school and high
school GPA for college preparatory courses (CAA). Somewhat unexpectedly, higher scores on
the Regents’ exams – New York’s high school exit exams – are associated with a very small
decrease in the probability of graduating among some groups.
College enrollment patterns and early college performance are also significantly
associated with graduation within 6 years of entry. Enrolling within a year of completing high
school and taking summer courses are associated with higher graduation rates for all racial/ethnic
groups. The latter has the highest magnitude for white and Hispanic students. Full-time
enrollment has a positive effect on graduation for whites and Asians but does not achieve
significance for blacks and Hispanics. A one standard deviation increase in first semester GPA
or credits earned is associated with a large increase in the likelihood of graduation for all groups.
Black Men, Black Women, Hispanic Men and Hispanic Women
Table 4 displays results for particularly at-risk groups of students, black and Hispanic
men, with black and Hispanic women included for comparison. Comparing gender subgroup
results for black and Hispanic students suggests that the positive association between Pell receipt
and graduation shown for these students in Table 3 was driven by women. Regardless of the
ethnic/gender category to which they belong, students who took more college preparatory units
12
and earned higher GPAs in high school are more likely to graduate. Attending summer school
has a uniformly positive association with graduation as do first semester GPA and credits
accumulated. The two enrollments variables that do differ in importance across groups are “no
delay after high school” and spring entrance. Entering college with a year of graduating from
high school has a stronger association with graduation for female students than male students,
within racial/ethnic group. Spring entrance has a negative effect for all groups, except black
women.
Conclusions and Discussion
The goal of this study is to identify risk factors that are most strongly associated with
student attrition. We also investigate whether risk factors for college drop out vary with student
race/ethnicity or gender. We find that many of the variables we examined were significantly
associated with college success across all groups of students. For example, we find that
academic performance during high school and early college are predictive of student success for
all students. High school and college GPA and the number of credits students earn in their first
semester are among the most influential variables we examined. Student enrollment choices,
such as the decision to enroll attend summer school, are also important for all groups.
Several of the predictors in our models have effects only for specific subgroups of
students. Receipt of Pell grant aid was not significantly associated with graduation for black or
Hispanic men. Enrolling in college within a year after high school was positively associated with
graduation for all groups, except black men. Spring entrance was negatively associated with
graduation for all groups other than black women. Higher Regents English test scores were
associated with a slightly reduced likelihood of earning a baccalaureate degree among black
women and Hispanic women.
13
Our research will assist CUNY administrators with the development of an early warning
system that tracks student progress and appropriately targets interventions to students at-risk of
drop out. Results suggest that baccalaureate students entering CUNY may initially be flagged as
“at-risk” based on demographic background. Descriptive statistics indicate that fewer than half
of black and Hispanic students graduate (33% and 36%, respectively), while more than half of
white and Asian students graduate (53% and 54%, respectively). Examining these at-risk groups
by gender, we see that only 28% of black men and 28% of Hispanic men who entered CUNY
four-year colleges had completed a baccalaureate degree six years later. Thus, an early warning
system aimed at reducing achievement gaps between racial and ethnic groups must focus on
black and Hispanic students, especially men.
Aspects of high school academic performance may also be used to initially identify
students at risk of drop out, because the number of college preparatory units and high school
grade point averages were both strongly associated with graduation. Our initial set of step-wise
models demonstrates that enrollment and academic performance at CUNY mediates
approximately one third of the high school GPA “effect”. Therefore, a comprehensive early
warning system would take these college factors into account as students progress through their
studies. First-year college GPA and number of credits accumulated are two variables that are
consistently associated with the probability of drop out for all racial and ethnic groups, and for
men and women. The models for black and Hispanic men also point to some “protective
factors” that may matter for these groups. Attending summer school was associated with a
slightly higher probability of graduating for both black and Hispanic men. Entering college after
high school without a delay was also positively associated with graduation for Hispanic men.
14
To be clear, the models we develop and estimate likely reflect many endogenous
relationships and we make no claim that the estimated coefficients are capturing causal
relationships. For example, those who attend summer school and complete more rather than
fewer credits their first year at CUNY are perhaps more likely to graduate in the first place.
Nonetheless the factors we identify can be used in at least three ways to improve the graduation
rates of at-risk students.
First, the statistically significant factors can be used to flag students at risk for special
attention. Second, and related, the flags used can be differentiated by subgroup. For example,
black and Hispanic students with below average high school grade averages need attention when
they enter CUNY. Third, the statistically significant factors may point to programs and
interventions that have the potential help students. These interventions, of course, would need to
be evaluated. As an example, black and Hispanic students who attend summer school after the
first year are more likely to succeed and, thus, incentives to help them do this could be helpful.
The number of credits accumulated during the first year at CUNY is important for all groups
and, thus, academic counselors may want to encourage students to take more credits and provide
them with support services so that they complete their courses rather than withdrawing mid-term.
There might, however, be a tradeoff between taking additional credits and first year GPA, which
would need to be monitored, since a higher GPA is also predictive of success.
The models we have developed point to the need to differentiate the systems by
subgroups, especially in urban post-secondary education systems with large numbers of at-risk
students. We also need to experiment with and rigorously evaluate alternative retention
programs that account for the varying importance of predictor variables across racial/ethnic and
15
gender groups. In future analyses, we expect to further explore both the student and institutional
characteristics that are associated with student success using duration models.
16
References
Adelman, C. (2006). The tool box revisited: Paths to degree completion from high school
through college. Washington, DC: U.S. Department of Education.
American Council on Education. (2010). Gender equity in higher education: 2010, Report #
312188. Washington, DC: Author.
Attewell, P. & D.E. Lavin. (2007). Passing the torch. New York: Russell Sage Foundation.
Bowen, W.G. & Bok, D. (1998). The shape of the river. Princeton, NJ: Princeton University
Press.
Bowen, W.G., Chingos, M.M., & McPherson, M.S. (2009). Crossing the finish line: Completing
college at America’s public universities. Princeton, NJ: Princeton University Press.
Bozick, R. & DeLuca, S. (2005). Better late than never? Delayed enrollment in the high school
to college transition. Social Forces, 84(1), 531-554.
Camburn, (1990). College completion among students from high schools located in large
metropolitan areas. American Journal of Education, 98(4), 551-569.
Carter, D.F. (2006). Key issues in the persistence of underrepresented minority students. New
Directions for Institutional Research, 130, 33-46.
Chen, R. & DesJardins, S.L. (2008). Exploring the effects of financial aid on the gap in student
dropout risks by income level. Research in Higher Education, 49, 1-18.
Cook, B.J. & Cordova, D.I. (2007). Minorities in higher education twenty-second annual status
report: 2007 supplement. Washington, DC: American Council on Education.
Goldin, C., Katz, L.F., Kuziemko, I. (2006). The homecoming of American college women: The
reversal of the college gender gap, The Journal of Economic Perspectives, 20(4), 133-
156.
Herzog, S. (2005). Measuring determinants of student return vs. dropout/stopout vs. transfer: A
first-to-second year analysis of new freshman. Research in Higher Education, 46(8), 883-
928.
Nora, A., & Cabrera, A.F. (1996). The role of perceptions in prejudice and discrimination and
the adjustment of minority students to college. Journal of Higher Education, 67(2), 119-
148.
McCormick, A. C. & Carroll, C. D. (1999). Credit production and progress toward the
bachelor’s degree: An analysis of postsecondary transcripts for beginning students at 4-
yearinstitutions.Washington, DC: U.S. Department of Education.
17
Moore, C. & Shulock, N. (2009). Student progress toward degree completion: Lessons from the
research literature. Sacramento, CA: Institute for Higher Education Leadership and
Policy.
Murtaugh, P.A., Burns, L.D., Schuster, J. (1999). Predicting the retention of university students.
Research in Higher Education, 40(3), 355–371.
O’Toole, D.M., Stratton, L.S., & Wetzel, J.N. (2003). A longitudinal analysis of the frequency of
part-time enrollment and the persistence of students who enroll part time. Research in
Higher Education, 44(5), 519-537.
Peltier, G.L., Laden, R., Matranga, M. (1999). Student persistence in college: A review of
research. Journal of College Student Retention, 1(4), 357-375.
St. John, E. (1991). The effects of student financial aid on persistence: A sequential analysis. Review of Higher Education, 14(3), 383-406.
St. John, E.P., Hu, S., Simmons, A., Carter, D.F., & Weber, J. (2004). What difference does a
major make? The influence of college major field on persistence by African American
and White students. Research in Higher Education, 45(3), 209-232.
Sum, A., Fogg, N., Harrington, P., with Khatiwada, I., Palma, S., Pond, N., & Tobar, P. (2003).
The growing gender gaps in college enrollment and degree attainment in the U.S. and
their potential economic and social consequences. Boston, MA: Center for Labor Market
Studies, Northeast University.
Velez, W. (1985). Finishing college: The effects of college type. Sociology of Education, 58(3),
191–200.
Zea, M.C., Reisen, C.A., Beil, C., Caplan, R.D. (1997). Predicting intention to remain in college
among ethnic minorities students. The Journal of Social Psychology, 137(2), 149-160.
18
Table 1: Descriptive Statistics for All Students
Baccalaureate Cohorts Entering CUNY in Fall 1999 and Fall 2004
Fall 1999 (N= 7,845) Fall 2004 (N= 10,349)
Variable Mean SD Mean SD % Change
6-year Grad Rate 0.426 0.495 0.481 0.500 0.055
White Student 0.328 0.469 0.323 0.468 -0.005
Black Student 0.221 0.415 0.194 0.395 -0.027
Hispanic Student 0.274 0.446 0.263 0.440 -0.011 Asian or Pacific Is. Student 0.176 0.381 0.218 0.413 0.042
Native American Student 0.001 0.037 0.002 0.043 0.001