The effectiveness of remedial education- addressing enhancement in student performance and retention in a post-1992 university Giorgio Di Pietro Westminster.

The effectiveness of remedial education- addressing enhancement in student

performance and retention in a post-1992 university

Giorgio Di Pietro

Westminster Business School

DEE annual conference

London School of Economics

6 September 2011

I would like to acknowledge the financial support received by the Economics Network of the Higher Education Academy

Background

The UK higher education system has undergone significant expansion in recent decades.

Alongside the benefits of this rapid expansion, there are also some concerns. One of these concerns is that a large number of students accepted into higher education institutions are unprepared for this level of study.

There is great evidence that the chances of obtaining a degree are significantly reduced for students that start university with inadequate academic skills.

Background-cont

Typically colleges and universities set up remedial courses in an attempt to address the problem of accepting unprepared students into higher education. These courses are specifically designed for first-year students and aim to help bridging the gap between students’ knowledge and the requirements for their degree courses.

Despite its growing importance, there is little evidence about the effectiveness of remedial courses. This is due to the paucity of data as well as to the difficulty of accounting for selection into remedial education.

Objective of the paper

This project aims at analysing the effectiveness of a math remedial course adopted by a single School of a post-1992 university.

It uses a regression discontinuity (RD) design in an attempt to address the problem related to selection into remedial education.

The RD design exploits the discontinuity induced by rules used for the assignment to remedial courses.

DataThe attention is focused on three cohorts of first-year students between the academic years 2007-2008 and 2009-2010. The total number of students is 1,521. Of these approximately 33.3% have failed the diagnostic test and hence have been assigned to a math remedial course.

The effectiveness of remedial education is measured against student performance on a math-based module that is compulsory for both remedial and non-remedial students. I consider two indicators of student performance on this module: 1) a binary variable that takes the value 1 if the student has successfully completed the module, and 0 otherwise; 2) the overall score (out of 100) received by the student on the module.

All students Nonremedial students Remedial studentsMean Mean Mean

Overall score on the compulsory first-year math-based course 53.239(16.375)

56.296(15.752)

47.109(15.890)

Successfully completed compulsory first-year math-based course 0.853(0.354)

0.900(0.299)

0.759(0.428)

Adjusted score on the math diagnostic test 2.380(4.283)

4.862(2.442)

-2.599(2.453)

Remediation 0.333(0.471)

- -

Compulsory first-year math-based course taken in the second semester 0.615(0.487)

0.603(0.490)

0.638(0.481)

Female 0.508(0.500)

0.490(0.500)

0.543(0.499)

Age< or =19 0.588(0.492)

0.621(0.485)

0.524(0.500)

Age >19 0.412(0.492)

0.379(0.485)

0.476(0.500)

Ethnicity

White 0.214(0.410)

0.218(0.413)

0.208(0.406)

Black 0.118(0.322)

0.081(0.273)

0.192(0.394)

Bangladeshi/Indian/Pakistani 0.217(0.412)

0.198(0.399)

0.255(0.436)

Chinese 0.021(0.144)

0.027(0.161)

0.010(0.099)

Other Asian 0.046(0.210)

0.044(0.206)

0.049(0.217)

Other ethnic group or Unknown 0.384(0.487)

0.433(0.496)

0.287(0.453)

Nationality

UK nationality 0.456(0.498)

0.412(0.492)

0.545(0.498)

Other EU nationality 0.258(0.438)

0.285(0.452)

0.204(0.403)

Non-EU nationality 0.286(0.452)

0.303(0.460)

0.251(0.434)

Socio-economic background

High socio-economic background 0.170(0.375)

0.175(0.380)

0.158(0.365)

Intermediate socio-economic background 0.105(0.306)

0.103(0.305)

0.107(0.309)

Low socio-economic background 0.150(0.357)

0.135(0.342)

0.180(0.384)

Unknown socio-economic background 0.576(0.494)

0.586(0.493)

0.555(0.497)

Observations 1,521 1,015 506

Differences in observable characteristics between remedial

and non-remedial students

In line with the findings obtained by similar studies (see, for instance, Attewell et al., 2006), remedial students are more likely to be older, more likely to come from an ethnic minority group and more likely to have a low socio-economic background relative to non-remedial students.

RD Design We focus our attention on students who either barely failed or

barely passed the math diagnostic test. The basic idea of the RD design is to compare the average performance of these two groups of students on the compulsory math-based module.

These groups of students are assumed to have the same observable (and unobservable) characteristics.

Difference in observable characteristics between remedial and non-remedial students within a 2- point interval around the cut-off on the math diagnostic test

Characteristic Remedial students Non-remedial students

Difference betweenremedial and non-remedial students

female 0.550 (0.498) 0.500 (0.501) 0.050 (0.045)Age< or =19 0.609 (0.489) 0.602 (0.491) 0.007 (0.044)UK nationality 0.543 (0.499) 0.509 (0.501) 0.034 (0.045)

Other EU nationality

0.211 (0.409) 0.241 (0.429) -0.030 (0.038)

White 0.218 (0.414) 0.236 (0.426) -0.019 (0.038)Black 0.163 (0.370) 0.106 (0.309) 0.057* (0.031)Chinese 0.010(0.102) 0.028 (0.165) -0.018 (0.012)

Bangladeshi/Indian/Pakistani

0.263 (0.441) 0.241 (0.429) 0.022 (0.039)

Intermediate socio-economic background

0.125 (0.331) 0.097 (0.297) 0.028 (0.028)

Intermediate socio-economic background

0.176 (0.381) 0.181 (0.386) -0.005 (0.345)

** indicates statistical significance at 5%;

Average score on the math-based module across remedial students

Average score on the math-based module across non-remedial students

Difference in average scores on the math-based module between remedial and non-remedial students

1- point interval around the cut-off on the math diagnostic test

48.667(15.753)

50.240(16.105)

-1.573(1.912)

2 -point interval around the cut-off on the math diagnostic test

47.937(15.875)

52.602(15.147)

-4.665***(1.400)

3- point interval around the cut-off on the math diagnostic test

48.104(15.886)

53.744(15.660)

-5.640***(1.193)

*** indicates statistical significance at 1%

Average proportion of remedial students successfully completing the math-based module

Average proportion of non-remedial students successfully completing the math-based module

Difference in the average proportion of remedial and non-remedial students successfully completing the math-based module

1- point interval around the cut-off on the math diagnostic test

0.784(0.412)

0.808(0.396)

-0.024(0.049)

2 -point interval around the cut-off on the math diagnostic test

0.782(0.414)

0.856(0.351)

-0.074**(0.035)

3- point interval around the cut-off on the math diagnostic test

0.775(0.418)

0.863(0.344)

-0.088***(0.029)

** indicates statistical significance at 5%; *** indicates statistical significance at 1%

Difference in average scores on the math-based module between remedial and non-remedial students once one controls for students’ characteristics

Number of observations

1- point interval around the cut-off on the math diagnostic test

-2.346(1.946)

308

2 -point interval around the cut-off on the math diagnostic test

-4.472***(1.399)

505

3- point interval around the cut-off on the math diagnostic test

-5.091***(1.192)

700

*** indicates statistical significance at 1%

Difference in the probability of successfully completing the math-based module between remedial and non-remedial students once one controls for students’ characteristics

Number of observations

1- point interval around the cut-off on the math diagnostic test

-0.034(0.050)

308

2 -point interval around the cut-off on the math diagnostic test

-0.077**(0.035)

505

3- point interval around the cut-off on the math diagnostic test

-0.083***(0.029)

700

** indicates statistical significance at 5%; *** indicates statistical significance at 1%

Conclusions

Our empirical analysis suggests that remediation does not matter!

This result is consistent with that obtained by Lagerlöf and Seltzer (2009) who use data from a pre-1992 university.

Other two findings (not shown in this presentations) are:

1) Timing of remediation does not matter either

2) Remediation does not affect the performance of students on other compulsory first-year modules