Edmund Cannon Electronic and traditional learning on an econometrics unit September 2009
Mar 28, 2015
Edmund Cannon
Electronic and traditional learning on an econometrics unitSeptember 2009
2 Introduction
Which students use Blackboard (BB) and how do
they use it?
Handouts (Word/pdf documents) versus Power-
point slides.
Possible consequences.
3Estimating treatment effects
Even with a perfectly-designed controlled
experiment, effects would need to be large for
them to appear statistically significant (sample
size 145 or fewer).
I have not performed a controlled experiment so
any correlations will not identify treatment effects.
But we can still learn something...
Nb in “idiot” regressions, the best explanators for
exam performance are lecture attendance and 1st-
year exam mark.
4Powerpoint etc.
Chen and Lin (2008, IREE) argue PP slides
improves student performance: other papers more
cautious.
Students like PP.
Need to think about how they use it.
5What are print-outs of PP slides for?
Probably helps dyslexic students.
May reduce the number of mistakes on some of the
pieces of paper in a student’s file.
May make students more passive (substitutes visual
for kinesthetic learning).
May change a student’s lecture attendance.
May substitute/complement other paper
resources.
i ii
X eé ùê úê úë ûåΕ
6What are (Word doc) handouts for?
Nb assuming a technical unit like
econometrics
Supplementary information to the text book.
Provides material to discuss in the lecture/class.
Lecturer too lazy / arrogant to use text book
properly.
7 QM3 Econometrics
Compulsory second-year unit for most Economics
students
Excluding Erasmus, etc, I analyse 145 students
(whole sample) or 115 students (omit EconMath).
All students have AAB+ at A-level, Maths A-level
grade A/B and have “passed” QM1 (Maths) and
QM2 (Statistics) in the first year.
Most students have a further compulsory
econometrics unit in the third year (not
EconMath).
8 QM3 Econometrics, cont'd
Textbook-based unit testing technical skills
(Murray: Econometrics: A Critical Introduction).
First term overlaps QM2 material - OLS (chapters
1-10).
Second term all new – heteroskedasticity, auto-
correlation, consistency, IV/2SLS, Demand and
Supply (chapters 11-14).
Exam: answer three questions on algebra, stata,
economic interpretation.
Statistics for total BB useage9
QM3 QM2 / Stat
Ever Term 2 Vac Term 3 Ever Term 2 Vac Term 3
I 47.0 8.1 2.5 4.9 47.7 9.6 2.1 4.9
II.i 44.3 8.8 2.0 5.2 41.0 7.6 1.8 4.6
II.ii 43.7 7.7 2.0 4.6 39.6 6.8 1.4 4.5
III 39.6 7.6 1.2 4.2 36.7 5.7 1.5 3.3
F 37.2 6.5 1.2 3.4 33.5 5.3 0.8 3.7
Ordered probit relating class of mark to BB use
is significant [p = 0.018].
10 What did I put on Blackboard?
Exercises (one electronic, the rest paper).
Handouts (Word/pdf documents).
Power-point files for lectures.
Lecture given
PP slides put on BB
Word doc put on BB
Lecture 15 30 Jan 7 Feb
Lecture 16 6 Feb 22 Feb
Lecture 17 13 Feb 22 Feb
Intro to IV 16 Feb
Sargan 16 Feb
Number of times students accessed teaching material
11
Tentative conclusion: students use BB as a
“mobile reference”
Number of times accessed this doc on BB12
QM3 Power-point slides (L15) Handout (Intro IV)
Ever Term 2 Vac Term 3 Ever Term 2 Vac Term 3
I 7.7 4.7 1.5 0.8 2.5 0.9 0.4 1.3
II.i 8.2 5.4 1.2 1.0 2.5 1.0 0.3 1.2
II.ii 7.1 4.5 1.2 0.7 2.2 0.9 0.4 0.8
III 6.7 4.5 0.7 1.0 1.8 0.8 0.1 0.8
F 6.4 4.0 0.7 1.0 1.7 0.8 0.2 0.8
Nb strong correlation between downloading
handout and downloading PP
Number of times accessed13
QM3 Power-point slides (L17) Handout (Sargan)
Ever Term 2 Vac Term 3 Ever Term 2 Vac Term 3
I 1.5 1.1 0.2 0.3 1.1 0.3 0.1 0.7
II.i 1.8 1.3 0.2 0.3 1.1 0.3 0.2 0.6
II.ii 1.3 1.0 0.2 0.1 0.9 0.2 0.2 0.5
III 1.4 1.1 0.1 0.2 1.0 0.4 0.0 0.4
F 1.6 1.0 0.3 0.4 0.7 0.4 0.0 0.2
14First time students accessed PP slides
First time students accessed handout
15
Proportion of students accessed on BB for the first time
16
QM3 Power-point slides (L17) Handout (Sargan)
Ever Term 2 Vac Term 3 Ever Term 2 Vac Term 3
I 0.80 0.70 0.05 0.05 0.70 0.30 0.10 0.30
II.i 0.81 0.68 0.08 0.05 0.68 0.30 0.14 0.24
II.ii 0.69 0.54 0.12 0.04 0.58 0.19 0.04 0.35
III 0.70 0.57 0.08 0.05 0.62 0.30 0.00 0.32
F 0.64 0.44 0.08 0.12 0.52 0.28 0.00 0.24
Conclusion: many students did not access material
that was needed to supplement the textbook.
17 Correlation with exam performance
QM3 mark QM3 mark Chen-Lin
Attend L15 21.7(6.5)
18.6(9.6)
0.064[0.029]
PP L15 26.9(8.5)
17.4(10.8)
0.0228[0.0149]
Attend * PP -19.8(8.8)
-16.5(10.7)
-0.0194[0.0316]
Lecture att 2.3(1.2)
1st year mark
4.9(1.3)
N 145 115
R-squared 0.16 0.31 Panel
18 Conclusions
Positive correlation between using BB and exam
mark – tells us nothing new;
Students access material on BB repeatedly
(reference);
Both good and bad students access material after
course is over (consistent with cramming);
What are students using material for?
19Estimating treatment effects
Consider a perfectly-designed controlled
experiment, where the treatment effect is a
binary variable and the explained variable is exam
mark.
{ }0,1 ; ,
i i i i
i i i i
Y d e
d d e
g= + +
Î ^
X b
X
Suppose the cross-section regression has an R2 of
0.15 (which would be considered reasonable),
that the standard deviation of exam mark is 15%
and the treatment effect raises performance by
5%.
Degrees of freedom must be about 150 for
statistical significance.