Blackboard Use and Exam Performace

Blackboard Usage and Exam Performance

Seamus CoffeyDept of Economics, UCC

s.coffey@ucc.ie 021-4901928

EC2101: Intermediate Microeconomics

Course began on 24th September 2008 and Blackboard website was available from 22nd September.

Exam took place on Wednesday 12th November and was for 10% of total marks available.

The one-hour exam comprised two questions each with two parts.

Each question was worth 50 marks and each part was worth 25 marks of the 100 available.

Exam Description

One week before the exam four questions were posted to Blackboard. Two of these appeared as the first part of the two questions. The other two questions posted were not used.

The second part of each question was unseen.1a – seen 1b – unseen; 2a – seen 2b – unseen Students were told that the second parts of the

questions would relate to material posted on the course blog on Blackboard from 25/09 to 04/11.

Overall Summary of Usage

Excluding Weekends and Bank Holiday

Number of people who visited the site for the first time on each day.9 people never visited the site prior to the exam.

Of the 6,068 “days” (164 students by 37 days) the site was visited on 1,358 days, or about 22%. Days exclude weekends and bank holidays.

Of these the bottom 50% accounted for 336 “days” and the top 10% accounted for 333 “days”.

The “top” student visited on 30 of the available 37 days which was matched by the bottom 25 students!

The average number of days the site was used was 8.3 (out of 37), the median was 7 and mode was 5.

There were 163 students registered for the module and 140 students sat the exam.

Student Use by Days0

51

01

52

0F

req

ue

ncy

0 5 10 15 20 25 30Number of Days Used

05

10

15

20

Fre

qu

en

cy

0 20 40 60 80 100Mark

Distribution of Marks

0.1

.2.3

Den

sity

0 5 10 15 20 25Question 1 Part A

0.0

5.1

.15

Den

sity

0 5 10 15 20 25Question 2 Part A

0.0

5.1

.15

.2D

ensi

ty

0 5 10 15 20Question 1 Part B

0.0

2.0

4.0

6.0

8.1

Den

sity

0 5 10 15 20 25Question 2 Part B

Distribution of Marks by Question

Variable | Obs Mean Std. Dev. Min Max

-------------+--------------------------------------------------------

Q1 Part A | 140 18.9 4.8 0 24

Q1 Part B | 140 9.6 5.4 0 24

Q2 Part A | 140 11.8 6.5 0 23

Q2 Part B | 140 9.6 5.4 0 24

Total mark | 140 47.9 14.7 10 90

-------------+--------------------------------------------------------

Part A | 140 30.7 9.3 6 47

Part B | 140 17.2 8.7 0 44

The average mark in the exam was just under 48%.

Students did substantially better in the questions that were seen (Part A: 30.7 out of 50) than in the questions that were unseen (Part B: 17.2 out of 50)

Variables

Mark v “Oldmark”0

20

40

60

80

10

0F

itte

d v

alu

es/

Exa

m m

ark

0 20 40 60 80 100Economics mark from Arts 1

02

04

06

08

01

00

Ma

rk

0 100 200 300 400 500Total Blackboard Usage from Start of Term

Mark v Total Blackboard Usage

“Oldmark” v Total Blackboard Usage0

20

40

60

80

10

0E

con

om

ics

ma

rk f

rom

Art

s 1

0 100 200 300 400 500Total Blackboard Usage for Term

Some Preliminary Regressions

Regression with robust standard errors Number of obs = 140 F( 1, 138) = 2.00 Prob > F = 0.1596 R-squared = 0.0085 Root MSE = 14.733

------------------------------------------------------------------------------ | Robust mark | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- totaltotal | .0274147 .0193897 1.41 0.160 -.0109246 .065754 _cons | 45.57679 2.137373 21.32 0.000 41.35055 49.80302------------------------------------------------------------------------------

Relationship is positive but not significant.

Regression of mark on “total usage of Blackboard site” from the beginning of term.

Source | SS df MS Number of obs = 140------------+------------------------------ F( 1, 138) = 3.05 Model | 653.283045 1 653.283045 Prob > F = 0.0829 Residual | 29555.6884 138 214.171655 R-squared = 0.0216-------------+------------------------------ Adj R-squared = 0.0145 Total | 30208.9714 139 217.33073 Root MSE = 14.635

------------------------------------------------------------------------------ mark | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- contotal | .0592053 .0338993 1.75 0.083 -.0078239 .1262345 _cons | 45.37734 1.907826 23.78 0.000 41.60499 49.14969------------------------------------------------------------------------------

Positive relationship and significant at the 10% level.

Regression of mark on “total content usage”, i.e. section of the site with lecture slides, readings and other handouts.

Regression with robust standard errors Number of obs = 140

F( 1, 138) = 5.32

Prob > F = 0.0225

R-squared = 0.0635

Root MSE = 14.318

------------------------------------------------------------------------------

| Robust

mark | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

anntotal | .1894081 .0821026 2.31 0.023 .0270664 .3517497

_cons | 42.90309 2.405475 17.84 0.000 38.14673 47.65945

------------------------------------------------------------------------------

Relationship is positive and significant at the 5% level. Graphed on next slide.Increased usage of the announcements section leads to a higher mark. Not surprising given that half of the exam was based on the 37 posts to this section.

Regression of mark on “total announcements (blog ) usage” from the start of term.

Mark v Total Announcement Usage0

20

40

60

80

10

0F

itte

d v

alu

es/

Exa

m m

ark

0 50 100 150Total Announcement Usage for Term

Regression with robust standard errors Number of obs = 140 F( 1, 138) = 0.90 Prob > F = 0.3442 R-squared = 0.0080 Root MSE = 9.281

------------------------------------------------------------------------------ | Robust a | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- contotal | .0226657 .0238778 0.95 0.344 -.0245479 .0698793 _cons | 29.70735 1.263107 23.52 0.000 27.2098 32.20489------------------------------------------------------------------------------

No significant findings.

Regression of combined mark from both part As (seen questions based on lecture content) and “total content usage”.

Regression with robust standard errors Number of obs = 140

F( 1, 138) = 6.90

Prob > F = 0.0096

R-squared = 0.0541

Root MSE = 8.4896

------------------------------------------------------------------------------

| Robust

b | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

anntotal | .1031929 .0392844 2.63 0.010 .0255157 .1808702

_cons | 14.50552 1.176658 12.33 0.000 12.17891 16.83213

------------------------------------------------------------------------------

Significant (at the 1% level) and positive relationship.

Regression with combined marks from part Bs (unseen questions based on blog posts to Announcements section) on “total Announcements usage”.

Source | SS df MS Number of obs = 140

-------------+------------------------------ F( 2, 137) = 4.00

Model | 579.53008 2 289.76504 Prob > F = 0.0206

Residual | 9935.69135 137 72.5232945 R-squared = 0.0551

-------------+------------------------------ Adj R-squared = 0.0413

Total | 10515.2214 139 75.649075 Root MSE = 8.5161

------------------------------------------------------------------------------

b | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

annall | .1209358 .0595487 2.03 0.044 .0031824 .2386893

annweek | .0708671 .0928607 0.76 0.447 -.1127586 .2544928

_cons | 14.77497 1.404544 10.52 0.000 11.99758 17.55236

------------------------------------------------------------------------------

Usage during term and not just prior to the exam is more important.But do the good students use Blackboard rather than students being good because they use Blackboard?

Regression of combined mark from part Bs on announcement usage from the start of term to November 4th (annall) and in the week prior to the exam (annweek) .

Source | SS df MS Number of obs = 109

-------------+------------------------------ F( 2, 106) = 34.38

Model | 8954.53176 2 4477.26588 Prob > F = 0.0000

Residual | 13803.0462 106 130.217417 R-squared = 0.3935

-------------+------------------------------ Adj R-squared = 0.3820

Total | 22757.578 108 210.718315 Root MSE = 11.411

------------------------------------------------------------------------------

mark | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

anntotal | .0502289 .0584326 0.86 0.392 -.0656194 .1660772

oldmark | .6348721 .0859628 7.39 0.000 .4644425 .8053017

_cons | 13.72065 4.2581 3.22 0.002 5.278557 22.16275

------------------------------------------------------------------------------

R-squared is now 0.39 and oldmark has a positive significant effect on mark. Total announcement usage which in the bivariate regression was significant is now very much insignificant. The dominant explanatory variable is oldmark.í

“Oldmark” is economics mark from Arts I for students. Reduces sample size to 109.