How to analyze data What do we do with the collected data? By Yun Jin Rho.
Mar 26, 2015
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
How to analyze data
What do we do with the collected data?
By Yun Jin Rho
How to analyze data l 09/24/113
Contents
1. Overview of data analysis MasteringAstronomy
2. Case studies - Group comparison (MyITLab) - Causal relationship (MyMathTest) - Item level analysis (MasteringEngineering)
3. Closing comments
How to analyze data l 09/24/114
Overview of data analysis
1
How to analyze data l 09/24/115
Raw data (MasteringAstronomy)
ID ADT-Pre Final Grade ADT-Post
1 15 0.863 182 3 0.832 83 11 0.685 154 6 0.855 8
5 0.853 16
6 10 0.894 167 2 0.823 4
8 0.773 10
9 4 0.787
10 5 0.238
11 0.842 13
12 12 0.841 1513 12 0.944 1814 7 0.952 15
15 0.749
… … … …… … … …
How to analyze data l 09/24/116
Descriptive statistics
N Mean SD Median Min Max Range SE
Pre 314 7.812 3.245 7 1 18 17 0.183
Post 291 11.825 3.909 12 2 21 19 0.229
Final 378 0.796 0.145 0.831 0 0.967 0.967 0.007
Improvement from the Pre scores (M = 7.812) to the Post scores (M = 11.825) was observed.
Is this amount of improvement statistically significant?
How to analyze data l 09/24/117
Distributions of test scores
How to analyze data l 09/24/118
Let’s assume the distributions of test scores are all normal.
How can we test if this difference of 4.263 is statistically significant? Dependent (paired) t-test
Hypotheses- H0: the improvement is not different from 0 (Pre = Post)- Ha: the improvement is significantly different from 0 (Pre ≠ Post)
Results95% Confidence Interval (C.I.) of the mean difference: [3.889, 4.637] t(250) = -22.44, p = 0.000 < 0.05 (significance level, )
ConclusionThe improvement of 4.263 was statistically significant.
Hypothesis testing
How to analyze data l 09/24/119
Then, how big is this improvement or effect? Effect size : measure of distance between the two different distributions
Cohen’s d = 1.36
ConclusionThere was a large learning effect.
Effect size
Cohen’s d < 0.3: small effect
Cohen’s d 0.5: medium effect
Cohen’s d > 0.8: large effect
How to analyze data l 09/24/1110
Was the students’ improvement in their test scores because of using Mastering? Is this improvement solely from using Mastering?
Pretest Posttest
Reading textbooks, Homework,
TA’s help, Quality of teaching,
Self-studying…
Pretest Posttest
Reading textbooks, Homework,
TA’s help, Quality of teaching,
Self-studying, Mastering…
Historical / Control without Mastering Experimental with Mastering
What does this large learning effect mean?
How to analyze data l 09/24/1111
Case studies
2
How to analyze data l 09/24/1112
Group comparison MyITLab
UIDReport
25Word
10Excel
15Exam
50Total 100
20111002 22.8 10 13.8 46.15384615 93
20111003 23 9.33 12.6 43.07692308 88
20111004 23.2 8.67 4.8 48.46153846 85
20111005 20.2 9.33 10.2 39.23076923 79
20111006 14 8.67 11.4 44.61538462 79
20111007 16.5 10 11.4 40.76923077 79
20111008 15 10 13.2 40 78
20111009 14 10 13.2 40.76923077 78
20111010 17.4 8.67 7.2 43.07692308 76
… … … … … …
UIDWord
10Report
30Excel
15Exam
45Total 100
20092002 8 27 13.5 27 76
20092003 7 24 10 31.84615 73
20092004 9 24 12.75 25.61538 71
20092005 9 23 13.4 24.92308 70
20092006 9 17.5 13 30.46154 70
20092007 7 26 12 24.92308 70
20092008 8 24 8 27.69231 68
20092009 8 21.5 11.75 26.30769 68
20092010 9 17 13 28.38462 67
… … … … … …
2011 1st semester using the labs 2009 2nd semester without the labs
.......
.......
How to analyze data l 09/24/1113
Group comparison With the labs vs. Without the labs
N Mean S.D. S.E.
Without LabsWord 123 75.58 16.89 1.52Excel 122 70.23 17.88 1.62Exam 124 55.77 10.27 0.92Total 124 61.37 9.73 0.87With LabsWord 228 79.88 20.13 1.33Excel 231 77.82 21.3 1.4Exam 233 64.86 12.41 0.81Total 233 69.39 12.09 0.79
How to analyze data l 09/24/1114
Path analysis (causal relationship) MyMathTest
IRB_ID PALG MALG PCALC ACTMATH 1109GRADE 1113GRADE
10766 0 32 A
10909 0 32 B
11291 97 16 F
10736 89 64 16 F
10747 28 21 W
10749 84 80 84 28 W
10944 13 20 B
10618 9 26 B
10902 34 24 B
10908 87 62 18 B
11254 54 27 D D
11264 54 27 D W
10582 93 92 91 29 A
10595 85 99 81 26 A
… … … .. … … …
How to analyze data l 09/24/1115
Path analysis Causal model
How to analyze data l 09/24/1116
Item level analysis MasteringEngineering
Pre-testStudent 1 2 3 4 5…
1 0 0 1 0 0…2 0 0 1 0 0…3 0 1 0 0 0…4 0 0 1 0 0…5 0 0 0 0 0…6 0 0 0 0 1…7 0 0 0 0 1…8 1 1 1 0 1…9 1 0 1 0 1…
10 0 1 0 0 0…11 0 0 0 0 0…12 1 0 0 0 0…13 0 0 0 0 0…14 0 0 1 0 0…15 0 0 0 0 0…16 1 1 1 0 0…17 1 0 0 0 0…18 0 0 0 0 0…19 0 0 0 0 0…20 0 0 0 0 0…21… … … … … …
Post-testStudent 1 2 3 4 5…
1 0 1 0 1 1…2 0 0 1 0 0…3 0 0 0 0 0…4 1 0 0 0 0…5 1 0 1 0 1…6 0 0 0 0 1…7 1 0 0 0 0…8 1 1 1 0 1…9 1 1 1 1 1…
10 0 0 0 0 1…11 0 0 0 0 0…12 1 0 0 0 1…13 1 1 1 0 0…14 1 1 1 0 0…15 0 0 1 1 0…16 0 0 0 0 0…17 1 0 1 0 0…18 1 1 1 0 1…19 0 0 1 1 1…20 0 1 1 0 1…21… … … … … …
How to analyze data l 09/24/1117
Item level analysis: Pre-test
How to analyze data l 09/24/1118
Item level analysis: Post-test
How to analyze data l 09/24/1119
Item level analysis: Posttest - Pretest
How to analyze data l 09/24/1120
Summary
3
How to analyze data l 09/24/1121
Closing comments
To analyze the data for efficacy studies, the most important thing is the study design to collect data.
Educational interpretation will be more important than the data analysis result.
How to analyze data l 09/24/1122
Thank you
Related Documents
