John Whitmer, Ed.D. Associate Director, Academic Technology Services California State University, Office of the Chancellor Society for Learning Analytics Research | LAK 2013 Case Study February 19, 2013 Using Learner Analytics to Understand Student Achievement in a Large Enrollment Hybrid Course slides posted:
30
Embed
Using Learning Analytics to Understand Student Achievement
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
John Whitmer, Ed.D.Associate Director, Academic Technology Services
California State University, Office of the Chancellor
Society for Learning Analytics Research | LAK 2013 Case StudyFebruary 19, 2013
Using Learner Analytics to Understand Student Achievement in
a Large Enrollment Hybrid Courseslides posted:
Outline
1. Context
2. Methods & Tools
3. Findings
4. Conclusions & Next Steps
1. CONTEXT
Founded in 1887
15,257 FTES, 95% from California, serves 12 counties
Primarily residential, undergraduate teaching college
Campus in California State University system (23 colleges, 44,000 faculty and staff, 437,000 students)
Conclusion: LMS Use Variables better Predictors than Student Characteristics
LMS Use
Variables
18% Average(r = 0.35–0.48)
Explanation of change in final grade
Student Characteristic
Variables
4% Average(r = -0.11–0.31)
Explanation of change in final grade
>
Smallest LMS Use Variable
(Administrative Activities)
r = 0.35
Largest Student
Characteristic
(HS GPA)
r = 0.31
>
Combined Variables Regression Final Grade by LMS Use & Student Characteristic Variables
LMS Use
Variables
25% (r2=0.25)
Explanation of change in final grade
Student Characteristic
Variables
+10%(r2=0.35)
Explanation of change in final grade
>
Question 3 Results:Regression by “At Risk” Population Subsamples
At-Risk Students: “Over-Working Gap”
24
Activities by Pell and Gradegrade / pelleligible
A B+ C C-
Pell-Eligible Not Pell-Eligible Pell-Eligible Not Pell-Eligible Pell-Eligible Not Pell-Eligible Pell-Eligible Not Pell-Eligible
0K
5K
10K
15K
20K
25K
30K
35K
Value
Content
Content
Engage
Engage
Assess
Assess
Admin
Admin
Content
Content
Engage
Engage
Assess
Assess
Admin
Content
Content
Engage
Engage
Assess
Assess
Content
Content Engage
Engage
Assess
Assess
Admin
Admin
Measure Names
Admin
Assess
Engage
Content
Extra effort in content-related activities
4. CONCLUSIONS & NEXT STEPS
Conclusions
1. At the course level, LMS use better predictor of academic achievement than student demographics (what do, not who are).
2. Small strength magnitude of complete model demonstrates relevance of data, but suggests that better methods could produce stronger results.
3. LMS data requires extensive filtering to be useful; student variables need pre-screening for missing data.
More Conclusions
4. LMS use frequency is a proxy for effort. Not a very complex indicator.
5. Student demographic measures need revision for utility in Postmodern era (importance to student, more frequent sampling, etc.).
6. LMS effectiveness for at-risk students may be caused by non-technical barriers. Need additional research!
Ideas & Feedback
Potential for improved LMS analysis methods: social learning activity patterns discourse content analysis time series analysis
Group students by broader identity, with unique variables: Continuing student (Current college GPA, URM, etc. First-time freshman (HS GPA, SAT/Act, etc)