YouEDU Staging Intelligent Interventions in MOOC Discussion Forums Akshay Agrawal Jagadish Venkatraman Shane Leonard Andreas Paepcke
Jan 18, 2016
YouEDU
Staging Intelligent Interventionsin MOOC Discussion ForumsAkshay AgrawalJagadish VenkatramanShane LeonardAndreas Paepcke
Agrawal 2
Agrawal 3
Agrawal 4
An Intelligent Forum
Agrawal 5
An Intelligent Forum
• Phase 1: Detect Confusion
Agrawal 6
An Intelligent Forum
• Phase 1: Detect Confusion• Phase 2: Intervene Automatically
Agrawal 7
Challenges
Agrawal 8
Challenges
• Scale:
Agrawal 9
Challenges
• Scale:
• Vocabulary of confusion is domain-dependent
Agrawal 10
Challenges
• Scale:
• Vocabulary of confusion is domain-dependent
• TA overloaded
Agrawal 11
Challenges
• Scale:
• Vocabulary of confusion is domain-dependent
• TA overloaded– lack of forum structure
Agrawal 12
• No tagged forum datasets• Availability of large-scale forum data
Agrawal 13
Why are forums still dumb?
YouEDU
• Pre-work: Dataset– Stanford MOOCPosts
• Phase 1: Confusion Detection– Classify individual posts w.r.t. confusion
• Phase 2: Intervention–Map confused posts to video snippets
Agrawal 14
MOOCPosts Dataset
• 30,000 posts,11 courses– Humanities/Sciences, Medicine,
Education
• Human-coded, 6 dimensions– confusion, sentiment, urgency [1-7]– opinion, question, answer [0/1]
Agrawal 15
http://datastage.stanford.edu/StanfordMoocPosts/
Agrawal 16
Phase 1: Detecting Confusion
Agrawal 17
Agrawal 18
Agrawal 19
Agrawal 20
Classifier Performance
Agrawal 21
Recap: Phase 1
Phase 2: Staging Interventions
Agrawal 22
Agrawal 23
Agrawal 24
Agrawal 25
Agrawal 26
Agrawal 27
Agrawal 28
Agrawal 29
Agrawal 30
Interventions: Evaluation
• 20 confused posts, course in statistics
• Six recommendations for each– randomize order
• 3 human raters– Label each snippet as relevant or
irrelevant
• k-precision– precision of retrieved documents,
limited to the first k recommendationsAgrawal 31
Agrawal 32
YouEDU Architecture
Agrawal 33
Classification Suitehttp://github.com/akshayka/edxclassify
MOOCPosts Datasethttp://datastage.stanford.edu/StanfordMoocPosts/
Agrawal 34
Agrawal 35
Agrawal 36
Agrawal 37
Agrawal 38