Educational Question Routing in Online Student Communities Jakub Macina Slovak University of Technology Ivan Srba Slovak University of Technology Joseph Jay Williams Harvard University / National University of Singapore Maria Bielikova Slovak University of Technology 11th ACM Conference on Recommender Systems, Como, Italy, 27th-31st August 2017
31
Embed
Educational Question Routing in Online Student Communities
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Educational Question Routing in Online Student Communities
Jakub Macina Slovak University of Technology
Ivan Srba Slovak University of Technology
Joseph Jay Williams Harvard University / National University of Singapore
Maria Bielikova Slovak University of Technology
11th ACM Conference on Recommender Systems, Como, Italy, 27th-31st August 2017
2 / 31
Online Student Communities
• Massive Open Online Courses (MOOCs)
Dropout rate up to 94%
Community Question Answering (CQA)
Discussion forum
3 / 31
Challenge for MOOC Discussions
• Up to 50% of unanswered questions
• Course instructors are overloaded with many students to serve
• Low participation of students in question answering
1. Lurkers who are not contributing
2. Willing to participate but overloaded with many questions
4 / 31
Our Idea: Question Routing
• Recommendation of new questions to users who are suitable to answer them
• Well-known research task from CQA systems
What is capital city of Italy?
5 / 31
Related Work
• Question routing in standard CQA
• Asker-oriented approaches
• Overloading small group of experts
• Based mainly on QA data
6 / 31
Related Work
• Question routing in standard CQA
• Asker-oriented approaches
• Overloading small group of experts
• Based mainly on QA data
• Question recommendation in MOOCs
• Constraints optimization framework (Yang et al 2014)
• Any question beneficial to user
• With significant time-delay
Not appropriate for MOOCs
7 / 31
Educational Question Routing
8 / 31
Educational Question Routing Task
Given new question 𝑞
find an ordered list of users 𝑢1, … , 𝑢𝑛who are most suitable to answer question 𝑞
Opportunities
Data from MOOC course (grades, accomplished exercises)
Constraints
Appropriate knowledge
Willingness to answer
Working capacity
9 / 31
Goals of Educational Question Routing
• G1: Decrease information load of users by accurate recommendations
• G2: Engage a greater part of the community in the question answering
• G3: Increase an average number of contributions
10 / 31
Educational Question Routing Framework
11 / 31
Educational Question Routing Framework
User modeling phase
12 / 31
Educational Question Routing Framework
Routing phase
13 / 31
1. Construction of Question Profile
• Question text profile 𝜃𝑞
• Captures question’s content
• Text pre-processing
• Bag-of-words model (tf-idf weights)
• Metadata
• Asker, category, etc.
14 / 31
2. Construction of User Profile
• User text profile 𝜃𝑢
• Captures topics of question user previously answered
(user’s interests)
𝜃𝑢 =
𝑞𝜖𝑄𝑢
(𝜃𝑞 + 𝜃𝑎,𝑞)
• Metadata about previous user activities
• Quantity, quality and time distribution
• In CQA and MOOC
15 / 31
3. Matching of Questions and Users
• Ranking of users given new question
• Ensemble of two classification tasks:
• Appropriate expertise to answer a new question
• Willingness to answer a new question
• Combination:
𝑃(𝑦 = 1) = 𝑃(𝑒𝑥𝑝𝑒𝑟𝑡𝑖𝑠𝑒 = 1) ∗ 𝑃(𝑤𝑖𝑙𝑙𝑖𝑛𝑔𝑛𝑒𝑠𝑠 = 1)
16 / 31
3. Matching of Questions and Users• Features derived from text and metadata comparison between
question and user profile
• Features for expertise classification (# of features = 11)
• Level of difficulty for a user to answer a new question - knowledge gap
• Portion of related lectures watched
• Grades
• Features for willingness classification (# of features = 14)
• Overall count of answers, questions and comments
• Amount of latest activity
• Response time on rec.
17 / 31
4. Optimization
• Balancing routed questions by considering current student’s workload 𝐿𝑢
18 / 31
Experiments – CQA system
• Educational and organizational CQA system Askalot
• Open source, developed at Slovak University of Technology
• Builds on diversity in students’ knowledge and educational/organizational specifics