- 1. Understanding In-Video Dropouts and Interaction Peaks in
Online Lecture Videos Juho Kim (MIT CSAIL) Philip J. Guo (MIT
CSAIL, U of Rochester) Daniel T. Seaton (MIT Office of Digital
Learning) Piotr Mitros (edX) Krzysztof Z. Gajos (Harvard EECS)
Robert C. Miller (MIT CSAIL)2014.03.04 Learning at Scale
2. Video Lectures in MOOCs 3. Classrooms: rich, natural
interaction dataMaria Fleischmann / Worldbank on Flickr | CC
by-nc-nd Love Krittaya | public domainarmgov on Flickr | CC
by-nc-sa unknown author | from pc4all.co.kr 4. liquidnight on
Flickr | CC by-nc-sa 5. How do learners use videos?Data-Driven
Approach: Analyze learners interaction with the video player 6. Why
does data matter? detailed understanding of video usage design
implications for Instructors Video editors Platform designers new
video interfaces and formatsImproved video learning experience 7.
How do learners use videos? Watch sequentially Pause Re-watch Skip
/ Skim 8. Collective Interaction Traces Student #7888 Student #7887
...... Student #4 Student #3 Student #2 Student #1 video time 9.
Collective Interaction Traces into Interaction Patterns interaction
eventsvideo timesecond-by-second activity tracking 10. ~40M video
interaction events from 4 edX coursesLearners 127,839Mean Video
Processed Videos Length Events 862 7:46 39.3MCourses: Computer
science, Statistics, Chemistry 11. Analyzing Clickstream Events:
play / pause In-video time and absolute time Learner ID: first-time
or re-watching Clickstream interaction logPer-learner watching
segmentslearner xxxlearner xxx0:00 play 0: 34 pause 0:57 play 1:47
pauseSegment 1 - 0:00-0:34 Segment 2 - 0:57-1:47Per-second stats
for views, re-watches, plays, & pauses 12. Collective
Interaction Patterns 1. In-video dropout viewershipvideo time2.
Interaction peaks interaction eventsvideo time 13. 1. In-video
dropout viewershipvideo time2. Interaction peaks interaction
eventsvideo time 14. In-video Dropout % watching sessions that end
before the video finishes.viewershipvideo time 15. 36%: dropouts
withinrate few seconds 55%: overall dropout first viewership36%
19%55%video timeWhy? Auto-play playing unwanted videos Misleading
video titles / interfaces 16. Longer videos lead to more dropouts.
17. Re-watching sessions lead to more dropouts than first-time
sessions. 18. Tutorial videos lead to more dropouts than lecture
videos. Lecture introduction to concepts continuous flowTutorial
supplementary examples step-by-step demos 19. 1. In-video dropout
viewershipvideo time2. Interaction peaks interaction eventsvideo
time 20. Interaction Peaks Temporal peaks in the number of
interaction events, where a significant number of learners show
similar interaction patternsinteraction eventsvideo time 21.
Analytic Workflow Bin data into per-second segmentsApply a kernel
smootherDetect peaks 22. Re-watching sessions show stronger and
more peaks than first-time sessions. 23. Tutorial videos show
stronger and more peaks than lecture videos. LecturesTutorials 24.
What causes an interaction peak? 25. Observation: Visual
transitions in the video often coincide with a peak. 26. lecture
videonumber of re-watching sessions 27. Analytic Workflow Step 1.
Visual Analysis second-by-second pixel differences between adjacent
frameshead slidehead slidehead slide 28. Analytic Workflow Step 2.
Peak Categorization Manually inspected 80 videos Interaction peak
Visual transition 29. Five Explanations for an Interaction Peak
Type 1. Beginning of new material Type 2. Returning to content Type
3. Tutorial step Type 4. Replaying a segment Type 5. Non-visual
explanation 30. Type 1. Beginning of new materialinteraction 31.
Type 2. Returning to contentinteraction 32. Type 3. Tutorial
stepinteraction 33. Type 4. Replaying a segmentinteraction 34. Type
5. Non-visual explanationinteraction 35. 61% of interaction peaks
involved a visual transition. Peak CategoryFrequencyType 1.
Beginning of new material25%Type 2. Returning to content23%Type 3.
Tutorial step7%Type 4. Replaying a segment6%Type 5. Non-visual
explanation39%61% 36. Can interaction data improve the video
learning experience? 37. Lessons for instructors, video editors,
and platform designers 1. Make shorter videos. 2. Add informative
titles and easy navigation. 3. Avoid abrupt visual transitions. 38.
Lessons for instructors, video editors, and platform designers 4.
Make interaction peaks more accessible.5. Enable one-click access
for tutorial steps. 39. Next Steps: More Data Streams What would
transcript / text add to the analysis? How about acoustic data? 40.
Next Steps: Scalability Reliably & automatically detect peak
types? How much data is needed until we see patterns? viewership5
learners video timeviewership5,000 learners video time 41. For
instructors & editors: Video Analytics 42. For learners:
Data-Driven Video UI 43. Contributions A first MOOC-scale in-video
dropout rate analysis A first MOOC-scale in-video interaction peak
analysis Categorization of learner activities responsible for an
interaction peak Data-driven design implications for video
authoring, editing, and interface design 44. Understanding In-Video
Dropouts and Interaction Peaks in Online Lecture VideosJuho Kim MIT
CSAIL [email protected] juhokim.com 45. Backup slides 46. Domain
educational videosTheory interactive learningHow-to videos MOOC
videosRole of interactivity Learner control Subgoal labelingMethod
scalable data collection to realize theory Crowdsourcing
Learnersourcing 47. Vision in learnersourcing Feedback loop between
Learners: natural, pedagogically useful activities System: improve
interaction using learner data Visualize and analyze large-scale
video learning activities Use data to inform learning platform
design 48. How can we design online video learning platforms that
are as effective as in-person classrooms?