1 7/22/2004 Carnegie Mellon Project LISTEN Some Useful Design Tactics for Mining ITS Data Jack Mostow Project LISTEN (www. cs . cmu . edu /~listen ) Carnegie Mellon University Funding: National Science Foundation ITS 04 Workshop on Analyzing Student-Tutor Interaction Logs to Improve Educational Outcomes, Maceio, Brazil
20
Embed
Carnegie Mellon Project LISTEN 17/22/2004 Some Useful Design Tactics for Mining ITS Data Jack Mostow Project LISTEN (listen)listen.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1 7/22/2004
CarnegieMellon
Project LISTEN
Some Useful Design Tactics for Mining ITS Data
Jack MostowProject LISTEN (www.cs.cmu.edu/~listen)
Carnegie Mellon University
Funding: National Science Foundation
ITS 04 Workshop on Analyzing Student-Tutor Interaction Logs to Improve Educational Outcomes, Maceio, Brazil
2 7/22/2004
CarnegieMellon
Project LISTEN
Outline
1. Project LISTEN’s Reading Tutor
2. Modify tutor to get mineable data
3. Map data stream to analyzable data set
4. Mine data set to discover insights
3 7/22/2004
CarnegieMellon
Project LISTEN
Project LISTEN’s Reading Tutor (video)
4 7/22/2004
CarnegieMellon
Project LISTEN
Project LISTEN’s Reading Tutor (video)
John Rubin (2002). The Sounds of Speech (Show 3). On Reading Rockets (Public Television series commissioned by U.S. Department of Education). Washington, DC: WETA.
Available at www.cs.cmu.edu/~listen.
5 7/22/2004
CarnegieMellon
Project LISTEN
Thanks to fellow LISTENers
Tutoring: Dr. Joseph Beck, mining tutorial data Prof. Albert Corbett, cognitive tutors Prof. Rollanda O’Connor, reading Prof. Kathy Ayres, stories for children Joe Valeri, activities and interventions Becky Kennedy, linguist
Listening: Dr. Mosur Ravishankar, recognizer Dr. Evandro Gouvea, acoustic training John Helman, transcriber
Programmers: Andrew Cuneo, application Karen Wong, Teacher Tool
Field staff: Dr. Roy Taylor Kristin Bagwell Julie Sleasman
Grad students: Hao Cen, HCI Cecily Heiner, MCALL Peter Kant, Education Shanna Tellerman, ETC
Plus: Advisory board Research partners
DePaul UBC U. Toronto
Schools
6 7/22/2004
CarnegieMellon
Project LISTEN
Project LISTEN’s Reading Tutor: A rich source of experimental data
Log operations at grain size and level of interest Click <x, y> at time t: motor control Click “Goldilocks”: item selection
Reify operations to log them analyzably Handwriting or speech typed input Freehand drawing graphical palette (Geometry Tutor) Free-form responses menu selection (Self 88) Natural language sentence starters (Goodman 03)
Time student and tutor actions Time allocation reflects motivation (ITS 02) Hasty responses indicate guessing (TICL 04) Latency reflects automaticity (TICL 04)
8 7/22/2004
CarnegieMellon
Project LISTEN
Modify tutor: add relevant data
Randomize tutorial decisions What skill to test, what help to give
Probe skills Assess cognitive development (Arroyo 00) Test vocabulary words (IJAIE 01) Insert automated comprehension questions (TICL 04)
Import student data Gender, age, IQ (Shute 96) Prior knowledge (Corbett 00) Pretest scores (TICL 04)
Hand-label when appropriate Transcribe (some) spoken input (FLET 04)
9 7/22/2004
CarnegieMellon
Project LISTEN
Modify tutor: an example
Randomize: explain some new words but not others. Probe: test each new word the next day.
Did kids do better on explained vs. unexplained words? Overall: NO; 38% 36%, N = 3,171 trials (IJAIE 01). Rare, 1-sense words tested 1-2 days later: YES! 44% >> 26%, N = 189.
10 7/22/2004
CarnegieMellon
Project LISTEN
Map data stream to data set:structure data into a single type
Data stream: heterogeneous events over time Data set: elements with the same features