Bridging Research and Practice in Distance Education A communication management approach to a state-of-the-art Online Learning Environment. Erin Shaw CARTE/DEN.

Bridging Research and Practice in Distance Education

A communication management approach to a state-of-the-art Online Learning Environment.

Erin Shaw

CARTE/DEN Collaboration Viterbi School of Engineering

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Goals and Progress

DEN OLE goals

Feature innovative teaching and learning facilities Increase learner-centered participation and collaboration Encourage instructor-initiated experimentation and assessment

Bridge research and practice in distance education

CARTE progress toward goal

Deploy ISI discussion board Automate registration via single sign on from Blackboard Render development at ISI independent of production at DEN

Integrate discussion and artifacts (course doc, webcasts) Allow students to attach and share documents Focus discussion and assessment

Expand communication capabilities Support IM (mobile), RSS feed, summarization and activity watches

Extend document capture capabilities Generate text, html, and pdf file formats and metadata Parse, segment, tile, and make searchable

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Integrated System Overview

ISI Discussion Board

Agent Response

ISI Document Database

DENBlackboard

System

StudentCollaboration

Tools

DENFile

System

InstructorAssessment

Tools

AgentAnalysis

Completed In Development Planned

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

DEN Blackboard Link to ISI Discussion Board

Native Blackboard discussion board link replaced with link to ISI board

Native Blackboard discussion board link replaced with link to ISI board

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

ISI Discussion Board

Blackboard sends user profile and discussion board automatically registers user

Blackboard sends user profile and discussion board automatically registers user

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Current State

Current state In use since Spring 2005

Two DEN courses currently online (CSci577, CSci402) Four archived courses available (AME599, 3xCSci402)

Extensive processing of 402 course materials completed

Current focus Utilize discussion board for research

Data benefits learning, mining, ped. agent and NLP research Spawned AIED paper, HTL demo, and AERA, IUI submissions

Experimental work (UCLA intern and CSci577 class) Proposal planning

Now is a critical time! Funding has ended (funds slated to run out ~12/2005) Minimal funding would assure continued support for

current course use research evaluation

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Project Rationale

Addresses instructional challenges Instructors want to experiment but have limited support

E.g. Can maintain own web site but not discussion board registration, security, and maintenance are obstacles

Students expect to use current communication technologies Evaluation of ‘Learner centered’ model difficult but necessary

Addresses technical challenges Building own OLE, duplicating DEN system impractical Blackboard’s Building Blocks solution incomplete

API incomplete, integration too tight ISI development within DEN production environment problematic

Our solution Encourages adoption (maximizes ease-of-use) Enables development at ISI (minimizes disruptions) Uses open source development model (maximizes code maintenance) Allows for evolution from Blackboard

18 open source CMSs currently available (EduTools, August, 2005)

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Discussion Board Deployment

Software phpBB, open source, latest version, running at ISI

• http://www.phpbb.com, http://melodic.isi.edu/phpbb2 ISI modifications adhere to community development standard

• modularized and self-documenting• we maintain changes, phpBB dev community maintains core

Courses online (2005) Spring: AME599, CSCI402 Summer: CSCI402 Fall: CSCI555, CSCI402 Archived 2004 discussion (CSCI402 and CSCI555)

Usage statistics ~130 past students, ~130 current students Over 2000 messages

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Summary of Accomplishments

Main accomplishments Single sign on from Blackboard (fully integrated) Document capture (Blackboard-integrated user interface) Successfully used in five courses

Discussion board details Auto-creation of lecture discussions

Integration of webcasts and lecture notes to follow

Annotation system Instant messaging/mobile learning integration Answer mining of archived discussions Archiving features

Document capture details Automate parsing of uploaded document Creates PDF, HTML, text and metadata files for all documents Makes documents searchable via discussion board

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Summary of New Research

Research thrusts SPADE rhetorical analysis (Shaw) Classummary evaluation (Liang Zhou, GRA) Q/A processing pipeline (Donghui Feng, GRA)

800 messages manually annotated for network analysis

Ontology creation (in planning with Kim)

Data share tools API for message export XML schema for message formatting API for agent message insertion

Evaluation tools Statistics viewer Questionnaire creation tool (auto-send to users) Instrumentation for Q/A evaluation (planned)

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Screen Shots

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Artifact-Based Discussion

Documents and images displayed next to discussionSupports all file formats, including PDF

Documents and images displayed next to discussionSupports all file formats, including PDF

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Topic-Based Annotation

Simple markup of HTML version of documentdirects processor to create links to auto-generated topics

Simple markup of HTML version of documentdirects processor to create links to auto-generated topics

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

BBS sends message Notification

BBS sends message Notification

User queries BBS for detailsUser queries

BBS for details

Two way communication via instant messaging

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Mobile Communication via IM

T*Mobile Sidekick

CingularNokia 3595

Working AssetsSprint LG 1200

AOL Instant Messenger communicationWorked on all phone types tested

AOL Instant Messenger communicationWorked on all phone types tested

AT&T Samsung

x427

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Statistics Viewer

Accessible via ‘Administrator Panel’ Allows instructors to access ISI dataAccessible via ‘Administrator Panel’ Allows instructors to access ISI data

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Question Answering Research

Text is incoherent, non-factoid

Questions barely identifiable, often include multiple sentences or paragraphs

Answers require explanations

Text is incoherent, non-factoid

Questions barely identifiable, often include multiple sentences or paragraphs

Answers require explanations

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Question Answering

Placeholder for results screen shot

Answer-bot refers questioner to most similar discussion and document segment.

Answer-bot refers questioner to most similar discussion and document segment.

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

XML Schema for Message Export

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Classummary Tool (Liang Zhou)

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Classummary Archive

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Classummary Notification Scheduling Interface

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Classummary Notification Email

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Research Potential

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Discourse Analysis (Shaw)

Apply SPADE (Soricut & Marcu) rhetorical parser to discussion board data (spring 2005) Run on over 1000 messages (Open University corpus) Used to validate differences in tutor and student messages AIED 2005 paper

Assessing and Scaffolding Collaborative Learning in Online discussions

Generalize work (to be proposed) Adapt for real-time analysis Run on DEN data Evaluate different types of messages

• Question vs. comment vs. answer• Tutor-instructor vs. student

Develop message classifier (done manually now) Develop rules for promoting discussions

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Pedagogical Interventions (Beal)

Analyze discussion board communication (spring 2005) Analysis of small corpus of messages (Beal’s UMass course) [More information here]

Generalize work (proposed in AIED poster, Beal, Kim, Shaw) Amalgamation of agent and ontology work, NLP, and

discourse analysis [More information here]

den.

usc.

edu

Lord Corporation Foundation and USC Viterbi School of Engineering

Learning Ontologies (Jihie Kim)

Analyze discussion board communication (summer 2005) Annotation of small corpus of messages (CSci555) Classify on-topic messages using keywords Compare technical vs general discussion messages

Generalize work (to be proposed) Acquire keywords automatically Identify keywords in messages Assess students’ contributions