Ltsa Slides

1998-05-21 LTSA 4.00 Overview, Frank Farance 1

Learning Technology Systems Architecture -- Overview

LTSA 4.00

Frank FaranceFarance/Edutool

[email protected], +1 212 486 4700

Slides and Document: http://www.edutool.com/ltsa


Overview

• Review of goals• Abstraction-Implementation Layers• Review of each layer• Demonstration of perspectives• What does conformance mean?• Open issues


Review of goals

• Identify common components• Identify critical interfaces• Widespread applicability:

– platform-neutral– content-neutral– pedagogically-neutral


Architecture DefinitionAnd Purpose

• LTSA 4.0 specification:http://www.edutool.com/ltsa

• [Excerpt:] “In general, the purpose of developing systems architectures is to discover high-level frameworks for understanding certain kinds of systems, their subsystems, and their interactions with related systems. ...”


Architecture DefinitionAnd Purpose (cont’d)

• [Excerpt cont’d:] “… An architecture isn't a blue print for designing a single system, but a framework for designing a range of systems over time, and for the analysis and comparison of these systems. …”


Architecture DefinitionAnd Purpose (cont’d)

• [Excerpt cont’d:] “… By revealing the shared components of different systems at the right level of generality, an architecture promotes the design and implementation of components and subsystems that are reusable, cost-effective, and adaptable.”


Abstraction-Implementation Layers

• The Groupings -- abstractions vs. implementations• The Priorities -- ordering• The Notations -- text, system, bus• The LTSA Layers -- five layers


Abstraction-Implementation Layers:The Groupings

• Organize engineering details into several groupings from high-level (coarse granularity) to low-level (fine granularity)

• Groupings are called layers• Layers above are called abstractions• Layers below are called implementations


Information InclusionOrganize raw details as: good, borderline, and bad examples

Raw Details Bad

Good

Borderline


Information ExclusionEssential vs. Non-Essential Details: Depends on Context

Browser ServerWeb Content


User Request(“Pull”)


Server Initiated(“Push”)


HTMLCGI

#1: Minimal Details, Only Data Flow

#3: (Again) More Details, Control Flow

#2: More Details, Control Flow

#4: More Details, Related Subsystems


Architecture Abstraction Hierarchy Reference Model (Belz, Suthers, Wheeler)

Application Model

Abstract System Model

- Interaction Model

- Conceptual Model

Abstract Implementation Model

Resource Model

LTSA Layer 1; IEEE 1484.1

LTSA Layer 2; IEEE 1484.1; AICC

LTSA Layer 4; IEEE 1484.1; IMS; AICC

LTSA Layer 5; IEEE 1484.1, 1484.2, 1484.4, 1484.6, 1484.7, 1484.10, 1484.11, 1484.12, 1484.13; IMS; AICC, ARIADNE

No category:IEEE 1484.3, 1484.9

LTSA Layer 3; IEEE 1484.1; IMS


Abstraction-Implementation Layers:The Priorities

• High-level design issues have most pervasive effect on design: first priority

• Low-level design issues are localized: consider these issues later

• Example: building a house– High-level (higher design priority): layout and number of rooms– Low-level (leave till later): choosing wallpaper


Iterative AbstractionDesign Priorities: Colors (high), Shapes (medium), Raw Details

Raw Details

Abstraction of Raw Details:Shapes (Arrow, Box, Star)

Abstraction of Shapes:Colored Polygons (Red, Green, Blue)


Iterative ImplementationDesign Priorities: Colors (high), Shapes (medium), Raw Details

Implementation of Colored Shapes: Constraints: Arrows Connected to Polygons

Implementation of Shapes withConstraints: Red, Green, Blue

Implementation of Colored Polygons with Constraints:Eight or fewer sides


Abstraction-Implementation Layers:The Notations

• High-level: systems (Yourdon) diagrams– Process (oval), store (box), flow (arrow)– Static connections (flows) identify functional decomposition

• Low-level: bus diagrams– Bus (bar), connector (line), system (box)– Dynamic, point-to-point connections for on-demand, arbitrary

component communication


Systems Notation:Processes, Stores, Flows

DataProcess

DataProcess

ControlProcess

One-Way Control Flow

One-WayData Flow

Two-Way Control Flow

Two-Way Data Flow

DataStore

ControlStore


Bus Notation

• Common naming• Common control flow (initiating transactions)• Non-homogeneous data flow (different transactions)• Dynamic, on-demand connections


Bus Notation:General Concepts

- Common naming

- Common control flow

- Non-homogeneous data flow

- Dynamic, on-demand connections

P

Q

R

S

Each member of bus has

unique name and shares namespace



- Common naming




P

Q

R

S

Each member uses same protocol to

request or respond to transactions



- Common naming




P

Q

R

S

Common control protocol, but different

data protocols

Example:PQ uses FTP, but

PR uses HTTP

FTP overTCP/IP bus

HTTP overTCP/IP bus



- Common naming




P

Q

R

S

No permanent connections among members of bus; each transaction is a connection

Examples:1. Q sends to P2. P broadcasts to all3. R sends to Q


Systems vs. Bus Notation

• Systems notation useful for:– Small number of components– Dedicated functions for each component– Useful for LTSA layers 1, 3, and 4

• Bus notation useful for:– Large number of components– Dynamic, on-demand connections between components– Useful for LTSA layer 5


Abstraction-Implementation Layers:The LTSA Layers

• LTSA from high-level to low-level:– Layer 1: Learner and environment interactions– Layer 2: Human-centered features– Layer 3: System components– Layer 4: Stakeholder perspectives– Layer 5: Operational components


Layer 1: Learner AndEnvironment Interactions

• Learner interacts with “environment” as part of a “learning experience”

Learner

Teacher

Other Students*Books

Library

CoachMentorCollaboration*

School

Other Employees*

Internet/Web

Television

Lab Parent

Computers

EmployerNewspaper

Multimedia

* See later slide: collaboration is internal to Learner


Layer 1: Learner and Environment:How Environment Effects Learner

• Primary Focus: the effect of environment on the learner, not the effect of learner on the environment

• Learner can have important effects on environment (research, teaching teachers, etc.), but is not primary technical focus of learning technology


Layer 1: Learner and Environment:Collaboration: Internal to Learner

• LTSA “learner” (abstraction) can represent one or more “students” (humans).

• Collaboration among students is internal to learner• Thus, from an information technology perspective, all

learner interaction is from the environment to the learner, i.e., inbound information


Layer 1: Learner and EnvironmentLearning Interactions: Input to Learner

• Primary Focus: technology issues, i.e., the flow of information, not pedagogy

Environment Student

Student

Student

Learner

CollaborationLearning

Interactions


Layer 2: Human-Centered Features Limitations => Effect on Design

• Humans receive the learning experience (human learning), not computers (machine learning)

• Human communication: fundamental limitations has pervasive effect on design

Environment LearningInteractions

LearnerLearner is not a computer.Human learners havelimitations -- complicatesreceiving information fromlearning interactions.


Layer 2: Human-Centered Features Why humans aren’t computers

• Humans aren’t reliable receivers of information:– Humans mishear or don’t hear at all– Retransmission is necessary

• Humans aren’t predictable receivers of information:– Humans hear different than desired– Different strategies are necessary


Layer 2: Human-Centered Features Why humans aren’t computers (cont’d)

• Humans learn differently (and over time):– Humans differently from each other– Same human may learn differently over time– Need intelligent customization, configuration– Need rich knowledge library for diverse humans

• Negotiate learner styles:– Learner alone or system coach alone cannot determine best

learning style


Layer 2: Human-Centered Features Constructing LTSA System Components

• #1: Start with human sensory input: multimedia delivery to learner (similar to entertainment)

• #2: Add feedback loop for human unreliability• #3: Add records database for varying teachers, infer

intelligently about student capability• #4: Add rich knowledge library, search, and lookup for

diversity of humans• #5: Add negotiated learning styles


Step #1: Transferring Information

• Transfer information to Learner• Sensory input/interactions for humans• Delivery of Multimedia: same as entertainment

Delivery

LearnerMultimedia


Step #2: Feedback/Coaching Loop

• Feedback necessary, humans are unreliable receivers, e.g., they don’t hear what you tell them

• Current state of feedback loop: Behavior, Evaluation, Assessment

Delivery

Learner

Evaluation

SystemCoach

Multimedia Behavior

A s s e s s m e n tL o c a t o r I n d e x


Step #2: Feedback/Coaching Loop (cont’d)

• System Coach: determines corrective actions• Locator Index:

– e.g., next lesson (Learning Content) towards objective– directs Delivery of Multimedia to Learner

• Loop recovers from errors, focuses on target

Delivery

Learner

Evaluation

SystemCoach

Multimedia Behavior



Step #3: Records Database

• Performance information– helps infer corrective action over long periods of time– nomadic students: allows student migration (handoff) to different

teachers (e.g, grade promotion) and different institutions • Records Database: Learner’s history, objectives

Delivery

Learner

Evaluation

SystemCoach

Performance (new)

Multimedia Behavior

RecordsDatabase

Performance (current)Performance

(history)



Step #4: Knowledge Library

• Humans are unpredictable and diverse• Knowledge libraries necessary to match human diversity• System Coach:

– directs different learning styles and strategies– based on Learner’s history, objectives, and Assessment– supported via queries to Knowledge Library for appropriate Learning

Content

Delivery

Learner

Evaluation

SystemCoachKnowledge

Library

Content Index(metadata)

Query Index

LearningContent

Performance (new)

Multimedia Behavior

RecordsDatabase

Learning Content


(history)



Step #5: Advice from the Learner

• Humans are self-aware: can give advice on learner styles• Learning Styles:

– interaction between Learner and system to negotiate different learning styles

– negotiation might actually be assertion: Learner may assert (e.g., experimentation), or system coach may assert (e.g., primary school might dictate learning style)

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Review: System Generated SolelyBy Human-Centered Features

• #1: Transferring Information• #2: Feedback loop: unreliable Learners• #3: Records Database: history supports intelligent inferences, history

allows Learner migration• #4: Knowledge Library: varying/diverse student capabilities• #5: Learning Style: negotiate between Learner/system

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)





allows Learner migration• #4: Knowledge Library: varying student capabilities• #5: Learning Style: negotiate between Learner/system

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)




• #1: Transferring Information• #2: Feedback loop: unreliable Learners• #3: Records Database: history supports intelligent inferences,

history allows Learner migration• #4: Knowledge Library: varying student capabilities• #5: Learning Style: negotiate between Learner/system

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)






Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)






Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Layer 3: System Components

• Processes: Learner, Evaluation, System Coach, Delivery• Stores: Records Database, Knowledge Library• Flows: Behavior, Assessment, Performance (past, present, future), Indexes

(query, content, and locator), Learning Content, Multimedia, Learning Style

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Learner(process, human)

• Abstraction of student• Can represent more than one student• Learner:

– Receives Multimedia– Emits Behavior– Negotiates Learning Style

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Learning Styles(data flow)

• Learner’s preference as input to system• System Coach can negotiate and/or restrict Learner’s preferences• Surrogates: e.g., parentlearner, teacher system• Other possible surrogates: e.g., parentsystem, student system,

mentorlearner

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Behavior(data flow)

• Observable Behavior of the Learner, coded, and sent to the Evaluation process

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Evaluation(process)

• Evaluation:– Receives Behavior– Receives Learning Content (context of Learner’s Behavior)– Stores Performance info based on Evaluation of Behavior– Produces Assessment given to System Coach

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Performance Information Stored By Evaluation (data flow)

• Evaluation of Behavior: can emit Performance info• Stored in Records Database

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Records Database(data store)

• Stores Performance information as Learner history• Retrieves and stores Learner history records for System Coach

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Performance Information Received BySystem Coach (data flow)

• System Coach gets Learner history by retrieving Performance information from Learner’s Records Database

• Can select strategies based on history

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Performance Information Stored BySystem Coach (data flow)

• System Coach can store Performance information, too:– “bookmarks” (incomplete sessions)– certifications (milestones, completion of module)– objectives (outlines for future sessions)

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Assessment Information(data flow)

• Evaluation process can emit Assessments: “where there student is at”• System Coach can use Assessment information, along with other

information, to select appropriate learning experience

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



System Coach(process)

• Sends, receives, and negotiates Learning Style with Learner

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)


1

1



• Receives current Assessment information and Performance information (history and objectives) for future learning experiences

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)


2 3

2

3



• Queries to the Knowledge Library:– Query Indexes sent to search for appropriate material– Content Indexes (metadata) returned as “found” Learning Content– Extracts the Locator Indexes (e.g., URLs) from the returned Content

Indexes (metadata)

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)


4

4



• Locator Indexes (similar to web URLs) are sent to Delivery process to identify (not transfer) Learning Content

• Note: Delivery is responsible for retrieving and Knowledge Library is responsible for transferring Learning Content

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)


5

5


Query Indexes(data flow)

• Query Index specifies “what to search for” in Knowledge Library • System Coach sends Query Indexes to search the Knowledge Library

for appropriate Learning Content, based on Assessment info and Performance info from Records Database

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Knowledge Library(data store)

• Database searched via Query Index• Returns Content Indexes (metadata) of Learning Content that matches

the Query Index• Emits Learning Content based on Locator Index from Delivery

process

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Content Index -- Metadata(data flow)

• Search of Knowledge Library returns Content Indexes (metadata) for Learning Content that matches Query Index

• Content Indexes (metadata) are similar to card catalog entries in a library

• Locator Indexes (e.g., URLs) can be extracted from Content Indexes (metadata)

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Locator Index(data flow)

• System Coach extracts Locator Index from the Content Index for appropriate learning experience

• Locator Index is similar to call number in a library, and similar to URL of the World Wide Web

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Learning Content Received ByDelivery (data flow)

• Delivery Process sends a Locator Index to the Knowledge Library pointing to Learning Content

• Knowledge Library returns Learning Content to Delivery process

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Delivery(process)

• Receives Locator Index (e.g., URLs) that identify learning experiences desired by the System Coach

• Transforms Learning Content to Multimedia• Sends Learning Content to Evaluation to give context of Learner’s

Behavior• Sends interactive Multimedia to the Learner

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Learning Content Sent ByDelivery (data flow)

• Delivery process sends Learning Content to Evaluation to give context of Learner’s Behavior

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Multimedia(data flow)

• Delivery process generates Multimedia learning experience based on Learning Content

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Functional vs. Actual Learning Technology Systems

• LTSA components identify functional boundaries and critical interfaces

• Actual systems integrate several components into individual products

• Functionality exists in actual systems but may be embedded within larger system (several components combined) or single component spread across several systems


Actual Implementation Example #1:Web-Based Learning

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)


Human

HumanInterface

(e.g., X WindowsWin95)

PresentationTool

(browser)CoursewareDatabase

(web servers)

StudentRecords

(database)

AbstractionImplementation


Actual Implementation Example #1:Web-Based Learning

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)


Human

HumanInterface

(e.g., X WindowsWin95)

PresentationTool

(browser)CoursewareDatabase

(web servers)

StudentRecords

(database)



Actual Implementation Example #2:Flight Simulator, Flight Instructor

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)


FlightInstructor

FlightSimulator

Pilot’sLogbook


Pilot


Actual Implementation Example #3:Non-Electronic, Traditional Classroom

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)


Teacher ReportCards,

Transcript


Student

SchoolLibrary

Student UsesLibrary Teacher Points Student To

Books (A Locator Index)

Retrieves Teaching Materials

Grades


Actual Implementation Example #4:Home Study Course

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)


SchoolHomeStudy

CourseStudent



Layer 4: Stakeholder Perspectives

• Each stakeholder has a view, but not complete picture• Show each stakeholder perspective within LTSA• Stakeholder perspectives used to validate views and

subsets of LTSA components• “It’s in there!” (if not, let us know!)


Layer 4: Stakeholder Perspectives (cont’d)

• Stakeholder perspectives analyzed by:– Primary design issues: LTSA components that greatly affect

design and implementation for stake holder (shown as red/bold)– Secondary design issues: LTSA components that affect design

and implementation of stake-holder, but not critical (shown as blue/dash ---)

– Other LTSA components: insignificant design issue or not applicable (no highlight/green fill)


Layer 4: Stakeholder Perspectives (cont’d)

• Stakeholders, typically, have different and/or incompatible priorities

• Differing priorities: obstacles to consensus• Non-technical (political, business) issues just as important

as technical issues• Cataloging stakeholder perspectives is critical for building

consensus


Abstraction-Implementation:System Components ==> Stakeholder/Design Perspectives

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Various Stakeholder/Design PerspectivesLearner, Assessment, Institution, Metadata, Intelligent Tutor, Mentoring, Collaboration, Ontology, Task, etc.


“Isolated” LTSA Stakeholders

• Isolated LTSA subsets -- little interaction with neighboring LTSA components

• Little overlap with other “isolated” stakeholders• Relatively “simple” integration• Can make use of isolated component standards• Best stakeholder examples: “Records”, “Metadata”,

“Multimedia”


Learner-Centered

• Primary: learner keeps his/her records, learner has influence on negotiated learning style

• Secondary: assessment and performance info

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Assessment-Centered

• Primary: educational standards, behavior, evaluation, assessment records, records database reporting

• Secondary: learning style, learner, system control (adaptive administration)

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Records, Certifications

• Primary: records database, performance info• Secondary: evaluation, assessment info, system coach

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Task Model, School-To-Work

• Primary: records database• Secondary: performance info

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Institution-Centered

• Primary: system control, administration, records, reporting, negotiating (many times: asserting) learning style

• Secondary: learner collaboration, teachers, evaluation (grading), delivery infrastructure (including multimedia and behavior observation correlated by learning content)

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Content-Centered

• Primary: delivery, multimedia, invocation via locator index (e.g., URLs), learning content format

• Secondary: assessment, content search (query index, content index), knowledge library, learner’s user interface

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Metadata

• Primary: query index (search), content index (metadata), locator index (e.g., URLs), and associating locator indexes with content indexes

• Secondary: knowledge organization, learning content (presentation)

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Ontologies, Expert Systems

• Primary: knowledge design and knowledge representation in knowledge library, learning content (presentation)

• Secondary: query index (search), content index (metadata), and locator index (e.g., URLs)

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Learning Objects

• Primary: knowledge library, query index (search), content index (metadata), learning content (presentation)

• Secondary: evaluation, performance info and database, assessment info, system coach, locator index (e.g., URLs)

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Multimedia

• Primary: delivery, multimedia, query index (search), content index (metadata), locator index (e.g., URLs), hardware limitations

• Secondary: system coach, knowledge library, learning content format, behavior

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Collaboration, Asynchronous Learning

• Primary: communication among students with same roles (collective learner), communication modes (multimedia), delivery tools, and differing “time zones”

• Secondary: system control by coordinator/moderator, learning style negotiated among participants and coordinator/moderator

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Multiple Role, Team Learning

• Primary: communication among students, but different roles (collective learner), communication modes (multimedia), delivery tools, and differing “time zones”

• Secondary: system control by coordinator/moderator, learning style negotiated among participants and coordinator/moderator

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



“Overlapping” LTSA Stakeholders

• Concerns many, most, or all LTSA components• Overlaps with other stakeholders• Differing, conflicting designs priorities• Relatively “complex” integration• Potential non-technical, political, or business obstacles to

consensus and interoperability• Best stakeholder examples: “Experimentation”,

“Intelligent Tutoring Tools”, “Distance Learning”


Mentoring, Coaching

• Primary: learner and mentor/coach jointly perform evaluation and jointly choose direction

• Secondary: tools and delivery support mentoring, coaching, experimentation, recordkeeping

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Interactive Environment

• Primary: learner directs style, learner is observed, delivery is responsive to evaluation

• Secondary: assessment info, system coach, locator index (e.g., URLs)

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Simulation

• Primary: delivery, multimedia, learning content to correlate multimedia and behavior, assessment info, system coach, locator index

• Secondary: behavior, performance info, records database

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Learning Tool-To-Tool Communication

• Primary: system coach, evaluation, assessment info, query index (search), content index (metadata), locator index (e.g., URLs), delivery, invocation

• Secondary: multimedia, learning style, performance info, records database, knowledge library, learning content

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Sequencing, Pre-Requisites, Co-Requisites

• Primary: system coach, evaluation, assessment info, query index (search), content index (metadata), locator index (e.g., URLs), delivery, invocation

• Secondary: multimedia, learning style, performance info, records database, knowledge library, learning content

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Experimentation, Discovery

• Primary: learner chooses direction, acquires knowledge during experimentation, learner, system coach, query index (search), content index (metadata)

• Secondary: learner does self evaluation, tools and delivery support experimentation, behavior, assessment info, locator index (e.g., URLs), delivery, multimedia

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Intelligent Tutoring Tools

• Primary: learner chooses direction, acquires knowledge during use of tutoring tool, knowledge library may be implicit (not explicitly defined) in tutor

• Secondary: tutor does evaluation, tools and delivery support experimentation

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Distance Learning, Distributed Learning

• Primary: distributed and nomadic communication of all “flows” (behavior, performance, multimedia, etc.)

• Secondary: processes and stores components• NOTE: Take note of primary design issues (red/bold)

– How does this slide differ from all others?– Answer: All flows are primary, everything else is secondary

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



“Parallel” LTSA Stakeholders

• Integration of multiple sessions, each with differing learning experiences and interactions

• Synchronization of multiple sessions: starting, stopping, coaching, learner interfacing

• Integration, collaboration, and synchronization of feedback and coaching strategies

• Best stakeholder example: “Student Teacher”


Parallel Sessions for Same Learner

• Primary: multiple, simultaneous learning experiences; multiple flows of behavior, learning style, multimedia

• Secondary: multiple evaluation, system coach, and delivery processes• Example: flight simulator -- simultaneous training of flying skills, navigation

skills, communication skills, cockpit resource management


Student Teacher

• Primary: student teacher as a teacher• Secondary: student teacher as a student

Student

Teacher


Multi-Tiered Process Improvement

• Student is evaluated by teacher, teacher coaches student• Teacher is evaluated by principal, principal coaches teacher • Principal is evaluated by school board, school board coaches principal• Teacher is also evaluated by performance of his/her students• Principal is also evaluated by performance of his/her students

Student

Teacher

Principal


Standards and Specification Development Organizations

ISOANSIIEEE

(LTSC)1484

AICC ARIADNEIMSADL

Requirements

Specs FromConsortia

IEEE, ANSI, ISO: Accredited Standards

Requirements

Specs

SpecsSpecs

Liaising


IEEE 1484 As A Stakeholder(IEEE 1484 Roadmap)

• IEEE 1484 working groups (WGs) and study groups (SGs) each involve certain areas of LTSA

• Some LTSA components should not be standardized, e.g., evaluation methods, delivery systems

• Some LTSA components should be standardized outside 1484, e.g., multimedia, education standards, cultural adaptation

• More info: http://www.manta.ieee.org/p1484


IEEE 1484.1:Architecture and Reference Model WG

• Primary: all components• Secondary: none

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



IEEE 1484.2:Learner Model WG

• Primary: learner, negotiated learning style, evaluation, performance info, assessment info, system coach

• Secondary: behavior

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



IEEE 1484.3:Glossary WG

• Primary: none• Secondary: all components

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



IEEE 1484.4:Task Model WG

• Primary: records database• Secondary: performance info

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



IEEE 1484.6:Course Sequencing WG

• Primary: system coach, performance info, assessment info, query index (search), content index (metadata), locator index (e.g., URLs), invocation of learning content

• Secondary: learning style, records database, knowledge library, delivery, learning content

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



IEEE 1484.7:Tool/Agent Communication WG

• Primary: system coach, behavior, evaluation, assessment info, performance info, records database, query index (search), content index (metadata), locator index (e.g., URLs)

• Secondary: query index (search), content index (metadata), knowledge library, delivery

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



IEEE 1484.9:Task Ontology WG

• Primary: knowledge library, learning content (presentation)• Secondary: query index (search), content index (metadata), locator

index (e.g., URLs)

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



IEEE 1484.10:CBT Data Interchange WG


• Secondary: learning style, behavior, records database, knowledge library, learning content, delivery

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



IEEE 1484.11:Computer Managed Instruction WG

• Primary: performance info, assessment info, query index (search), content index (metadata), locator index (e.g., URLs), invocation of learning content

• Secondary: behavior, records database, system coach, knowledge library, learning content, delivery

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



IEEE 1484.12:Learning Objects Metadata WG

• Primary: query index (search), content index (metadata), locator index (e.g., URLs)

• Secondary: system coach, knowledge library, learning content, delivery

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



IEEE 1484.13:Student Identifiers SG

• Primary: performance info, records database• Secondary: learner, system coach

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Other Specification Development Organizations As Stakeholders

• Educom’s Instructional Management Systems (IMS) Projecthttp://www.imsproject.org

• Aviation Industry CBT Committeehttp://www.aicc.org

• DoD Advanced Distributed Learninghttp://www.adlnet.com

• European Union ARIADNE Project


Educom’s Instructional Management Systems (IMS) Project

• Main areas of development:– Metadata: learning technology metadata– Content objects: learning content, course structure– Management systems: collaboration, reporting– Profiles: student records– External services: interface to institution

• More info: http://www.imsproject.org


IMS Metadata

• Primary: content index (metadata) and associating locator indexes with content indexes

• Secondary: query index (search), locator index (e.g., URLs), knowledge organization in knowledge library

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



IMS Content Objects

• Primary: knowledge library, content index (metadata), learning content (presentation)

• Secondary: evaluation, performance info, assessment info, system coach, locator index (e.g., URLs), delivery

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



IMS Management System

• Primary: learner, system coach, performance info, locator index (e.g., URLs)

• Secondary: negotiated learning style, records database, content index (metadata), knowledge library, delivery

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



IMS Profiles

• Primary: records database, performance info• Secondary: evaluation, assessment info, system coach

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



IMS External Services

• Primary: records database, student registration, institution back office• Secondary: performance info

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Aviation Industry CBT Committee (AICC)

• Develops guidelines and recommendations for CBT in aviation applications

• Subject areas: Courseware Delivery Stations, Digital Audio, Operating Windowing System, CBT Peripheral Devices, Computer Managed Instruction (CMI), Courseware Interchange, Digital Video, Icon Standards

• More info: http://www.aicc.org


AICC AGR003:Digital Audio

• Primary: multimedia• Secondary: delivery

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



AICC AGR005:CBT Peripheral Devices

• Primary: behavior, delivery, multimedia• Secondary: learner

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



AICC AGR006:Computer Managed Instruction

• Primary: performance info, assessment info, query index (search), content index (metadata), locator index (e.g., URLs), invocation of learning content

• Secondary: behavior, records database, system coach, knowledge library, learning content, delivery

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



AICC AGR007:Courseware Interchange


• Secondary: learning style, behavior, records database, knowledge library, learning content, delivery

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



AICC AGR008:Digital Video

• Primary: multimedia• Secondary: delivery

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



AICC AGR009:Icon Standards

• Primary: learner, learning content• Secondary: delivery, multimedia

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



AICC CRS002:Glossary of Terms Related to CBT

• Primary: none• Secondary: all components

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



DoD Advanced DistributedLearning (ADL)

• Research and development areas:– Promote widespread collaboration. – Exploit Internet technologies. – Develop next generation learning technologies. – Create reusable content, and lower costs, with object-based tools.

• Supplies requirements to IMS• More info: http://www.adlnet.org


ADL Collaboration

• Primary: all components, ADL addresses collaboration of learning technologies and learning technology requirements

• Secondary: none

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



European Union - ARIADNE

• Collaborative browsing• Learning technology metadata• Working with IMS• More info:

http://tina.lancs.ac.uk/computing/research/cseg/projects/ariadne/


ARIADNE Metadata

• Primary: content index (metadata) and associating locator indexes with content indexes

• Secondary: query index (search), locator index (e.g., URLs), knowledge organization in knowledge library

Delivery

Learner

Evaluation


Library


Query Index

LearningContent

Performance (new)

Multimedia Behavior

LearningStyle

RecordsDatabase

Learning Content


(history)



Layer 5: Operational Components

• Common components that build the system• Operational components may serve several functions• Several system components may be implemented via the

same operational components, e.g., performance info and learning content both coded as XML (an operational component)


Layer 5: Operational Components(cont’d)

• Operational components are identified by:– Systems, e.g., clients and servers– Connectors (interface protocols), e.g., HTTP, PPP– Busses (namespaces), e.g., URLs, telephone numbers

• Components (systems) on a bus can “talk” to each other via interface protocols


Abstraction-Implementation:Stakeholder/Design Perspective => Operational Components

Various Stakeholder/Design PerspectivesLearner, Assessment, Intelligent Tutor, Distance Learning, Mentoring, Collaboration, Ontology, Task, etc.

Protocols

Interchange DataStorage

Processes

HumanInterface

Control Flow



Information Busses:Example: Internet Client-Server Bus, Client Perspective

- Client connects to many servers

- On-demand connections

- Several servers may have same purpose, but varied resources; like libraries: same interface, but varied collections of books

Server

Server

Server

Client


Information Busses:Example: Internet Client-Server Bus, Server Perspective

- Server handles requests from multiple clients

- On-demand connections

- Clients may appear homogenous, but have different identities

Client

Client

Client

Server


Information Busses:Example: Combining/bridging two busses

User

WebClient

WebServer 1

WebServer 2

WebServer 3

GUIApp 1

GUIApp 2

GUIApp 3

Window ManagerApplication “Bus”

Examples:X Windows, Win95

World Wide Web“Bus” of Browsersand Web Servers

Web Browser: “sits” on two busses: both window manager application & web application


Information Busses:Example: Combining/bridging three busses

User

WebClient

WebServer 1

WebServer 2

GUIApp 1

GUIApp 2

WindowManager

Application“Bus”

World Wide Web“Bus” of Browsersand Web Servers

Web Browser:GUI<->Web

ChatClient 1

ChatClient 2

ChatServer

Chat Session:GUI<->Chat

Chat “Bus”of Sessions


Information Busses:Commonly Used Busses/Protocols

• Layer 5 is: collection of common bus protocols, no specific requirements

• Examples:– Transport, Network, Data Link: TCP, UDP, IP, Ethernet,

telephone– Application, Presentation, Session: X11, HTTP, IRC, SMTP,

NNTP, FTP, TELNET– Data, Control, Security: HTML, XML, RDF, RPC, CORBA,

DCOM, HLA, Kerberos, HTTPS, SSL, HLA


Summary of Five LTSA Abstraction-Implementation Layers

Environment

Interactions

Learner

Five Human-Centered Features

40+ Stakeholder Perspectives

Layer 3

Layer 4

Layer 5

Layer 2

Layer 1Learner

EnvironmentInteractions

SystemComponents

Protocol & Interface

Busses

LICILC

LS

BM

P PA

DL

E

SCKL RD

LC

QI


LTSA Layer Summary (cont’d)

• Layer 1:– Learning experience: learners interacting with the environment– A learner (abstract) may represent several students (humans)– From information technology perspective, flow is one-way:

environment ==> learner– Collaboration is within the learner abstraction



• Layer 2:– Because the learner abstraction is “implemented” with “humans”

(human learner), rather “implemented” with “computers” (machine learning), the “nature of humans” has pervasive effect on system design

– Fundamental limits of human communication



• Layer 2 (cont’d):– Humans aren’t reliable receivers of information: retransmission– Humans aren’t predictable receivers of information: different

strategies– Humans learn differently (and over time): customizable,

configurable



• Layer 3:– System notation: process, store, flow– Human-centered feedback loop: learner, behavior, evaluation,

assessment, system coach, delivery, multimedia– More “intelligence” from learning content: need records database

and performance records– Learner has different “teachers” during his/her lifetime: records

database facilitates “hand-off”



• Layer 3 (cont’d):– Humans each learn differently (and learn differently over time):

rich knowledge library required; query indexes and content indexes (metadata) make best use of knowledge library

– Learners and their surrogates (e.g, parent, mentor) negotiate learning styles with system coaches and their surrogates (e.g., institution, teacher, courseware developer)



• Layer 4:– Stakeholders each have own, incomplete perspectives– All stakeholders’ perspectives are incorporated in LTSA system

components diagram– IEEE 1484: working groups specifying critical interfaces; some

technical standards outside of 1484; “education standards” are necessary but outside scope of 1484 and “technical standards”



• Layer 5:– Bus notation: bus, connector, system– Operational components: actual implementation– One operational component (e.g., HTTP) might be used by several

system components (e.g., records database, knowledge library)– One system component (e.g., delivery) might use several

operational components (e.g., HTTP, HTML, MPEG)



• Layer 5 (cont’d):– Bus may be used for several purposes– Systems on bus share common namespace, e.g., URLs, process

IDs, object IDs– Systems may be connected, simultaneously, to several busses



• Layer 5 (cont’d):– Systems dynamically connect, on-demand to other systems (e.g.,

clients to servers)– System connectivity is not pre-arranged– Connections (protocols) may serve several purposes (e.g., XML

used for performance records and learning content; MIDI used for behavior and multimedia)


LTSA Conformance

• LTSA conforming products/services– provide the critical interfaces (functionality) that conform to one

or more components of LTSA: Learner (Collaboration), Behavior, Evaluation, Performance, Records Database, System Coach, Query Index (search), Content Index (metadata), Locator Index (e.g., URLs), Knowledge Library, Delivery, Multimedia

– identify the protocols that describe the bus interface– identify the interface point


LTSA Conformance

• LTSA conforming products/services• Measuring conformance: the system under test is called the

“implementation under test”• A system that conforms to the LTSA specification is called

a “conforming implementation”• “A conforming implementation shall conform to all five

layers.”


LTSA Layer 1 Conformance

• Validating Learner-Environment interactions• “The implementation under test shall identify the

boundaries of the Environment, the flows of Interactions, and the Learner.”

• “The implementation under test shall identify the Learner's relationship to human students.”

• Examples of Learner/Student relationship:– Is the Learner represented by more than one Student?– Is there collaboration among the Students?



• Validating human-centered features• “The implementation under test shall identify the methods

for addressing the human-centered features:”– unreliable learners– unpredictable and diverse learners– nomadic learners– influence on learning style



• Validating decomposition into LTSA system components• “The implementation under test shall identify the available

LTSA system components: Learner, Learning Style, Behavior, Evaluation, Performance, Records Database, Assessment, System Coach, Query Index, Content Index, Locator Index, Knowledge Library, Learning Content, Delivery, Multimedia.”



• Identifying which LTSA subset (stakeholder) is applicable -- stakeholders can be any subset, not limited to the examples in the LTSA specification

• Mapping conceptual to actual system components• “The implementation under test shall identify the subset of

applicable LTSA system components.”• “The implementation under test shall identify the

boundaries and mapping of LTSA system components.”



• Identifying the information busses -- bus protocols and topology not limited to examples in LTSA specification

• “The implementation under test shall identify the information busses in use, their namespace, control protocols, and mapping to LTSA system components.”


Open Issues

• Verifying stakeholders perspectives• Identifying and refining operating components: protocols,

interfaces• Public review in IEEE 1484.1 and other fora


Summary

• Yourdon (systems notation): tried and true• Bus notation used for larger number of component• Five abstraction-implementation layers:

– Learner and environment interactions– Human-centered features– System components– Stakeholder perspectives– Operational components


Summary (cont’d)

• Wide range of implementations supported• All stakeholders’ perspectives identified• Reflects commercial systems

Ltsa Slides

Documents

learning stylesystem

system generated

negotiated

learning style

associating

learner chooses

records database

records database