The Valladolid Presentation - Nov, 16, 2011

ARISTOTLE UNIVERSITY OF THESSALONIKI

Workshop on Spanish Technology Enhanced Learning R&D projects -- Valladolid, Nov, 14-16, 2011

Ubiquitous OrchestrationPerspectives from the Pedagogical Space and

Tools for the CSCL teacher

Stavros DemetriadisAssistant Professor

[email protected]://mlab.csd.auth.gr/sdemetri

@sdemetri

Aristotle University of Thessaloniki,Department of Informatics

Greece

ARISTOTLE UNIVERSITY OF THESSALONIKIStavros DEMETRIADIS

Workshop on Spanish Technology Enhanced Learning R&D projects -- Valladolid, Nov, 14-16, 2011 2

...boiled down to a Special Theory of Orchestration



A more modest objective...

(a) Ubiquitous Orchestration A comment on Orchestration linking to the Pedagogical Space Implications (linking to EEE...) Flexibility: “Adaptation Patterns”

(b) Orchestrating the CSCL classroom Extending IMS-LD capabilities to cater for adaptation MentorChat: Orchestrating students’ dialogue Augmented Reality: The PROTEAS project Scripting: What happens in scripted collaboration when it is not

scripted?



The C&B vs. the CSCL classroom I Did the “chalk & blackboard” teacher have orchestration

problems? No chalk (sends a student to bring some) Wet sponge Wet board (has to wait before writing) No curtains (striking sun writings on the board difficult to read)

But look at the poor CSCL teacher: A student is missing Three students (instead of two) have to

work together – Is the script efficient for a group of three also? Some students have problems logging in the web system – What exactly is the problem? Students surfing the Internet / others playing a computer game – What do I do? (while trying to help the others with logging in) Supervising the computer-supported activity– Do they work as instructed/scripted?

A comment on orchestration



The C&B vs. the CSCL classroom II ... the teacher can easily overcome problems in the C&B

classroom Interaction in the C&B classroom is low Students sit quiet and read or write individually No communication .....

But ...you need Collaboration and Technology to make it really a mess! C & T multiply the interactions Teacher - Student // Student – Student // Student – Content // Student – Technology




Orchestration I

Orchestration: Design, Enactment and Adaptation of rich pedagogical scenarios

Over... Multiple Spaces (f2f, Web, 3D Worlds, Augmented reality) Multiple Digital Tools (in the above spaces) Multiple Learning Activities (using various tools in spaces) Multiple Levels (Individual, Small Group, etc.) Multiple Agents (teacher, student, technology) Multiple Synergies ..................................................................




Orchestration II

Ubiquitous: Continuous and over many dimensions Distributed: to many agents (off-loading..)


Example: Nick misses the Monday lesson because he is ill. The teacher has designed a dyad-based script but now student number is odd (23). The system supporting the activity adapts the activity so that a triad can also work productively. Orchestration duties are off-loaded to the system.

Example: The teacher asks the students to form 4-member small groups and enact a jigsaw-based scenario. The teachers says “We work in groups of four as we did the other time”. Students experienced with the scenario understand the hint for the Jigsaw activity.

Orchestration duties are off-loaded to the students



Reflections

Type I Reflection

Technology Spaces (TS)

Data-based

Pedagogical Space (PS)

Pattern-based

Cool technology! I know a good pattern that I want to implement. How can this technology help me?

TEACHER

Type II ReflectionCool technology! I want to use it. But what patterns fit well with these tools?

Mapping TS onto PS

Mapping PS onto TS




An example: Twitter

Type II Reflections: Twitter allows students to send short messages. Is this of some use?

...Well, I put another screen in my classroom and project there students’ Twitter messages. Lets see what happens...

Type I Reflections: a good learning “pattern” is that my students communicate with domain experts. I can use Twitter to support this expert-novice communication by enabling students to “follow” experts.




Reflections (cont’d)

Linking Spaces: Opportunities to teachers to reflect on

PS (Physical + Multiple TSs) relationship

in at least three modes:

(a) “Complementary” Mode Distribute a pattern over various Spaces

(b) “Constrain” Mode Good practices “constrain” the use of novel

tools/services/representations in a TS

(c) “Construct” Mode Reflect on activities successfully distributed

over many Spaces and construct an abstract understanding enriching

the Pedagogical Space and possibly the affordances of TSs




A Model of Reflections

Technology-based Space (TS)

Pedagogical Space (PS)

Pattern-based

TEACHER

Complement? (Type I)

Technology-based Space (TS)

Physical Space

Construct? (Type I & II)

Constrain? (Type II)


ReflectionsData-based

Continuity Discontinuity



Implications I

Support Type I Reflection

Simple – easy to handle – tools to allow intuitive integration

into pattern enactment (i.e. support mapping of PS onto TS) cf. “Awareness” tools (Dillenbourg in Stellar Report, Dillenbourg et al. 2011)

Implications

Example: Simple interaction analysis tools can be easily used by teachers for monitoring and adaptation purposes



Implications II

Support Type II Reflection

Provide teachers with good practices to make optimal use

of new tools/technologies (i.e. help mapping TS onto PS) (cf. Dimitriadis in STELAR Report - Dillenbourg et al. 2011)

Implications

Example: Provide scenarios (“scripts”) for using educational robots in the classroom to help improve (a) domain learning, (b) students’ collaboration and metacognitive skills development



Implications III An Architecture for Orchestration Tools

Easy to use (low complexity) pattern-supported, tools “Pattern-intuitive” level

Interface Level

Specific to Spaces/Tools


Rule Level Advanced users define the Rules for

Interface Level tools Information-based

DataFlow Level Defining DataFlow (protocols,

models, etc.) Data-basedLinking Spaces

Modified across Spaces



Implications IV More on the proposed Architecture ...

What Representations? What Patterns?

DataFlow Level

DataFlow streaming between spaces What Data? How to stream data across multiple Spaces? Semantics to describe Data? Flexibility: Cater for future extensions of data-relevant

protocols?

Linking SpacesModified over Spaces


Rule Level

What Information? (from Data) What Rule? (from Information) Tools to define Rules

Interface Level


Relevant to EEE



Adaptation patterns

Introducing Flexibility Exemplifying the reflection-based

approach & Tool architecture



Flexibility

Pre-defined: I can integrate in my design alternative activity flow to be enacted if conditions occur

On-the-fly: I (or the system) can trigger alternative activity flow during the activity alerted by specific events



Adaptation Patterns

A core idea of pedagogical value on how to adapt the collaborative learning activity when specific conditions occur

…an AP suggests a valuable alternative (to the whole or part of the initial design) depending on conditions

IF [ something specific occurs] THEN [adapt your design]

Example: Students work in dyads on a collaborative task. An “advanced” student works in a dyad with a “beginner” student. The task is easy for the advanced and has lost interest.

Adaptation: the teacher adjusts the task to address the advanced level of student, i.e. makes the task more demanding and interesting for the advanced student (of course without failure of the beginner).

Adaptation Pattern: “Advance the Advanced”

Adaptation patterns



Adaptation Pattern Life-CycleAdaptation patterns



List of Adaptation Patterns

Karakostas & Demetriadis, 2010

Adaptation patterns



FlexCoLab: Implementing Adaptation Patterns 1/2 Adapt the resources Adapt the activity

Novice Intermediate

Advanced

Adaptation patterns



PPR: PyramidPeer Review

Implementing Adaptation Patterns 2/2



Overall Adaptation patterns “live” in Pedagogical Space

Identifiable through Teacher activity analysis & Interaction analysis

Tools like FlexCoLab & PPR aim to help teachers identify favorite adaptations and integrate them in their design (pre-defined)

The tools support mainly Type I Reflections at Interface & Rule level known patterns to implement with technology

Next steps Better interface: Difficulties with complex pattern setting interface Evaluation in context Clearly define Interface vs. Rule vs. DataFlow levels What about On-the-fly adaptations?

Adaptation patterns



IMS-LD

Enhancing IMS-LD to cater for adaptivity



Adaptation Pattern Specification Input, Rule(-s), Model(-s), Output (IRMO)

RULESINPUT

InteractionAnalysis

MODELLED ENTITIES

OUTPUT

On Screen Representation

A D A P T A T I O N P A T T E R N

ordb manifest

During design: Define monitored parameters (e.g. from interaction analysis tools) Rules (the adaptation model of the pattern) are hard-wired to the pattern Define Output (form, content, etc.)

IMS-LD



Extending IMS-LD capabilities

IMS-LD modeling language has limited capabilities in expressing and

implementing complex information processing (necessary when

complex adaptive interventions need to be made by the CSCL

system)

How IMS-LD, a de-facto standard in the CSCL area, could facilitate

the required adaptive behavior of a CSCL system through

communication with external software components? such as web services, communication tools or virtual learning

environments (VLEs).

IMS-LD



MAPIS: A reference architecture for implementing ACLS systems with IMS-LD

IMS-LD



MentorChat

Orchestrating Students’ Dialogue



MentorChat Design Rationale

Dialogues between students and tutors: a prominent component of effective tutoring.

In CSCL: a supportive conversational agent may model and intelligibly trigger the peer dialogue.

The MentorChat agent acts as a peer discourse facilitator enabling the teacher to define what type of intervention to implement.

Not a teacher substitute, but a teacher‘s tool to orchestrate intervention/support strategy during students’ discussions.

MentorChat



MentorChat



MentorChat Architecture

MentorChat

Teacher defined



Teacher Orchestration using MentorChat Define a set of rules to guide the agent’s behavior

A rule can be divided into three parts: (a) an event

A set of keywords that can be used in conjunction with regular expressions in order to recognize specific phrases and language structures

(b) a triggering prompt When an event occurs the agent posts an appropriate triggering

prompt as a participant that joins the dialogue (c) a final prompt

is posted by the agent at the end of the conversation to display some important aspects of the problem‘s solution that

have not been mentioned in the group discussion

MentorChat



Dialogue Sample (Yannis and Nelly discuss online issues of cognitive theory for Multimedia

learning)

<YANNIS>: ...in the brain some connections are formed with the prior knowledge and stored in long-term memory

<Tutor>: What do you think Nelly about the role of long-term memory? <NELLY>: In order to store information in the long-term memory, it should... <YANNIS>: ...we should also mention the concepts of Selecting and

Organizing... <Tutor>: Do you think Nelly that the concept of Organizing can be part of your

answer? Why? <NELLY>: Of course, because the better the two models (verbal and pictorial)

are organized in memory the better the integration with prior knowledge. Is this a satisfactory answer?

<YANNIS>: Wait Nelly, I will write down the answer and then you can check if something is missing

<Tutor>: Before you submit your final answer please consider commenting on the concepts of: verbal and pictorial model.

<NELLY>: ok, then. Lets discuss these issues and I think also we need to mention the redundancy and contiguity principles

MentorChat



Two studies: School and University classes Students discussing online in dyads in lab

conditions MentorChat posts messages whenever a concept

modeled in the vocabulary appears in students’ discussion

Teacher monitoring students conversation

Preliminary Evaluation I

MentorChat



Preliminary Evaluation II MentorChat interface has been evaluated as user friendly and easy to

use Students mention that agent‘s messages helped their group discussion

to advance and improve their collaboration Also: the prompts before the final answer very helpful to refresh their

domain knowledge School: The teacher found very useful the ability to monitor the

discussion from a simple panel She stated that “…a MentorChat activity, ..., can motivate students to

actively participate in a collaborative task…‘ She also said that agent‘s messages can promote the critical thinking of students.

University: Preliminary statistical analysis indicates learning benefits for the treatment condition (discussion in MentorChat)

MentorChat



Implications for Orchestration I MentorChat can help teachers to orchestrate multiple

students’ discussions by predefining interventions (rules) Beneficial interventions may be offered to multiple groups

PROBLEMS What is the necessary agent intelligence to intervene in a

human-like way? Students after using MentorChat model the behaviour of

the agent and may not pay attention to what is said Sometimes students ask the agent for further support

Unavailable at the moment

MentorChat



Implications for Orchestration II

Relevant to EEE? An agent in a space can become mediator for teacher

interventions and orchestration actions Once these actions are predefined then multiple students

groups could be benefited (or not?) from interacting with this agent

MentorChat



The PROTEAS project

PROgramming TangiblE Activity System



Tangible

(AR)

Introductory programming constructs

PROTEAS



Virtual (Isomorphic to AR)

PROTEAS



Some preliminary research results

Participants and Method 61 children (36 boys, 25 girls) 3 age groups (5-6, 7-8, 11-12) They played with both systems Qualities and Quantities

recorded Pictorial questionnaire

Likeness (Total) Tangible 80,0% Virtual 20,0%

Easiness (Total) Tangible 64,4% Virtual 35,6%

Play again With Friends (Total) Tangible 75,0 Virtual 25,0%

Tangible easiness between ages 5-6 years 7-8 years 11-12 years 94,1% 73,1% 18,8%

This is significant and confirms that tangible interface should be considered as easier for the younger

PROTEAS



Implications for Orchestration

Relevant to EEE? Students may understand faster and deeper the

affordances of a space if they use it in parallel with another isomorhic space (?)

Restrictions in one space may help students “discover” and apply self-orchestration rules that can be transferred to other spaces (?)

PROTEAS



What happens in scripted collaboration when it is not

scripted?

Orchestrating scripted student groups



A common orchestration problem

If a script works well when it guides the collaborative activity of a dyad...

...does it work equally well when it guides a triad of students?

Research Questions: 1) How is the script efficiency dependent on group size? 2) What happens when the script leaves unscripted a

part of the activity for the three students?

Scripting



Study design

Learning domain: Algorithms + Algorithm visualization tools Control: Students working in triads to solve an algorithm

problem unscripted Simple Prompt: “Collaborate”

Treatment: Students working also in triads on the same problem Additionally: a peer-tutoring script assigned successively each

student in the triad to become tutor and pose an issue to his/her colleagues relevant to solving the problem

Simple Prompt: “Collaborate” When the Student-tutor posed the issue the two other peers were

unscripted on how to collaborate

Scripting



Alcolab tool

Scripting



“Issue” level analysis

Suppose S1 poses an issue Control: while discussing with others Treatment: Triggered by the script

S2 and S3 are in both conditions unscripted as to what to do next - What do they do?

Issue level analysis: Refers to the analysis of peer interaction (between S2 and S3) after S1 posed an issue

S1

S2 S3

S1 poses an issue

Scripting



Results Control behavior

Treatment behavior

S1 poses an issue

Peer interaction for answering: Directly to S1

Leaving the other peer (S3) in a low-participation condition

Peer interaction for answering: Interaction between S2 & S3

Then answering the issue by S1

This behavior resulted to triads improved learning outcomes (port-test)

S1

S2 S3

S1

S2 S3

Scripting



Why? When a script leaves student interaction unscripted then students’

internal scripts are activated In this case internal scripts responding to prompt “collaborate” were

activated CONTROL: All Students identified themselves as peers

Therefore directly answering the issue by S1 was considered as collaboration

TREATMENT: assigning the role of tutor to S1 resulted in the other two students S2 & S3 identifying themselves as peers Therefore “collaborate” meant interact with peer before answering to the

tutor

Scripting



Implications for Orchestration

Assigning roles to students may create social hierarchies and activate peer interaction among certain partners

Assigning such roles can be a teachers’ handy technique to activate peer interaction between peers

A peer tutoring script effective when implementing with dyads may also be effective when implemented with triads

Relevant to EEE? Social hierarchies established in a space (for example

with virtual partners) may direct and trigger collaboration (discource activity) between students (in this or other spaces)

Scripting



Final Remarks

Orchestration: Ubiquitous & Distributed Pedagogical Space: establishes continuity Support teachers’ reflections while linking (mapping)

various Physical space & Technology-based Spaces on Pedagogical Space

A proposed architecture for orchestration tool entails three levels: Interface, Rule, DataFlow

...but will these ideas and guidelines help making orchestration simpler and easier for the average teacher?



Ευχαριστώ!

Muchas Gracias!



Invitations (1) IEEE Learning Technology Newsletter http://www.ieeetclt.org/content/newsletter Next issue: Adaptive & Intelligent Systems for Collaborative Learning Guest Editor: Stavros Demetriadis Critical Dates: Deadline for submission of articles: 15 December 2011

1000 words submission

(2) 4th International Conference on Intelligent Networking and Collaborative Systems (INCOS-2012) IEEE Technical Sponsorship // Proceedings by IEEE CPS September 19-21, Bucarest, Romania

(3) 3rd International Workshop on Adaptive & Intelligent Systems for Collaborative Learning (IWASCL-2012) In conjunction with INCoS-2012 Previous: IWASCL-2009 in Barcelona, IWASCL-2010 in Thessaloniki

The Valladolid Presentation - Nov, 16, 2011

Education

orchestration problems

orchestration duties

orchestrating students

missingthree students

twitter type

support type

multiple agents teacher

pedagogical space implications