A Foundation for the Study of Computer-Supported Collaborative Learning Requiring Immersive Presence (CSCLIP) For Next Generation Elearning

A Foundation for the Study of

Computer-Supported Collaborative Learning

Requiring Immersive Presence (CSCLIP)

For Next Generation Elearning

Ramesh Sharda1, Nicholas C. Romano, Jr. 1, Joyce Lucca1, Mark Weiser1, George Scheets 2, Jong-Moon Chung2, and Catherine M. Sleezer3

1Management Science and Information Systems 2Electrical and Computer Engineering

3Human Resource and Adult Education Oklahoma State University

Contact E-mail: [email protected]

2

A Foundation for the Study of

Computer-Supported Collaborative Learning

Requiring Immersive Presence (CSCLIP)

For Next Generation Elearning

ABSTRACT

The dramatic increase in distance learning (DL) enrollments in higher education is likely to continue. However, research on DL that includes psychomotor, cognitive, and affective skills is virtually non-existent. Indeed, DL for psychomotor skills has been viewed as impossible. Laboratory coursework, which we define as including learning of psychomotor, cognitive, and affective skills, has become a limiting factor in the growth of DL. What is needed is a synergistic integration of technologies and Human-Computer Interface (HCI) principles from Computer Supported Collaborative Learning (CSCL), collaborative learning systems, and immersive presence technologies to enable achievement of psychomotor learning objectives. This paper defines the Computer Supported Collaborative Learning requiring Immersive Presence (CSCLIP) research area, provides a theoretical foundation for CSCLIP, and develops an agenda for research in CSCLIP to establish a foundation for the study of this emerging area. It also briefly describes a CSCLIP-based telecommunications lab currently under development. CSCLIP is presented as a major research opportunity for IS researchers interested in empirical research as well as technical development.

KEYWORDS

Computer-Supported Collaborative Learning Requiring Immersive Presence (CSCLIP),

Distance Learning (DL), Situated Learning, Immersive Presence (IP), Psychomotor

Learning Objectives, Laboratory Equipment Manipulation and Control.

3

I. INTRODUCTION

The use of Distance Learning (DL) to provide education and business instruction is increasing. A

recent National Center for Education Statistics (NCES) survey found that over the next 3 years

most higher-educational institutions plan to start using or to increase using asynchronous Internet

instruction, two-way interactive video, and synchronous Internet instruction [61]. The 2000

NCES survey also reports that higher education institutions are using a number of different

technologies to deliver distance education including various video and Internet-based

technologies, and other modes such as CD-ROMs [61]. Businesses are increasing their use of DL

for corporate training and education [40, 84, 98]

DL technology advancement stems from educator awareness and use and also from provider

application development [61]. Technological advances create awareness and demand among

users, while use pushes providers to enhance existing technologies and develop new ones [43.]

Increasingly adult educators recognize that to be effective, on-line courses cannot simply

replicate classroom instruction [34]. Research-based strategies are needed that improve DL in the

three learning domains: the cognitive domain that focuses on intellectual content and problem

solving, the affective domain that focuses on learning emotions and values, and the psychomotor

domain that focuses on learning the characteristics of physical movement. Many DL studies focus

on cognitive skills (e.g., [1, 27, 63, 102 ].) A few DL studies focus on affective learning (e.g., [63,

77, 97]). However, DL research that includes cognitive, affective, and psychomotor skills is

virtually non-existent. Indeed, using DL for psychomotor skill acquisition has been viewed as

impossible. Newton [71] states, “Psychomotor learning is …. outside the domain of online distant

learning.” Laboratory (lab) coursework, which we define as including psychomotor, cognitive,

and affective skills acquisition, has become a limiting factor in DL growth. Current knowledge in

instructional development (ID) and information technology (IT) are insufficient for learning

modules that employ hands-on DL with equipment in a group setting.

Similarly, MIS research on systems and on Human-Computer Interface (HCI) princ iples as

applicable to education has thus far focused on cognitive or affective objectives. See for

example, [2]. Sherron and Boettcher [89] argue that DL has advanced through four different

generations in terms of predominant technology, time-frame, media, and communication aspects.

Their discussion shows a historical movement towards interactivity in DL technologies and

4

processes. It does not however address lab modules that require small teams to collaborate on

solving tasks requiring psychomotor skills.

The proposed research agenda builds on the body of knowledge about virtual labs. Many virtual

lab projects are immersive, but target individual learning rather than group interactions. Some

immersive systems are too expensive for typical individual DL students, or they require travelling

somewhere to use them, defeating the purpose of DL. Other systems use virtual reality to

illustrate concepts (e.g., in chemistry), but they do not include psychomotor skills or group

settings. Thus far, little progress has been made in developing virtual labs that 1) enable

individuals and groups to engage in distributed learning of cognitive , affective, and psychomotor

skills, 2) provide a DL setting that is typical of traditional lab courses, where the instructors and

other students are engaged, and 3) use a practicable virtual lab system with reasonably-priced,

available hardware and software. Research that contributes to developing such labs will enable

lab learning to extend to new locations and populations. It will also contribute to basic ID and IT

knowledge on distributive group learning at various levels of virtuality. This is one domain where

MIS researchers can truly build upon their strengths and contribute to the next generation of

eLearning.

Accomplishing such research requires a synergistic integration of learning and IT scholarship that

includes HCI principles from Computer-Supported Collaborative Learning (CSCL), collaborative

learning systems, and immersive presence technologies. We refer to this specific eLearning

domain as Computer-Supported Collaborative Learning requiring Immersive Presence (CSCLIP).

Learning environments can be categorized based on the dimensions of temporal, spatial, and

learning objectives. DeSanctis and Gallupe’s [31] 2 by 2 framework for Group Support Systems

(GSS) has been applied to understanding IT usage in learning environments. While this

framework enables us to classify learning settings based on

5

Figure 1. A typology of learning environments

dimensions of space and time, it does little to improve our understanding of the technologies

required to support the learning objectives in different settings. We propose extending DeSanctis

and Gallupe’s framework by adding a third dimension, learning objectives, that differentiates

between typical classroom and lab instruction. Figure 1 presents our extended typology of

learning environments with three dimensions, temporal, spatial, and learning objectives. Our

typology shows that a traditional lab setting requires Same-Time/Same-Place (STSP) interactions

and has cognitive, affective, and psychomotor learning objectives. CSCLIP uses IT to extend lab

instruction to support Same-Time/Different-Place (STDP) environments through DL while

maintaining the level of presence and support for cognitive, affective and psychomotor learning

objectives. Lab activities can also be conducted in a Different-Time-Same-Place (DTSP)

environment through the use of sequential experiments As technology improves, some lab

experiences are being offered in a Different-Time/Different-Place (DTDP) environments through

the use of video, computer models, and simulations [35, 50]. The focus of CSCLIP is to support

lab courses that require group collaboration on lab exercises at a distance in the STDP mode.

Examples of such exercises include manipulating and controlling equipment, assembling and

connecting components, disassembling components, and communicating effectively among group

members.

6

In this section of the paper, we defined CSCLIP, highlighted the need for CSCLIP research and

practices, and framed CSCLIP as a type of learning environment. The remainder of the paper

builds on this information. In Section II we discuss the theoretical foundations for CSCLIP, in

Section III we describe a CSCLIP system currently under development and briefly mention other

CSCLIP-based systems, in Section IV we outline a research agenda for CSCLIP, and in Section

V we conclude with directions for future research and implications.

II. THEORETICAL FOUNDATIONS: THE EVOLUTION OF CSCLIP

The theoretical basis for CSCLIP derives from several areas including learning theories,

collaborative computer systems, and technology theories. In this section we overview the

literature on psychomotor learning objectives. Then we summarize other learning theories that

form the basis for CSCLIP principles. Finally, we discuss how the various collaborative learning

technologies come together to support the complete set of all three types of learning objectives.

Our review is representative of the vast and growing literature in this area and by no means

exhaustive.

7

2.1 Psychomotor Learning Objectives

Bloom’s [10] behavioral learning objectives span the domains of cognitive, affective, and

psychomotor. Each learning domain is subdivided into hierarchical categories with increasing

levels of complexity [46]. Researchers have developed hierarchical taxonomies for each domain:

Bloom’s Taxonomy of Cognitive Behaviors [10] Krathwohl’s Taxonomy of Affective Behaviors

[59] and Simpson’s Taxonomy of Psychomotor Behaviors [91]. The cognitive domain refers to

intellectual learning and problem solving. Cognitive levels of learning include knowledge,

comprehension, application, analysis, synthesis, and evaluation [10]. The affective domain is

associated with emotional learning, feelings, being, relationships, and the ability to deal with

situations. Affective levels of learning include receiving, responding, valuing, organizing, and

characterizing by a value [59] The psychomotor domain refers to movement characteristics and

capabilities including physical types of learning [91]. Harrow’s [45] taxonomy for the

psychomotor domain is organized according to the degree of coordination including involuntary

responses as well as learned capabilities. Psychomotor levels of learning include perception,

simulation, confirmation, production, and mastery.

Figure 2. Relationship among Cognitive, Affective, and Psychomotor domains (Hauenstein

1998)

Hauenstein [46] posits that all three domains (i.e. cognitive, affective, and psychomotor) are

essential for each level of learning and that behavior results from the combination of learning in

all three domains. Figure 2 shows how the force and magnitude of the cognitive and affective

domain vectors combine to result in a psychomotor domain vector. People act in accord with the

8

strength of what they know (cognitive) and the strength of how they feel about it (affective) and

the strength of what they can do [46.] Seels and Glasgow [87] describe this further: perceptual

abilities require the learner to make adjustments to his/her environment based on interpreting

various information sources.” Examples include the ability to catch, write, balance, distinguish,

and manipulate. Skilled movements result from acquiring a degree of efficiency when performing

a complex task. Physical activities can also require strenuous effort for long periods of time so

endurance, strength and vigor may be factors. Non-discursive communication includes body

postures, gestures, and facial expressions. CSCLIP instruction requires that learners achieve

perceptual, skilled movement, and non-discursive communication learning objectives through

DL.

2.2 Learning Theories

An initial review of the literature on learning theory in general, and on technology-supported

learning specifically, reveals that no one theory exists that adequately explains how people learn,

how instructional systems should be designed, how social interaction affects learning, or how

people and technologies function best together [58.] These are important dimensions for DL

instruction. The following sections overview three major categories of learning theories: 1)

collaborative learning theories; 2) group theories; and 3) technological theories.

2.2.1 Collaborative Learning Theories

Caroll’s [16] early study of computer-aided instruction (CAI) indicated that tasks should be

meaningful, active, and build on the learner’s experience. This finding supports Knowle’s [57]

theory of andragogy that highlights the learner’s need to know why he or she should learn

something. As technology improved, collaborative tools were developed that supported

distributed interaction and that led to educators using the tools to achieve both cognitive and

affective objectives. Theories that focused on the individual and the social aspects of learning

became important such as shared cognition and situated cognition theory {Brown, 1989

#284;Lave, 1988 #535;Lave, 1990 #536. Situated Cognition theory stresses learning where

students learn particular concepts in real world situations, and the concepts can actually be used.

Collins [22] defined situated learning as the notion of learning knowledge and skills in a context

that reflect the way the knowledge will be useful to the learner in real life. Stein [96] states that

“to situate learning means to create the conditions in which participants will experience the

complexity and ambiguity of learning in the real world. Situated learning often allows peers to

understand how the knowledge and skills they have developed can be used in new situations.”

9

While socio-cognitive theory focuses on individual learning within a social situation, socio-

cultural theory focuses on the causal relationship between social interaction and an individual’s

cognitive development [32]. Vygotsky’s [101] work in the area of Zone of Proximal

Development (ZPD) relates to the theory. Vygotsky defined ZPD as an area of learning activities

that individuals can complete with the help of more capable peers, teachers, or artifacts. He also

said that interaction and scaffolding can aid in individual cognitive growth. Essentially, an

individual who learns improved problem-solving skills under guidance of or in collaboration with

more capable individuals can then apply the improved skills when working independently to

solve similar problems.

Socio-constructivist theory recognizes that knowledge is not a fixed object, but rather constructed

by an individual through working and practicing with that object [88]. The theory supports

learning through authentic, challenging, and collaborative projects [33]. It extends Piaget’s [79]

work on individual cognitive development to adult learners. Instruction based on socio-

constructivist theory relies on collaborative learning environments that closely reflect real world

experiences. Students working together in authentic activities bring to the learning their own

frameworks and perspectives. The experience enables them to see problems from other students’

perspectives, to negotiate and to create new meanings and explanations through shared

understanding. The collaborative and social theories in this section relate to the group theories

that are discussed in the next section.

2.2.2 Group Theories

According to group composition theory, many factors affect the productivity of collaborating

groups. Two factors that are especially important for CSCLIP are group size and level of group

homogeneity. With respect to the group size, small groups seem to function better than larger

ones. In large groups, negative behaviors such as “social loafing” can occur, and, furthermore,

some members can be excluded from interesting activities [69, 86]. The dominant theory relative

to homogeneity vs heterogeneity of group members is the similarity-attraction theory [78], which

indicates that more homogeneous groups have less conflict, fewer differences in opinions, faster

communication, and more frequent interactions. Heterogeneous groups on the other hand

generate more varied opinions, and more creative group decisions. Research findings indicate that

“optimal heterogeneity” exists with some differing opinions that trigger interactions that are

within group norms [19, 65].

10

Adaptive structuration theory explains why and how computer systems impact group behavior

[81]. The theory’s four basic dimensions are 1) control – Is the group led by the technology or

does it try to alter the system for its own purposes?; 2) attitude – What is the level of comfort with

the system and the degree of respect the group develops for the technology?; 3) faithful or ironic

usage of the technology – Is the system used in the “spirit of the technology” i.e., its intended

purpose?; and 4) level of consensus – What collective beliefs and group social structure are

needed for purposeful action?

Zigurs and Kozar [103] provide an integrated research framework using an input-process-output

(IPO) model. Input variables include group task performed, group composition, and the

technological environment. Process variables include interaction and intermediate role outcomes.

Outcome variables consist of effectiveness and efficiency, satisfaction, and cohesiveness. As

CSCLIP involves extensive use of technology, it is necessary to look at those theories as well.

2.2.3 Technology Theories

Daft and Lengel’s [25] media richness theory proposes that a rich medium facilitates rapid

clarification of ambiguous issues, while a “lean” medium can require a longer time to improve

understanding. Face-to-face (F2F) is considered the richest form of media because in can provide

immediate feedback, multiple cues including voice inflection and body gestures, and a range of

meanings that can be expressed using natural language to convey personal feelings and emotions.

Leaner media tends to be more impersonal and include written memos or formal reports. This

theory recognizes that users have a mix of information requirements and communicate with

varying degrees of uncertainty and equivocality that requires a range of richness in the media.

Davis [28] theorized in the Technology Acceptance Model (TAM) that two main criteria predict a

user’s attitudes towards use of an IT system, perceived usefulness and perceived ease-of-use.

Perceived usefulness is defined as the extent to which individuals believe a system “will help

them perform their job better.” Perceived ease-of-use, in contrast is defined as “the degree to

which a person believes that using a particular system would be free of effort.” TAM also posits

that perceived usefulness is impacted by perceived ease-of-use because the easier a system is to

use, the more useful it can be at improving job performance. Technology has been moving toward

lower cost, personal computers, and immersive presence-related technology. While theories

11

relating to these issues are in short supply and are largely untested, in the next section we look at

some attempts to explain this phenomenon.

2.2.4 Presence Theories

Presence theory is based on earlier work in the area of social presence. Short, Williams and

Christie [90] emphasized the need for social presence, which they defined as the quality of the

communication medium that is required? to understand person-to-person telecommunications.

Recent work by Biocca et al. [7] argues that the level of satisfaction and the productive

performance in teleconferencing and collaborative virtual environments is based largely on the

quality of social presence. Unlike physical environments, social virtual environments might

require communication with minimal or constrained social cues. Social presence theory can

provide insights into the nature of nonverbal and interpersonal communication and how this

affects productivity and the transfer of skills from a distributed learning environment to a real

world setting.

CSCLIP can thus be defined as a theory-based system involving hardware, software, and people

that support the achievement of cognitive, affective and psychomotor skills in a distributed

environment. Figure 3 summarizes many theories that inform CSCLIP.

Figure 3. Theories that inform CSCLIP

12

13

2.3 Collaborative Learning Systems

In this section, we relate CSCLIP to existing learning systems. The idea of employing computer

systems to facilitate both individual and team-based learning has been around since the 1940’s,

when Vannevar Bush described his famous “Memex” [14]. Over the past four decades

researchers, educators, and corporate trainers from many varied disciplines have explored using

computer systems in teaching and learning and several areas of research and practice have

emerged. More recently many universities have begun to offer full degree programs online

through distance education [20, 29, 56, 61, 64].

Three types of computer-based systems have been employed individually and in pairs to achieve

various DL objectives: Computer-Supported Learning Systems, Collaborative Systems, and

Immersive Presence Systems. Each system has evolved independently with researchers and

practitioners from several disciplines making great strides toward the use of the computer to

“augment” the human intellect [37]. When two of these systems are integrated, higher-order

learning objectives may be achieved. However, we assert that integrating all three will enable

learning objectives to be supported in a DL context that have typically required co-located

interactions. We briefly describe each type of system relative to CSCLIP.

Computer-Supported Learning Systems have traditionally been labeled Computer-Aided/Assisted

Instruction (CAI) systems (i.e. [9, 18, 26, 51, 80]. These systems contributed significantly to the

use of computers in education. However, they traditionally focused on individual learners

working on a local computer to accomplish cognitive learning objectives. DL, at its most basic

level, is an extension of CAI to enable remote students to access course content [20, 29, 64].

Several different technologies and methods, ranging from simple downloading of text to

sophisticated streaming of digital video, have been used for DL [3, 11, 60]. Traditional DL still

focuses on content delivery to individual students to accomplish cognitive learning objectives

[10, 64, 89]

Collaborative Systems are often referred to by the all-encompassing term “GroupWare”, that was

coined by MIS researchers Paul and Trudy Johnson-Lenz Circa 1980 [55]. Collaborative systems

can range from email to online discussion groups and Internet chat rooms to sophisticated Group

Decision Support Systems [21, 54]. Most GSS research for education has involved same-time,

same-place classroom situations [100].

14

Immersive Presence systems commonly referred to as Virtual Reality (VR) [38] have emerged as

a result of advances in networking technology and processing power coupled with decreasing

costs in desktop audio and video equipment Slater et al. [94] define immersion as “an objective

description of what any particular system does provide. Presence is a state of consciousness, the

(psychological) sense of being in the virtual environment, and corresponding modes of behavior.”

Slater and Wilbur [93] elaborate on this and argue that immersion can be assessed by the

characteristics of technology independently of presence. Immersion can lead to presence (i.e., a

participant’s psychological sense of “being there”). Furthermore, immersion is a necessary rather

than a sufficient condition for presence [95]

Succinctly put, immersion is wholly a product of the system, while presence is wholly a product

of the subject’s psyche [8]. Lombard and Ditton [62] extend the concept of “being there” to

include the idea of transportation. They identify three distinct types of transportation: "You are

there," whereby the user is transported to another place; "It is here," in which another place and

the objects within it are transported to the user; and "We are together," in which two (or more)

communicators are transported together to a place that they share.” These concepts are also

relevant in CSCLIP environments.

Immersive Presence user interfaces that are available today range from highly rich and immersive

to the much leaner medium of desktop videoconferencing down to text-based interactive

communication. At the high end is the Cave Automatic Virtual Environment (CAVE) [85],

which is a VR system with a rear projected 10 foot-cubed room display and stereoscopic images

that creates the illusion that 3D objects appear to co-exist with the user in the room [85]. In the

middle is PC technology that uses Virtual Reality Markup Language (VRML) and multimedia

technology. Finally, text can considered being immersive, like when one is “immersed” in a good

book [42].

Figure 4 illustrates how the three system types are combined to support teaching and learning as

well as for business processes, entertainment, or other purposes. We discuss them in paired

combinations below.

15

Figure 4. Overlap of the three areas that CSCLIP support

The intersection of computer-supported learning systems and collaborative systems includes

systems that extend DL by integrating collaborative learning and information technology, which

is commonly referred to Computer-Supported Collaborative Learning (CSCL) [75]. Many MIS

researchers have used Group Support Systems (GSS) in the classroom to enhance learning [99,

100], while others in IS and related fields have developed Asynchronous Learning Networks

(ALNs) [6, 23, 36, 44, 49]. Combinations of these two system types have enabled affective

learning objectives related to interactive communication and teamwork to be achieved, in

addition to more traditional cognitive learning objectives.

The intersection of computer-supported learning systems and immersive systems includes many

systems of virtual learning environments that have resulted from their integration. Researchers

have explored virtual classrooms [47, 48, 70] and using Video teleconferencing or streaming

video to present lectures via the web or on CD-ROM to remote students [11, 53, 82]

The intersection of immersive presence and collaborative systems includes categories: 1)

entertainment, 2) simulation, and 3) visualization [67]. The first category, entertainment,

includes of Multi User Domains (MUDs), which were originally designed to facilitate such

action-packed adventure games on the Internet as Doom and Dungeons and Dragons [92] Later a

new MUD-system was developed called MOO (MUD Object Oriented) where participants create

16

virtual ‘selves’ and ‘lives’ using text and then interact with other participants in a virtual world in

real-time sometimes over an extended period of time, leading to long-term relationships. The

second category, simulations, are widely used in the area of Virtual Training (VT) for military

purposes [5, 17, 24]. This form of training provides individuals and groups training in real-world

situations in surroundings that do not restrict safety. In addition, participants can review their

actions and discuss with other group members better ways to improve performance. In the third

category, visualization, engineers, architects, and designers use combinations of immersive and

collaborative systems extensively to solve problems that require large, complex models and data

sets. Collaborative Virtual Design Environments (CVDEs) use VR to view and review complete

systems, assembly processes, and individual parts [83.] CVDEs provide realistic 3D displays and

enable rotational capability for complete 360-degree visualization as well as views from top,

bottom, inside, and underneath objects.

CSCLIP represents the intersection and integration of these three system types. We propose that

this integration which enables the achievement of psychomotor learning objectives facilitates

learning in STDP group-situated learning environments for all the learning domains, and enables

immersive presence for the next generation of eLearning.

III. CSCLIP in ACTION

3.1 A Virtual Telecommunications Lab Under Development

A CSCLIP-based system is being developed at Oklahoma State University (OSU) to support the

DL capabilities of its highly successful Master of Science in Telecommunications Management

(MSTM) program (www.mstm.okstate.edu). Although most required MSTM coursework is

delivered through a combination of Web and video technologies, the program requires all

students to travel to OSU to receive significant learning via a hands-on lab on the use of technical

equipment. This lab course covers various aspects of voice, video, and data networking. The goal

of the lab is to ground the learning previously obtained in several theory-based lecture courses in

reality, to familiarize students with telecommunications equipment, and to enlighten them about

challenges that technicians face.

The lab experiments mimic the order in which communications might be installed in a small

company. Groups of students first install telephone services using Digital Multiplex Systems

(DMS) 10 (DMS-10 and Private Branch Exchange (PBX.) Next they use Microsoft Windows to

establish peer-to-peer Ethernet Local Area Networks (LANs) and to enable file sharing and

17

e-mail services within their group. The groups also establish PC-based video conferencing. They

then interconnect their individual group networks (LAN) via a backbone packet or ATM network,

and connect the resulting network to the rest of the world. Student groups use the installed

communications to conduct experiments to better understand the intricacies of Internet

Protocol-based networking, client-server networking, Web-server hosting, and circuit- and

packet-switched voice and video. They also investigate carrier-type, wide area network issues

using simulation software.

Although feedback on the lab course is highly favorable, compelling travel to a common site is

often viewed as undesirable and may force some students not to enroll in the program. Continued

advances in wide area and last mile communications and other integrated technologies show

promise for making a virtual lab experience as effective as a co-located interaction. Additionally,

leveraging these emerging support mechanisms may even improve the experience of those who

attend the lab in person. The MSTM lab course is an excellent candidate for implementing

CSCLIP because student interactions with many functions in the hardware occur via personal

computers. Given today's technology, there is no reason that students’ PCs have to be phys ically

located in the lab for them to participate. However, enabling the interactions that normally occur

in a STSP lab requires good quality audio and video communications between the lab and the

remote students' sites.

To orient remote students to the physical structure of the lab and the available equipment, a

simulated three-dimensional virtual environment has been constructed, through which all students

can tour the lab prior to the first day of class. It is possible to “walk” the halls and investigate a

variety of equipment, without regard to student location, physical accessibility of the room, or

equipment availability. In some ways, this virtual tour is better than a physical tour because

equipment from the lab is linked to detailed technical information from manufacturers and

protocol developers. For example, someone could virtually walk around the telephone switch,

select and remove a critical component for closer inspection, and then pull up technical details for

that device, all without disrupting the function of the physical switch.

Additionally, the lab has several live cameras that can be remotely accessed by students via a

point-and-click virtual map. Almost all areas of the lab can be viewed by selecting a part of the

lab to “be in” through the various camera nodes. To prevent eavesdropping and spying, whenever

someone connects to a camera, his or her image is displayed on a co-located monitor, so it

18

appears as if they entered the room. The video link also includes an audio connection to the local

monitor, allowing a remote student to verbally interact with students or the instructor in the

immediate area of any pre-positioned video node.

The integration of the virtual tour with a network of conferencing nodes allows participants to

move around the lab in the virtual world, manipulate components of live equipment without

disrupting operations, and engage in communication with the students and instructors who are

physically present in the lab or who are at other remote locations. We expect the virtual

environment to enhance the experience by creating an “off-line” environment that can be used for

between-class learning, as well as a route to move from one synchronous environment to the next.

Figure 5 shows the navigational map of the lab, the camera controls, and a telephone switch.

Students can view all the lab equipment through “hot spots” that display components in 360

degrees and contain detailed documentation.

Figure 5. Virtual Tour of a DMS-10 Telephone Switch

19

Remote students cannot physically manipulate cables, therefore we provide a similar experience

that allows remote students to work through the same logical processes toward a solution and to

have pedagogical results similar to the local experience. We accomplish this using physically

connected links over which remote students establish logical connections. For instance, if a

connection to a hub would normally be made with a category 5 twisted pair cable with RJ-45

connectors, that cable will be installed and tested in the physical lab. A remote student selects the

appropriate cable from a “bin” of virtual cables available through the interface and plugs it into

the hub using the mouse and drag-and-drop action. See Figure 6.

Figure 6. Virtual cable bin for a cabling exercise

If the cable ends are physically correct, the software allows the student to graphically connect to

the hub. See Figure 7. If the correct media (between the terminations) are selected, there is a high

probability that a Simple Network Monitoring Protocol message will be sent to the hub activating

the existing physical connection already in place. This virtual cable connection is then made

available to all students just as if a physical connection had been established. However, virtual

cabling errors are randomly generated to simulate the faults that are experienced in F-2-F lab

20

settings and in production networks. Students trouble-shoot the errors with virtual versions of

cable testing equipment. Success is based on probabilities just as in the physical world : bad

cables are mixed with the good cables in the cable bin creating a real-world chance of failure due

to a bad cable.

Figure 7. Virtual cable connection in process

These are just a few example virtual applications from the many that have been developed and

tested to help groups of local and remote students work together to achieve learning objectives in

a telecommunications lab course.

3.2 Other CSCLIP Systems

To provide a more complete understanding of CSCLIP, we conducted an extensive literature

review and Web search to identify educational CSCLIP applications that include the components

of collaboration, learning, and immersive presence. We used a broad definition of immersion that

includes both the highly immersive systems that employ large room technology and the less

immersive desktop systems. The systems range from the complex CAVE systems that was

21

described earlier in the paper to the PC-based 24/7 physics lab developed at University of

Lancaster, UK [12, 13] Some systems are in the very early stages of development and

information was insufficient to include them in the comparison. Lucca et al. (2003) discusses the

comparative technology and human interface features of these systems. It is evident that new

work is beginning to take shape in this area. In the following paragraphs we highlight some

efforts that are further along in the development lifecycle .

Researchers at the University of Washington are doing some highly innovative work with

augmented reality that has application in the medical field. Their project “Viewing the Future”

(https://www.fastlane.nsf.gov/servlet/showaward?award=0226323) uses augmented reality that

has application in the medical field. It builds on the network performance of Internet2 to deliver

video, simulations, animation, VR movies, audio sound tracks, and other digital media . This

project even seeks to enable remote control of microscopes and other traditional laboratory

equipment. However, these and other similar projects rely on the enormous bandwidth Internet2

provides.

Projects also exist that deal with remote instruction. For example, Old Dominion University’s

TELETECHNET (http://www.cs.odu.edu/~dlibuser/nsf/nie/iri/). seeks to refine and provide a set

of basic features that faculty and students of all disciplines can use to enable interactive remote

instruction. Their system is based on shared operation of X-windows tools and applications

However, it is not designed to be used in a real-time environment and does not facilitate or

require all the components of immersive presence.

IV. CSCLIP RESEARCH AGENDA

CSCLIP offers major research opportunities for IS researchers along both technical and

organizational dimensions. In this section we identify some possible directions where IS

researchers can contribute to further development, understanding, and evaluation of CSCLIP next

generation eLearning systems. Figure 8 illustrates one possible path for research in this domain.

One can begin by considering the theoretical perspectives of learning, groups, and immersive

presence, available technologies, and the specific domain knowledge for a particular set of lab

experiences to be offered in a distributed environment. Researchers could operationalize this

conceptual and technical knowledge by designing specific lab exercises and processes for

distributed learning environments. This step also leads to determining what specific technologies

need to be developed or integrated to create a usable lab experience. As the lab exercises and the

22

technologies are developed, there is a need to design and conduct rigorous controlled experiments

to assess the effectiveness of the learning exercises and processes. The results from those

experiments will produce the generalizeable distributed learning technologies and processes and

also contribute to the CSCLIP theory base.

Redesign of Lab Experiences

And Desired Outcomes

DomainExpertiseAvailable,

AppropriateIT Modules

Develop and Assess New Technologies

LessonsLearned

Learning Theories

Collaborative Theories

ImmersivePresence Theories

Theory

Design and Conduct Rigorous, Controlled Experiments

ContributeTo Refinement

Figure 8. A General research agenda in CSCLIP

4.1 Learning Process and Outcomes Research

Developing next generation eLearning systems presents an opportunity for IS researchers to

contribute to the theory of distributed cognitive, affective, and psychomotor learning through

rigorous empirical testing. This section presents a research agenda to guide this undertaking.

Figure 9 presents a framework for the empirical study of CSCLIP systems. We identify four

classes of input variables that affect the learning outcomes and present possible research issues

for them below.

23

Figure 9. Framework for research in CSCLIP

Researchers need to measure the learning impacts of STDP lab instruction on individual and team

learning compared to STSP lab instruction, especially the Instructional Design (ID) and

Information Technology (IT) factors that affect STDP acquisition of lab skills through DL.

Gandolfo [41] states that technology applications must be consonant with what is known about

the nature of learning and must be assessed to ensure that they are indeed enhancing learners’

experiences and notes that this is not happening in the current polarized environment. One barrier

to the use of IT by university faculty is “the lack of information about good practice” [4]. This

research needs to provide “good practice” guidelines for the use of technology in ways that are

consonant with learning processes and objectives.

Validation of learning and the verification of student assessment have been issues in DL since its

inception [52]. The challenge is to match the assessment to the mode of delivery and to

simultaneously assure that assessment consistently measures competence. The cognitive domain

of DL is most commonly assessed in higher education because the assessments can be easily

created and interpreted. Holzl [52] observed that psychomotor objectives are the most difficult to

measure and noted a lack of literature on psychomotor assessment in DL. Morley [68] stated that

measuring affective objectives are difficult because of their general nature, but that DL “does not

exacerbate the problematic nature of affective measures, and in some ways, might even increase

the validity of results” (Cognitive, Affective, and Psychomotor evaluation in Distance Education

section 3).

24

The next sections examine the four classes of variables that affect learning outcomes in CSCLIP

environments: Human dimension; instructional design (ID), instructional technology (IT), and

learning processes.

4.1.1 Human Dimension

Some variables are inherent to individual students, teams, and instructors. Gagné et al. [39]

identify conditions internal to learners (e.g., the learners’ intellectual skills, cognitive strategies,

verbal information, attitudes, and motor skills.) ID deliberately arranges external events to

support internal learning processes.

4.1.2 Instructional Design and Learning Processes

Instructional design involves using a systematic five-phase process to create instruction: analyze;

design; develop; implement; and evaluate. All the phases are connected by learning objectives;

each ID phase contributes to creating instruction that results in the learning specified by the

objectives. CSCLIP instruction can be designed and developed to provide three support

mechanisms: process support, task structure, and task support [72., 74] These mechanisms

improve the effectiveness, the efficiency, and the satisfaction of distributed groups by increasing

process gains and reducing process losses. Process gains are activities that improve group

performance over individual performance. Examples of group process gains that lead to better

overall performance include the generation of more alternative solutions, improved error

detection, and increased synergy [73.] Process losses diminish group performance compared to

individual performance. Examples of group process losses include fragmentation or turn taking

when speaking is necessary, domination by one or a few individuals, fear of negative evaluation

by other group members, and information overload. Process gains are increased through

improved communication channels that reduce fragmentation, dominance, and social loafing and

enable members to stay focused on the task. Process support for CSCLIP may be provided

through the use of audio, video and chat using basic TCP/IP networking technology, as well as

more sophisticated VR technologies. Task support refers to the instructional infrastructure (e.g.,

the virtual tours of the lab and equipment). Task structure refers to the instructional lab modules

that provide information for task completion.

4.1.3 Instructional Technology

25

Technology plays a key role in implementing and assessing CSCLIP. Based on theoretical

underpinnings, the quality, reliability, ease of use, usefulness, and accessibility of the technology

are important concerns. The effects of alternative technologies on learning outcomes and their

potential application to CSCLIP remain to be studied. Many technology options to be

investigated are discussed in Section 4.2.

4.1.4 Outcomes

The variables we have identified may influence the quality and quantity of achievement outcomes

in the learning environment and the transfer of learning to the performance environment.

Evaluation includes summative and formative measures. Summative measures determine whether

learning objectives were met. Cognitive objectives can be measured with paper-and-pencil tests,

essays, and oral tests or presentations; affective objectives can be measured with role plays, paper

and pencil tests, or exercises; psychomotor objectives can be measured with performance tests,

on-the-job observations, or product reviews [15.] Formative evaluation identifies needed

improvements and changes to make program design and delivery more effective. Multiple

measurements of the outcome variables as proposed by DeLone and Mclean [30] are extremely

critical.

4.1.5 Research Agenda, Questions, and Approach

Synthesized, the background information reveals the need for basic research on DL for lab skills.

It also reveals that the scholarship in the areas of Education and Information Systems provide

language and frameworks that will support such research.

One key research issue is to determine the effect of various ID and IT variables on lab skills at

three different levels of “virtuality”, using Hauenstein’s [46] conceptual framework. We need to

test for DL environments whether learning improvements in all three domains will result in

higher levels of performance than learning improvements that are limited to any one or two

domains. We also need to test various characteristics of individuals, teams, instructor, tasks, IT

Tools, ID strategy to understand specifically how they affect lab learning.

Examples of some specific issues that need to be studied are:

• Measuring how improvements in Cognitive and Affective skills can affect the acquisition

and transfer of psychomotor lab skills and performance behavior among teams and

individuals in DL situated environments.

26

• Examining effectiveness of proposed IT tools to ascerta in which give the most impact in

terms of improving learning processes and outcomes

Because the research in this area necessarily involves studies of distributed groups, multiple

research paradigms can make important contributions. In addition to positivist studies,

interpretive studies of group interactions and instructor-student interactions are needed to

understand how the ideal learning systems can be constructed [66, 76.]

4.2 Technology Development

IS researchers interested in technology development will find CSCLIP to be a fertile ground for

developing either completely new applications and/or modifying existing technologies.

Improvements in such related areas as the increasing availability of reliable network access, high

quality video and audio, and computer processing power will indirectly improve the CSCLIP

learning experience. Application specific advances are also required to both improve the

immersive experience and increase the ease of interaction. Short term, these will involve using or

modifying existing or under development hardware and software platforms to provide at least

minimal solutions. Medium term, emphasis will likely focus on making the high-end systems of

today low cost and readily available to the masses. Long term, many new technologies need to be

invented to enable the CSCLIP experience of the future.

Predicting long-term requirements necessitates speculation that is all too often incorrect. We

focus instead on noting some selected short and medium term enhancements yet to be fully

realized.

Ideally, a remote student will be able to interact with other students and the instructor as easily as

if he/she were in the local lab environment. Fixed cameras scattered about a lab only partially

solve this problem, as the cameras may not be optimally positioned, if for example, the instructor

needs to demonstrate something not in a camera field of view. A Wireless Instructor system that

enables virtual “over-the-shoulder” viewing is critical to improving the remote student’s “you are

there” experience.

Considerable further development is needed in applying the presence awareness technologies in

CSCLIP. In a STSP lab, it is easy for the students to locate the instructor who might be in a

particular room with a sub-group of lab students. In the CSCLIP environment, a capability is

needed to allow the remote students to easily find the instructor at any given moment. A context

27

mapping (CM) scheme and virtual presence registry (VPR) needs to be developed so that the

students and instructor know where everybody is, and when someone (student or instructor)

enters or leaves a room. Such context mapping and virtual presence registry would be akin to a

continuous updating of “buddy lists” in the instant messenger products. However, the instructor

entry/departure to different modules may be noted through wireless nodes. Wireless technology

may also be used in combination with wireless cameras so that the instructor can draw attention

of all students, local and remote, to explain something about a specific equipment or process in

more detail. Students at remote sites need to be able to feel that they are getting the same level of

attention and detail that local students do. Several technologies including blue tooth, wireless

video, presence-awareness systems, etc. may need to be integrated to make this work. The

Bluetooth system may consist of a radio unit, a link control unit, and a support unit for link

management and host terminal interface functions. The Bluetooth system provides a point-to-

point connection, or a point-to-multipoint connection, in which the channel is shared among

several units. Two or more units sharing the same channel form a piconet, in which one unit acts

as a master of the piconet, and others act as slaves. When the instructor enters a room, the

designated receiver in that corresponding piconet would capture the signal from the PDA

transmitter carried by the instructor. The instructor gets registered in the piconet of the room.

Figure 10 displays one architecture of the application. Though initial work on tackling these

issues is underway at OSU, much work remains to be accomplished.

28

Transmitter and powersupply (with belt clip)

Receiver

Audio/Video Receiver placed inthe instruction labInstructor's Headset with a

Camera, Microphone & Wireless transmitter

Wireless Link

Hub

Computer Computer Computer

On siteStudents

Internet

Laptop computer Computer Computer

RemoteStudents

CCD Camera

Microphone

LAN

InternetGateway

Audio/Videodata

Audio/VideoGateway

Audio/Video Encoderand Data Compressor

Figure 9. WI instructor AV headset network connections

The virtual lab under development for CSCLIP currently limits each lab group to one virtual

(remote) lab member. The technical requirements to enable multiple virtual lab members per lab

team, or lab teams completely composed of remote members who interact with local instructors

and equipment are mostly unknown. This is a huge area that is largely unexplored.

Lastly, we note that ongoing areas of IS and Networking research have direct bearing on the

experience of the remote students. Research to reduce voice and video delivery times, improve

the responsiveness of camera controls, enable reliable unicast and multicast data communications,

secure equipment from unauthorized access will all help reduce the isolation that may be felt by

remote students.

4.3 User Interfaces for Immersive Systems

CSCLIP systems use various technologies to enable the immersive presence of a user, including

virtual movement through a facility and real-time interaction with physical lab equipment and

other participants and the instructors. Improving the cumbersome transitions between these

technologies will provide remote students a richer experience of the target environments and,

simultaneously, eliminate their active management of various connections to the resources. An

29

overall interface for remote users needs to be developed and refined that uses various resources

from a single intuitive interface to provide content, context, and connectivity for the entire

learning experience. Objects within the lab will have a common set of attributes that can be

accessed (e.g., any piece of equipment will be able to be viewed, and manipulated.

Documentation for that unit will be available to the user through a common set of tools.

Some specific research issues that will require both technical development and empirical testing

include:

• Develop seamless transition mechanisms between virtual navigation and real-time audio/video interactivity

• Integrate live and VR components of lab objects in a database, creating a resource library for remote students’ immersion.

• Finalize and test an? overarching interface that is intuitive and simple, despite facilitating access to the required complex support systems.

• Analyze bandwidth and QoS requirements and specify minimum requirements for remote sites

V. CONCLUSIONS

Our integrated CSCLIP framework provides guidelines for system development, empirical

testing, and observational research that will aid in understanding distributed learning for lab

groups. The CSCLIP framework also establishes a foundation for future areas of study and

provides key constructs that when operationalized will add to this research stream. As Alavi and

Leidner [2] argue quite eloquently, MIS researchers have a unique opportunity to contribute to

theory-driven development of distributed learning which include our strengths in “applications of

information technologies to cognitive processes,… long tradition of research in the appropriate

structures and processes to enhance information system success, organizational level vision on

structures and processes to effectively implement (such) initiatives… knowledge of information

technologies to help determine appropriate instructional applications.” (2001). Certainly the

group support systems research stream is a key ingredient in distributed learning environments,

whether or not it includes psychomotor components. We assert that opportunities are even

greater in distributed learning environments that do incorporate psychomotor learning objectives,

because this area is just beginning to build momentum. Only recently has the bandwidth been

available to develop and deliver virtual lab coursework at a distance.

CSCLIP involves a complex combination of technology, people, and learning processes. Though

our own application is in the area of teaching a telecommunications lab course, similar

30

opportunities exist in many domains where considerable training and learning needs to be

imparted to a diverse group of users scattered widely. MIS researchers have a major role to play

both in developing such applications and in conducting empirical studies to generate the

evaluative evidence to guide the design, development, and implementation of such systems. We

have the opportunity to take a lead in this area. Otherwise, as Alavi and Leidner [2] warn: “In the

absence of intellectual leadership, profit motives may start to drive the field; as researchers and

educators, we may find ourselves on the sidelines of our own game.” The proposed CSCLIP

framework and research agenda serve as a foundation for MIS researchers and educators who

want to participate in developing theory for, designing, building, and evaluating next generation

CSCLIP eLearning systems.

31

VI. REFERENCES 1. Alavi, M. Computer-mediated Collaborative Learning: An Empirical Evaluation. MIS

Quarterly, 18, 2, (1994), 159-174. 2. Alavi, M., and Leidner, D. Research Commentary: Technology-Mediated Learning - A Call for

Greater Depth and Breadth of Research. Information Systems Research, 12, 1, (2001), 1-10.

3. Aniebona, M.C. Effective Distance Learning Methods as a Curriculum Delivery Tool in Diverse University Environments: The Case of Traditional vs. Historically Black Colleges and Universities. Communications of the Association for Information Systems, 4, October, (2000),

4. Baldwin, R.G. Technology's impact on faculty life and work, in Kay Herr Gillespie, ed., New Directions for Teaching and Learning, Vol. 76, San Francicso: Jossey-Bass, 1998, 7-22.

5. Bell, H.H. The Effectiveness of Distributed Mission Training. Communications of the ACM , 42, 9 September, (1999), 73-78.

6. Benbunan-Fich, R., and Hiltz, S.R. Correlates of Effectiveness of Learning Networks: The Effects of Course Level, Course Type, and Gender on Outcome, in Proceedings of

Thirty-Sixth Hawaii International Conference on System Sciences, Wikoloa Village, Kona, HI, 2002 IEEE Computer Society Press CDROM.

7. Biocca, F.; Harms, C.; and Burgoon, J.K. Criteria for a theory and measure of social presence. Paper under review for Presence, Online at http://www.mindlab.msu.edu/Biocca/papers/2002_scope_conditions_social_presence.pdf, (2002.),

8. Blake, E.; Casanueva, J.; and Nunez, D. Presence as a Means for Understanding User Behaviour in Virtual Environments. South African Computer Journal, 26, November, (2000), 247-251 Online at http://www.cs.uct.ac.za/Research/CVC/Publications/sacj00_1blake.pdf.

9. Blitzer, M.D., and Boudreaux, M.C. Using a computer to teach nursing. Nursing Forum, 8, 3, (1969), 234-254.

10. Bloom, B.S. Taxonomy of Educational Objectives: The Classification of Educational Goals, handbook 1: Cognitive Domain . New York, NY USA: McKay, 1956.

11. Brackett, J.W. Satellite-Based Distance Learning Using Digital Video and the Internet. IEEE Multimedia, 5, 3 (July-September), (1998), 72-76.

12. Brna, P., and Aspin, R., Collaboration in a Virtual World: Support for Conceptual Learning? online at: http://cbl.leeds.ac.uk/~paul/papers/hci-et97paper/hci-et.html:

13. Brna, P., and Aspin, R. Collaboration in a Virtual World: Support for Conceptual Learning? Education and Information Technologies, 3, 3-4 December, (1998), 247-259.

14. Bush, V. As We May Think. The Atlantic Monthly , 176, 1, (1945), 101-108. 15. Cafarella, R.S. Planning programs for adult learners. San Francisco, CA, USA: Jossey-Bass,

1994. 16. Carroll, J.M. The Nurnberg Funnel. Cambridge, MA, USA: MIT Press, 1990. 17. Carroll, L. Multimodal Integrated Team Training. Communications of the ACM , 42, 9

September, (1999), 69-71. 18. Chambers, J.A., and Sprecher, J.W. Computer-assisted instruction: current trends and critical

issues. Communications of the ACM , 23, 6 June, (1980), 332-342. 19. Chatman, J., and Flynn, F. The influence of demographic heterogeneity on the emergence and

consequences of cooperative norms in work teams. Academy of Management Journal, 44, 5 October, (2001), 956-976.

20. Cleveland, P.L., and Bailey, E.K. Organizing for distance education, in Proceedings of Twenty-seventh Annual Hawaii International Conference on System Sciences, Maui, Hawaii, 1994 IEEE Computer Society Press 134-141.

21. Coleman, D. Chapter 1: Groupware: Technology and Applications: an overview of groupware, in D. Coleman and R. Khanna, ed., Groupware: Technology and Applications, Prentice-Hall, 1995, 3-41.

22. Collins, A. Cognitive Apprenticeship and Instructional Technology., Technical Report No. 6899, BBN Labs Inc., (1988).

32

23. Coppola, N.; Hiltz, S.R.; and Rotter, N. Becoming a Virtual Professor: Pedagogical Roles and ALN, in Proceedings of Thirty-fifth Hawaii International Conference on System Sciences, Wailea, HI, USA, 2001 IEEE Computer Society Press CDROM.

24. Crane, P. Implementing Distributed Mission Training. Communications of the ACM , 42, 9 September, (1999), 91-94.

25. Daft, R.L., and Lengel, R.H. Organizational Information Requirements, Media Richness and Structural Design. Management Science, 32, 5 May, (1986), 554-571.

26. Daniel, J.I. Computer-Aided Instruction on the World Wide Web: The Third Generation. The Journal of Economic Education, 30, 2 (Spring), (1999), 163-170.

27. Daniel, J.I. Using the Web to Improve Computer-Aided Instruction in Economics. The Journal of Economic Education, 30, 3 (Summer), (1999), 225-241.

28. Davis, F. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly , 13, 3 September, (1989), 319-340.

29. Dede, C. The Evolution of Distance Education: Emerging Technologies and Distributed Learning. American Journal of Distance Education, 10, 2, (1996), 4-36.

30. DeLone, W.H., and McLean, E.R. Information Systems Success: The Quest for the Dependent Variable. Information Systems Research, 3, 1, (1992), 60-95.

31. DeSanctis, G., and Gallupe, R.B., . A foundation for the study of group decision support systems. Management Science, 33, 5 May, (1987), 589-609.

32. Dillenbourg, P.; Mendelsohn, P.; and Schneider, D. The distribution of pedagogical roles in a multi-agent learning environment, in R. Lewis and P Mendelsohn, ed., Lessons from Learning, Amsterdam, The Netherlands: North-Holland, 1994, 199-216.

33. Doise, W., and Mugny, G. The social development of the intellect. Oxford, UK: Pergamon Press., 1984.

34. Driscoll, M. Web-based training: Using technology to design adult learning experiences. San Francisco, CA, USA: Jossey-Bass, 1998.

35. Duarte, M., and Butz, B. An Intelligent Universal Virtual Laboratory (UVL), in Proceedings of IEEE 34th Southeastern

Symposium on System Theory, Huntsville, AL, USA, 2002 IEEE Computer Society Online at: http://www.temple.edu/imits/shock/Ed571ee5.pdf.

36. Dufner, D.; Park, Y.-T.; Kwon, O.; and Peng, Q. Asynchronous Team Support: Perceptions of the Group Problem Solving Process When Using a CyberCollaborator, in Proceedings of Thirty-Sixth Hawaii International Conference on System Sciences, Wikoloa Village, Kona, HI, 2002 IEEE Computer Society Press CDROM CD ROM.

37. Engelbart, D. A conceptual framework for the augmentation of man's intellect, in P. D. Howerton and D. C. Weeks, ed., Vistas in information handling, Washington D.C., USA: Spartan Books, 1963, 1-29.

38. Fisher, S.S. Environmental Media: Linking Virtual Environments to the Real World in Creative Digital Media: Its Impact on the New Century., in Proceedings of Keio University COE International Symposium, Tokyo, Japan, 2001 Keio University Press.

39. Gagné, R.; Briggs, L.; and Wager, W. Principles of Instructional Design. 4 ed., Fort Worth, TX, USA: Holt, Rinehart and Winston, Inc., 1992.

40. Galvin, T. 2002 Industry Report: Training magazine’s 21st annual comprehensive analysis of employer sponsored training in the United States, Training, No. October, (2002), 24-33.

41. Gandolfo, A. Brave new world? The challenge of technology to time-honored pedagogies and traditional structures, in K.H. Gillespie, ed., New Directions for Teaching and Learning, The Impact of Technology on Faculty Development, Life, and Work , Vol. 76 (December), San Francisco, CA USA: Jossey-Bass, 1998.

42. Gerrig, R.J. Experiencing narrative worlds. New Haven, CT: Yale University Press, 1993. 43. Gladieux, L., and Swail, W.S., The Virtual University and Educational Opportunity: Issues of

Equity and Access for the Next Generation. 1999, Policy Perspectives. Washington, DC: The College Board Online at http://www.collegeboard.org/policy/html/virtual.html:

44. Hardless, C.; Lundin, J.; and Nulden, U. Mandatory Participation in Asynchronous Learning Networks, in Proceedings of Thirty -fifth Hawaii International Conference on System Sciences, Wailea, Maui, HI, USA, 2001 IEEE Computer Society Press CDROM.

45. Harrow, A.J. A Taxonomy of the Psychomotor Domain . New York, NY: David McKay Company, Inc., 1972.

33

46. Hauenstein, A.D. A Conceptual Framework for Educational Objectives: A Holistic Approach to Traditional Taxonomies. Lanham, MD, USA: University Press of America, 1998.

47. Hiltz, S.R. Correlates of learning in a virtual classroom. International journal of man-machine studies, 39, 1, (1993), 71-98.

48. Hiltz, S.R. The virtual classroom. learning without limits via computer networks. Norwood, NJ: Ablex Publishing Corporation, 1994.

49. Hiltz, S.R., and Wellman, B. Asynchronous learning networks as a virtual classroom. Communications of the ACM, 40, 9 September, (1997), 44-49.

50. Hites, M.; Sekerak, M.; and L., S. Implementing and Evaluating Web-Based "Hands-On" Laboratories for Undergraduate Education, in Proceedings of ASEE IL/IN Sectional Conference, DeKalb, IL, 1999.

51. Hoffer, E.P.; Methewson, H.O.; Loughery, A.; and Barnett, G.O. Use of computer-aided instruction in graduate nursing education: a controlled trial. Journal of Emergency Nursing, 1, 2, (1975), 27-29.

52. Holzl, A. How do we Assess Graduate Attributes?, in Proceedings of Effective Teaching and Learning at University, Brisbane, Queensland Australia, 2000 ACE Group, Teaching and Educational Development Institute The University of Queensland Online at: http://www.tedi.uq.edu.au/conferences/teach_conference00/papers/holzl-2.html.

53. Johannsen, A.; van Diggelen, W.; de Vreede, G.J.; and Krcmar, H. Effects of Video Communication and Telepresence on Cooperative Telelearning Arrangements, in Proceedings of Thirty-Third Annual Hawai'i International Conference on System Sciences, Wialea, Maui, HI, USA, 2000 IEEE Computer Society Press CDROM.

54. Johansen, R. Computer Support for Business Teams. New York, NY, USA: The Free Press, 1988.

55. Johnson-Lenz, P., and Johnson-Lenz, T. Groupware: The emerging art of orchestrating collective intelligence., in Proceedings of First Global Conference on the Future, Toronto, Canada, 1980.

56. Kaye, A.R. Computer-supported collaborative learning in a multi-media distance education environment, in C. O'Malley, ed., Computer Supported collaborative learning, Berlin: Springer Verlag, 1995, 3-21.

57. Knowles, M., and Associates Andragogy in Action. San Francisco: Josey-Bass Publishers, 1984.

58. Koschmann, T.D.; Myers, A.C.; Feltovich, P.J.; and Barrows, H.S. Using technology to assist in realizing effective learning and instruction: A principled approach to the use of computers in collaborative learning. The Journal of the Learning Sciences, 3, 3, (1993/94), 227-264.

59. Krathwohl, D.R.; Benjamin S. Bloom; and Masia, B.B. Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook II: Affective Domain . New York, NY, USA: David McKay Co., Inc., 1964.

60. Lawless, N.; Allan, J.; and O'Dwyer, M. Face-to-face or distance training: two different approaches to motivate SMEs to learn. Education + Training, 42, 4/5, (2000), 308-317.

61. Lewis, L.; Snow, K.; Farris, E.; Levin, D.; and Greene, B. Distance Education at Postsecondary Education Institutions: 1997-98. NEA Higher Education Research Update, 6, 2 April, (2000), Online at: http://www.nea.org/he/heupdate/vol6no2.pdf.

62. Lombard, M., and Ditton, T. At the Heart of it all: The Concept of Telepresence. Journal of Computer Mediated Communication, 3, 2, (1997),

63. Makkonen, P. Do WWW-based Presentations Support Better (Constructivistic) Learning in the Basics of

Informatics?, in Proceedings of CSCL a Nordic perspective: Nordic Workshop on Computer Supported Collaborative Learning, Göteborg, Sweden, 199957-66.

64. Mason, R., and Kaye, A. Mindweave: Communication, Computers and Distance Education. Oxford, UK: Pergamon, 1989.

65. McLeod, P.; Lobel, S.A.; and Cox, T.H. Ethnic diversity and creativity in small groups. Small Group Research, 27, 2 May, (1996), 827-847.

66. Mingers, J. Combining IS Research Methods: Towards a Pluralist Methodology. Information Systems Research, 12, 3, (2001), 240-259.

67. Monnet, J., Virtual reality: The technology and its applications. 1995,

34

68. Morley, J. Methods for Assessing Learning in Distance Education Courses. Education at a Distance (ed), 13, 1 January, (2001), Online at http://www.usdla.org/html/journal/JAN00_Issue/Methods.htm.

69. Mulryan, C.M. Student passivity during cooperative small group in mathematics. International Journal of Educational Research, 15, 5, (1992), 261-273.

70. Neal, L. Virtual classrooms and communities, in Proceedings of International ACM SIGGROUP conference on Supporting group work: the integration challenge,, Phoenix, AZ, USA, 1997, 1997 ACM Press.

71. Newton, G., Introduction to Developing Online Electrical Course Ware. 1999, Electrician.COM Online at: http://www.electrician.com/articles/dlearn.htm:

72. Nunamaker, J.F., Jr.; Briggs, R.O.; Mittleman, D.D.; and Balthazard, P.B. Lessons from a dozen years of group support systems research: a discussion of lab and field findings. Journal of Management Information Systems, 13, 3 Winter, (1996-97), 163-207.

73. Nunamaker, J.F., Jr.; Dennis, A.R.; Valacich, J.S.; Vogel, D.R.; and George, J.F. Electronic meeting systems to support group work: theory and practice at Arizona. Communications of the ACM, 34, 7 July, (1991), 40-61.

74. Nunamaker, J.F., Jr.; Dennis, A.R.; Valacich, J.S.; Vogel, D.R.; and George, J.F. Group support systems research: experience from the lab and field, in L. M. Jessup and J. S. Valacich, ed., Group Support Systems: New Perspectives, New York, NY, USA: McMillan Publishing Company, 1993, 125-145.

75. O'Malley, C. Computer Supported collaborative learning. Berlin: Springer Verlag, 1995. 76. Orlikowski, W.J., and Baroudi, J.J. Studying Information Technology in Organizations:

Research Approaches and Assumptions. Information Systems Research, 2, 1-28, (1991), 77. Pate, J.; Martin, G.; Beaumont, P.; and McGoldrick, J. Company-based lifelong learning:

what's the pay-off for employers? Journal of European Industrial Training, 24, 2/3/4, (2000), 149-157.

78. Pfeffer, J. Organizations and organizational theory. Marshfield, MA: Pittman, 1982. 79. Piaget, J. The moral judgment of the child. Gencoe, IL: Free Press, 1932. 80. Piemme, T.E. Computer-assisted learning and evaluation in medicine. The Journal of the

American Medical Association, 260, 3 July, (1988), 367-373. 81. Poole, M.S., and DeSanctis, G. Understanding the use of group decision support systems:

The theory of adaptive structuration, in J. Fulk and C. Steinfie ld, ed., Organizations and communication technology, Newbury Park, CA, USA: Sage Publications, 1990, 173-193.

82. Price, M.A. Designing Video Classrooms. Instructional Technology, January, (1991), 15-19. 83. Ragusa, M., and Bochenek, G. Collaborative virtual design environments. Communications of

the ACM, 44, 12 (December), (2001), 40-43. 84. Roberts, J.M. The Story of Distance Education: A Practitioner's Perspective. Journal of the

American Society for Information Science, 47, 11, (1996), 811-816 Online at: http://polaris.umuc.edu/~skerby/omde_603/op_read_roberts.html.

85. Roussos, M.; Johson, A.; Moher, T.; Leigh, J.; Vasilakis, C.; and Barnes, C. Learning and Building Together in an Immersive Virtual World. Presence, 8, 3 June, (1999), 247-263.

86. Salomon, G., and Globerson, T. When teams do not function the way they ought to. International Journal of Educational Research, 13, 1, (1989), 89-98.

87. Seels, B., and Glasgow, Z. Exercises in instructional design. Columbus OH: Merrill Publishing Company, 1990.

88. Sherman, L.W. A Postmodern, constructivist and cooperative pedagogy for teaching educational psychology, assisted by computer mediated communications, in Proceedings of The first international conference on Computer support for collaborative learning, Bloomington, IN, USA, 1995 Lawrence Erlbaum Associates, Inc.

89. Sherron, G., and Boettcher, J., Distance Learning: The Shift to Interactivity, in CAUSE Professional Paper Series #17. 1997, Boulder, CO, USA. Online at: http://www.educause.edu/ir/library/pdf/PUB3017.pdf.

90. Short, J., Williams, E., and Christie, B. The Social Psychology of Telecommunications. London, UK: John Wiley & Sons, 1976.

91. Simpson, J.S. The classification of educational objectives, psychomotor domain, Office of Education Project No. 5-85-104, University of Illinois, (1966).

92. Singhal, S., and Zyda, M. Networked Virtual Environments Design and Implementation. New York, NY, USA: Addison-Wesley, 1999.

35

93. Slater, M., and S., W. A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments. Presence: Teleoperators and Virtual Environments, 6, 6 December, (1997), 603-616.

94. Slater, M., Sadagic, A., Usho, M., and Schroeder, R. Small Group Behaviour in a Virtual and Real Environment: A Comparative Study. Presence: Teleoperators and Virtual Environments, 9, 1 February, (2000), 37-51.

95. Slater, M., and Usoh, M. Representations Systems, Perceptual Position and Presence in Immersive Virtual Environments. Presence: Teleoperators and Virtual Environments, 2, 3, (1994), 221-233.

96. Stein, D. Situated Learning in Adult Education. ERIC Digest, 195, Online at: http://www.ericfacility.net/databases/ERIC_Digests/ed418250.html, (1998),

97. Stocks, J.T., and Freddolino, P.P. Enhancing Computer-Mediated Teaching Through Interactivity: The Second Iteration of a World Wide Web-Based Graduate Social Work Course. Research on Social Work Practice, 10, 4 July, (2000), 505-518.

98. Strehle, G.P., Distance Learning in America: How Institutions and Corporations are Stimulating Growth. 1998, Delivered to London conference: "The School for Life, the electronic future of higher education and life-long learning." Sponsored by the Ingenta institute: Online at: http://www-caes.mit.edu/headquarters/report-20000926.html

99. Tyran, C.K. GSS to Support Classroom Discussion: Opportunities and Pitfalls, in Proceedings of Thirtieth Annual Hawaii International Conference on Systems Science, Maui, HI, USA, 1997 IEEE Computer Society Press.

100. Tyran, C.K., and Shepherd, M. GSS and Learning Research: A Review and Assessment of the Early Studies, in Proceedings of Proceedings of the 31st Annual Hawaii International Conference on System Sciences, Maui, HI, USA, 1998 IEEE Computer Society Press.

101. Vygotsky, L.S. Mind in Society: The development of higher psychological processes. Cambridge, MA: Harvard University Press, 1978.

102. Webster, J., and Hackley, P. Teaching effectiveness in technology-mediated distance learning. Academy of Management Journal, 40, 6 December, (1997), 1282-1309.

103. Zigurs, I., and Kozar, K. A. An exploratory study of roles in computer-supported groups. Management Information Systems Quarterly, 18, 3, (1994), 277-297.