Remote Khamis AR 2003 Ps

©

Remote Interaction with Mobile Robots

ALAA M. KHAMIS, FRANCISCO J. RODRIGUEZ AND MIGUEL A. SALICHSDepartment of Systems Engineering and Automation, Carlos III University of Madrid,

Avda. Universidad, 30, 28911 Leganes, Madrid, [email protected]@[email protected]

Abstract. This paper describes an architecture, which can be used to build remote laboratories to interact remotelyvia Internet with mobile robots using different interaction devices. A supervisory control strategy has been usedto develop the remote laboratory in order to alleviate high communication data rates and system sensitivity tonetwork delays. The users interact with the remote system at a more abstract level using high level commands. Thelocal robot’s autonomy has been increased by encapsulating all the robot’s behaviors in different types of skills.User interfaces have been designed using visual proxy pattern to facilitate any future extension or code reuse. Thedeveloped remote laboratory has been integrated into an educational environment in the fiel of indoor mobilerobotics. This environment is currently being used as a part of an international project to develop a distributedlaboratory for autonomous and teleoperated systems (IECAT, 2003).

Keywords: mobile robotics, remote interaction, online robots, internet robotics, remote laboratories

1. Introduction

The new trends in robotics and automation systemshave helped to develop many robotic systems, whichaim to spread the use of the robot in human daily life.Most of these new trends have been made possible bythe evolution of the personal computer (in terms ofcost, power, and robustness) and the Internet (in termsof security, speed, and reliability) (Brugali and Fayad,2002). The Internet has become a major global tool forcommunication and information sharing. It providesa global, integrated communication infrastructure thatenables an easy implementation of distributed systems.Recently, considerable research efforts in the fiel ofmobile robotics are addressing the use of the Internetas a communication medium to facilitate remote in-teraction with mobile robots. Remote interaction is aspecial type of human-robot interaction, where the hu-man and the robot are separated by physical barriers butlinked via telematic technologies. Such type of inter-action can be used in many useful applications such as

remote experimentation, teleoperation, teleperception,teleprogramming, etc.Remote laboratories can be considered innovative

environments, which can be used to provide remoteinteraction with mobile robots for educational and re-search purposes. Remote laboratories can be defineas network-based laboratories where the user and thereal laboratory equipment are geographically separatedand where telecommunication technologies are usedto give users access to laboratory equipment (Khamiset al., 2003). Such laboratories have the advantage thatthey are not restricted to synchronized attendance byinstructors and students: thus they have the potential toprovide constant access whenever needed by students.In recent years, many researchers have built remote

laboratories for mobile robots using similar architec-tures and implementation tools (Archive, 2003). Manysuch remote facilities can be put together to form aframework or a distributed laboratory that can be usedto provide a coordinated set of experiments for stu-dents with hardware facilities physically spread over

1

different locations, but accessible via the Internet. Theproject IECAT (Innovative Educational Concepts forAutonomous and Teleoperated Systems) in which weare participating is an example of such frameworks inthe f eld of mechatronics (IECAT, 2003). Such frame-works aim to be an electronic workspace for distancecollaboration and experimentation in research or in an-other creative activity, to generate and deliver resultsusing distributed information and communication tech-nologies. They assist in the exchange of existing hard-ware resources and educational materials between thepartners.

In the designing of these distance laboratories forrobotic systems, a number of challenges must beaddressed, particularly the telematics infrastructurewhich gives access to experiments, as well as the userinterface which provides the necessary interactivitywith the remote hardware supporting the learning pro-cess of students through appropriate feedback. In thispaper, these considerations are discussed.

This paper is divided into seven sections. The f rstsection gives a brief description of the remote labora-tories. The advantages and the disadvantages of usingInternet as communication medium are presented inSection 2. Section 3 discusses the remote interactionwith a mobile robot with a special focus on the selectionof control strategy. The system architecture is describedin Section 4 followed in Section 5 by the implementedpedagogical scenarios, which incorporate the proposedremote laboratory in an educational environment formobile robotics. Section 6 presents user feedback andf nally the paper is concluded in Section 7.

2. Internet as Communication Medium

Although the Internet provides a cheap and readilyavailable communication medium for remote interac-tion systems, there are still many challenges that need tobe solved before successful real-world applications canbe achieved. These problems include restricted band-width, random time delay and data loss, which inf u-ence system performance. Internet performance as acommunication medium has nondeterministic charac-teristic, which depends mainly on the network load.The performance of a computer connection is basedon the speed and reliability with which data are trans-mitted over that connection. The speed and reliabilityof the entire Internet cannot be measured at present.Very little quantitative performance data is available,but a number of projects have been created recently

to analyze performance. Currently, the Internet Traf-f c Report (ITR, 2003) and Internet Weather Report(IWR, 2003) provide an approximate measure of In-ternet performance by measuring the performance ofconnections among a small group of monitored sitesdistributed throughout the world. Delay and packetlosses are monitored as measures of connection speedand reliability respectively. Other projects are tryingto develop metrics to analyze overall Internet delayand packet loss such as the Internet Performance Mea-surement and Analysis Project (IPMA, 2003) and theInternet Protocol Performance Metrics (IPPM, 2003).

Many researchers have studied the use of Internetas a communication medium from different points ofview. Brady and Tarn (1998) discuss Internet-based re-mote teleoperation including a theoretical foundationfor modeling communications delays. Han et al. (2001)have proposed a control architecture, which guaranteesthat a personal robot can avoid obstacles and reducepath error and time difference between a virtual robotat the remote site and a real robot at the local site.The proposed architecture is insensitive to the inherenttime delay where the personal robot is controlled usinga simulator provided at a remote site.

Others try to use the Quality-of-Service (QoS) modelto improve the eff ciency of Internet-based systems(Fung et al., 2002; Gillet and Salzman, 2002). On theInternet and in other networks, QoS is the idea thattransmission rates, error rates, and other characteristicscan be measured, improved, and, to some extent, guar-anteed in advance (Whatis, 2003). QoS is usually de-scribed by four parameters, time delay, bandwidth, jit-ter and packet loss. The implementation of QoS modelin today’s networks is still in process, thus giving theopportunity to propose new dynamic tuning parameters(Hirche, 2002). Internet-based systems can take advan-tage of the implementation of the QoS model not onlythrough the guaranteed end-to-end behaviour, but alsoby the control of network parameters through a perfor-mance criterion in order to adapt the performance tothe task requirements.

Other researchers study the Internet effects from userperspectives. A quantitative evaluation of operabilitythat depends on communication time delay in teleoper-ation and time perceptions of human operator presentedin Ando et al. (1999). The experimental results of thisstudy showed some tendency that from 0.6 s to 1 s indelay there was some change of operator’s operationtime. Other qualitative studies show that people seemto be able to compensate for (learn) small added delays,

2

but cannot learn large ones (>100 ms.) and the operatorperformance decreases signif cantly as the time delayincreases beyond 0.2 s (Rogers and Murphy, 2001).

A survey was conducted during the period from7/1/02 to 5/2/02 to evaluate the network performancebetween the University of Applied Sciences (FH-Weingarten) in Germany, and the University Carlos IIIin Madrid (UC3M), which are partners in the IECATproject. The results of the study have been used asguidelines for selecting the suitable control strategyfor the remote laboratory. Many tools can be used tomeasure the network performance such as ttcp (2003),pingER (2003) or hp netperf (2003). In this study theNetperf tool was used, which consists of two differentexecutables: the netperf client in a 200MHz LinuxMachine at FH-Weingarten, and the netperf serverinstalled in a 300 MHz PIII Linux Machine at UC3M.The test was re-run several times using different testdurations and it was being run two times per day (at11 AM and at 6 PM) and the average was calculatedto insure high accuracy.

2.1. Time Delay

TCP Connect/Request/Response Test has been usedto measure the round trip time between the two sites.This test mimics the http protocol used by most webbrowsers. Instead of simply measuring the performanceof request/response in the same connection, it estab-lishes a new connection for each request/response pair.The measured round trip time has been used to calcu-late the total time delay between the client and serverbased on the TCP analysis model described in Kuroseand Ross (2001). Figure 1 shows a comparison betweenthe local and remote host time delay, which is the time

Figure 1. Time delay variation in ms.

Figure 2. Average time delay measured from different sites.

elapsed from when the client (at Fh-Weingarten) ini-tiates a TCP connection until the client receives a re-quested 10 kB web page stored in UC3M server.

The results show that the Internet imposes serioustime delays, which are highly unpredictable and in-evitable, unlike traditional teleoperation systems wherea dedicated transmission link is used to guarantee thetime delay and the bandwidth.

Another study has been carried out to compare theresponse time from different hosts. Figure 2 shows theresults of the comparison between the average responsetime calculated during one week from FH-Weingarten,Germany as local host and four remote hosts, whichare UC3M, Spain, University of Aalborg, Denmark,Helsinki University of Technology, Finland and UtahState University, USA. The used model supposes thata 10 Kbytes html f le with one referenced object betransferred from the remote host to the local host.

The results show that the physical distances betweenthe hosts have not dominant effect on the time delay.The Internet time delay is caused by the queuing de-lay, the processing delay and the transmission delayin the switches and the propagation delay in the links.The queuing delay def nes the time; a packet waits inthe buffer of a switch for transmission onto the nextlink. Therefore its value varies with the network load.The propagation delay depends on the physical distancedue to the speed of light. With regard to all other delaysadding to the total communication delay, the queuingdelay represents the major portion. The queuing timedepends mainly on the network load, which causes thearbitrary feature of the delay. The Internet delay vari-ation is the most important problem to handle whenimplementing remote interaction systems. Dependingon the network load, the delay can vary from a few

3

milliseconds to hundreds. This delay affects the relia-bility of the remote operation. Beyond a certain delay,manual control of a vehicle may become highly errorprone or impractical (McGovern, 1990). A maximumtime delay of 1 s is usually taken as a reference ofoperability in remote control systems.

2.2. Bandwidth

Bandwidth has a general meaning of how much infor-mation can be carried in a given time period (usuallya second) over a wired or wireless communicationslink. In data transmission, throughput is the amount ofdata moved successfully from one place to another ina given time period (Whatis, 2003). To evaluate thebandwidth between UC3M and FH-Weingarten, theNetperf Stream test has been used. In TCP Stream Testnetperf client sends messages of selected size to thenetserver daemon which receives them. There is noprotocol beside the TCP/IP protocol but all messagessent are also considered as correctly received hencethe term throughput can be used for the bandwidth. Atimer makes the test stop and the throughput is calcu-lated in both sides by using the message size, numberof messages and elapsed time. The test was re-run sev-eral times using different test durations and the averagethroughput was calculated. Figure 3 shows the resultsof this test.

In Bapna et al. (1998) a teleoperation system for themobile robot Nomad has been presented, which givesus the opportunity to operate Nomad safely from thedistant control centers. Images and data from Nomadwere also immediately available on the Internet. Bapnaet al. have proved that the 1.4 Mbps bandwidth is suf-f cient enough to transport real imagery from the roverto a local control station and then to remote mission

Figure 3. Bandwidth in Mbps.

control sites. The obtained results (average = 1.61Mbps) can be considered satisfactory but it is neces-sary to take into account the nondeterministic featureof network performance.

In Internet-based systems, high-level commands arerecommended to be used in remote interaction withthe mobile robot because these commands require lessbandwidth. The idea -of overcoming communicationconstraints by communicating at more abstract leveland increasing the robot’s autonomy- is fundamental toremote control via constrained communications. Thelimitation in bandwidth also limits the refreshing rateof the video images, which prohibits experiments inwhich the scene might vary at high speeds. Otherwiseabrupt breaks and jumps in the visualization of theevents will occur. Virtual reality images of a renderedsimulation have much lower needs in throughput;therefore they pose a better alternative to real cameraimages in these kinds of situations (Khamis et al.,2002). If real time images must be provided fromremote sites, the system should react dynamically tochanging bandwidth and computational resources andonly transmit those pixels that are actually needed byusing intelligent techniques (intelligent fragmentation,intelligent frame rate, intelligent task rate and bruteforce compression) described in Sayers (1999). Inmulti-user systems, system resources have to be sharedand some types of interaction should be detrimental orprohibitive, particularly those that require a high anddedicated communication bandwidth.

2.3. Jitter

A crucial component of end-to-end delay is the ran-dom queuing delays in the network devices. Becauseof these varying delays within the network, the timeelapsed from generation of a packet at the source untilit is received can f uctuate from packet to packet. Thisphenomenon is called instantaneous variability or jitter.Jitter is the variance in one-way latency. The previousround trip measurements can be used to measure thejitter. Let the i-th measurement of the round trip time(RTT) be Ri , then the “jitter” is taken as being the In-ter Quartile Range (IQR) of the frequency distributionof R (Cottrell, 2003). From descriptive statistics of themeasured round trip time, jitter can be calculated asfollowing:

Jitter = Inter Quartile Range (IQR) = third quartile(Q3) − f rst quartile (Q1) = 1911 − 340 = 1571 ms.

4

This value represents high jitter, which can neg-atively affect the performance of data transmissionespecially in speech applications. Although speechtransmission does not need a high bandwidth, it is verysensitive to delay changes. Gateways can be used todelete jitter in such applications. Many mechanismshave been proposed to remove the jitter in some spe-cial applications such as voice applications or Internettelephony (Kurose and Ross, 2001).

2.4. Packet Loss

Package loss is caused by exceeding the network capac-ity causing a network device to drop a packet. Probablythe greatest concern of any Internet-based systems isthe nondeterministic system behaviour that would re-sult during packet loss or total dropout. One possibilityin order to prevent package loss is implemented in TCP,where, as soon as package loss is detected, a resend isrequested by the receiver. That results in higher latencyregarding TCP compared to UDP, so there is a trade-off between the parameters package loss ratio and timedelay (Hirche, 2002).

3. Remote Interaction

The remote interaction is a special type of human robotinteraction, where the human and the robot are sep-arated by physical barriers but linked via telematictechnologies. Such type of interaction can be used inmany useful applications such as remote experimen-tation, teleoperation, teleperception, teleprogramming,etc. However, regardless of the application, most of theremote interaction systems consist of the following ba-sic components as shown in Fig. 4.

Figure 4. Remote interaction.

• An operator interface, incorporating an interactiondevice that the operator uses to send control com-mands to the remote system. There are many typesof interaction devices such as PCs, mobile devicessuch as Personal Digital Assistants (PDA) and mo-bile phones, speech-based communication, joysticksand haptic interfaces.

• A mobile robot that performs the operator’s com-manded actions at the remote site.

• A communication scheme between sites. It is rec-ommended that the communication scheme have arobust signal communication link with an accept-able time delay; dedicated data links with suff cientthroughput; and an effective data loss-recovery ap-proach. As mentioned previously, although Inter-net is a cheap, readily accessible communicationmedium, its performance has nondeterministic char-acteristics.

• Feedback interfaces. Video transmission is com-monly used to provide visual feedback for the op-erator. Video transmission demands high bandwidthavailability. When this is not possible, computer-generated imagery supplies the operator with a vir-tual interface that combines low bandwidth sensorydata to form a realistic image. These virtual inter-faces are also useful to overlaying computer predic-tions or visual clues onto video images. Auditoryfeedback can also be used to enhance visual expe-rience and human robot interaction. Although audiotransmission requires low bandwidth, it is very sen-sitive to time delay and jitter. Other type of feedbackcan be provided using kinesthetic aids. Using hapticsystems the operator can sense the response of itscommands directly in the control interface.

The control strategy of the remote interaction systemplays an important role in the performance and stabil-ity of the system. When there is signif cant time delayin the communication link, instability occurs and man-ual closed-loop control is no longer suitable (Sheridan,1992). Two control strategies are discussed here, whichcan be used to develop Internet-based systems.

3.1. Manual Closed Loop Control

Most of the Internet-based systems use the manualclosed loop control model because of its simplicity.These systems tolerate delays by using the human op-erator as an intelligent link to the control system. Anerror in the position of the remote device is visible in the

5

Figure 5. Manual closed loop control model.

returned imagery, and the operator sends commands tothe remote device to compensate. This model has thecomponents shown in Fig. 5.

It is based on a simple protocol commonly used indistributed computation “The Request/Response Pro-tocol”. The client interacts with the system using anyWeb browser to make the request. Client requests aretranslated into HTTP requests, which are satisf ed bythe Web server. These requests are converted to high-level control requests that are received by the robotcontroller, which transmits them as low level control re-quests to execute the required task. Sensory feedback isrequired to give the user information about the remoterobot’s environment and the consequence of his/hercommands. While these systems are not certainly opti-mal, they are simple and they work. Predictive displayscan be used to improve this model. Because of the com-munications delay, immediate feedback cannot comefrom the remote site, and instead it must be generatedat the operator station (Sayers, 1999).

3.2. Supervisory Control

One technique that can improve operator performanceand avoid the instability problem is a shared or super-visory control scheme, where control of the robot isshared between a remote control loop and the local hu-man operator (Sheridan, 1992). The goal is not to makea robot that can realise the full task autonomously butto enable the robot to perform some simple subtasksthat the operator can sequence. By supervisory con-trol, the user can communicate with the remote systemat a more abstract level using high or task-level com-mands sent to a robot, which has an increased level ofautonomy in order to alleviate high communicationsdata rates. Limiting the remote interaction to high levelcommands helps to decrease the bandwidth needs andincreasing robot’s autonomy can help to decrease tele-operated task sensitivity to the delay. Figure 6 shows

Figure 6. Supervisory control.

the main components of the control model used to de-velop the remote laboratory. The following subsectionsdescribe the developed model in more details:

3.2.1. Robot Autonomy. To increase the local au-tonomy of the robot the concept of skill has beenused. A skill represents the robot’s ability to perform aparticular task. They are all built-in robot action andperception capacities (Alami et al., 1998). The AD(Automatic Deliberative) control architecture has beenproposed to develop different robotic skills (Barber andSalichs, 2001). This architecture has two main levels.The automatic level contains low-level control mod-ules, which act directly upon the actuators, as well asthe modules that collect data from the different sensorsof the system. The automatic skills have been classif edinto motor, perceptive and sensorimotor skills. MotorSkills are those skills which generate actions for therobot’s actuator such as direct control skill. PerceptiveSkills are those skills which interpret the informationobtained from sensors or other perceptive or sensor-motor skills. Sensorimotor Skills obtain as input theinformation provided by sensors or other perceptiveskills, and based on this information choose the mostadequate actions for the actuators.

The other level is deliberative, which contains mod-ules that require reasoning capacity. Those modules donot produce immediate responses. They need a longertime to process the information they work with. Thosemodules form the deliberative skills such as path plan-ner, the environment modeler and the task supervisor.Skills have been implemented in form of client-servermodules (Boada, 2002; Boada et al., 2002). As shownin Fig. 7, each module contains an active object, anevent manager object and data objects.

6

Figure 7. Skill structure.

Each skill is a distributed object that has an identity,interfaces and state. The active object of the skill hasits own thread of control and it is in charge of pro-cessing. The processing results are stored in the dataobjects. These objects contain different data structuresdepending on type of data stored but the interfaces aresimilar. During the processing, the active object cangenerate events. Events are sent to the event managerobject, which is in charge of notifying them to otherskills, which have registered on it. In order to commu-nicate among objects of the same module or differentmodules Common Object Request Broker Architecture(CORBA) is used. When a skill is activated, it connectsto data objects of other skills or to sensors’ servers asrequired by the skill. Then, it processes the receivedinput information, and f nally, it stores the output re-sults in its data objects. If the skill is sensorimotor, itcan connect to actuators’ servers in order to send themmovement commands. A skill can send a report aboutits state while it is active or when it is deactivated.For example, the skill called go-to-goal can inform onwhether the robot has achieved the goal or not. Whenthis skill is deactivated it might inform about the errorbetween the current robot position and the goal (Boadaet al., 2002).

3.2.2. User Intervention. In this model, the user has asupervisory role by which he can activate or deactivatethe robotic skill. The error detection and recovery isperhaps the most signif cant challenge to time-delayedtelerobotics. Errors can be handled by using a three-stage process: autonomous detection, shared diagnosis,and manual recovery. Errors are detected by using avisualization mean such as streaming video, graphicalmodels, sensory data or connection status panels. Thediagnosis task is shared by the user and the system. Inrecovering from the error, the user can telecollaborate

Table 1. Total response time in ms.

Host Min. Av. Max.

UC3M 131.89 138 148.65

FH-Weingarten 283.89 324.89 264.89

with a human at the remote site or can ask the necessaryprivileges to be able to telnet the remote servers toreboot them.

3.2.3. Response Time. By limiting the remote inter-action to high-level commands and by increasing thelocal autonomy of the robot using the concept of skills,the system sensitivity to the communication delay canbe decreased dramatically. The total response time,which is the time elapsed between sending the skillactivation command until the start of skill execution,has been obtained by measuring the time delay fromclient to middleware server and then from middlewareserver to the skill server by using software clocks. Thetotal response time is the sum of the two round tripdelays.

The results showed that the connection between themiddleware and the server (radio connection 1 Mbits/s)doesn’t impose signif cant delay. The average value(10.89 ms) can be considered as a constant round tripdelay between the middleware and the server. The ma-jor part of the delay was the client-middleware delaydue to the Internet. Table 1 shows the total responsetime.

The measured values (138 ms from UC3M and324.89 from Weingarten) represented acceptable val-ues for remote laboratories applications taking into ac-count that a maximum communication delay of 1 sec isusually taken as a reference of operability in teleopera-tion systems. These delays will be noticed by the usersbut they are accepted for an educative application asremote laboratory.

4. System Architecture

A remote laboratory for mobile robotics can be built us-ing two kinds of distributed systems models. The f rstchoice is to use a two-layer client/server structure, andthe second option includes a three-layer model that in-cludes a middleware. Two layer architectures are veryeasy to implement at f rst, but it is very diff cult tomaintain the application up to date with them. In thiscase, the client side is responsible for the data access,the business logic implementation, formatting the data,

7

Figure 8. System architecture.

the user interface and data input. Because all the detailsinvolved on the client side, programs are very diff cultto adapt to Internet, as they do not scale well. As shownin Fig. 8, the proposed architecture is three-layer archi-tecture based on the concept of supervisory control. Inthis architecture, the client layer is responsible for thepresentation of system interfaces, the middleware layerserves as controller to communicate the client with theremote robot servers and the server layer, which con-tains the abstraction model of the system representedby robot’ servers. The following subsections describein more details the architecture layers.

4.1. Client Layer

The visual-proxy pattern has been used to de-velop system interfaces (Houlb, 1999). This patternis in some ways a specialization of the Presenta-tion/Abstraction/Control (PAC) architecture (Rohbert

et al., 1996), which can be used to build user interfacesfor object-oriented systems and can guarantee high de-gree of extensibility and reusability of the softwarecomponents). The objective of this pattern is to sep-arate the generation of the user interface entirely fromthe abstraction layer objects to provide the reusabilityand extensibility facilitates (Houlb, 1999). The controlobject is passive with respect to message f ow. The mes-sages go directly from the visual proxy (presentationlayer) to the abstraction-layer object that manufacturedthe proxy. In the visual proxy architecture, the encapsu-lation is still intact in the sense that the implementationof the abstraction-layer object can change without theoutside world knowing about it.

As shown in Fig. 9, the ExperimentTool class im-plements UserInterface so it can produce visual prox-ies when asked. It asks the other SkillController andthe DynamicRepository classes for visual proxies assimple JComponents. This class contains a constructor

8

Figure 9. Client layer.

and implementation of all methods required by the in-terface but it is not a God-like class that controls ev-erything from above. It positions the solicited proxieswithin the panel but does absolutely nothing else withthem. This class is simply a passive vehicle for hold-ing visual proxies. The proxies communicate directlywith the abstraction-level objects that creates them andthese abstraction layer objects can communicate witheach other. If the state of an abstraction-level classchanges as a result of some user input, it sends a mes-sage to another of the abstraction-level classes, whichmay or may not choose to update its own user interface(its proxy) as a sequence.

ExperimentApplet instantiates the ExperimentToolsto form the experiment interface seen by the user viaany Web browser. SkillController class is used to acti-vate/deactivate the skill and it can be an automatic or adeliberative skill or a combination of different types ofcontrol classes in the same panel. The DynamicRepos-itory class provides information about a sensor’s statusand can be used to remotely acquire the sensory data.This class is implemented as a thread by implementingthe Runnable interface to provide updated sensory datain real time. This data may be odometry, which indi-cates the actual robot position and its translational androtational velocities or ultrasonic sensor data or lasersensor data.

SkillController may use the DynamicRepositoryclass to invoke sensory data, which may be neces-sary to complete the control task, such as in the caseof obstacle avoidance skill by using sensory data.Both SkillController and DynamicRepository classes

interact with the middleware using http protocol to sendthe user requests to the robot servers.

As shown in Fig. 8, different interface agents canbe developed for different interaction devices by cus-tomizing the proposed design according to the capabil-ities of the interaction device. For example, in the caseof mobile devices such as PDAs and Mobile Phones(MP), the same design can be used with some modi-f cations necessary to deal with specif c problems re-lated to the limitations of handheld devices (slow pro-cessors, limited memory/storage, and small displays)and wireless network (restricted bandwidth, high la-tency, low connection stability, and low predictableavailability).

4.2. Middleware Layer

In this layer, there is a PC linked internally to an Eth-ernet LAN and externally to the university’s Intranetwhich runs an Apache Web server, which hosts twoagents:

• User agent: contains two main servlets to man-age user access. The user authentication servlet col-lects and verif es the user registry. The authorizationservlet manages the authenticated user access andnotif es the user that the access to the robot agent isgranted or rejected based on round robin approach.When only a single user is connected, no restrictionson usage are given. However, if multiple users aresimultaneously connected, only one is given accessfor a specif ed amount of time. Once the time has ex-pired, the next user in line will be given opportunityto control the robot and the f rst user will be movedto the end of the waiting list. A chat room is pro-vided as a communication mechanism between thedifferent users. Also this agent holds other servletsnecessary for the teleeducation environment as sim-ulation servlet, system evaluation servlet, online testcorrector servlet, etc. These services do not requireconnection with the real robot and they are accessibleto all connected users.

• Robot agent: deals with the real robot when therobot is actually running. It provides the requiredinformation to process a certain skill selected bythe user or required in step of an experiment. Theagent contains two groups of java servlets, dynamicrepository servlets to invoke the sensory data froma resource agent in the server layer and skill con-troller servlets that send control commands to the

9

skill agent in the server layer. The communicationbetween these servlets and the remote robot serversis done via the Object Request Broker (ORB) ofthe Common Object Request Broker Architecture(CORBA) where the Java servlet acts as a client tothe robot server. The ORB provides the communi-cation via the unif ed interface language InterfaceDef nition Language (IDL) and based on the Inter-net Interoperable ORB protocol (IIOP). The decisionto use CORBA as the distributed object architectureof the remote laboratory is based on a qualitative andquantitative comparison between the two most com-monly used architectures, CORBA and RMI (Juricet al., 2000). This study concluded that CORBA issuitable for large scale or partially Web-enabled ap-plications where legacy support is needed and goodperformance under heavy client load is expected.Moreover, CORBA servers can be located at any In-ternet site. RMI, on the other hand, is suitable forsmall-scale fully Web-enabled applications wherelegacy support can be managed with custom buildor pre-built bridges, where ease of learning and easeof use is more critical than performance. Integrat-ing Java RMI with IIOP can provide another solu-tion that combines RMI easy programming featureswith CORBA interoperability (Schaaf and Maurer,2001).

4.3. Server Layer

This layer contains four main agents. Resource agentrepresents the servers of robot’s resources implementedin C++ such as base server (actuator server, odometryserver and sonar server), laser server, etc. This agentis also responsible for managing the users’ requests toavoid any data inconsistency. The skill agent representsskill server, which may be simple automatic skill ordeliberative skill server. The simple skills can be com-bined to form complex skill by using the sequenceragent (Rivero et al., 2003). The sequencer is respon-sible for deciding which skills have to be activated ineach moment to avoid simultaneous activation of skillswhich act upon the same actuator. The data base agentprovides access to a database, which contains plansof sequences. This layer also contains a commercialnetwork camera (Sony SNC-RZ30P) with embeddedstreaming video and pan tilt zoom control capabilitiesto provide real imagery transmission from the remoteenvironment to the local site of the operator.

5. Pedagogical Scenarios

Robotics education provides an ideal f eld for teleed-ucation systems because of its f exibility. Unlike tra-ditional f elds, robotics is still an emerging area. Rel-atively few programs exist at the graduate level, andeven fewer at the undergraduate level. The courses inexistence are still new and are open to rapid change andnew approaches. Different educational activities can becombined in order to develop an innovative teaching en-vironment in the f eld of indoors mobile robotics. Theseactivities are classif ed into instructional activities andconstructional activities. The instructional activities areused to map traditional teaching and learning activitiesand to solve traditional teaching constraints of curricu-lum, class size and limited resources. The construc-tional activities give the students the opportunity tophysically construct and implement the ideas derivedfrom the course. The following subsections describethese activities in more details.

5.1. Instructional Activities

The instructional activities of the course are imple-mented based on the Web-based education model. Inthe traditional education model, lectures and labora-tories are the ways commonly used in any educationsystem. This model of education is quite entrenchedand the f ow of knowledge is largely unidirectional,especially in large classes, with the exception of occa-sional questions and discussion. The amount of ques-tions and discussion is usually inversely proportionalwith the class size, resulting in large classes often be-coming the academic analog of watching an informa-tional video (Bourne et al., 1996). Many instructionalactivities such as following can be implemented basedon a web-based education model to map traditionalteaching and learning activities and to solve the tradi-tional teaching constraints of curriculum, class size andlimited resources.

5.1.1. Online Classes. Online lecture is used to mapthe traditional classroom. Bourne et al. proposed the or-ganization of online lecture material according to well-known taxonomies in education (Bourne et al., 1996).Barrett’s Taxonomy proposed that learning should bedivided into four categories: literal, inferential, ap-plicative and evaluative. Merrill’s Taxonomy uses aperformance or remember, use, or f nd (create). Thecontent is classif ed as fact, concept (classif cation),

10

procedure or principle. Many online classes have beendesigned based on these taxonomies to present thebasic knowledge of robots as a general issue and in-door mobile robotics as main subject. This knowledgeincludes introductory concepts, robots classif cations,mobile robot anatomy, control architectures of mobilerobots, sensors commonly used in mobile robots andmobile robots applications. The class also has beensupplemented by FAQ pages, a search engine, a digitallibrary and downloadable materials.

5.1.2. Remote Laboratory. An online laboratory ina f eld such as mobile robotics must have live perfor-mance characteristic, not just virtual reality or sim-ulation programs. The multi-layered architecture, de-scribed in the previous section, has been implementedin order to reach this goal. The knowledge is presentedto the user by using three types of tutoring tours. Thesetours are classif ed according to the level of guidance ofthe tutor into: fully guided, unguided, and guided tour.The fully guided tour is a demonstration tour, whichdemonstrates basic concepts without any interventionon the part of the student. The unguided tour or freetour does not determine any order and tasks at all. Thistour provides generic tools to the user and let him/hercustomize the experiment according to his/her needs.These generic tools include 2D model for the robotand the lab, odometry data panel, sonar data panel,laser data panel, motion controller and low-level tele-programming editor. The guided tour aims to presentdifferent specif ed courses with direct interaction be-tween the student and the tutor. The experimental is-sues of the courses are provided by using interfaces ofdifferent types of skills as shown in Fig. 10.

The following examples are some of the experimentsthat have been implemented and are presented throughthe guided tour.

Figure 10. Screenshot of skill interfaces.

• Direct Control. This experiment aims at familiariz-ing the user with the mobile robot motion control andpositioning by using different interaction elementssuch as PC or any MIDP (Mobile Information De-vice Prof le)-enabled device such as handheld PDAor cellular phones. The remote user can send directcontrol commands to move the robot B21 from RWIforward, backward or to turn it left or right. Usinga network camera, a 2D graphical model, and theodometry data (actual location with respect to theinitial point, translational and rotational robot veloc-ities), the remote user will be able to view the effectof the sent commands.

• Go-to-Goal Skill. By using this automatic skill, theuser can send the robot to a certain point in the lab. Toactivate this skill the user has to set the skill parame-ters as goal point, robot velocity and maximum error.The skill can provide information as to whether therobot has achieved the goal or not. When this skillis deactivated it might supply information about theerror between the current robot position and the goal.This skill also has been combined with a perceptiveskill called detect obstacle to avoid the obstacle dur-ing motion.

• Wall Following. By this skill, the robot can followcertain contour using sonar data. The distance be-tween the robot and the contour and the robot’s ve-locity are controlled by the user.

• Round Trip Skill. This skill is a complex skill formedby combining go-to-goal skill and orientation controlskill. It commands the robot to travel to a certainposition and then return to the initial point.

• Environment Modelling and Robot Localization. Theskill modelling skill can be used to construct an en-vironment map using sensory data. The constructedmap is then used to remotely estimate the robot’sposition by computing sets of poses which providea maximal-quality match between a set of currentsensor data and the map. The remote user can com-pare the result of the localization algorithm withthe odometry data to determine the error. Figure 11shows the experiment interface.

5.1.3. Assessment Tools. Methods of assessment areimportant aspect of every course. Using telematic-based model, it will be easy to construct systems thatautomatically correct and handle quizzes for the teach-ing staff and that reduce the time taken to carry outother forms of assessment. Evaluative tests can be usedto provide online assessments. A student f rst takes

11

Figure 11. Modelling & localization experiment.

an exam and then, based on the results of the exam,customized learning activities can be generated to re-inforce the areas in which the student scored poorly.Moreover practice tests can be designed to cover theexperimental issues as a simulation to traditional lab-oratories. Three types of tests are designed as onlineassessment tools. Online quizzes are used to evaluatethe student’s background in mobile robotics in general.Time restricted theoretical tests are used to evaluatethe student’s knowledge in the theoretical topics of thecourse. Time restricted experimental tests are designedto help the student to understand practical problems.

5.1.4. Communication Tools. An essential part ofany learning experience is communication. Twocommunication mechanisms (synchronous such astext/audio/video chat, Internet telephony and videocon-ference and asynchronous as E-mail, newsgroups andmailing lists) are in plans for implementation to providecommunication between students and teaching staff.

5.2. Constructional Activities

Constructionism is an active learning process in whichstudents construct things that are personally mean-ingful to themselves or others around them (Papertand Harel, 1991). Instead of being served informationin the traditional one-way setting, students developtheir own knowledge and understandings of a subjectthrough physical construction and implementation oftheir ideas. The construction activities give the studentsthe opportunity to physically construct and implement

Figure 12. Environment perception experiment.

the ideas derived from the course. They consist of thefollowing activities:

5.2.1. Off-Campus Activities. The off-campus con-structional activities are active learning activities asteleprogramming, telemonitoring, telediagnostic, tele-maintenance and teleperception, etc. . . , which can beincorporated into the remote laboratory.

• Teleperception. The objective of this experiment isthe perception of environment using multi-sensordata (sonar and laser). The experiment is divided intotwo parts: without robot motion and with robot mo-tion. The objective of the f rst part is to understandthe operation of sonar and laser sensors and to be fa-miliar with these readings. The second part aims atrecognizing the real environment using sensor data.Figure 12 shows the use of accumulative readings ofsonar and laser sensors in determining the obstaclezone remotely.

• Teleprogramming. A low level commands editor isnow under development to facilitate interacting withthe remote robot using low level commands or agroup of commands in the form of a programmingscript. The user will be able to teleprogram therobot or to send low level commands via Internetto the robot’s base server to perform many taskssuch as adjusting the robot control parameters, in-creasing/decreasing robot velocity, informing aboutbattery state, setting the Watch Dog timer, whichcan be used for safety or diagnostic purposes, etc.Also a graphic user interface used to def ne plans ofsequences and store them in a database is under de-velopment to develop complex skills by combiningsimple ones.

12

Table 2. Student feedback.

Agree Strongly agreeQuestion (%) (%)

The learning objectives were clear 43 57

The course was well presented 29 71

The lab was easily accessible 29 71

The corrected exercises contained 43 57helpful comments

The system was essential to the course 14 86

The system was reliable 43 57

The system was easy to use 29 71

5.2.2. On-Campus Activities. On-campus activitiessuch as the design and construction of new prototypesor reverse-engineering of existed systems can be usedto provide face-to-face interaction between the studentsand teaching staff and between the student and the ma-chine, and therefore increase student motivation anddecrease students’ feeling of isolation. These construc-tional activities also help to increase student creativ-ity and team work. Students are asked to build roboticprototypes using lab kits such as Lego or Fischertech-nik parts (including motors) as hardware components,the handy board (Motorola 88HC11 based or 16-bitSiemens 80C167) micro controller board for control,commonly used sensors and other components. A tele-operated rover also can be built and programmed bythe students.

6. User Feedback

The proposed teaching environment was used duringthe academic year 2001–2002 to update a postgraduatecourse on intelligent autonomous robots. Student feed-back was gathered using an online questionnaire. Thestudent responses were uniformly positive as to the useof the different proposed teaching activities speciallythe use of the remote laboratory. Most of the studentsfelt that the online experiments helped them to achievea deeper understanding of the subject material. Table 2shows examples of students’ feedback.

7. Conclusions

A three-tier architecture was proposed to build mobilerobotics remote laboratories to facilitate remote inter-action with mobile robots using different interaction

devices for educational and research purposes. The su-pervisory control strategy used to implement the sys-tem helps to alleviate high communications data ratesand to decrease system sensitivity to network delays.The visual proxy pattern used to build the user inter-faces of the remote laboratory provides f exible user in-terfaces with minimal coupling relationships betweensubsystems. The generation of the user interface is en-tirely separated from the abstraction layer object to pro-vide the reusability and extensibility facilitates. Basedon the described architecture, many interfaces havebeen implemented for simple automatic skills such asdirect control (using PC or PDA or Mobile Phone), Goto goal, orientation control and wall following skills.A sequencer has been used to combine simple skills toobtain complex skills such as go to point with obstacleavoidance and round trip skill. The developed remotelaboratory has been integrated in an educational envi-ronment for mobile robotics. As a further work, moreinterfaces for other interaction means such as speech-based interaction, joystick or haptic-based interactionwill be developed.

References

Alami, R., Chatila, R., Fleury, S., Ghallab, M., and Ingrand, F.1998. An architecture for autonomy. The International Journalof Robotics Research, 17(4):315–337.

Ando, N., Lee, J., and Hashimoto, H. 1999. A study on inf uence oftime delay in teleoperation. In Proceeding of the 1999 IEEE/ASMEInternacional Conference on Advanced Intelligent Mechatronics,Atlanta, USA, pp. 317–322.

Archive of Online and Internet Robots. 2003. Available in:http://ford.ieor.berkeley.edu/ir/

Bapna, D., Rollins, E., Foessel, A., and Whittaker, R. 1998. Antennapointing for high bandwidth communications from mobile robots.In Proceedings of the 1998 IEEE International Conference onRobotics & Automation, ICRA98, pp. 65–70, Leuven, Belgium,vol. 4, pp. 3468–3473.

Barber, R. and Salichs, M.A. 2001. A new human based archi-tecture for intelligent autonomous robots. In The Fourth IFACSymposium on Intelligent Autonomous Vehicles, Sapporo, Japan,pp. 85–90.

Boada, M. 2002. Control system for autonomous mobile robots basedon reactive skills. Ph.D. Thesis (In Spanish), Universidad CarlosIII de Madrid.

Boada, M., Barber, R., and Salichs, M. 2002. Visual approach skill fora mobile robot using learning and fusion of simple skills. Roboticsand Autonomous Systems, 38:157–170.

Bourne, J. et al. 1996. A model for on-line learning networks in engi-neering education. Journal of Engineering Education, 85(3):253–262.

Brady, K. and Tarn, T. 1998. Internet-based remote teleoperation.In Proceedings of the 1998 IEEE International Conference onRobotics & Automation, ICRA98, Leuven, Belgium, pp. 65–70.

13

Brugali, D. and Fayad, M. 2002. Distributed computing in roboticsand automation. IEEE Transactions on Robotics and Automation,18(4):409–420.

Cottrell, L., Matthews, W., and Logg, C. 2003. Tutorial on In-ternet Monitoring & PingER at SLAC. Available in: http://www.slac.stanford.edu/comp/net/wan-mon/tutorial.html#jitter

Fung, W., Xi, N., Lo, W., and Liu, Y. 2002. Improving eff ciencyof internet-based teleoperation using network QoS. In Proceed-ings of the 2002 IEEE International Conference on Robotics &Automation, ICRA02, Washington, DC, pp. 2707–2712.

Gillet, D. and Salzman, C. 2002. Real-time interaction over the Inter-net. In Proceedings, the 15th IFAC World Congress on AutomaticControl (b’02), Barcelona, Spain.

Han, K., Kim, S., Kin, Y., and Kim, J. 2001. Internet control archi-tecture for Internet-based personal robot. Journal of AutonomousRobots, 10:135–147.

Hireche, S. 2002. Control of teleoperation systems in QoS commu-nication networks. Thesis, Technische Universitat Berlin.

Houlb, A. 1999. Building User Interfaces for Object-OrientedSystems. JavaWorld. Available in: http://www.javaworld.com/javaworld/jw-07-1999/jw-07-toolbox.html

IECAT (Innovative Educational Concepts for Autonomousand Teleoperated System) Web Page. 2003. Available in:http://www.ars.fh-weingarten.de/iecat/index.html

IPMA, Internet Performance and Measurement Analysis Project.2003. Available in: http://nic.merit.edu/ipma/

IPPM, Internet Protocol Performance Metrics. 2003. Available in:http://www.advanced.org/ippm.html

ITR, Internet Traff c Report. 2003. Available in: http://www.internettraff creport.com/

IWR, Internet Weather Report. 2003. Available in: http://www.mids.org/weather

Juric, M., Rozman, I., and Hericko, M. 2000. Performance compar-ison of CORBA and RMI. Information and Software Technology,42:915–933.

Khamis, A., Perez Vernet, M., and Schilling, K. 2002. A remote ex-periment on motor control of mobile robots. In the 10th Mediter-ranean Conference on Control and Automation, MED 2002,Lisbon, Portugal.

Khamis, A., Rivero, D.M., Rodriguez, F., and Salichs, M. 2003.Pattern-based architecture for building mobile robotics remotelaboratories. In IEEE International Conference on Robotics andAutomation (ICRA’03), Taiwan.

Kurose, J. and Ross, K. 2001. Computer Networking. Addison-Wesley.

McGovern, D. 1990. Experiences and results in teleoperationof land vehicles. Report SAND87-0646, Sandia National Lab,Albuquerque, PA.

netperf Web Page. 2003. Available in: http://www.netperf.org/netperf/NetperfPage.html

Papert, S. and Harel, I. 1991. Constructionism. Cambridge, MA:MIT Press.

pingER Web Page. 2003. Available in: http://www-iepm.slac.stanford.edu/pinger/

Rivero, D.M., Khamis, A., Rodriguez, F.J., and Salichs, M.A. 2002.A patterns-oriented framework for the development of automaticand deliberative skills for mobile robots. In The 11th Interna-tional Conference on Advanced Robotics, ICAR03, University ofCoimbra, Portugal.

Rogers, E. and Murphy, R. 2001, Human-robot interaction. Final Re-port for DARPA/NSF Study on Human-Robot Interaction. Avail-able in: http://www.csc.calpoly.edu/∼erogers/HRI/HRI-report-f nal.html.

Rohbert, H., Sommerlad, P., and Stal, M. 1996. Pattern OrientedSoftware Architecture: A System of Patterns. John Wiley & Sons.

Sayers, C. 1999. Remote Control Robotics, 1st ed. Springer Verlag.Schaaf, M. and Maurer, F. 2001. Integrating Java and CORBA: A

programmer’s perspective. IEEE Internet Computing, 5:72–78.Sheridan, T. 1992. Telerobotics, Automation, and Human Supervi-

sory Control. Cambridge, USA: MIT Press.ttcp Web Page. 2003. Available in. http://ftp.arl.mil/∼mike/ttcp.htmlWhatis Web Page. 2003. Available in: http://whatis.techtarget.com/

Alaa M. Khamis received his B.Sc. degree with “Distinction withHonor’s Degree” in Electrical Engineering from Alexandria Univer-sity, Egypt, in 1993, the M.Sc. degree in Electrical Engineering fromSuez Canal University, Egypt in 1998, and is currently pursuing aPh.D. degree in Industrial Technology, at Carlos III University ofMadrid, Spain. From 1995 to 1999, he was a teaching assistant inthe Engineering Sciences Department at Faculty of Petroleum andMining Engineering, Suez Canal University. In 2001, he was se-lected as distinguished scholar in the DAAD-Programme “Interna-tional Quality Network” (IQN) to share in research projects related toremotely controlled experiments at Faculty of Applied Science, FHRavensburg-Weingarten, Germany. His present research is primarilyfocused on Internet robotics.

Francisco J. Rodrıguez is an Associate Professor of System Engi-neering and Automation at Carlos III University of Madrid, Spain.He received his Ph.D. degree in Artif cial Intelligence and Roboticsfrom the Polytechnic University of Madrid, Spain, in 1993. His cur-rent research interests are in robotics, online robots and intelligentsystems.

14

Miguel A. Salichs is a full professor and head of the Systems Engi-neering and Automation Department at the Carlos III University ofMadrid. He received Electrical Engineering and Ph.D. degrees fromPolytechnic University of Madrid. His research interests include cog-nitive architectures, personal robots and multimodal human-robotinteraction. He is a member of the Policy Committee of the Interna-tional Federation of Automatic Control (IFAC).

15