Providing an ’Interaction Interface' for networked robots using Squeak ITB844 Bachelor Project Presentation by Svein-Magnus Sørensen (n4607048) Supervised.

Providing an Providing an ’Interaction Interface' ’Interaction Interface' for networked robots for networked robots

using Squeakusing Squeak

ITB844 Bachelor Project PresentationITB844 Bachelor Project Presentation

by Svein-Magnus Sørensen (n4607048)by Svein-Magnus Sørensen (n4607048)

Supervised by Joaquin SitteSupervised by Joaquin Sitte

Objectives and GoalsObjectives and GoalsProject objectives:Project objectives: Create a generic robot interaction interface in Squeak, based Create a generic robot interaction interface in Squeak, based

on the Java-implementation by Erik Berglund. It should enable on the Java-implementation by Erik Berglund. It should enable the passing of messages to remote objects and execute them.the passing of messages to remote objects and execute them.

Use the Agora Distributed Object-architecture as a basis for Use the Agora Distributed Object-architecture as a basis for message passing to remote objects over any TCP/IP network.message passing to remote objects over any TCP/IP network.

Install and run the Squeak VM on the Koala PC/104 system.Install and run the Squeak VM on the Koala PC/104 system. Using the generic interface, implement and test a specific Using the generic interface, implement and test a specific

controller for the K-Team ’Koala’-robot.controller for the K-Team ’Koala’-robot.

Learning goals:Learning goals: Learn to use and become familiar with the Squeak-Learn to use and become familiar with the Squeak-

environment and Smalltalk for application development.environment and Smalltalk for application development. Become familiar with the K-Team ’Koala’-robot and its Become familiar with the K-Team ’Koala’-robot and its

extension, the PC/104 architecture running Linux. extension, the PC/104 architecture running Linux. Be able to perform and manage a medium sized programming Be able to perform and manage a medium sized programming

project for on time delivery.project for on time delivery.

SmalltalkSmalltalk

What is it?What is it? A programming language and development environment A programming language and development environment

developed by Alan Kay in the 1960s.developed by Alan Kay in the 1960s. The first pure object oriented environment created.The first pure object oriented environment created. A very simple but powerful language. Can usually do three times A very simple but powerful language. Can usually do three times

as much as conventional languages with the same amount of as much as conventional languages with the same amount of code, and is grammatically comparable to English.code, and is grammatically comparable to English.

What is it?What is it? A Smalltalk-80 implementation with all source code available for A Smalltalk-80 implementation with all source code available for

changing on the fly, where everything is written in Squeak itself, changing on the fly, where everything is written in Squeak itself, including the Virtual Machine!including the Virtual Machine!

SqueakSqueak

Interactive programming with a late-bound protocol shortens the Interactive programming with a late-bound protocol shortens the Edit-Compile-Test-Debug cycle considerably.Edit-Compile-Test-Debug cycle considerably.

Effective programming in a simple but powerful language that is Effective programming in a simple but powerful language that is fully object oriented.fully object oriented.

Highly portable. Ports are available for many different platforms, Highly portable. Ports are available for many different platforms, and can easily be ported to new platforms when needed.and can easily be ported to new platforms when needed.

The Virtual Machine gives completely bit-identical running of any The Virtual Machine gives completely bit-identical running of any Squeak applications on any available platform.Squeak applications on any available platform.

It is fast. Comparable in speed to programs written in C.It is fast. Comparable in speed to programs written in C. It is small. The image can be shrunk to just a few hundred It is small. The image can be shrunk to just a few hundred

kilobytes which allows it to run on small mobile devices.kilobytes which allows it to run on small mobile devices. Extendable. Supports C primitives to run drivers for almost any Extendable. Supports C primitives to run drivers for almost any

kind of hardware.kind of hardware. It is available for free with a liberal license.It is available for free with a liberal license.

Why Squeak ?Why Squeak ?

K-Team’s Koala-robotK-Team’s Koala-robot Is a powerful mid-sized robot designed for real-world applications. Is a powerful mid-sized robot designed for real-world applications.

It rides on 6 wheels for indoor all-terrain operation. It rides on 6 wheels for indoor all-terrain operation. Sports 16 infra-red and ambient light sensors arranged around the Sports 16 infra-red and ambient light sensors arranged around the

robot in all directions.robot in all directions. It is modular, and a range of other sensors and various utilities can It is modular, and a range of other sensors and various utilities can

be added to improve its functionality. be added to improve its functionality. Supports serial-communication over a wired or wireless RS232 link Supports serial-communication over a wired or wireless RS232 link

following aschii sequences of the SerCom-protocol.following aschii sequences of the SerCom-protocol.

Extensions to the Koala robot provided by Smart Devices Laboratory at QUT-FIT:

• A PC/104 computer running Linux• A optical pan-tilt turret-camera• Wireless network capabilities

PC/104 ArchitechturePC/104 Architechture A IEEE standard for PC compatible embedded systems.A IEEE standard for PC compatible embedded systems. Has a compact form factor and rugged connectors.Has a compact form factor and rugged connectors. Uses a unique self-stacking bus to eliminate the need for Uses a unique self-stacking bus to eliminate the need for

backplanes and card cages.backplanes and card cages. A relaxed bus drive gives very low power requirements.A relaxed bus drive gives very low power requirements. PC/104+ incorporates the PCI-bus into the PC/104 form factor.PC/104+ incorporates the PCI-bus into the PC/104 form factor.

The Koala-addon PC/104 from K-Team features:The Koala-addon PC/104 from K-Team features: Pentium MMX 266mhz CPUPentium MMX 266mhz CPU 64mb RAM64mb RAM Onboard video and Ethernet controllers.Onboard video and Ethernet controllers. Ultra Slim 20gb Hard-driveUltra Slim 20gb Hard-drive

AgoraAgora A highly modular and easy maintainable system for point-to-A highly modular and easy maintainable system for point-to-

point communication between microcomputers.point communication between microcomputers. Inspired by the Transputer, it is an extension of the same ideas Inspired by the Transputer, it is an extension of the same ideas

about modular systems for smart appliances.about modular systems for smart appliances. Provides transparent and dynamic usage of remote objects in a Provides transparent and dynamic usage of remote objects in a

consistent way over any architecture.consistent way over any architecture. An implementation in Squeak was created last year by Johannes An implementation in Squeak was created last year by Johannes

Jansson at QUT’s Smart Devices Laboratory.Jansson at QUT’s Smart Devices Laboratory.

How it all works togetherHow it all works together1.1. A robot and a client-computer must both be running a Squeak-VM with all A robot and a client-computer must both be running a Squeak-VM with all

the necessary Robot and Agora classes installed.the necessary Robot and Agora classes installed.2.2. An Agora-server must be started in the Squeak-VM on the robot to accept An Agora-server must be started in the Squeak-VM on the robot to accept

incoming connections from clients.incoming connections from clients.3.3. The client must instantiate and start a RemoteClient object on the local The client must instantiate and start a RemoteClient object on the local

Squeak-VM.Squeak-VM.4.4. The starting client tries to make an Agora-connection to the specified The starting client tries to make an Agora-connection to the specified

AgoraServer at the specified or default port (40000).AgoraServer at the specified or default port (40000).5.5. When a connection is established, the client tries to instantiate a When a connection is established, the client tries to instantiate a

RobotController on the remote VM through the Agora architecture, and a RobotController on the remote VM through the Agora architecture, and a local ProxyObject is also created.local ProxyObject is also created.

6.6. Now the client can transparently send messages to, and receive return-Now the client can transparently send messages to, and receive return-values from, the ProxyObject like with any other local object, while the values from, the ProxyObject like with any other local object, while the execution of the messages are taking place in the actual RobotController execution of the messages are taking place in the actual RobotController object on the Robot-Server.object on the Robot-Server.Any control-functions or sensor-values from the robot can now be Any control-functions or sensor-values from the robot can now be accessed directly from the ProxyObject in a uniform manner. accessed directly from the ProxyObject in a uniform manner. The values can then be processed on the client, lowering the CPU and The values can then be processed on the client, lowering the CPU and memory requirements of the robot itself. memory requirements of the robot itself.

TCP/IP Network TCP/IP Network

How it all works with the Koala:How it all works with the Koala:Instance of class ”KoalaRemoteClient” on client-computer

Understands messages from the generic control protocol.The control message in turn sends the relevant SerCom message.

AgoraORB ”RobotController” Passes messages transparently through a network

AgoraController on PC/104

Instance of class ”KoalaRobotController”Connects to the Koala processor through a serial-link and passes SerCom-commands and return-

values between the Koala and the AgoraController

SerCom command

SerCom return value

Koala processorUnderstands and replies to SerCom commands

Connects to AgoraController and creates a local ProxyObject

Creates instance of RobotController

K-Team Koala Robot system

Client system

The Robot-ServerThe Robot-ServerAgoraControllerAgoraController The primary component of the server functionality. Controls the networking The primary component of the server functionality. Controls the networking

functionality of the provided implementation of the Agora architecture.functionality of the provided implementation of the Agora architecture. When instantiated it uses the AgoraLinkManager to start an Agora-server When instantiated it uses the AgoraLinkManager to start an Agora-server

listening for TCP connections (default port is 40000).listening for TCP connections (default port is 40000). Closes the TCP-port and terminates when stopped.Closes the TCP-port and terminates when stopped.

RobotControllerRobotController Is an AgoraObject usually instantiated remotely by a connected client.Is an AgoraObject usually instantiated remotely by a connected client. Contains the sendCommand interface between the native robot-control Contains the sendCommand interface between the native robot-control

functions (eg. SerCom at the Koala) and Squeak-messages.functions (eg. SerCom at the Koala) and Squeak-messages.

KoalaRobotControllerKoalaRobotController Inherits the RobotControllers functionality of AgoraObjects.Inherits the RobotControllers functionality of AgoraObjects. Implements the sending of SerCom commands over a serial link between Implements the sending of SerCom commands over a serial link between

the PC/104 and the Koala-chip through the sendSerComCommand the PC/104 and the Koala-chip through the sendSerComCommand message, and returns the responses from the Koala-chip.message, and returns the responses from the Koala-chip.

UML DiagramUML DiagramRobot-Server package

AgoraControlleralm, port

Class: newClass: startUpClass: shutDowninitstartControllerstopController

RobotControllerClass: newinitstartControllerstopControllersendCommand: (string) ; boolean

KoalaRobotControllerserialPort, robotSerial

Class: newstartControllerstopControllersendCommand: (string) ; stringsendSerComCommand: (string) ; string

Object – Basic Squeak Class AgoraObject – by Johannes Jansson

In Squeak all methods (messages) are public and all attributes are private.Method Attributes in this font

Methods in this font (method attribute) ; return value

The Robot-ClientThe Robot-Client

RemoteClientRemoteClient The primary component of the client functionality.The primary component of the client functionality. When started it uses the AgoraLinkManager to connect to an existing Agora-When started it uses the AgoraLinkManager to connect to an existing Agora-

server running on a specified address (default port is 40000).server running on a specified address (default port is 40000). Instantiates a remote RobotController object on the server.Instantiates a remote RobotController object on the server. Interfaces a series of general commands for robot-control.Interfaces a series of general commands for robot-control. Destroys the RobotController and closes the TCP-port when stopped.Destroys the RobotController and closes the TCP-port when stopped.

SerComRemoteClientSerComRemoteClient Extends the functionality of the RemoteClient, using SerCom specific control Extends the functionality of the RemoteClient, using SerCom specific control

messages to implement many of its methods.messages to implement many of its methods. Understands separate control-messages for the entire SerCom protocol.Understands separate control-messages for the entire SerCom protocol. Implements some easy to use commands to control a compatible robot.Implements some easy to use commands to control a compatible robot.

KoalaRemoteClientKoalaRemoteClient Inherits all of the SerCom functionality from SerComRemoteClient.Inherits all of the SerCom functionality from SerComRemoteClient. Initializes itself with control-values specific for the KoalaInitializes itself with control-values specific for the Koala

UML DiagramUML DiagramRobot-Client package

SerComRemoteClientactualMotorPositions, stepsPrDegree,

stepsPrCentimeterhalt , move: (int) , turn: (int)moveStraight: (int) ; booleanmoveStraightCM: (int) ; booleansetSpeedControllerMaxSpeed:andAcceler

ation: (int, int) ; boolean

In addition implements each of the 18 defined commands of the SerCom-protocol as separate Squeakmessages.Also re-implements relevant Control-messages from class ’RemoteClient’ to use SerCom for their tasks.

RemoteClientalm, robot, cr, tolerance,

Class: newInit , stopClient , stop ; booleanstartClientTo: (string)startClientTo:atPort: (string, int) getStepperPositions : booleansetAndWaitForStepperPositionsLeft:

andRight: (int, int) ; booleanis:differentFrom: (OrderedCollection, OrderedCollection) ; booleansetPositionTolerance: (int)setSpeedControllerMaxSpeed:

andAcceleration: (int, int) ; booleanturnDegrees: (int) ; booleanzeroMotorCounter ; boolean...And many more for a total of 24 methods.

KoalaRemoteClientClass: newInit , startClientTo:atPort: (server, port)

Object – Basic Squeak Class

Changes to the Agora-classesChanges to the Agora-classes

The provided Agora software implementation by Johannes The provided Agora software implementation by Johannes Jansson did not work as required out of the box, so I had to Jansson did not work as required out of the box, so I had to make the following changes to the code for it to be useable:make the following changes to the code for it to be useable:

Modified the AgoraTCPProtocol and AgoraSocket to support Modified the AgoraTCPProtocol and AgoraSocket to support the changed Socket implementation in Squeak 3.6the changed Socket implementation in Squeak 3.6

Extended the AgoraLinkManager and AgoraTCPProtocol to Extended the AgoraLinkManager and AgoraTCPProtocol to allow for connections to custom ports and servers.allow for connections to custom ports and servers.

Corrected some small inconsistencies in the code.Corrected some small inconsistencies in the code.

AchivementsAchivements Created generic RobotServer and RobotClient Created generic RobotServer and RobotClient

interfaces in Squeak, extendable to fit many interfaces in Squeak, extendable to fit many kinds of embedded devices and robots.kinds of embedded devices and robots.

Made and is testing an implementation of Made and is testing an implementation of the new interfaces for a ’Koala’-robot made the new interfaces for a ’Koala’-robot made by K-Team, extended with an PC/104 system.by K-Team, extended with an PC/104 system.

Used the ”Agora Distributed Object Used the ”Agora Distributed Object architecture” as a base for transparent architecture” as a base for transparent message passing between the RobotServer message passing between the RobotServer and RobotClient over a TCP/IP network.and RobotClient over a TCP/IP network.

EvaluationEvaluation Had some difficulty adapting to the way of doing things in Had some difficulty adapting to the way of doing things in

Squeak when I was used to the language structure of C and Squeak when I was used to the language structure of C and Java.Java.

Development and debugging in Squeak was amazingly fast Development and debugging in Squeak was amazingly fast and easy once I got the hang of how things are done.and easy once I got the hang of how things are done.

Squeak is an excellent environment for embedded systems. Squeak is an excellent environment for embedded systems. Agora was a perfect base for interacting with robots in real-Agora was a perfect base for interacting with robots in real-

time.time.

Due to its development model, Squeak programs can only be Due to its development model, Squeak programs can only be guaranteed to run on the version it was developed on, as the guaranteed to run on the version it was developed on, as the implementation of base classes can be changed without implementation of base classes can be changed without warning.warning.

The socket primitives has given me troubles during testing.The socket primitives has given me troubles during testing.

Ideas for future workIdeas for future work

• Create a generic graphical interface for Create a generic graphical interface for the RemoteClient in Squeak.the RemoteClient in Squeak.

• Develop methods to handle sensor-input Develop methods to handle sensor-input and video feeds at a higher and more and video feeds at a higher and more intuitive level.intuitive level.

• Use several concurrent remote objects Use several concurrent remote objects on the Robot-Server to perform time-on the Robot-Server to perform time-critical tasks like collision detection and critical tasks like collision detection and autonomous navigation.autonomous navigation.

More informationMore information

Online References:Online References:

Squeak CentralSqueak Central www.squeak.orgwww.squeak.org

Smalltalk: The Pure Object Enviroment Smalltalk: The Pure Object Enviroment www.smalltalk.orgwww.smalltalk.org

K-Team CorporationK-Team Corporation www.k-team.comwww.k-team.com

PC/104 Consortium PC/104 Consortium www.pc104.orgwww.pc104.org

Books and papers:Books and papers:

The Agora Software Implementation, Johannes Jansson, The Agora Software Implementation, Johannes Jansson, 20032003

Squeak: Object Oriented Design (...), Mark J. Guzdial, Squeak: Object Oriented Design (...), Mark J. Guzdial, 20002000

The EndThe EndAny Questions ?Any Questions ?