MultiCross – A programming Library for multi-party Collaborative Applications Bachelor Thesis Silvan Troxler University of Fribourg 2011 Professor: Prof. Rolf Ingold Head of the DIVA Research Group Advisors: Dr. Denis Lalanne Dr. Bruno Dumas Matthias Schwaller
38
Embed
MultiCross – A programming Library for multi-party ... · A programming Library for multi-party Collaborative Applications Bachelor Thesis Silvan Troxler University of Fribourg
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
MultiCross –
A programming Library for multi-party
Collaborative Applications Bachelor Thesis
Silvan Troxler
University of Fribourg 2011
Professor: Prof. Rolf Ingold Head of the DIVA Research Group
Advisors: Dr. Denis Lalanne Dr. Bruno Dumas Matthias Schwaller
3 Development Process .............................................................................. 9 3.1 Client/server architecture .......................................................................................... 9
4.1.1 Class App ............................................................................................................ 17 4.1.2 Class AppPanel ................................................................................................... 17 4.1.3 Client-Class ......................................................................................................... 18 4.1.4 Thread-Classes ................................................................................................... 18
4.2 Package Server ........................................................................................................ 19 4.2.1 Class Server ........................................................................................................ 19 4.2.2 Class ClientThread .............................................................................................. 19 4.2.3 Class User ........................................................................................................... 20 4.2.4 Class AppInfo ...................................................................................................... 20
4.3 Extending the MultiCross Library ........................................................................... 21
2
4.3.1 Adding a new device type .................................................................................... 21 4.3.2 Adding a new event ............................................................................................. 21
5 Validation ................................................................................................ 22 5.1 Use case 1: „Ligretto“-Game .................................................................................. 22
5.1.1 Game Description ................................................................................................ 22 5.1.2 Creating a Server ................................................................................................ 22 5.1.3 Creating a Client .................................................................................................. 23 5.1.4 Adding behavior ................................................................................................... 25 5.1.5 Adding Images ..................................................................................................... 25
5.2 Use case 2: “DiffusionBarrier”-Game .................................................................... 26 5.2.1 Game description ................................................................................................. 27 5.2.2 Display technique ................................................................................................ 27 5.2.3 Simultaneous actions ........................................................................................... 27
5.3 Performance Test ..................................................................................................... 27 5.4 Validated Features ................................................................................................... 28
6 Discussion .............................................................................................. 30 6.1 Review Development Process ................................................................................ 30 6.2 Review MultiCross Library ...................................................................................... 30
3 Development Process In order to develop the library, two applications were programmed. Even if this required a lot
of work and most notably a lot of revision during the whole process, it was a good way to see,
which functions the library should provide and how it can be improved in order to be useful for
different applications.
The development of the library and the two applications can be divided in several parts:
1. Designing and implementing of a client/server architecture
2. Designing and implementing application 1 and the library
3. Designing and implementing application 2 and improving the library
In this chapter, the whole development approach with its major problems is going to be shown
and explained. The results of this process, thus the library and the applications themselves
are then presented in the following two chapters.
3.1 Client/server architecture
3.1.1 Design
For a collaborative application that can be executed on several computers in a network at the
same time, it is important to have a good client/server architecture.
There are several possibilities of how the server and the clients can communicate with each
other. All machines could be interconnected and therefore pass messages to any other client
or server.
Figure 3.1: Network with interconnected clients1
1 inspired by http://decode-abap.blogspot.com/
10
Another approach would be to have one central server and have all messages pass through
this server in order to get to another client. In this case, for every change a client wants to
share with another client, it has to send a message to the server and is not able to inform the
client directly. The server handles all incoming messages, executes the desired action and
then informs the other clients.
Figure 3.2: Network with central server1
For this project, the second approach was chosen with the following advantages:
• This implementation provides a better control of what the application is actually doing,
as all the messages have to pass through the server.
• Imagine the clients could communicate directly to each other. It could happen, that
two clients are sending messages at the same time, but the other clients in the net-
work are not going to receive the messages in the same order and therefore, they
would have different program states at the same time.
Additionally, for a multi-party application, we want to have more than one device per client.
Beside the “server” and the “client”, we therefore need to have a third entity “user”. One client
could have multiply users and handles all user interactions and communicates them to the
server if necessary.
To sum up, let’s have a look at the most important entities for the collaborative application:
1 http://decode-abap.blogspot.com/
11
Entity Description
Server The central unit which handles the whole application
Client One single machine
User One single Input Device, such as a hand on the DiamondTouch or a
Wiimote at the communication board. One Client can have more than
one User.
Table 3.1: Most important entities for client/server structure supporting multi-party interaction
3.1.2 Implementation Inspired from an existing open source code1 found on the Internet a solution, which includes
all the needs mentioned in the section before, was implemented.
This solution utilises the well-known observer pattern. The server is the observer, observing
all registered clients. In the java.util package, the two abstract classes Observer and Observ-
able are available and can be subclassed in order to use this design pattern.
The following figure shows the observer pattern and the adjusted implementation for the cli-
ent/server architecture used for this library.
Figure 3.3: Observer Pattern (top) and adjusted implementation (bottom)
Every time a Client connects to the Server, a new instance of the class ClientThread is creat-
ed. This is mandatory, since in Java a ServerSocket object can only receive a connection re-
quest but not handle any messages sent from the client. For each new connection, a sepa-
rate socket is created. Over this socket, messages can be passed from one end to the other.
In this case, the socket is forwarded to the clientThread which can then receive and send 1 http://www.developer.com/print.php/1356891
12
messages from and to one client. Since the clientThread is observed by the server, it can also
notify the server in order to send a message over the different clientThreads to all clients.
3.1.3 Messages
Thus, a clientThread is responsible to receive messages from the client. Also, it parses the
messages and decides how to respond or what to do.
All messages are composed the same way. The first (and sometimes the second) word is the
keyword with the help of which the clientThread decides what to do. All other words or num-
bers are additional information which might be needed. All the words and numbers are sepa-
rated by a semicolon.
The following table lists all possible messages between client and server and explains what
they stand for. A practical example is given afterwards.
Client to Server - Messages
IP;<IP-address>
This is always the first message sent from the client to the server. It is a registration on the
server, telling the server that a new client wants to join the application. The server will then
respond to this request by giving the client a specific client number.
USER;NEW; device_nr>
For each input device at a client’s machine, a user should be created. For this purpose, the
client sends a message to the server, informing the server about the device. The device
number can be chosen by the client itself, but has to be unique. Therefore, it is recommended
to use the client number multiplied by 10 and add 1 for each new device so that we have for
example the device number 21 and 22 on the client number 2.
POSITION;<device_nr>;<x>;<y>
This is the most basic message. It consists of the keyword “POSITION”, the device number
as well as the x and y position of the cursor. This message is used to indicate a change of the
coordinates of the cursor of a device.
Server to Client - Messages
RESP;<client_nr>
Whenever a new client wants and is allowed to join the application, the server respond with
this message. <client_nr> would then be replaced by the actual client number which is used
in messages afterwards. This message is only sent to the client, who wanted to register on
the server.
USER;NEW;<client_nr><device_nr>
Whenever a new user is created, the server responds to all its clients by sending this mes-
sage, in order to ensure that all clients are aware of the fact that a new user has joined the
application.
13
POSITION;<dev_nr>;<x>;<y>
After receiving a cursor position change from a client, the Server automatically sends this
message to all its clients in order to inform them about the change.
ERROR;<message>
If an error occurs while handling the received message, a error message can be passed to
one or all clients by using this message. After the keyword “Error” the error message is
passed as a string.
Here is an example of how a usual communication between the server and its clients could
look like. Suppose we have the server and two clients communicating with each other.
Figure 3.4: client/server communication
In this case, the first client registers itself on the server. Then the second client is doing the
same thing. Client 1 then tells the server, that there is a new user with device id 11. The serv-
er responds to all known clients and informs them that a new user at client 1 with device nr 11
was created. A moment later, the position of the user with device id 11 is changed and client
1 who is aware of that change, sends an adequate message to the server which again in-
forms all its clients about the position change by sending exactly the same message to all
clients.
3.2 Developing the Library The Library and the first application were developed in parallel. Therefore, the first step of
developing the library was actually to design the first application. After having dealt with all
the needed classes and functions, the application specific parts were separated from the
more general parts which could also be used for other applications. Also the client/server-
implementation was included in the library.
14
Since the Library as well as the Applications are explained in detail later, only the structure
and the important points in the development process are treated at this point.
3.2.1 Application Data and Logic
One of the major decisions was to determine whether the application is running on the server
or independently on each client and therefore, where to store the application data. Since for
the first application a shuffle-function is needed, it makes more sense to have run it on the
server side. Suppose each client would shuffle an array itself. Every client would then have a
different order in this array and synchronisation would not be possible. However, if the server
shuffles the array and then passes the order to all its clients, they will all have the same state
at the end. For this reason, the decision was made to run the application on the server, or at
least to let the server keep track of the most important things in order to ensure synchronisa-
tion.
In the shuffle example above, the server would shuffle the array and pass the order to all its
clients. However, whenever the clients need the next element from the array, they don’t have
to ask the server again, since all of them have exactly the same elements in the same order.
Thus, some data is stored both on the server and at each client, but the data on the server is
effective in case of an asynchronous state.
While the server is therefore in charge of the application data, the visualisation of the data is
the clients’ business.
With this approach, another great advantage arises. Since the server knows all about the ap-
plication, it can restore the application state at any time. If, for example, a client is joining the
application later, the server can send all the needed information in order to update this client
about the current application state.
3.2.2 Application specific messages
Of course, in addition to the messages used for the client/server architecture, the developer
should be able to define application specific messages. Since any message can be sent from
and to a client, one only has to be sure the client or the server can decode the message. For
this purpose, the parsing-function of the ClientThread as well of the Client class has to be
adjusted, adding the keywords to recognize the message type and assigning an action to per-
form.
3.2.3 Adding multiple devices support
The library should be able to handle different devices, such as a normal mouse, the Dia-
mondTouch Table as well as Wiimotes. For this purpose the Client class was sub classed for
each type of input device in order to add device specific functions. The client then creates a
separate thread that listens to all inputs on the device. The performed actions on a given in-
put (for example a click or a dragging), have to be specified in the run-method of the thread.
15
Figure 3.5: Simplified class diagram of the client package
3.3 Library Improvements While programming the second application, the strength and the weakness of the existing
library could easily be seen. Since the aim was that the library code has not to be changed to
write an application, but can easily be extended, all the changes, which must have been
made to the library code while implementing the second application, were the library’s weak
points and required a revision.
The following list shows the main problems and their solutions or rather adjustments made to
the library.
The application specific messages had to be added to the ClientThread class. An additional Parser class was created. Any subclass of this Parser class can now be set as
the parser of the application. Every time the server or the client receives a message, the
evaluate-method of this object is called.
To set the application specific parser, an instance of the subclass has to be passed as a con-
structor argument when instantiating a new client or accordingly assigned to the parameter
“parser” of the server object.
Duplicated code in the different client classes
A parent class was created for the different Client classes. This can avoid duplicated code on
the one hand and provide an easy way to integrate new devices on the other hand.
16
In order to have a convenient way to create one of the client’s subclasses, a ClientFactory
class was created.
Inflexible thread classes in order to extend the event based behaviours The thread classes were revised. For each device, the input is analysed by this thread class
and click, press, drag and release events are filtered. Whenever one of this event occurs, a
corresponding method is called. This method can also be changed when subclassing the
thread class. This approach gives more freedom to the developer and is hiding the device-
specific input treatment.
Thus, the programmer is free to add the same behaviour for all devices or to react differently
to the events depending on the device type.
Not enough freedom in using the library
At some points the programmer had to choose between using the library and to disclaim of
his freedom or to program it himself. Since this is not desirable some hooks were included
were the developer can add own behaviour to some functions, but still has all the library’s
advantages.
Extra-Features
To simplify the programmer’s life, a few extras were included in the library, such as an ex-
tended JFrame which provides a good starting point for the applications’ graphical output as
well as some useful functions and a standard interface for choosing the device type.
Also, a logging was included in order to simplify the debugging. All messages received over
the network communication are added in the log-file. For the server as well as for each client,
it can be determined whether the logging is turned on or off.
17
4 MultiCross Library The Library is divided into two packages, the client and the server package. While the classes
of the client package mainly serve the application’s client, the classes of the server class are
important for the server of the application. While only the main classes and functions are pre-
sented in this chapter, examples how to use the different classes are given in chapter 5 of this
report. Furthermore, the complete source code and the javadoc of either the library and the
two use cases can be found on the enclosed CD.
Figure 4.1: Simplified UML diagram of MultiCross Library
4.1 Package Client
4.1.1 Class App
Every application which makes use of the MultiCross library, has to be started by creating an
App object or an object of an App’s subclass. This Class provides some useful functions to
the developer such as opening the interface to choose a device type or to connect to the
server.
The JFrame class is extendend by this class. Therefore, a new instance automatically creates
a JFrame. The content pane of this window has to be of the type AppPanel (or a subclass
thereof), and can either be passed as an argument to the constructor or set afterwards by
calling the method setAppPanel(appPanel).
4.1.2 Class AppPanel
The AppPanel class extends a JLayeredPane of the java.swing package. The only mandatory
thing is to specify the size of the JLayeredPane by setting the variable dimension of type Di-
mension as well as to define a JPanel named cursorPanel as one Panel of the JLayered-
Pane, on which by default, the cursor is displayed afterwards. Apart from that, the developer
is absolutely free in creating his own layout and his own design.
18
4.1.3 Client-Class
The Client class describes a usual client of an application. Such a client is capable of con-
necting to, as well as communicating with, a server.
When instantiating a new Client Object, an application panel of type AppPanel, a parser of
type Parser and a Boolean, to indicate whether logging is on or off, has to be passed as ar-
guments to the constructor.
Here is a list of the most important and most often needed methods of the Client class.
connect(String hostName, int port)
is called to connect to the server. This method needs to be executed before messages can be
sent to the server. After calling this method, the client is listening to all messages comming
from the server.
sendmessage(String msg)
sends a message to the server if it is connected.
afterUserCreation(User u)
hook method to specify an action after a new user has been created.
Table 4.1: The most often needed methods of the class Client
The MultiCross library already includes three subclasses for the three different device types
“Standard Mouse”, “DiamondTouch Table” and “Wiimotes”. Each of this subclass includes
some methods especially needed for the corresponding device. Also, each Client class cre-
ates a new Thread object (see section 4.1.4) and starts it in order to listen to the input on the
specific device.
4.1.4 Thread-Classes
For each device, a separate thread class has to be written in order to listen to the device input
and filter the different events which can occur. The library already includes the thread classes
for the supported device mentioned in the section before. For all of these devices, the follow-
ing events inspired from the standard mouse events are handled:
• Click
• Press
• Release
• Drag
• Moved
If one of this event occurs, the standard actions are performed and the corresponding handle-
function is called. For example if a click event occurs, the position of the device is also updat-
ed automatically by sending an appropriate message to the server. Then, the method
handleMouseClick() is called which, by default, does not perform any action. However, this
19
method can be filled with actions when subclassing the class and thus can be adapted to re-
alise the programmer’s intentions.
4.2 Package Server
4.2.1 Class Server
As explained in section 3.1, the server class is responsible for the clients’ registrations and for
creating clientThreads to listen to this clients’ messages. There are only a few methods which
could be useful to use in your server’s Parser class.
hasUser(int device_id)
returns true if a User with the passed device_id exists. Returns false otherwise.
getUser(int device_id)
Returns an object of type User with the device_id passed as an argument. Before using this
method, the method hasUser(device_id) should have been called in order to get sure that the
user exists.
hasClient(InetAdress s)
returns true if a client with the InetAdress s exists, false otherwise.
getClient(InetAdress s)
Returns an object of type Client with the InetAdress s. Before using this method, the method
hasClient(InetAdress s) should be used in order to get sure that the client exists.
Table 4.2: The Server class' most useful methods
After instantiating the server, the start()-method has to be called. Before calling the start()-
method the server’s appInfo and parser should be set using the two function setAppIn-
fo(AppInfo info) and setParser(Parser parser).
4.2.2 Class ClientThread
For each client a clientThread instance is created on the server. One clientThread is always
listening to one particular client, receives the messages of this client and can send a mes-
sage to this client as well as to all clients registered on the server.
Whenever a message is received, the evaluateString()-method of the specified parser is
called automatically. In this method the field clientThread is a reference of the clientThread
which received the message. Over this variable, messages can be sent back to the client us-
ing the method sendToClient(String msg) or to all clients using the method sentToAll(Srting
msg).
20
4.2.3 Class User
The User class holds information about the users using the application. Usually no change
has to be made to this class. Nevertheless, you can find a brief overview of the most im-
portant methods below.
setVisible(Boolean visible)
This method sets the variable visible to the value passed as an argument. This variable can
then be used in your AppPanel class to specify whether the user’s cursor should be visible or
not.
getClientNr()
Returns the number of the client on which the user is working.
getDeviceNr()
Returns the number of the user’s device.
setColor(Color c)
The first eight users automatically get a color from the MultiCross library. The following user
will all get the color black. Using this method, the color can be set manually.
Table 4.3: the most important methods of the class User
4.2.4 Class AppInfo This class is an empty abstract class. It’s only advantage is to have a new type which can be
subclassed. For a new application, this class has to subclassed and can be filled with both
fields containing all the data needed by the application’s server and functions used to manipu-
late the data.
Figure 4.2: UML of the Server Package
21
4.3 Extending the MultiCross Library
4.3.1 Adding a new device type
To include a new device type, two classes have to be written. First of all a new Thread class
has to be created in order to listen to the new device’s input. This class has to implement the
interface Runnable and therefore provide a run-method. At least the events click, press, re-
lease, drag and depending on the device the event move should be catched and the corre-
sponding handle-functions have to be created.
Additionally a new subclass of the Client class has to be created, which is in charge of start-
ing a new thread with the before created Runnable object.
As well as the ClientFactory class, the chooseDevicePanel class must also be updated.
4.3.2 Adding a new event
Maybe a developer wants to add a new event such as entered or exited. In this case, all
thread classes have to be updated, specifying for each device type when the new event ap-
pears and what should happen by its occurrence. It is also recommended to create a new
handle function for each new event type.
22
5 Validation Even if the two applications and the library were developed in parallel meaning both the li-
brary and the application grew as an entity, the two applications are a good indication for the
library’s gain. The first use case, the Ligretto game, is in need of an efficient system since it
depends on fast actions. The second use case on the other hand validates the collaborative
capabilities of the library.
5.1 Use case 1: „Ligretto“-Game Use case 1 is the famous card game Ligretto, which was implemented using the MultiCross
library.
5.1.1 Game Description
The rules of the game are pretty easy. Every player has 40 cards, numbered from 1 to 10 in 4
different colors. The aim of the game is to get rid of all the cards as fast as possible. All play-
ers are playing simultaneously and therefore attention to the cards being played by others as
well as one's own cards is needed.
When the game starts, players simultaneously discard cards in the middle of the table, build-
ing colored piles in ascending numerical order according to color, and only starting new piles
with a '1' card.
Every player has three “open cards”, a “stack” of ten cards and the rest of the 40 cards re-
main in the player’s hand. While the cards on the stack are covered, the “open cards” are vis-
ible and can be played. If an “open card” is played, a card from the “stack” replaces it. As
soon as the stack of one player is empty, the round is finished. Whenever a player can’t play
any of his “open cards”, he can look at every third card from the card in his hand and always
play the card on the top of this pile. Every played card counts one point while each remaining
card on the “stack” at the end of the round counts minus two points for this player.
5.1.2 Creating a Server
At first, let us have a look in how one can use the MultiCross library to create a new server for
its own application. To do this, a new class has to be created which includes the main func-
tion and therefore is the starting point of the application.
For the Ligretto Application, this class is called LigrettoGameServer and was created as a
singleton class such that the main-function looks like that:
LigrettoGameServer g = LigrettoGameServer.getInstance();
The constructor of this class then does actually do the main task.