Run-time conformance checking of mobile and distributed systems using executable models

Run-Time Conformance Checking of Mobile andDistributed Systems Using Executable Models

Ahmad A. SaifanSchool of ComputingQueen’s University

Kingston, Ontario, [email protected]

Ernesto PosseSchool of ComputingQueen’s University

Kingston, Ontario, [email protected]

Juergen DingelSchool of ComputingQueen’s University

Kingston, Ontario, [email protected]

ABSTRACTThis paper describes an approach for conformance testingof mobile and distributed systems. The approach is basedon kiltera — a novel, high-level language supporting the de-scription and execution of models of concurrent, mobile, dis-tributed, and timed computation. In our approach, a kil-tera model of the system is constructed from a high-levelmodel which describes system behaviour using, e.g., a suit-able UML profile. Check points are identified in the imple-mentation under test (IUT) and the kiltera model and bothare instrumented appropriately at these check points. Dur-ing execution, relevant information flows from the IUT tothe kiltera model which signals any non-conformance de-tected. Unique features of our approach include the sup-port for mobility, distribution, time, dynamic creation anddeletion of agents, and distributed monitoring. We describethe approach and a prototype implementation using a run-ning example for illustration. Results of first, preliminaryexperiments are reported.

Categories and Subject DescriptorsD.2.5 [Software Engineering]: Testing and Debugging,Testing tools; I.6 [Computing Methodologies]: Simula-tion and Modeling

General TermsDesign, Experimentation, Verification, Languages

KeywordsSoftware testing, conformance testing, runtime monitoring,mobile agents, process algebra, π-calculus, Java, Aglets

1. INTRODUCTIONCode mobility refers to the “capability to reconfigure dy-

namically, at run-time, the binding between the softwarecomponents of the application and their physical location

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.Workshop on Parallel and Distributed Systems: Testing, Analysis, and De-bugging, (PADTAD - VII) , July 19-20, 2009Copyright 2009 ACM 978-1-60558-656-4/09/07 ...$10.00.

within a computer network” [5]. Mobility occurs naturallyin many distributed system applications such as telecommu-nications and electronic commerce. Moreover, mobility mayreduce bandwidth consumption and coupling and increaseflexibility [24]. The use of mobility has reached a certaindegree of maturity: Several different development platformsare available (e.g., Aglets [15, 2], Voyager, and Grasshopper)and comparative performance evaluations have been con-ducted [29]; agent-oriented software engineering (AOSE) hasproduced tool-supported development methodologies (e.g.,Prometheus [22]) and Tropos [4]), promising commercial ap-plications exist [21], and standardization is being consid-ered [23]. However, it seems that relatively little work hasbeen done to support quality assurance techniques such astesting and verification of mobile systems [7, 30].

In this paper, we present an approach for checking the con-formance of a mobile, distributed application with respect toan executable model at runtime. The approach is based ona novel high-level modeling language for mobile, distributed,and timed systems called kiltera [25]. Application of the ap-proach starts with the creation of a high-level model (HLM)of the system using, e.g., a UML profile; the HLM is assumedto capture the most relevant aspects of the system behavioursuch as descriptions of the movement of agents, their interac-tion with hosts and other agents, and any results computed.Next, the HLM is translated into a kiltera model (KM); kil-tera’s direct support for many relevant features (e.g., supportfor concurrency with (a)synchronous message passing, move-ment of processes, and time- and site-dependent behaviour)makes this translation relatively straight-forward; kiltera’ssimulation environment allows early analysis. The HLM isthen also used to identify suitable “check points” at whichconformance between the implementation under test (IUT)and the KM is to be checked; check points typically occurright before or after the sending or receipt of messages oragent movement; after these check points have been locatedin the IUT and the KM, both are instrumented at these checkpoints to allow relevant information to flow from the IUT tothe KM. During the last step of our approach, the IUT andthe KM are executed, both possibly in a distributed fashion.The KM will report any non-conformance that arises duringexecution.

Our work benefits from the fact that kiltera allows a suc-cinct, accessible expression of many features of mobile anddistributed systems. Moreover, model analysis is possibleusing kiltera’s simulation environment. Finally, our approach

does not assume a central monitoring component; instead,the KM can be arbitrarily distributed, just like the IUT,which helps reduce any performance penalty. Complete au-tomatic code generation from the KM seems possible, but isleft for future work. We have implemented the approach ina prototype and used it to conduct first, preliminary exper-iments with promising results.

This paper is structured as follows: Section 2 providesthe necessary background: it briefly describes a UML profileused to express the HLM, it gives an informal description ofkiltera, and sketches how to go from the HLM to a kilteramodel. Section 3 describes the approach using a runningexample for illustration. Section 4 discusses the results offirst, preliminary uses of our prototype. Related work isreviewed in Section 5. Section 6 concludes the paper.

2. BACKGROUNDIn this Section we will describe briefly our high-level mod-

elling formalism (Subsection 2.1), our executable modellingformalism (Subsection 2.2) and an outline of the transforma-tion of high-level models into executable models (Subsection2.3).

2.1 Mobile Agents in UMLIn order to describe distributed systems and mobile agents

we use a UML profile introduced in [14] as our high-levelmodelling language. This profile extends the UML with fournew types of Sequence Diagrams. Here we use only one ofthese types, called “Swimlaned Mobility Diagrams” (SMDsfor short). These diagrams are intended to represent agentlocation, agent creation and agent movement.

An SMD consists of one or more swimlanes representingnodes (a.k.a. hosts or locations). Each swimlane is visu-ally represented by a column labelled with the name of thenode. Within each swimlane there is a Sequence Diagramwith a life-line for each agent in that node. In addition tothe standard message arrows for Sequence Diagrams, SMDscan have two new types of arrows between agent life-lines:1) arrows that represent agent creation, labelled new, and2) arrows that represent agent movement between nodes, la-belled move. Message arrows between life-lines in differentswimlanes represent remote communication. Figure 1 showsa small example where an agent P1 located at a node A cre-ates, in the same node, a new agent P2 which subsequentlymigrates to a node B.

2.2 An executable modelling language: kiltera

In this Subsection we describe our executable modellingformalism, kiltera [25]. This language is used to describe thebehaviour of timed, concurrent, interacting processes whichmay be distributed over several sites. It provides operatorsto compose processes in parallel, to describe communicationvia events, or equivalently via message-passing over chan-nels, to limit the scope of events, to delay processes and toobserve the passage of time, as well as to move processes toremote sites.

The core of the language is a process algebra which wecall πklt, an extension of the π-calculus [18]. Unlike otherbasic process algebras kiltera provides some higher-level con-structs to facilitate development. In particular, we allow the

Figure 1: A simple Swimlaned Mobility Diagram.

use of complex expressions and data-structures in messages,and use pattern-matching as a mechanism to extract infor-mation from data.

The language has a formal semantics and a meta-theory(see [25]) which serve as the basis for formal analysis of mod-els. Furthermore it has been implemented, supporting bothuniprocessor and truly distributed simulation.

In the following we introduce informally a significant sub-set of the language and a brief description of the simulator.

OverviewA model or specification in kiltera consists of one or moremodules, the smallest “movable”processing unit. Each mod-ule has the syntax:

module A[x](y) : P or module A[x](y) : sites s in P

Here P ranges over process terms, defined below. We usex, xi, ... for port/channel/event names, and A, B, ... for pro-cess/module names, s, si for site names, and y, yi, ... for anyother variable name. The notation x denotes a list of namesor values x1, ..., xn. In the definition of a module, the namesx represent the interface of the module, this is, its ports,or equivalently, the events which it can use to communi-cate with other modules. The names y represent local statevariables and the (optional) s represents the names of sitesknown by this module. The process body P is a process termwhich describes the structure and behaviour of the module.

The syntax for process terms P is shown in Figure 21.Here E ranges over expressions, F ranges over patterns, op ∈{+,−, ∗, /, mod, and, or, not, <, >, =, <=, >=, ! =}, n rangesover floating point numbers, s ranges over strings, x rangesover variable names, and f ranges over function names, withfunction definitions having the form: function f(x) : E.

The process done simply terminates. The term“trigger xwith E” triggers an event x and associates this event withthe value of expression E. Alternatively, one can say that itsends the message E through channel x (a channel and an

1In the presentation of the syntax we use braces { and }to denote syntactic nesting for the par and seq operators,but in the actual implementation and the examples we useindentation-based nesting.

P ::= done

| trigger x with E| when β1 → P1 | · · · |βn → Pn

| event x in P| wait E → P| par {P1, ..., Pn}| process A[x](y) : P1 in P2

| A[x](E)| move A[x](y) to s| here s in P| dchannel x in P

β ::= x with F after yE ::= n | true | false | “s” | x | op E

| E1 op E2 | f(E1, ..., Em) | (E1, ..., Em)F ::= n | true | false | “s” | x

| (F1, ..., Fm)

Figure 2: kiltera syntax.

event are synonymous). The expression E is optional. Thisprocess performs communication by unicasting : if there aremultiple listeners, only one of them accepts the message,and the choice is non-deterministic. Channel mobility isachieved in the same way as in the π-calculus since even-t/channel names are expressions, and so they can be sentto other processes as messages. The process “when β1 →P1 | · · · |βn → Pn”is a listener, consisting of a list of alterna-tive input guarded processes βi → Pi. Each input guard βi isof the form“xi with Fi after yi”, where xi is an event/chan-nel name, Fi is a pattern, and yi is a variable (the suffixes“with F” and “after y” are optional). This process listensto all events (channels) xi, and when xi is triggered with avalue v that matches the pattern Fi, the corresponding pro-cess Pi is executed with yi bound to the amount of time thatthe listener waited, and the alternatives are discarded 2. Alistener process represents, thus, a process in a state withexternal choice. Pattern-matching of inputs means that theinput value must have the same “shape” as the pattern, andif successful, the free names in the pattern are bound to thecorresponding values of the input. For example, the value(3, true, 7) matches the pattern (3, x, y) with the resultingbinding {x 7→ true, y 7→ 7}. The scope of these bindingsis the corresponding Pi.

3 The process “event x in P”, alsowritten “channel x in P”, hides the names x from the envi-ronment, so that they are private to P . Alternatively, we cansay that it creates new events/channels x whose scope is P .The process “wait E → P” delays the execution of processP by an amount of time equal to the value of the expressionE. The process “par {P1, ..., Pn}” is the parallel composi-tion of P1, ..., Pn. The process “process A[x](y) : P1 in P2”declares a new process definition A with ports x, (optional)state variables y and body P1. The scope of this definition isthe process P2. The term “A[x′](E)” creates a new instanceof a process (or module) named A, whose definition is inthe current scope, where the ports x and variables y of thedefinitions are substituted in the body of A by the eventsor channels x′ and the values of E respectively. The pro-

2Note that to enable an input guard it is not enough forthe event to be triggered: the event’s value must match theguard’s pattern as well.3This is essentially the same as pattern-matching in func-tional languages like ML or Haskell.

cess “move A[x](y) to s” creates an instance of the processdefined by module A in site s. The process “here s in P”binds the name of the local site to s in P . Finally, the pro-cess “dchannel x in P” creates a channel to communicatewith modules on remote sites. Note that modules are essen-tially the same as process definitions, except that they donot have a sourounding lexical context and therefore are self-contained, since their only external references are its ports.This is why we only allow modules to be moved to othersites.

There are several derived process terms, such as sequen-tial composition, timeouts, conditionals, etc. Here we onlymention a few of them. The process “seq {P1, ..., Pn}” isthe sequential composition of P1, ..., Pn, this is, P1 mustterminate before beginning P2, and so on. The processterm “when β1 → P1 | · · · |βn → Pn timeout E → P” as-sociates a timeout with a listener. If after an amount oftime determined by the value of the expression E none ofthe events have been triggered, control passes to P . Theprocess “match E with F1 → P1| · · · |Fn → Pn” evaluates theexpression E and attempts to match it with each patternFi. If a pattern Fi matches then the corresponding processPi is executed. If more than one pattern matches the choiceis non-deterministic. The process “if E then P else Q” isshorthand for match E with true→ P |false→ Q. Finally,we also have terms of the form “let y = E in P” whichdefine local names in a process P .

SimulationOur implementation of kiltera supports both uniprocessorand truly distributed simulation. Both are based on eventscheduling. The basic idea is that each term in the languageis treated as a simulation event to be executed by the event-scheduler (and not to be confused with a communicationevent in the language itself). The result of a simulation is adetailed event trace.

The event-scheduler contains a queue of simulation events(terms) to be executed, but rather than store them all in asingle linear queue, we divide them into time-slots, i.e., se-quences of all simulation events to be executed at a given in-stant in time. Hence the global event queue is a time-orderedqueue of time-slots, each of which is a queue of terms. Ex-ecution proceeds by taking the first time-slot in the queue,and taking the first term in the time-slot and perform its ac-tion. Once the first time-slot becomes empty, the simulatorproceeds to the next time-slot.

Each action executed depends on the specific construct.The par construct for example, simply adds the subtermsto the current time-slot. The action for event creates anew communication event object in the heap. Interaction isdone by means of the observer design-pattern. The actionfor a when creates a listener for the appropriate events andregisters them with the corresponding event objects. Thetrigger construct notifies the communication event object,which then selects one of its listeners and executes the cor-responding continuation (while discarding other branches ofthe original listener). The delay construct wait simply addsthe term in the appropriate time-slot of the global queue.

Distributed simulation is achieved using the TimeWarpalgorithm [11]. Briefly, this is an optimistic simulation algo-

rithm, where multiple event-schedulers (such as the one de-scribed above) run on different sites, but rather than block-ing to wait for external messages, each simulator proceedsas fast as it can, and whenever an external message arrives,its time-stamp is compared with the local clock (the timeof the first time-slot). If the external message has a futuretime-stamp relative to the local time, it is simply scheduled.If it is in the past, then the simulator rolls back to a timebefore the message’s time. This algorithm is guaranteed toyield the same behavior of a single global event-scheduler.

In the case of kiltera whenever we move a module to a re-mote site we send a copy of the module to that site (the IPaddress for the site is specified in a configuration file). Thesite must be running a daemon which, whenever it receivesa module, it starts a new simulator locally which runs thenewly arrived module. Each simulator executes the Time-Warp algorithm, and has a “dchannel” manager to handleremote communications through the dchannels created lo-cally.

2.3 From high-level to executable modelsNow we sketch how to go from high-level models specified

as SMDs to executable models specified as kiltera models.First we discuss briefly how standard Sequence Diagramscan be represented in kiltera. Then we describe how SMDsare represented.

Emulating standard Sequence Diagrams in kiltera

In a normal Sequence Diagram we have several life-lines fordifferent active objects. Arrows between life-lines representmessage passing. In kiltera, active objects are processes: aprocess definition corresponds to the class of an active objectand object creation is achieved by process instantiation. Theports in a process definition determine the messages that aprocess can send or receive. The parallel composition opera-tor par is used to spawn parallel life-lines. Message-passingis achieved with trigger (to send a message) and when

to wait for a message. Communication in kiltera is asyn-chronous. Synchronous messages can be modelled by meansof an acknowledgment/response protocol. This is typicallydone as follows: the sender of a message creates a local “re-sponse” channel and sends this channel together with themessage, and then waits for the answer on this new localchannel. The receiver uses this private channel to send anacknowledgment, or the response to the message. The fol-lowing listing shows such an example.

1 process Sender [ x ] :2 channel re sponse in3 seq4 trigger x with ( ”message ” , re sponse )5 when re sponse with r e s u l t −>6 // Do something with r e s u l t7

8 process Rece iver [ x ] :9 when x with ( data , re sponse ) −>

10 // Do something with data11 trigger re sponse with answer12

13 in14 channel a in15 par16 Sender [ a ]17 Rece iver [ a ]

Emulating SMDs in kiltera

In kiltera, sites play the role of nodes or hosts. Within akiltera model, sites have symbolic names, introduced by thesites keyword. These symbolic names can be associatedwith actual IP addresses in a separate configuration file.Agents are represented by modules.

Before an agent is able to create agents in a remote siteor move to a remote site, it needs to know the target site.There are three ways in which an agent can know a site. Thefirst is if the site was given in its sites declaration. Thesecond is by using the here operator to know the name ofthe local site. The third is by receiving a site name sent byanother agent elsewhere. This is possible since site namesare considered first-class values and therefore they can betransmitted in messages through channels.

Modeling the creation and movement of agents can bedone in several ways, and it depends on how we assign re-sponsibilities of moving agents. This is, who initiates move-ment: should an agent tell another agent to move, or shouldan agent move itself. Also, there are different approaches totransfer the state of an agent. Furthermore, we can createan agent first locally and then move it, or we create it at theremote site. The latter is directly captured by the semanticsof the move construct: move A[x](y) to s creates a new in-stance of A in a (possibly remote) site s, linked through thechannels x and with initial state values given by y. Creat-ing a copy of an agent (module) locally can be achieved byhere s in move A[x](y) to s. This shows how new arrows ofSMDs can be directly modelled with the move operator. Soin this case, the process/module creating the agent is alsoresponsible for moving the agent to the target site.

How can we emulate an agent migrating on its own? Thesimplest way is by capturing all the necessary state, test ifit is already in the destination, and if not, move a copy ofitself there with the required initialization state, and stop.For example, agent P2 from Figure 1 could be emulated asfollows:

1 module P2 [ x ] ( s t a t e ) :2 s i tes A, B3 // Use/modify s t a t e y i e l d i n g new state4 here s in5 match s with6 A −>7 move P2 [ x ] ( new state ) to B8 | B −>9 // . . . do what needs to be done in B

3. DESCRIPTION OF THE APPROACHOur approach consists of the following four steps:

1. Construction of the high-level model (HLM) of the sys-tem using SMDs,

2. Translation of the HLM into a kiltera model (KM),

3. Instrumentation of the implementation under test (IUT)and KM using the HLM, and

4. Execution of the instrumented IUT and the instru-mented KM in parallel.

See Figure 3 for illustration. We will now describe eachof these steps in more detail using a running example.

Figure 3: Steps of our conformance testing approach

3.1 Step 1: Construction of the HLM

Our approach starts with describing the desired behaviourof the system using SMDs described in Section 2.1. Consider,for instance, an online shopping agent application. In thisapplication, an agent is searching for a specific item (e.g.,a camera) by traveling to different online shopping malls inorder to find the lowest price for this item and return theresult of the search to the original site. Figure 4 shows theSMD of a scenario in the shopping application. We begin bysetting up the scenario, creating two “Malls” and a client.The first one is Mall1 created at Mall1site with two shops(Shop1 and Shop2) and the second mall is Mall2 created atMall2site that has one shop (Shop3). The client createsan agent that is responsible for finding the lowest price. Inthis scenario, the agent is sent to Mall1 and then it goes toMall2. In each mall, the agent queries the mall’s informa-tion kiosk for a list of shops in the mall. Then, it asks eachshop for the price of the camera and updates the currentbest price, if necessary. After the agent has finished visitingall malls, it sends the best price and the corresponding shopback to the client.

In order to clarify the model, we divide the HLM presentedin Figure 4 into HLMs for the client, agent, malls and the“main” (i.e., a component that sets up the entire scenario).We will only show the HLMs for the main, the client, andthe agent. The HLM for the main is shown in Figure 5.The initialization consists of three steps: creating the client,creating the malls by specifying their sites (in this exam-ple we have two malls: Mall1 and Mall2 created in sitesMall1site and Mall2site respectively) with different shopsand by providing the addresses of these malls to the clientto start the search process.

Once the client gets all mall list from the main, it createsan agent at Mall1site, provides it with the addresses of theother malls, and waits for the result to be returned from the

Figure 4: The SMD of lowest price for camerasearcher scenario

agent. Figure 6 shows the HLM of the client.

After the agent has been created at Mall1site, it sends amessage to the Mall1 asking it for the shop list (i.e., the listof shops in this mall). After that, it starts sending messagesto the shops one by one and in a specific order asking andwaiting for their prices of the camera. Then, the agent movesto the second mall located at Mall2site and does the samething. Once the agent has queried all shops and visitedall malls, it sends a message to the client telling him theminimum price and in which shop and mall the cheapestcamera can be found. Figure 7 shows the HLM of the agent.

3.2 Step 2: Construction of KM from HLM

In the second step of our approach, the HLMs are trans-lated into kiltera models (KMs); kiltera’s direct support formany relevant features (e.g., support for concurrency with(a)synchronous message passing, movement of processes, time,and site-dependent behaviour) makes this translation rela-tively straight-forward. Figure 8 shows the Main KM of thecorresponding HLM presented in Figure 5. The three sitesare declared in line 2 and the channels to the client and themalls’s information kiosk are introduced in line 3. The state-ments in lines 5-8 are executed in parallel: create an instance

Figure 5: The SMD of the initialization process

Figure 6: The SMD of the client

of Client with one channel (to_cust) at site Home (line 5),create an instance of Mall1 with channel mall1info at siteMall1site (line 6), create an instance of Mall2 with chan-nel mall2info at site Mall2site (line 7), and send the mallsites and their channels to the client through the channelto_cust (line 8).

Figure 9 shows the Client KM of the corresponding HLMpresented in Figure 6. In this figure, after the Client hasreceived all mall addresses (line 2), it starts its process bycreating the Agent at site mall_site (line 5-6). The Agent isconnected to channels from_agent (to send back the result)and mall_info (the link to the information desk where theAgent is located). Variable mall_site is bound to the firstmall in the list of malls received from Main. Furthermore,the client sends the agent the remaining mall addresses rest(lines 2). Then, the Client waits for the result from theAgent (line 7). The result is the price, and a link to theshop where the agent found the minimum price of the cam-era.

Figure 10 shows the agent KM of the corresponding HLMpresented in Figure 7. The Agent consists of four subpro-cesses: GetShopList, QueryShops, EnterShop and GoToNextMall.When an agent arrives at a mall it executes the processGetShopList (line 48). This process sends a message to themall information booth asking for the shop list. When itreceives the shop list (line 8), it starts querying the shops(invoking process QueryShops in line 9) one by one search-ing for the camera and updating the current lowest price ifnecessary. This is done by invoking EnterShop (lines 24-37)which asks the shop if it has the camera or not; if this is thecase and the shop sends a price less than the best_price

that we have so far, then the new best price is updated;

Figure 7: The SMD of the agent

otherwise, the best_price is left unchanged (line 34). If theAgent did not get a response from the shop within some timet (the time the agent is allowed to stay in each shop), thenthe Agent continues with the next shop. After the Agent

exits a shop it inokes recursively QueryShops (line 22) withthe remainder of the shop list. Once the Agent has fin-ished the shop list (which means the shop list is empty inline 14), then it executes the GoToNextMall process whichchecks whether there are more malls to visit or not. If thereare more malls (line 43), the Agent travels to the next mallby creating an instance of Agent at that mall with an up-dated state (lines 44-45) and behaves in the same way. If theAgent has visited all malls (mall_list is empty, line 41), itsends the result back to the Client telling him the lowestprice and where the cheapest camera was found (line 42).

As we can see from example in Figure 10, the move state-ment sends an instance of the process Agent in the remotesite mallsite and any state information is provided as pa-rameters to the Agent. Furthermore, while the Agent istrying to find the best price of the product camera, it usesa timeout in the shop because if the shop does not respond(e.g., because it is down), then the Agent should not waitforever.

1 module Main :2 s i tes Home, Mal l1 s i t e , Ma l l 2 s i t e in3 dchannel to cus t , mal l1 in fo , ma l l 2 in f o in4 par5 move Cl i ent [ t o cu s t ] to Home6 move Mall1 [ ma l l 1 in f o ] to Mal l 1 s i t e7 move Mall2 [ ma l l 2 in f o ] to Mal l 2 s i t e8 trigger t o cu s t with [ ( Ma l l 1 s i t e , ma l l 1 in f o ) ,9 ( Mal l2 s i t e , ma l l 2 in f o ) ] .

Figure 8: The KM of the initialization process of theshopping example

1 module Cl i ent [ from main ] :2 when from main with [ ( ma l l s i t e , ma l l i n f o ) ; r e s t ] −>3 dchannel f rom agent in4 par5 move Agent [ from agent , ma l l i n f o ] ( r e s t , nu l l , nu l l )6 to ma l l s i t e7 when f rom agent with ( pr i ce , shop ) −>8 // pr in t where the Agent found the min pr ice

Figure 9: The KM of the client

3.3 Step 3: Instrumentation of IUT and KM us-ing HLM

After we have constructed the HLMs and translated theminto KMs of the shopping example, we start the instrumen-tation step. Before we talk about the instrumentation wehave to mention here that we use the Aglets platform [15, 2]in order to implement the online shopping application illus-trated in Figure 4. The Aglets Software Development Kit(ASDK) is a framework and environment for developing andrunning mobile agents. It is originally developed at the IBMTokyo Research Laboratory. The Aglets system is one ofthe most popular Java-based open source mobile agent sys-tems [16]. An Aglet is a Java agent able to autonomouslyand spontaneously move from one host to another. In theinstrumentation step, we instrument both the IUT (imple-mented in Aglets) and the KM to allow relevant informationto flow from the IUT to the KM. We instrument the IUTfirst because it is going to send information to the KM. Weuse HLM presented in Section 3.1 to start the instrumen-tation step. First we identify a collection of “check points”at which conformance between the (IUT) and the KM is tobe checked. Typically, check points involve the sending orreceipt of messages or agent movement. The numbers andlocations of checkpoints are determine by the constraints tobe enforced. More precisely, the users must enforce that theinformation required for enforcing the constraint is commu-nicated to the KM at the appropriate time. Figure 11 showsthe HLM with 5 check points numbered from 1 to 5 pre-sented as a circle.

Once we have decided on the checkpoints in the HLM welocate them in the IUT and the KM. Figure 12 shows thelocation of check point 5 in the IUT (move the agent to amall site) and Figure 13 shows the corresponding locationof check point 5 in the KM.

Next, at each check point in the IUT, we insert appro-priate instrumentation code which transmits relevant stateinformation to the KM via sockets through a process we call“Python connector”. In addition, at each check point in KM,

1module Agent [ to customer , to ma l l ]2 ( ma l l l i s t , b e s t p r i c e , bes t shop ) :3

4 process GetShopList [ ma l l i n f o ] :5 channel re sponse in6 par7 trigger mal l i n f o with ( ‘ ‘ get shop l i s t ’ ’ , r e sponse )8 when re sponse with s h o p l i s t −>9 QueryShops [ s h o p l i s t ] ( b e s t p r i c e , bes t shop )

10

11 process QueryShops [ s h o p l i s t ]12 ( b e s t p r i c e h e r e , b e s t shop he r e ) :13 match s h o p l i s t with14 [ ] −>15 GoToNextMall [ ] ( b e s t p r i c e h e r e , b e s t shop he r e )16 | [ shop ; r e s t ] −>17 event s h op v i s i t e d in18 EnterShop [ shop , s h op v i s i t e d ]19 ( b e s t p r i c e h e r e , b e s t shop he r e )20 when s h op v i s i t e d with21 ( new bes t pr i c e , new best shop ) −>22 QueryShops [ r e s t ] ( new bes t pr i c e , bes t shop )23

24 process EnterShop [ shop , s h op v i s i t e d ]25 ( b e s t p r i c e , b e s t shop he r e ) :26 channel re sponse in27 par28 trigger shop with ( ‘ ‘ camera ’ ’ , r e sponse )29 when re sponse with p r i c e −>30 i f p r i c e < b e s t p r i c e then31 trigger s h op v i s i t e d with ( pr i ce , shop )32 else33 trigger s h op v i s i t e d with34 ( b e s t p r i c e , b e s t shop he r e )35 timeout t −>36 trigger s h op v i s i t e d with37 ( b e s t p r i c e , b e s t shop he r e )38

39 process GoToNextMall [ ] ( b e s t p r i c e , bes t shop ) :40 match m a l l l i s t with41 [ ] −>42 trigger to customer with ( b e s t p r i c e , bes t shop )43 | [ ( ma l l s i t e , ma l l i n f o ) ; r e s t ] −>44 move Agent [ to customer , ma l l i n f o ]45 ( r e s t , b e s t p r i c e , bes t shop ) to ma l l s i t e46

47 in48 GetShopList [ t o ma l l ]

Figure 10: The Agent KM

we insert instrumentation code that receives state informa-tion from the IUT and compares it with the expected inter-nal information; if they are different, then non-conformanceis signalled; otherwise, the KM continues. The comparisonmay require the addition of a specific routine that implementthe constraint that is to be enforced at that checkpoint. Con-straints may restrict individual messages or entire sequences.For instance, suppose each shop in a mall is to be queriedexactly once. The instrumentation needs to ensure that theIUT sends the name of the shop about to be queried to theKM while the KM keeps track of the shops queried so far.

Figure 14 and Figure 15 show the instrumentation codeinserted in IUT and in KM respectively.

3.4 Step 4: Execution of IUT and KM

In the last step of our approach we execute the instru-mented IUT and the instrumented KM and check the con-formance between them. While the IUT is running, it sendsmessages possibly including state information to the KMthrough a “Python connector”. The Python connector isa Python script [1] that serves as a “bridge” between the

Figure 11: Check points locations in SMD

1 // Check point 52 dest = new URL ( de s t i n a t i on [ j ] ) ;3 di spatch ( dest ) ;

Figure 12: Location of check point 5 in the IUT

IUT with the KM. When the KM receives a message fromIUT through the Python connector, it compares it with itsexpected internal state. In the case of non-conformance,the KM stops its execution and outputs an error messagetogether with the trace of the execution. Figure 16 showsthe architecture of the monitoring infrastructure of our ap-proach.

4. EXPERIMENTATIONWe now describe some of the experiments we have con-

ducted.

4.1 Checking sample properties of the shop-ping application

To implement a particular conformance check in the KM,it is important that all necessary information is sent fromthe IUT. For instance, to check that an agent in the IUThas moved properly to a new node it suffices for the IUTto send a string in some format to the KM describing the

1 // Check point 52 move Agent [ to customer , ma l l i n f o ]3 ( r e s t , b e s t p r i c e , bes t shop ) to ma l l s i t e

Figure 13: Location of check point 5 in the KM

1 //Check point 52 // Inser ted code begin3 Socket s = new Socket ( ‘ ‘ l o c a l h o s t ’ ’ , 60002 ) ;4 PrintStream s t r= new PrintStream ( s . getOutputStream ( ) ) ;5 s t r . p r i n t l n ( ‘ ‘ Agent Traveled to Next Mall ’ ’ ) ;6 // inser t ed code end7 dest = new URL ( de s t i n a t i on [ j ] ) ;8 di spatch ( dest ) ;

Figure 14: Code inserted into IUT at check point 5

movement. However, to check more complex behavioral con-straints (e.g., that a sequence of actions has been performedin the right order), the KM may have to maintain and queryadditional state information. Finally, to check that timingconstraints have been met, timing information needs to becommunicated and timeouts may have to be used. Exam-ples for each of these kinds of conformance check will nowbe given. At the end of the section, another experiment willbe discussed briefly.

Property 1: Agent visits next mall at expected time.At the checkpoint shown in Figure 15, the KM expects a

message from the IUT containing the string "Agent trav-

eled to next mall" (lines 3-5). If such string is received,the KM continues its execution by creating an instance ofAgent and moves it to the next mall site mall_site (line 6-7). If any other kind of message is received (because, forinstance, the number of malls or the number of shops in amall in the IUT and the KM do not match, or the IUT didnot query all shops in a mall) the behavior of the IUT isnon-conformant (line 9). In that case, the KM outputs anerror message and terminates.

Property 2: Agent queries each shop in a mall exactlyonce.

Suppose each shop in a mall is to be queried exactly once.The instrumentation needs to ensure that the IUT sends thename of the shop about to be queried to the KM, whilethe KM keeps track of the shops queried so far. In Fig-ure 17, after receiving a message in line 14, the KM checkswhether the IUT has already queried this shop or not bychecking the list Jshoplist (using the process Search inline 17). If the shop name received from the IUT (Jshopname)is already contained in Jshoplist (Search sends a message‘‘Found’’ to the listener in line 18), non-conformance is sig-nalled (line 20); otherwise, the shop is added to Jshoplist

(line 22) and execution continues.

Property 3: Timing constraints.Figure 18 shows the KM that is used to check time con-

formance. More precisely, we check that shops respond toqueries in a timely fashion (within t time units) and that

1 //Check point 52 // Inser ted code begin3 HandleJavaMessages [ ex t e rna l ]4 when ex t e rna l with data−>5 i f data = ‘ ‘ Agent Traveled to Next Mall ’ ’ then6 move Agent [ to customer , ma l l i n f o ]7 ( r e s t , b e s t p r i c e , bes t shop ) to ma l l s i t e .8 else9 //Error , stop execut ion with non−conformance

10 // inser t ed code end

Figure 15: Code inserted into KM at check point 5

Figure 16: The architecture of the monitoring in-frastructure

the agent reacts appropriately to delayed shop responses.The instrumentation needs to ensure that the agent in theIUT informs the KM how long it had to wait in responseto a query. The agent in the KM will receive this time inrsponsetime in line 5. If rsponsetime is greater than t

(line 5), non-conformance is reported. Also, if the KM agentdoes not receive a response from the shop in the KM within t

time units (line 13), but a proper response has been receivedfrom the shop in the IUT(line 20), we have non-conformance.

Table 1 compares the size of the IUT and the KM beforeand after the instrumentation.

File Name IUT KMBefore Inst. After Inst. Before Inst. After Inst.

Main 26 26 8 8Client 57 60 7 20Agent 212 262 44 151

Mall1 with 4 shops 272 272 23 23Mall2 with 2 shops 152 152 21 21

Table 1: Size in lines of code of the IUT and KMbefore and after the instrumentation

We make the following observations:

1. The instrumentation increases the size of the agent themost. This is because the agent carries out most of thebehavior and contains the most check points. Instru-

1 process Search [ answer ] ( l i s t , item ) :2 match l i s t with3 [ ]−>4 trigger answer with ‘ ‘ Not Found ’ ’ .5 | [ hd ; r e s t ]−>6 i f hd=item then7 trigger answer with ‘ ‘ Found ’ ’ .8 else9 Search [ answer ] ( r e s t , item )

10

11 process loop [ Sh op l i s t ] ( J shop l i s t , . . . )12 . . .13 HandleJavaMessages [ ex t e rna l ]14 when ex t e rna l with Jshopname−>15 event checkitem in16 par17 Search [ checkitem ] ( J shop l i s t , Jshopname )18 when checkitem with data−>19 i f data= ‘ ‘Found ’ ’ then20 // Error shop v i s i t e d twice21 else22 l et J s h op l i s t=enqueue ( J shop l i s t , Jshopname )23 in24 . . .

Figure 17: Checking Property 2 in KM

1 trigger shop with ( ‘ ‘ camera ’ ’ , r e sponse ) .2when re sponse with ( shoppr ice , . . . ) −>3 par4 HandleJavaMessages [ ex t e rna l ]5 when ex t e rna l with rsponset ime−>6 i f ( f l o a t ( rsponset ime)<=t ) then7 i f shoppr i ce < min pr i ce then8 // New min price , v i s i t next shop with t h i s pr ice9 else

10 // check next shop with the current min pr ice11 else12 // Time non−conformance , stop execut ing13 timeout t −>14 par15 HandleJavaMessages [ ex t e rna l ]16 when ex t e rna l with data−>17 i f ( data= ‘ ‘Not Found ’ ’ ) or ( data= ‘ ‘No Respond ’ ’ ) then18 trigger Next with ( ‘ ‘ Get Next Shop ’ ’ , . . . ) .19 else20 //non−conformance ,\ i u t \ f ind the camera but21 \km\ did not

Figure 18: Checking Property 3 in KM

mentation to the other files is much smaller. Neitherthe Main nor any of the malls require any instrumen-tation.

2. The instrumentation of the Client KM is larger thanthe corresponding instrumentation on the IUT side.This is because the Aglets platform offers a primitivewhich allows the easy determination of how many pro-cesses reside on a node; in the KM, on the other hand,this primitive does not exist and needed to be imple-mented.

3. Both, before and after the instrumentation, the sizeof the kiltera code is considerably smaller than thatof the corresponding Aglets code. We view this asan indication that kiltera does indeed allow a succinctexpression of mobile and distributed computation.

We do not yet have comprehensive performance data in-dicating, e.g., how much of a performance penalty was in-troduced by the instrumentation. However, all of the con-

formance checks described above completed in less than aminute.

4.2 Other examplesWe have also applied our framework to two other exam-

ples: a centralized distributed mutex algorithm, and Lam-port’s distributed mutex algorithm (implementations of thesetwo algorithms was taken from [9]). Our prototype was ableto detect seeded faults in both implementations. For in-stance, we modified Lamport’s algorithm allowing a processto send a “release” message although it did not hold a to-ken and was not in the critical region. Interestingly, thismodification caused the code to examine “intermittent” (ortemporary) inconsistency which would disappear after con-tinued execution. The example shows that a system mayrecover from failure “on its own” and that, therefore, partialexamination of traces may not always be sufficient.

5. RELATED WORKRelevant related work seems to fall into three categories:

run-time monitoring, testing of mobile code, and testing ofagent systems.

1. Run-time monitoring: Many approaches and tools forrun-time monitoring exist (e.g., Java MaC [13, 12],Java PathExplorer [10], Java Run-time Timing-constraintMonitor [19], decentralized monitoring [26], and JavaMonitoring-Oriented Programming (MOP) [6]). Withthe exception of Java MOP, these approaches are basedon the same idea: The monitored code is instrumented(possibly automatically) such that it emits sequencesof events during execution; event sequences are ana-lyzed by a central analysis component for specificationviolations. Approaches differ with respect to the ap-plication domain (e.g., real-time systems [19, 12], dis-tributed systems [26], and sequential and concurrentsystems [13, 6]), the degree of automation of the in-strumentation phase (e.g., automatic instrumentationfrom specifications [13, 26], automatic instrumentationusing scripts [10], and manual instrumentation [19]),and the specification formalism supported (e.g., tem-poral logic [13], timing constraints [19], and rewritelogic [10]). Our approach is unique in that it sup-ports the possibly distributed analysis of mobile code.Moreover, specifications are expressed using an exe-cutable process algebra; compared to more declarativespecifications (e.g., using temporal logic), this kind ofoperational specification appears more suitable for thecomprehensive description of agent interactions with-out sacrificing mathematical rigor.

2. Testing of mobile code: A more limited amount of ex-isting work is devoted to testing of mobile code. In [7],a framework is presented to support testing mobileJava code with respect to the standard code cover-age criteria (e.g., statement coverage, branch coverage,etc). In [8], test patterns are obtained from design pat-terns for mobile agents. Finally, a formal frameworkfor conformance testing of mobile code is presentedin [17]. The work extends the conformance testing the-ory for distributed systems developed by Tretmans [28]and uses labeled transition systems. Our approach,on the other hand, is based on an extension of the

π-calculus, which appears much more suitable. How-ever, our work currently lacks a formal definition ofconformance.

3. Testing of agent systems. In this context, agents fea-ture not only autonomy and mobility, but also someform of planning based on, e.g., the “beliefs”, “desires”and “intentions” (BDI) model. Our work is not con-cerned with planning at all. However, it is possible toadd planning to the Kiltera model and thus also makeit subject to the conformance check. Agent-orientedsoftware engineering (AOSE) is concerned with sup-porting the effective construction of reliable agent sys-tems. Most AOSE methodologies (such as Prometheus[22]) advocate the use of models (e.g., sequence dia-grams and state machines) in early stages of develop-ment. Several papers suggest leveraging these modelsfor test case generation [27, 32, 31, 20]. The work in [3]discusses conformance testing and thus is closer toours: agents are monitored with respect to“interactionconstraints” that capture properties of interactions be-tween agents as logical formulas; constraint checking isimplemented using constraint logic programming; tim-ing constraints are supported, but support for mobilityand distributed monitoring appears to be missing.

In summary, to the best of our knowledge, our approachis unique in that it simultaneously supports mobility, dis-tributed execution and time and uses models which can bedirectly executed in a simulation environment.

6. CONCLUSION AND FUTURE WORKAn approach for runtime conformance checking of mobile

and distributed systems has been described. The approachis based on kiltera, a novel modeling language for concur-rent, distributed, mobile, and timed systems. Due to kil-tera’s simulation environment, early analysis of the modelsis possible. Moreover, execution of the kiltera model may bedistributed. Our use of the approach in two case studies hasbeen sketched. The results are preliminary, yet encouraging.

The most important topic for future work is a more thor-ough evaluation of the approach. In particular, we are inter-ested in determining how effective it really is for detectingfaults in mobile and distributed systems. To this end, weare currently considering the use of mutation testing. Atthe moment, our approach does not help the user find ap-propriate test inputs. It would thus be more comprehensive,if the kiltera models could also be used to generate test inputs— a topic that is well-researched in the context of model-based testing (albeit not on kiltera models). Finally, anotherpromising idea to pursue is the automatic generation of, e.g.,Java code from kiltera. Our approach could then be used todevelop techniques for testing code generators.

7. REFERENCES[1] Python website. http://www.python.org/.

[2] Aglets website, 2002. http://www.trl.ibm.com/aglets/.

[3] M. Alberti, F. Chesani, M. Gavanelli, E. Lamma,P. Mello, and P. Torroni. Compliance verification ofagent interaction: a logic-based tool. Applied ArtificialIntelligence, 20(4-5), April 2006.

[4] P. Bresciani, P. Giorgini, F. Giunchiglia,J. Mylopoulos, and A. Perini. An agent-orientedsoftware development methodology. Journal ofAutonomous Agents and Multi-Agent Systems,8:203–236, May 2004.

[5] A. Carzaniga, G. Picco, and G. Vigna. Designingdistributed applications with mobile code paradigms.In 19th International Conference on SoftwareEngineering (ICSE’97), Boston, MA 1997.

[6] F. Chen, M. d’Amorim, and G. Rosu. Checking andcorrecting behaviors of Java programs at runtime withJava-MOP. Electronic Notes in Theoretical ComputerScience, 144(4):3–20, 2006.

[7] M. Delamaro and A.M. Rizzo Vincenzi. Structuraltesting of mobile agents. In 3rd InternationalWorkshop on Scientific engineering of distributed Javaapplications (FIDJI’03), LNCS 2952, November 2003.

[8] A. Figueiredo, A. Almeida, and P. Machado.Identifying and documenting test patterns frommobile agent design patterns. In First InternationalWorkshop on Mobility Aware Technologies andApplications (MATA’04), pages 359–368, 2004.

[9] V. K. Garg. Concurrent and Distributed Computing inJava. John Wiley & Sons, 2004.

[10] K. Havelund and G. Rosu. An overview of the runtimeverification tool Java PathExplorer. Formal Methodsin System Design, 24(2):189–215, March 2004.

[11] D. R. Jefferson. Virtual Time. ACM-TOPLAS,7(3):404–425, July 1985.

[12] M. Kim, I. Lee, U. Sammapun, J. Shin, andO. Sokolsky. Monitoring, checking, and steering ofreal-time systems. In 2nd International Workshop onRun-time Verification (RV’02), volume 70 ofElectronic Notes in Theoretical Computer Science,Copenhagen, Denmark, July 2002.

[13] M. Kim, M. Viswanathan, S. Kannan, I. Lee, andO. Sokolsky. Java-MaC: A run-time assuranceapproach for Java programs. Formal Methods inSystem Design, 24(2):129–155, 2004.

[14] M. Kusek and G. Jezic. Extending UML sequencediagrams to model agent mobility. In 7th InternationalWorkshop on Agent Oriented Software Engineering(AOSE’06), pages 51–63, 2006.

[15] D. B. Lange and M. Oshima. Programming andDeploying Java Mobile Agents with Aglets.Addison-Wesley, Reading, MA, 1998.

[16] Y-H. Li, C-R. Dow, K-H. Chang, and K-H. Chen.Design and implementation of an open sourcedlocation service system for the aglets mobile agentsystem. International Conference on Integration ofKnowledge Intensive Multi-Agent Systems(KIMAS’07), pages 85–90, 2007.

[17] M. Marche and Y.-M. Quemener. A model forconformance testing of mobile agents in a MASIFframework. In 2nd International Workshop on FormalApproaches to Agent-Based Systems (FAABS’02),LNAI 2699, October 2002.

[18] R. Milner, J. Parrow, and D. Walker. A calculus ofmobile processes, parts I and II. ReportsECS-LFCS-89-85 and 86, Computer Science Dept.,University of Edinburgh, March 1989.

[19] A.K. Mok and L. Guangtian. Efficient run-time

monitoring of timing constraints. Third IEEEReal-Time Technology and Applications Symposium,pages 252–262, June 1997.

[20] F. Mokhati, M. Badri, L. Badri, F. Hamidane, andS. Bouazdia. Automated testing sequences generationfrom auml diagrams: a formal verification of agents’interaction protocols. International Journal ofAgent-Oriented Software Engineering, 2(4):422–448,2008.

[21] S. Munroe, T. Miller, R.A. Belecheanu, M. Pechoucek,P. McBurney, and M. Luck. Crossing the agenttechnology chasm: Experiences and challenges incommercial applications of agents. KnowledgeEngineering Review, 21(4):345–392, 2006.

[22] L. Padgham and M. Winikoff. Developing IntelligentAgent Systems: A Practical Guide. John Wiley andSons, 2004.

[23] L. Padgham, M. Winikoff, S. DeLoach, andM. Cossentino. A unified graphical notation of AOSE.In 9th International Workshop on Agent OrientedSoftware Engineering (AOSE’08), 2008.

[24] G. Picco. Mobile agents: An introduction. Journal ofMicroprocessors and Microsystems, 25:65–74, 2001.

[25] E. Posse. Modelling and simulation of dynamicstructure discrete-event systems. Ph.D. thesis, Schoolof Computer Science – McGill University, October2008.

[26] K. Sen, A. Vardhan, G. Agha, and G. Rosu. Efficientdecentralized monitoring of safety in distributedsystems. In 26th International Conference on SoftwareEngineering (ICSE’04), pages 418–427, 2004.

[27] H.-S. Seo, T. Araragi, and Y.R. Kwon. Modeling andtesting agent systems based on statecharts. InFORTE’04 Workshops, LNCS 3236, 2004.

[28] J. Tretmans. Conformance testing with labelledtransition systems: Implementation relations and testgeneration. Computer Networks and ISDN Systems,29(1):49–79, 1996.

[29] R. Trillo, S. Ilarri, and E. Mena. Comparison andperformance evaluation of mobile agent platforms. In3rd IEEE International Conference on Autonomic andAutonomous Systems (ICAS’07), Athens, Greece,2007.

[30] M. Winikoff. Future directions for agent-basedsoftware engineering. International JournalAgent-Oriented Software Engineering, 2008. Acceptedfor publication.

[31] Z. Zhang, J. Thangarajah, and L. Padgham.Automated unit testing intelligent agents in PDT. InDemonstration at the 7th International Conference onAutonomous Agents and Multi-Agent Systems(AAMAS-08), Estoril, Portugal, 2008.

[32] M. Zheng and V.S. Alagar. Conformance testing ofBDI properties in agent-based software. In 12thAsia-Pacific Software Engineering Conference(APSEC’05), December 2005.