Programmable agents for flexible QoS management in IP networks

1Programmable Agents for FlexibleQoS Management in IP NetworksHermann De Meer(1), Aurelio La Corte(2), Antonio Pulia�to(3), Orazio Tomarchio(2)(1)Dept of Electrical EngineeringCenter for Telecommunications ResearchColumbia University, New York, USA(2)Istituto di Informatica e Telecomunicazioni, University of Catania, Italy(3)Dipartimento di Matematica, University of Messina, Italy

Correspondence address :prof. Antonio Pulia�toDipartimento di MatematicaSalita Sperone, Contrada Papardo98166 Messina - ItalyPhone: +39 090 393229 { fax: +39 095 338280e-mail: [email protected]

2 AbstractNetwork programmabillity seems to be a promising solution to network management andQoS control. Software mobile-agents technology is boosting the evolution toward applicationlevel control of network functionalities. Code may be deployed in the network dynamically andon-demand for the bene�t of applications or application classes. Agents support a dynamic dis-tribution of control and management functions across networks, thus increasing exibility ande�ciency. We propose to use mobile-agent technology to overcome some of the problems inherentin current Internet technology. We focus our attention to QoS monitoring, being locally signi�-cant in network sub-domains, and realize a QoS management strategy in response to variations ofuser, customer of application requirements and of the network state. We describe our experienceand the results obtained from our test-bed, where software agents are instantiated, executed,migrated and suspended in order to implement exible QoS management in IP networks.KeywordsSoftware Agents, RSVP, Quality of Service, Underreservation, Adaptation, Customer PoliciesI. IntroductionSaltzer, Reed and Clark have coined the famous \end-to-end" arguments in system design in1984, which shaped the Internet signi�cantly as we see it nowadays [26]. The arguments werebased on the observation that functions placed at low levels of the network may be redundantand the cost may therefore not be su�ciently compensated by their value. As a result, networkcontrol and management were separated from application level control functions. One of the moreprominent reasons to challenge this legacy wisdom is inherent in multimedia networking, wherequality-of-service (QoS) requirements of applications have to be exibly met by networks in realtime. Revisiting their original arguments, Reed, Saltzer and Clark point out that the drivingforce was in fact the need for exibility which led to the stated, well-accepted design principles ofmonolithic systems and networks [25]. But with the possible advent of programmable networks,the whole picture might change. Network programmability, as the ultimate form of exibility,might in fact open up the real possibility \to defer design choices upward in the layering, closerto the application, and later in time, even though the resulting functions may actually take placedeep inside the network" [25]. It is therefore no contradiction to the end-to-end argument, ratherto the contrary, if code is deployed in the network dynamically and on-demand for the bene�t of

3applications or application classes, as long as this would not compromise usability, performanceand security of the network and its services as a whole.While network programmability is attracting increasing attention, its state of development isstill at its infancy [16], [23], [9]. Meanwhile, pragmatic approaches have been suggested to useJAVA as an, albeit ine�cient, vehicle for rapid prototyping of network protocols and softwareagent systems. Based on a system independent application platform, code can be dynamicallydownloaded and can actually roam the network in order to perform network functions on behalfof customers, users or applications. This approach has the advantage of gaining experience incustomizing network functionalities on the y as long as programmability has not become trueon a broader scale. While computational e�ciency is of less concern under these circumstances,management, control and transport can be customized on demand in a smaller time scale. Theexisting approaches reach from �ne grain in-band capsules, which replace IP packets and containcode and data, to router and switch extensions that are "plugged in" on demand on a somewhatcoarser time scale, or any combination in between [23].The motivation for using mobile agents in support of network transport, management, or con-trol is to perform dedicated processing on behalf of customers inside the network at particularlinks or in particular regions of node clusters which may not implement the required algorithmsand services. In this paper we present a strategy for exible QoS management in IP networksbased on the use of mobile agents. The main part of the paper is initiated in Section II by ashort review of the QoS problem in the Internet and a motivation for the introduction of pro-grammability in networking, in general, and software agents, in particular. In Section III, theadopted reference architecture is described, our software-agent concept is introduced in moredetail, and the coordination and communication mechanisms among agents are illustrated. Sec-tion IV presents the algorithm for implementing the resource reservation strategies within RSVPsignaling. Section V introduces a heuristic approach to determine �gures of merit for bandwidthallocation. Section VI shows the experimental results obtained by measurement in our networktestbed based on our Mobile Agent Platform. Section VII comments about fairness of the pro-posed reservation algorithm with regards to the best-e�ort tra�c. Section VIII, �nally, presentsthe conclusions and points out to future work and possible extensions.

4 II. QoS in the Internet and Programmability of NetworksQoS is related to multimedia applications and is characterized by quantitative measures, ex-pressing subjective and objective criteria in terms of user satisfaction. User satisfaction by andlarge depends on the timeliness by which continuous media like audio and video data are pre-sented in a distributed setting, exposing the data to transport and processing of various kind [].There are no dedicated resources provided in traditional internetworking data communicationenvironments which are largely packet switched, allowing for highly exible and e�cient datatransport. The degree of resource sharing and multiplexing implies best-e�ort quality with re-spect to a timely delivery of service, meaning that there is no timeliness guarantee. Highlyutilized networking and processing resources, which is desirable from a service provider's pointof view, may result in temporary delay increases, relative throughput decreases or signi�cantdelay variations, all deteriorating QoS in a substantial way.Generally, two types of counter measures can be taken: provisioning of dedicated resourcesor adaptation of applications. There has been a long and painful struggle in the Internet com-munity towards the royal way of resource reservation since the early nineties. Initial optimismwas embodied in the ATM technology, implementing early QoS-architecture models and allowingfor resource reservations and QoS guarantees to a large extend. Nevertheless, serious scalabiltyand exibility concerns were soon raised, resulting in a somewhat lower enthusiams. While theIETF standardization body responded by rolling back to the IP networking layer, the ow-based(state-aware) resource reservation concept was pertained in the resulting intserv model [5]. Con-sequently, while part of the exibility problem appeared to be solved in the face of heterogeneousnetworking, scalability was not. More recently, the IETF community added another perspec-tive to their spectrum of measures for QoS support in the Internet, namely, the provisioningof di�erentiated service classes [27], thereby giving up on explicit resource reservations. Sincethe resulting di�serv model does not rely on any ow-based state representation inside the corenetwork, the approached solution appears to be better scalable than the earlier proposals [3], [4].Still, concerns remain about the scalability and complexity of the remaining signaling procedureinvolved, in particular, in the face of the limited QoS predictability of the resulting services.While some of the cost factors have become more clearly visible, the bene�t of service di�eren-tiation for QoS support is still more or less a matter of belief.Reviewing the historical process in providing QoS support across the Internet, some observa-

5tions can be made. It is becoming increasingly clear that ow-based reservation schemes suchas RSVP/intserv, or ATM for that matter, do not scale well in core networks. On the otherhand, it is questionable if QoS can be provided within a stateless approach. As a result, one maywonder if there is a single solution to all of the challenging questions at all. Wouldn't it be ratherdesirable to leave the decision of how much state to implement and what state to implement tothe customer? Following that vision, each customer could implement its own policy, trading costwith service quality according to its own preferences and capacities. But to that end, networkservice must be open, exible and con�gurable. Our vision is to provide such customized servicesin parallel upon demand. For example, some customers, who may represent larger organizationssuch as companies or universities or subsidiaries thereo�, may wish to rely on guaranteed ser-vices provided by the intserv model (and are willing to pay for all implied resources), or maybe satis�ed with di�serv model, accepting occational quality disturbances, or may even preferto implement a type of service of its own that has not been standardized at all. A frameworkfor architecting network resources based on programmable networks has been suggested earlier,applying the paradigm of spawning network [10], [9], [8]. We follow a similar perspective, butuse our agent-based platform MAP [24] instead for rapid prototyping of network services, thusallowing for a faster time scale of service deployment.A major incentive for an agent-based approach is that policies can be implemented dynami-cally, allowing for a resource-state-based admission and reservation strategy. Agents are used todiscover about resources available inside the network and claim resources on behalf of customersaccording to some (later de�ned) \�gures of merit". Figures of merit capture trade-o�s betweenbandwidth claimed and loss risk incurred due to high utilization. Loss, although not being amonetary quantity, can be seen as a cost measure for claiming resources at a particular stateof the network. Di�erent customers may follow di�erent �gures of merit so that a negotiationbecomes possible. Implicitly, of course, this concept is based on the assumption of customers'varying willingness to pay for resources under di�erent circumstances. The willingness may varyfrom customer to customer but also with time from a single customer's perspective, resulting ina non-homogeneous approach.Another advantage of agents is their capability to trigger adaptation of applications inside thenetwork on behalf of customers. This allows for an immediate response to resource shortages,diminishes the amount of useless data transported, and reduces signaling overhead. The agents

6perform adaptation along the lines of �gures of merit mentioned earlier, thus allowing again theincorporation of customer policies and preferences. This concept is combined with underreser-vation of resources (which is going to be explained later in more detail), allowing for a minimumquality of service (which is related to customer requirements) to be guaranteed. Adaption isperformed based on sharing of excess (bandwidth) resources.Resources are usually claimed on a much coarser scale than ow-based resource reservationswould do. Therefore, we are inclined to talk about virtual links and virtual networks thatare customized on behalf of customers. Granularity of time and space for resource reservationthrough agents are performed on a much coarser scale than with ows. Flow adaptation isperformed relative to a virtual network claimed by a particular customer.Mobile agents [18], [20], [19] provide the highest possible degree of exibility and can carryapplication speci�c knowledge into the network at locations where it is needed. This philosophyis similar as the one recently expressed in the active networks community within the SwitchWareproject at UPen [2], where active extensions or software \plug-ins" are complemented by activemessages for a full scale programmability of networks. But we do not investigate programmabilityon the level of lower operation system APIs, but rather think in terms of programmable networkAPIs, which are provided by an underlying network programming environment. Similarly as inthe DARWIN project [11], we also leverage the complete Internet protocol stack and build on topof it. Note that this does not preclude changes of the underlying architecture, being triggeredby other projects from the programmable networking communities [9].In the rest of the paper, we focus on de�nition and implementation of basic mechanisms,allowing agents to interact with the available reservation protocol. RSVP [6], [30] is used forsignaling of reservation, measurement-based admission control or QoS adaptation control. Wealso limit our studies to experiments involving controlled load service classes [29] rather thanguaranteed services [28]. Controlled load services require more attention due to their loosede�nition and are also closer to service classes that are propelled within the di�serv framework.III. A Mobile-Agents EnvironmentQoS management through the use of mobile agents is being currently investigated by di�erentresearchers dealing with distributed systems programming, monitoring and management [12], [7].We have already applied such technology in similar scenarios [15], [14]. We have also developeda fully programmable environment, named MAP (Mobile Agent Platform) [24], that allows the

7development and the management of mobile agents. It has been fully created using the Javalanguage, and provides all the mechanisms needed for creating, running, suspending, waking up,deactivating and activating the agents, and allows them to communicate and migrate throughthe network, in order to run the task assigned to them. MAP is suitable for the creation ofdistributed applications as it provides security mechanisms based on the use of asymmetric keysfor authentication and access control and moreover it is compliant to the MASIF standard [13],[22].VL

VL

VL

CoreNetwork

Edgerouter

routerEdge

routerEdge

Customer

Customer

Customer

RR

user

RR

user

user

Fig. 1. Agents on a serverGenerally speaking, in order to take advantage of agent technology for QoS improvements,the approach we propose is based on the use of a set of specialized agents which dynamicallydecide the amount of resources to be assigned to each virtual link and use the RSVP signalingmechanisms for allocating the requested bandwidth. The reference scenario, as depicted in�gure 1, assumes a set of customers each constituted of users wishing to exchange data. Acustomer can be a company, a university campus, a public or private entity which groups varioususers. Each customer accesses the core network through an edge router, which represents theaccess point (for ingress and egress tra�c) to the external world. The core network is madeof a set of routers which allow to establish virtual links (VLs) among di�erent customers. AVL groups physical connections between users belonging to di�erent customers. A set of VLsconstitutes what is nowadays referenced as a virtual programmable network.An edge router may receive a request for VL creation from the customer which, according to

8the behavior of the users it is hosting, may decide to set up a new VL with a remote customer.Conversely, a request for setting up a new VL may arrive to an edge router from the core network.To better specify the role of the edge routers, we refer to Figure 2, which shows the softwaremodule to be executed on them: we assume router A as the originator of a request for VLcreation and router B as the receiver of the request. The following agents will be involved onthe edge router A:� coordinator: it receives the opening requests of a new VL from the customer. For eachrequest, it creates a clconnect agent, that manages the request itself.� clconnect: it is the manager of the VL. As soon as it is created, it sends a message of openingrequest to the remote edge router (B) and waits for the arrival of the collector agent.On the router B the following agents are executed:� coordinator: as soon as it receives a 'request for VL creation' a new connector agent is createdin charge of managing the VL.� connector: for each VL there is an associated connector which contains the identi�er of theVL and the identi�er (unique) of the agent associated with the same VL but executing onthe remote edge router (A).� collector: it is periodically created from the connector with the task of traveling through allthe routers between the two edge routers (A and B) to collect useful informations on thestate of the network.Customer

coordinator

RSVP

connector

connector

connector

collector

collector

collector

request of VL creation

VL

VL

VL

Network

Core

B A

RSVP

clconnect

network status

collector

VL creationrequest of

coordinator

Fig. 2. Agents on edge routersFinally, in each router along the VL in the core network there are two agents with the task tocollect some statistics about the conditions of the network (Figure 3). These two agents are:� rsvpstat: it collects the information about the bandwidth reserved through RSVP, and givesit to the collector agent each time it requires.

9� ipstat: it collects the information about the total bandwidth occupied on the output portsof the router, and provides it to the collector agent each time it requires.rsvpstat

ipstat

Router

rsvpstat

ipstat

Router

collector collector collector

Edge router Edge router

Core networkFig. 3. Agents present in each routerTo better clarify the whole process of setting up a VL and in order to specify the way theagents coordinate each other, in the following we illustrate the steps to be followed, referring to�gures 2 and 3.Once the coordinator agent of the edge router A has received the message for setting up anew VL, it creates a new clconnect agent and runs it. This clconnect agent sends a messageto the coordinator agent present in the edge router B, which sends the data contained in themessage to a new connector agent just created. The connector discovers the path between theedge routers A and B, and sends the list of crossed routers to a new collector agent just created.Once the collector receives the list, it migrates towards the �rst router in the core network, whereit requires the local rsvpstat agent for the information about the bandwidth reserved by RSVP,requires the local agent ipstat (see Figure 3) for the information concerning the total bandwidthoccupied, and calculates the free bandwidth that could still be assigned. The collector agent alsostores the identi�ers of all the edge routers with VLs sharing the same output interface of theVL being established 1. Once all the necessary data have been collected, the collector migratestowards the following router, on which it repeats the same operations previously described. Thisway the collector collects the information about the state of all the routers in the core networkalong the path beween the edge routers A and B.When the collector reaches the edge router A, it passes all the collected information to the1The VLs crossing the router but using a di�erent output interface than the one used by the connection servedby the collector do not take bandwidth away to it, and may therefore be ignored.

10clconnect agent (see Figure 2), which is responsible for applying the customer policy for resourcereservation. Each customer may adopt its own policy, by delegating to the agents in the networkthe role of applying and enforcing it. These policies do not need to be �xed in advance, but can becustomized and changed on demand, thanks to the agent framework that brings them deep insidethe network. The following section describes a particular policy that we have implemented forour testbed. The idea behind the algorithm is that, according to the conditions of the network,a VL can be fully-reserved or under-reserved, that is all the resources it needs (according to whatis speci�ed in [29]) or a percentage of these resources are reserved, respectively.With regard to updating the parameters concerning the reservation according to the changesin the network, we need to outline how the migration of the collector agent from the edge routerB to the edge router A does not take place only once, but is periodically executed. Besides, foravoiding problems of inconsistency, the collector agents are programmed to kill their executionif an open (but not reserved) VL already exists in a router to which they migrate. This way, weavoid to collect potentially unreliable informations as a consequence that the existing VL is stillbeing set up and the data related to the status are being modi�ed.IV. Algorithm for Resources ReservationThe algorithm executed by the clconnect agent in order to implement the reservation policyconsists of two distinct phases, as shown in Figure 4. In the �rst phase (for example accordingto what has been �xed in [6], or according to the quality a customer wishes to assure to itsusers) the clconnect agent �rstly tries to reserve all the bandwidth originally required by theapplication. If there are not enough resources available, it veri�es if an under-reservation canbe executed according to some �gures of merit which will depend on the network parameters(speed, latency, technologies) and also on some other parameters such as the willing to pay for auser to obtain a high and guaranteed QoS. More speci�cally, the idea is to check the feasibilityof providing an amount of reserved bandwidth lower than the one originally needed, as long asthe performance (e.g. average loss ratio) is maintained within admissible values.If the resources of the network are still not su�cient for meeting the request of the under-reserved connection, the second phase of the algorithm is started. The clconnect agent requiresthe clconnect agents managing other VL to give up a part of the resources used. This requiresa strong coordination action with the clconnect agents managing the VL already established.In fact, the clconnect agent builds (according to the information communicated by the collector

11Start

Is it possible to execute a full reservation?

Is it possible to executean under-reservation?

Yes

No

Phase 2

Yes

No

Yes

Phase 1

NoacceptedThe VL is

Is it possible to collect the necessary

crossing the routers?bandwidth from some of the VLs

The VL creation is refused

Coordination with other agentsto reduce the bandwidth assignedto existing VLs

Fig. 4. Under-reservation algorithmagent) a list of the routers that do not have the resources needed for creating the VL. Eachrouter of this list is then associated to a record which stores all the reserved VL that cross therouter itself. For each router of the list, the clconnect agent sends a resource request messageto the clconnect agents related with the reserved VL crossing the router, asking to reduce theamount of bandwidth used. If it succeeds in collecting the necessary bandwidth, the router isremoved from the list and the same procedure is repeated on the subsequent router, if any. Ifthe list is empty, it indicates that the agent has already obtained all the necessary resources,and may create the under-reserved VL. Conversely, if even a single router on the path has noVL to which the clconnect could require resources, or the sum of those given up by the otheragents is not su�cient, the VL creation is refused and the QoS required by the customer has tobe renegotiated.

12 V. Figure of merit for under-reservationAccording to the previous discussion, a way to determine the amount of reduction in bandwidthreservation has to be identi�ed. The proposed approach is mainly heuristic as it aims to identifya relation between the loss ratio and the bandwidth allocated to a VL. A �gure of merit whichexpresses the relation between allocated bandwidth and average packet delay, although theo-retically important, has not a practical utility for two main reasons: a) it is extremely di�cultto obtain a precise measure of the delay between two end points due to clocks synchronization;b) as we are not dealing with guaranteed services [28], there is not a feasible way to impose adelay constraint to a VL, but reservation has to be carried out in terms of allocated bandwidth.In this section we describe the experimental trials we have carried out and the �gure of meritwe have evaluated and used in executing the algorithm described in the previous section. Wede�ne as the under-reservation factor fur the percentage of assigned resources with respect tothe total amount of resources required by a VL. So, fur = 100% means that no under-reservationis performed and the VL is seen as a reserved connection according to what is speci�ed in [29],whereas fur = 0% means that VL is seen as a best-e�ort connection.Figure 5 shows our experimental environment. The edge routers of customer A and B arerepresented in our testbed by two hosts TX and RX. These edge routers are connected througha core network represented in our experimental network by the two routers R1 and R2 which areconnected among them by a 10 Mbit/s dedicated link.In the performed experiments we used the RSVP protocol to signal resources reservation. Thehosts used in our testbed network are SUN workstations with the Solaris 2.5.1 operating system,in which the RSVP protocol for Sun hosts [21] has been installed. R1 and R2 are equipped withtwo network interface card and are con�gured as RSVP-enabled routers. The TX and RX nodeemulate the aggregate tra�c that is generated by a customer: therefore, since the testbed is areal network, in the following description of the experiments we will use the term connection todenote the previously de�ned Virtual Link.The implementation of the RSVP protocol for Sun hosts used allows exploiting some of themain functionalities of the protocol, even if it is only able to support Best E�ort and ControlledLoad services. The mechanism used to provide link sharing between the accepted tra�c typesis the Class Based Queuing (CBQ) mechanism [17]. It consists of a Classi�er, an Estimatorand a Packet Scheduler which, according to the reference RSVP architecture [6], [30], classi�es

13B

CustomerRXTX

A

Customer

Core network

R1 R2routerEdge Edge

routerRouter Router

Fig. 5. Network architecture used for experimental evaluation of the under-reservation mechanism.the incoming packets into pre-de�nited classes, estimates bandwidth usage of each class andselects the next class to send a packet, respectively. Moreover, the Packet Scheduler supports aborrowing mechanism de�ned by the Integrated Service Working Group for the Controlled Loadservices [29]. By means of this mechanism, a connection should have the availability of extrabandwidth from other tra�c classes when it is unused, that must be immediately returned tothe owner category, if it is needed.The subdivision of network bandwidth on the di�erent classes, that is the partition of thelink between routers among classes is set by the network administrator during the RSVP devicecon�guration, according to the estimated tra�c that will load the device. In the performedexperiments, the 10 Mbit/s of bandwidth supplied by the link connecting the two edge-routersis subdivided as follows:� 35% (3.5 Mbit) for reserved connections;� 35% (3.5 Mbit) for the best-e�ort tra�c and the one that can be lent to reserved connections;� 30% (3 Mbit) for the best-e�ort tra�c that cannot be lent to reserved connections.According to what described in [6], this repartition cannot be dynamically modi�ed; in fact,changing it would cause the dropping of connections and their reactivation.For generating information streams sent from the TX workstation to the RX workstation weused the Mgen v3.1 tool [1], which allows generating tra�c with speci�c statistical features,de�ning the parameters needed for RSVP and performing measurements about throughput,percentage of loss ratio and end-to-end delay.In order to relate the loss ratio to the amount of under-reserved bandwidth and apply thereservation policy described in Section IV, we performed several experiments during which theTX workstation sends to the RX workstation a certain number of UDP information streams,

14each with exponentially distributed departing times. In the core routers R1 and R2, a reservedconnection (that is a VL) is assigned to each information stream (which forms the aggregate ofthe tra�c provided by a customer). The parameters of the tra�c related to each connection havebeen varied, so that the reserved connections in any case fully occupy the bandwidth assignedin the routers (3.5 Mbit/s). In particular, the number of reserved connections has been variedbetween 2 and 7, the emission rate of reserved connection between 10 and 100 frame/sec, thenumber of packets/frame between 1 and 10, and the UDP packet payload size has been set 512or 1024 bytes.Routers have been also loaded with best e�ort tra�c, which is the sum of two contributions:the �rst one has an interarrival time with exponential probability distribution, while the secondhas a constant interarrival time. In both cases, the UDP packet payload size has been set to 512bytes. By varying the number of packets/sec of each contribution, the best-e�ort load has beenvaried between 0% and 50% of the dedicated link capacity (10 Mbit/s).

Fig. 6. Average loss ratio versus under-reservation factor by varying the BE tra�c load (L)By measuring the loss ratio during several experiments and averaging the obtained results, wehave depicted the graph of Figure 6, where the average loss ratio versus under-reservation factorby varying the best e�ort load is shown. Even if the performances depend on the distribution ofthe load supplied by reserved connections, such graph can give an indication about the minimumunder-reservation factor acceptable as a function of the total load: it can therefore be used asthe �gure of merit according to which the reservation algorithm is executed.The various experiments carried out have allowed verifying how a certain level of quality ofservice (in terms of average loss probability) can be maintained even without reserving all the

15resources required by a virtual link. The increase in the amount of reserved VL that can beaccepted becomes an increase in pro�ts for a network manager who wants to make users pay forthe use of the reserved bandwidth.VI. Experimental results on QoS management policyIn order to verify the e�ectiveness of the proposed QoS management strategy, we have set upan experimental trial where software agents implement the algorithm described in Section IV.The network architecture used is again the one shown in Figure 5. The core routers are loadedby reserved connections (that are VL), each of which derives from a constant bit rate tra�csource which emits a frame of 10 1-Kbyte packets every 100 ms, and best e�ort tra�c, whichconstitutes the so-called additional load. The bandwidth supplied by the link between the twocore routers is again divided as described in the previous section, that is into 35% (3.5 Mbit)for reserved connections, 35% (3.5 Mbit) for the best-e�ort tra�c that can be inherited by thereserved connections, and 30% (3 Mbit) for the best-e�ort tra�c that cannot be inherited by thereserved connections.Three sets of experiments have been carried out. The �rst with the aim to show how the systembehaves when a new VL is required and the overall reserved bandwidth is already allocated butthe total load is low. The second experiment introduces an extra load in the network due to thepresence of VL that load the network with best e�ort tra�c. The third experiment checks howthe system behaves when more stringent QoS requirements are imposed.Test 1: By using the agents, we have made 4 connections, each of which required a value ofloss ratio not higher than 5%. The additional load has been set to 5% of the total bandwidth.Since the load o�ered by the four reserved connections is 3.3 Mbit/s, which is less than the valueforeseen for this kind of connections (3.5 Mbit/s), they have been completely reserved by theagent system. Then we have created (always with the agents) another connection with the samerequests in terms of maximum loss ratio. Even if the additional load is very low, the systemcould create neither a completely reserved connection nor a single under-reserved connection (inthis case, in fact, the under-reservation factor should be 25%, which, according to the graph inFigure 6, provides a loss ratio higher than 5%).The second phase of the under-reservation algorithm (see Figure 4) has to be activated, byrequiring resources to the agents of the other connections. In this case the under-reservationfactor is calculated by using the curve in Figure 6. In particular, the variation in the reservation

16 Reservation (a) Reservation (b)Connection 1 100 Kbyte/s 57.3 Kbyte/sConnection 2 100 Kbyte/s 92.1 Kbyte/sConnection 3 100 Kbyte/s 100 Kbyte/sConnection 4 100 Kbyte/s 100 Kbyte/sConnection 5 non-reserved 57.3 Kbyte/sTABLE ITest 1: values of reservation of the 4 connections (a) before adding the fifthconnection and (b) after adding the fifth connectionPloss (%)Connection 1 0.4059Connection 2 0.3119Connection 3 0.2327Connection 5 0.3684TABLE IITest 1: percentage of packet loss ratio (Ploss) for connections 1-2-3-5assigned to the 5 connections is shown in Table I, from which we may notice that only the�rst two connections have lent part of the bandwidth assigned to them to the new incomingconnection (connection 5). This is due to the fact that the algorithm tries to minimize thenumber of connections which it interact with, for varying their assigned resources, thus reducingthe signaling overhead. This way only a few connections are envolved by this process, but theyget a maximum resource under-reservation according the Figure 6.By measuring average end-to-end packet delay and loss ratio of each connection, we noted thatthe reserved connections are only slightly a�ected by the decrease of the reservation, that is car-ried out with the creation of the new connection (see, for example, Table II, where the percentageof packet loss ratio is shown for the �rst three connections and for the added connection). Thisobviously, is due to the fact that the total load of the network is low.Test 2: In this test we have re-examined the same scenario used in the previous case by

17increasing the additional load to the network. It is represented by two components: one thatgenerates 200 pkts/s of 512 bytes with exponential distribution of interarrival times, and anotherone that generates 200 pkts/s of 512 bytes with uniform distribution. The total load represents20% of the total bandwidth supplied by the links (10 Mbit). Even in this case we use 4 connectionsthat require 5% as a maximum admissible value of loss ratio, to which we will add a �fth onewith the same features.As in the previous case, the system reduces, according to the graph in Figure 6, the bandwidthreserved for the �rst two connections, in order to create the �fth one (Table III). By observingthe progress of the end-to-end delay (Figure 7), we notice how the connections that reduce thereservation value have a period of settlement and then become stable, but their performancesof course degrade in comparison with the condition of full reservation. However, the quality ofservice (in terms of required packet loss ratio) is maintained (see Table IV).Reservation (a) Reservation (b)Connection 1 100 Kbyte/s 57.3 Kbyte/sConnection 2 100 Kbyte/s 92.1 Kbyte/sConnection 3 100 Kbyte/s 100 Kbyte/sConnection 4 100 Kbyte/s 100 Kbyte/sConnection 5 non-reserved 57.3 Kbyte/sTABLE IIITest 2: values of reservation for the 4 connections (a) before adding the fifth one and(b) after adding the fifth connection (additional load 20% of the total bandwidth)Ploss (%)Connection 1 3Connection 2 4Connection 3 2.4Connection 5 3.6TABLE IVTest 2: percentage of packet loss ratio (Ploss) for connections 1-2-3-5

180 0.5 1 1.5 2

x 104

0

50

100

150

200

250

300

(a)

Rita

rdo

(ms)

0 0.5 1 1.5 2

x 104

0

50

100

150

200

250

300

(b)

Rita

rdo

(ms)

0 0.5 1 1.5 2

x 104

0

100

200

300

(c)

Rita

rdo

(ms)

0 0.5 1 1.5 2

x 104

0

100

200

300

(d)

Rita

rdo

(ms)

Del

ay (

ms)

Del

ay (

ms)

Del

ay (

ms)

Del

ay (

ms)

Fig. 7. Test 2: progress of the delay for the connections (a) 1 (b) 2 (c) 3 (d) 5 (added)Test 3: In this test the network is loaded as in Test 1, but reserved connections require a lossratio not higher than 2%. In this case, from the graph in Figure 6, the under-reservation factormust be higher than 64.5%. Table V shows the value of reservation for all the connections afterthe under-reservataion algorithm of Figure 4 has been applied.Even in this case, the progress of the end-to-end delay su�ered by the �rst two connections afterlenting their bandwidth (Figure 8) crosses a transient interval, and then settles. The quality ofservice required for the various connections (loss ratio not higher than to 2%) is still maintained,as shown in Table VI.In conclusion, by using mobile agents, we have created a system that allows the dynamic intro-duction of new QoS management services. Agents dinamically decide the amount of resources tobe assigned to each virtual link using the RSVP signaling mechanisms for allocating the reservedbandwidth. But since we are focusing on virtual link, granularity of time and space for resourcereservation is on a much coarse scale than with a ow based approach. In the experimentsshown, performance is not of primary interest. Besides, performances greatly depend on the

19Reservation (a) Reservation (b)Connection 1 100 Kbyte/s 64.5 Kbyte/sConnection 2 100 Kbyte/s 77.8 Kbyte/sConnection 3 100 Kbyte/s 100 Kbyte/sConnection 4 100 Kbyte/s 100 Kbyte/sConnection 5 non-reserved 64.5 Kbyte/sTABLE VTest 3: values of reservation for the 4 connections (a) before adding the fifth one,and (b) after adding the fifth connectionPloss (%)Connection 1 2Connection 2 2Connection 3 1.2Connection 5 1.2TABLE VITest 3: percentage of packet loss ratio (Ploss) for connections 1-2-3-5

implementation using software tools for emulating the hardware functionalities of routers. Butwe think we need to outline some points arising from the tests carried out. Only three agentsare needed for managing a VL: two stationary ones on the edge routers, and a third mobile onein the network. For each new VL, these three agents are always created, and their task is tomanage all of the aspects concerning the quality of the VL. The tests carried out on the agentsystem have con�rmed that such system (with the architecture seen in Figure 5) carries out itstask adequately, in particular with regard to the collection of the informations about the stateof the network and the application of the algorithm of underreservation. The average valuesreported by the collector agent with regard to the bandwidth used in the routers are always closeto the values of tra�c actually generated. According to the conditions of the network and the�gures of merit of Fig. 6, the system can deduct which value of reservation must be applied toeach virtual link. Besides, the exchange of messages between the agents for lending the resources

200 0.5 1 1.5 2

x 104

0

50

100

150

200

(a)

Rita

rdo

(ms)

0 0.5 1 1.5 2

x 104

0

50

100

150

200

(b)

Rita

rdo

(ms)

0 0.5 1 1.5 2

x 104

0

50

100

150

200

(c)

Rita

rdo

(ms)

0 0.5 1 1.5 2

x 104

0

50

100

150

200

(d)

Rita

rdo

(ms)

Del

ay (

ms)

Del

ay (

ms)

Del

ay (

ms)

Del

ay (

ms)

Fig. 8. Test 3: progress of the delay for connections (a) 1 (b) 2 (c) 3 (d) 5 (added)results to be functional and e�ective.VII. Impact of the proposed reservation strategy on best-effort trafficPrevious tests have shown how, by using software agents a customer policy such as underreser-vation can be applied in a network. Let us note that, by using our agent approach, the policyfor resource reservation may be based not only on the availability of resources (as it happens inRSVP), but also on other network informations of interest for the customer such as congestion,low utilization of some links, and so on. If addition of resources is not possible or not wanted ortoo expensive, alternative adaptation strategies could be considered. Using the heuristic resultspresented in the �gure of merit of Fig. 6, our agent approach can also be used to incorporate cus-tomer policies for application adaptation inside the network. The adaptation triggered by agentsenhances the fairness of the reservation scheme with regard to the best e�ort tra�c. When thetra�c o�ered by the reserved VL increases, and the performance of the VLs which o�er beste�ort tra�c are excessively degraded, agents are used to adapt reserved tra�c: a portion of datastreams should be discarded (by dropping enhancement layers in the case of a layered codingscheme) or the emission rate of the users of a customer should be decreased.

21To this aim, we considered the same communication scenario of the previous tests (see Fig-ure 5), but the borrowing mechanism has been disabled and the bandwith in the core routersR1 and R2 has been subdivided in a di�erent manner: 3.5 Mbit out of 10 have been assignedto the reserved connections, while the remaining 6.5 Mbit/s have been used for the transmissionof best-e�ort tra�c. The two routers have been loaded with two data streams (a reserved anda not-reserved one), which consist of exponential tra�c with packets size equal to 1024 bytesand with a variable average rate shown in Figure 9, where the bold line in the graph of reservedconnection indicates that the decrease in the emission rate is driven by our agent-based system,which interacts with the Mgen tool.

0 1 2 3 4 5 6 7 80

1

2

3

4

5

6

7

Mbi

t/s

minutes

Best-effort streamReserved stream

Fig. 9. Time progress of the average load o�ered by the two data streamsIn the test, the average end-to-end delay, the average loss ratio and the average rate of thepackets received by the RX node during observation windows of 1 second have been measured(see Figure 10).As long as the reserved tra�c is fully contained in the 3.5 Mbit assigned to it, the performancesof the best-e�ort tra�c are not in uenced by the presence of reserved tra�c. Conversely, whenthe reserved tra�c increases, the portion of it that cannot be included in the 3.5 Mbit is managed

22as best-e�ort tra�c and, obviously, contributes to increase its loss ratio. During the �rst 2minutes of transmission the average delay and loss probability of both streams remain low, whilethe graph of the average rate of received packets re ects the time progress of the tra�c o�eredby both streams.

0 200 40024

24.2

24.4

24.6

24.8

25

(ms)

(a)

average delay

0 200 40024

24.2

24.4

24.6

24.8

25

(ms)

(d)

average delay

0 200 4000

200

400

600

800average pk−rate

(b)

0 200 4000

200

400

600

800average pk−rate

(e)

0 200 400 6000

0.1

0.2

0.3

0.4

0.5

0.6

0.7average loss

(c)

0 200 400 6000

0.1

0.2

0.3

0.4

0.5

0.6

0.7average loss

(f)Fig. 10. Average end-to-end delay, average loss probability and average rate of received packets by the RXnode during observation windows of 1 second for reserved stream (a),(b),(c), and best-e�ort stream(d),(e),(f)After 2 minutes the best-e�ort tra�c increases till it reaches 4.5 Mbit/sec: as long as thereserved tra�c remains constant, the average delay of both streams slightly increases, while theloss ratio remains low. The average rate of the received packets still re ects the real progress ofthe o�ered tra�c, even if with a wider uctuation in comparison with the previous 2 minutes.Four minutes after the beginning of the transmission, the reserved tra�c increases up to 5

23Mbit/sec, and this causes a considerable worsening in the performances concerning both streams.The average delay increases considerably, as well as the amount of lost packets. Consequently, thepacket rate measured at the destination node does not re ect the time progress of the tra�c sentby the source. At this point, when the performance of best e�ort tra�c are considered excessivelydegraded, an adaptation mechanism starts by activating an agent that (in our case) interactswith Mgen tool adjusting its frame rate, by progressively reducing, during a time interval of 1minute, the average emission rate of the reserved connection from 5 Mbit/s to 3.5 Mbit/s. Bymeans of this reduction, packet loss ratio of best-e�ort tra�c is reduced and a certain level offairness is assured.The experiment shows how, by using this agent-based approach, the adaptation policy isestablished by the customer (or user), but its activation depends on dynamic information thatagents monitor inside the network. On the other side some application-level informations (how toadapt, enhancement layers to drop, etc) which are needed at network level and that are di�cultto preliminarily know during the setup of a VL, are embodied by agents which transport this\knowledge" in the edge routers. Thus, the exibility of the agent approach is the fact that wedo not need to "hardwire" in the network any prede�ned solution: di�erent strategies can beapplied on demand. VIII. ConclusionsSoftware agents allow for dynamic customization of QoS management functions and a highdegree of exibility that is increasingly required in heterogeneous networks where a high vari-ability of application requirements must be met. In this paper a framework for QoS managementbased on mobile agents has been presented and implemented. Using this agent based approachwe have shown how customer policies can be implemented dynamically, allowing di�erent reser-vation strategy. Agents operate inside the network on behalf of customers according to some�gures of merit which express the trade-o� between bandwidth requested and loss risk incurred.As a particular policy we have introduced the underreservation concept, which assures a mini-mum quality of service, relying on the sharing of extra resources. Agents have been also used fortriggering adaptation of applications inside the network.For a proof of concept, a testbed consisting of ordinary Unix workstations, emulating (viasoftware) the operation of routers, was developed. Several experiments have been carried outon this testbed, in order to validate our previous ideas. Performance was not an issue under

24these circumstances. The results obtained are promising, and demonstrate the feasibility of QoSmanagement complemented by software agents.Acknowledgment This work has been partially funded by the Italian Ministero dell' Uni-versit�a e della Ricerca Scienti�ca e Tecnologica (MURST) in the framework of the MOSAICOproject (Design Methodologies and Tools of High Performance Systems for Distributed Applica-tions). References[1] B. Adamson and S. Gallavan. MGEN tool. http://manimac.itd.nrl.navy.mil/MGEN, 1997.[2] D.S. Alexander, W.A. Arbaugh, A.D. Keromytis, and J.M. Smith. The SwitchWare Active Network Archi-tecture. IEEE Network, 12(3):29{36, May/June 1998.[3] Y. Bernet, R. Yavatkar, P. Ford, F. Baker, L. Shang, K. Nichols, and M. Speer. A Framework for Use ofRSVP with Di�-serv Networks. Internet Draft, draft-ietf-di�serv-rsvp-01.txt, November 1998.[4] Y. Bernet, R. Yavatkar, P. Ford, F. Baker, and L. Zhang. A Framework for End-to-End QoS CombiningRSVP/Intserv and Di�erentiated Services. Internet Draft. draft-bernet-intdi�-00.txt, March 1998.[5] R. Braden, D. Clark, and S. Shenker. Integrated Service in the Internet Architecture: an Overview. RFC1633, June 1994.[6] R. Braden, L. Zhang, S. Berson, and S. Herzogand S. Jamin. Resource ReSerVation Protocol (RSVP) {version 1 Functional Speci�cation. RFC 2205, September 1997.[7] M. Breugst and T. Magedanz. Mobile Agents - Enabling Technology for Active Intelligent Network Imple-mentation. IEEE Network, pages 53{60, May/June 1998.[8] A.T. Campbell, M.E. Kounavis, D.A. Villela, J. Vicente, H.G. De Meer, K. Miki, and K.S. Kalaichelvan.Spawning Networks. IEEE Network, pages 16{29, July/August 1999.[9] A.T. Campbell, H.G. De Meer, M.E. Kounavis, K. Miki, J. Vicente, and D. Villela. A Survey of ProgrammableNetworks. ACM Computer Communication Review, 29(2):7{23, April 1999.[10] A.T. Campbell, H.G. De Meer, M.E. Kounavis, K. Miki, J. Vicente, and D.A. Villela. The Genesis Kernel: AVirtual Network Operating System for Spawning Network Architectures . In 2nd IEEE Int. Conf. on OpenArchitectures and Network Programming, OPENARCH99, New York, March 26{27 1999.[11] P. Chandra, A. Fisher, C. Kosak, T.S. Eugene Ng, P. Steenkiste, E. Takahashi, and H. Zhang. Darwin: Re-source Management for Value-Added Customizable Network Service. In Sixth IEEE International Conferenceon Network Protocols (ICNP'98), Austin, October 1998.[12] D.M. Chess. Mobile Agents: Are they a Good Idea. IBM Technical Report RC19887, October 1994.

25[13] Crystaliz, General Magic, GMD Fokus, and IBM. Mobile Agent System Interoperability Fa-cility. Joint submission to OMG. OMG TC Document ORBOS/97-10-05, also available throughftp://ftp.omg.org/pub/docs/orbos/97-10-05.pdf, November 1997.[14] H. de Meer, A. Pulia�to, J.P. Richter, and O. Tomarchio. Tunnel Agents for Enhanced Internet QoS. IEEEConcurrency, 6(2):30{39, April-June 1998.[15] H. de Meer, A. Pulia�to, and O. Tomarchio. Management of QoS with Software Agents. Cybernetics andSystems: An International Journal, 27(5), 1998.[16] D. Tennenhouse et al. A Survey of Active Networks Research. IEEE Communication Magazine, 35(1):80{85,January 1997.[17] S. Floyd and V. Jacobson. Link-sharing and Resource Management Models for Packet Networks. IEEE/ACMTransactions on Networking, 4(4):365{386, August 1995.[18] M.R. Genesereth and S.P. Ketchpel. Software agents. Communications of the ACM, 37(7):48{53, July 1994.[19] K. Rothermel and R.Popescu-Zeletin Eds.,. Mobile Agents. Lecture Notes in Comp. Science, LNCS1219,1997.[20] T. Magedanz, K. Rothermel, and S. Krause. Intelligent Agents: An Emerging Technology for Next GenerationTelecommunications? In Proceedings of INFOCOM'96, San Francisco, CA, USA, March 1996.[21] Sun MicroSystem. RSVP { ReSerVation Protocol Release ver. 0.4.11. Porting for Sun Workstation of ISIRSVP ver. 4.1a6, http://www.isi.edu/rsvp, 1998.[22] D. Milojicic, M. Breugst, S. Covaci, and al. MASIF: the OMG Mobile Agent System Interoperability Facility.In Second International Workshop on Mobile Agents, (MA'98), Stuttgart (Germany), September 1998.[23] IEEE Network. Special Issue on Programmable Networks. 12(3), May/June 1998.[24] A. Pulia�to, O. Tomarchio, and L. Vita. MAP: Design and Implementation of a Mobile Agent Platform.Journal of System Architecture, 1999. to be published.[25] D.P. Reed, J.H. Saltzer, and D.D. Clark. Commentaries on `Active Networking and End-to-End Arguments'.IEEE Network, 12(3):66{71, May/June 1998.[26] J.H. Saltzer, D.P. Reed, and D.D. Clark. End-to-End Arguments in System Design. ACM Transaction ofComputer System, 2(4):277{288, November 1984.[27] S.Blake, D. Black, M. Carlson, E. Davies, Z. Wang, and W. Weiss. An Architecture for Di�erentiated Services.RFC 2475, December 1998.[28] S. Shenker, C. Patridge, and R. Guerin. Speci�cation of Guaranteed Quality of Service. RFC 2212, September1997.[29] J. Wroclawski. Speci�cation Controlled-Load Network Element Service. RFC 2211, September 1997.[30] L. Zhang, S. Deering, D. Estrin, S. Shenker, and D. Zappala. RSVP: A New Resource Reservation Protocol.IEEE Network, 7(5), September 1993.