Research Article QoS Aware Middleware Support for ...downloads.hindawi.com/journals/ijdsn/2016/2702789.pdf · applications QoS when a dynamic system recon guration is triggered. Section

Research ArticleQoS Aware Middleware Support for Dynamically ReconfigurableComponent Based IoT Applications

Aitor Agirre,1 Jorge Parra,1 Aintzane Armentia,2 Elisabet Estévez,3 and Marga Marcos2

1Embedded Systems, IK4-Ikerlan, 20500 Arrasate, Spain2Department of Automatic Control and Systems Engineering, University of the Basque Country (UPV/EHU), 48013 Leioa, Spain3Department of Electronics and Automation Engineering, University of Jaen, 23071 Jaen, Spain

Correspondence should be addressed to Aitor Agirre; [email protected]

Received 20 November 2015; Revised 25 February 2016; Accepted 10 March 2016

Academic Editor: Wen-Huang Cheng

Copyright © 2016 Aitor Agirre et al. This is an open access article distributed under the Creative Commons Attribution License,which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Sensor web systems, cyber-physical systems, and the so-called Internet of Things are concepts that share a set of commoncharacteristics. The nature of such systems is highly dynamic and very heterogeneous and issues such as interoperability, energyconsumption, or resource management must be properly managed to ensure the operation of the applications within the requiredquality of service level. In this context, base technologies such as component based software engineering or Service OrientedArchitecture can play a central role. Model driven development and middleware technologies also aid in the design, development,and operation of such systems. This paper presents a middleware solution that provides runtime support for the complete lifecyclemanagement of a system consisting of several concurrent applications running over a set of distributed infrastructure nodes. Themiddleware builds up on top of a general purpose component model and is driven by a quality of service aware self-configurationalgorithm that provides stateful reconfiguration capabilities in face of both internal (application triggered) and external (applicationunaware) reconfiguration events. The platform has been deployed over an automated warehouse supervision system that serves asa case study.

1. Introduction

Several paradigms have emerged over the last decade withsimilar characteristics and challenges. Internet of Things(IoT), cyber-physical systems (CPS), and sensor web systemsdemand a common set of characteristics such as geographicaldistribution, interoperability, dynamicity, or system hetero-geneity. Aspects such as quality of service (QoS) and resourcemanagement are particularly relevant in this kind of systems,as far as they affect user experience and energy efficiency[1–3]. To cope with these issues, different technologies havebeen proposed. Component frameworks and Service Ori-ented Architecture (SOA) were initially developed to providesystem decoupling, promote code reuse, and ease applicationdeployment. But the existence of other requirements suchas runtime reconfiguration [4, 5] or QoS assurance [6] hashighlighted the need to integrate other technologies.

More specifically, this work is focused on systems formedby heterogeneous infrastructure nodes (in terms of hardware

and operating system) that dynamically allocate componentbased stateful applications which are as well diverse in termsof implementation language, communication protocols, andQoS requirements. An example of such kind of systemsis detailed in Section 6, where an automated warehousecomposed of tens of heterogeneous hardware nodes andhundreds of software components needs to be supervised inreal time in order to avoid downtime due to new applicationdeployment or to unexpected component faults which wouldbe otherwise difficult to diagnose. Such distributed system iscomposed of several distributed (legacy) applications that useheterogeneous communication protocols.Theproposedmid-dleware supervises the complete system, provides fault toler-ance support for specific critical components, and aids in theregistration and deployment of new system configurations.

Several key challenges arise from such needs, namely, (1)the adoption of a suitable component framework, (2) the sup-port for dynamic reconfiguration, and (3) themanagement ofthe system resources to provide the required QoS level to the

Hindawi Publishing CorporationInternational Journal of Distributed Sensor NetworksVolume 2016, Article ID 2702789, 17 pageshttp://dx.doi.org/10.1155/2016/2702789

2 International Journal of Distributed Sensor Networks

demanding applications. This work proposes a middlewaresolution that addresses these challenges. It inherits the advan-tages of a standard component model such as SCA [7] andextends it to provide the required QoS aware reconfigurationsupport.TheproposedQoS characterization is inspired in [8].It has been adapted to support component based distributedapplications, extending it to model additional aspects as stateor distribution. The concepts of service and service imple-mentation [9] have inspired the concepts of logical componentand physical component. The middleware architecture is theresult of previous work [10, 11], as well as the low level QoSmanagement mechanisms [12, 13], the definition of the basicservices [14], and the basic fault tolerance mechanisms forthe middleware components [11, 15].This paper is focused onthe QoS aware dynamic reconfiguration algorithm proposedin Section 5. An approach to implement a fault tolerantdistributed state recovery mechanism is also described inSection 6.

The middleware support that is proposed in this paperextends the DAMP (Distributed Applications ManagementPlatform) platform introduced in [16], which provided basiclifecycle management support for component based dis-tributed applications, as well as node level (local) QoSenforcement in face of components self-QoS reconfigurationrequests. This work goes further and proposes a holisticapproach to the entire systemQoS enforcement through spe-cific resource management policies, considering the applica-tion as a unique entity that shares the available resources withthe other applications that are running on the system. Thisoverall QoS enforcement of the demanding applications takesinto account the functional dependency between applicationcomponents, alongside their QoS requirements and actualnode allocation. Also, the eventual availability of redundantcomponent replicas in alternative hardware nodes is alsoconsidered.

More specifically, self-reconfiguration mechanisms tosupport stateful system reconfiguration in face of external(application unaware) as well as internal (application trig-gered) reconfiguration events are proposed. These mech-anisms rely on four specific functionalities provided bythe extended DAMP platform, that is, (1) infrastructureresource monitoring, (2) application state monitoring, (3) areconfiguration API, and (4) a QoS aware reconfigurationalgorithm, the latter being themain contribution of this work.

The infrastructure resource monitoring updates the sys-tem database with actual node utilization and livelinessinformation, thus enabling the detection of eventual nodeoverload and physical faults.The application statemonitoringcollects actual component execution and response times.Thisway, component crash or deadlinemiss events can be detectedandmanaged.The system reconfigurationAPI provides awayfor the application components to trigger functional eventssuch as self-QoS change requests and also allows the systemoperator to issue application start and stop operations into thesystem.Therefore, the monitoring capabilities of the middle-ware enable the detection of external (nonfunctional, applica-tion unaware) reconfiguration events, while the reconfigura-tion API can be used by already running applications to trig-ger internal (functional, application specific) reconfiguration

events. Finally, theQoS aware reconfiguration algorithm allo-cates the available infrastructure resources to the demandingapplications, taking into account their QoS related require-ments.This set of functionalities enables the runtime supportfor the complete lifecycle management of distributed anddynamically reconfigurable component based stateful appli-cations.

The rest of the paper is as follows: Section 2 presentsa brief survey of research works that focus on dynamicallyreconfigurable systems and QoS support. Section 3 depictsthe overall middleware design, its architecture, and thedescription of the provided services. Section 4 describes thecomponents, applications, and infrastructure that composethe system model, with special attention to its QoS charac-terization. Section 5 details the mechanisms that enforce theapplications QoS when a dynamic system reconfiguration istriggered. Section 6 describes the mechanisms that enable astateful system recovery and Section 7 presents a case studythat builds a distributed supervision system of an automatedwarehouse, upon some of the most representative function-alities of the middleware presented in this paper. Finally,Section 8 presents the conclusions andpoints out futurework.

2. Related Work

A component framework usually comprises a componentmodel and a set of related tools and methodologies that aidin the overall process of application development lifecycle,from initial specification and design, through development,deployment, runtime support, and maintenance. Neverthe-less, not all the component frameworks cover all these steps.This fact, alongside their different level of abstraction, toolsupport, or scope, hinders the process of selection of asuitable one for a specific need. In this sense, [17] presentsa good classification framework that eases the selectionof a component model. In previous works [7, 16] severalcomponent frameworks were analyzed, and Service Compo-nent Architecture (SCA) was adopted as the most suitableone for the considered target systems: on the one hand,the availability of different (commercial and open source)implementations of the standard and the annotations basednature of the component model makes the adaptation oflegacy code relatively easy. On the other hand, the standardprovides native support for diverse communication protocolbindings. Thus, the required requirements related to flexibil-ity, interoperability, and system heterogeneity are addressedby the adoption of a standard componentmodel such as SCA.

With respect to the reconfiguration capabilities, most ofthe considered component frameworks consider only staticreconfiguration support. It means that an application mustbe stopped before reconfiguration is accomplished and thenrestarted. This is the case of SCA standard itself, whichrelies its configuration capabilities on XML configurationfiles and an architecture description language (ADL) thatdefines the application components interfaces and bindingsat design time. But as far as authors know it does notdefine (nor does, e.g., Wright, Darwin, or ACME [18]) anyruntime environment for managing its architectural entitiesduring runtime. Opposite to that, some other component

International Journal of Distributed Sensor Networks 3

Table 1: Related work feature comparison.

Statefulreconfiguration Distributed state Resource

management Standard component model

Li [4, 6] Yes (functional) No — NoGarcıa-Valls [9, 21, 22] No No Yes NoRomero and Garcıa-Valls [23] Yes No Yes NoAlmeida et al. [8] No No Yes NoSLAstic [24, 25] No No Yes NoSchmidt et al. [26–28] No No Not native YesOrlic et al. [31] No No Yes NoGCM [32] Yes No No No

Hammer and Knapp [34] Yes (appspecific) Yes No No

DAMP (this work) Yes Yes Yes Yes

models provide support for dynamic reconfiguration, forexample, SOFA [19] or FraSCAti [20], which exploits itsfractal nature to provide a dynamically reconfigurable SCAimplementation. However, regarding QoS support, none ofthese general purpose component frameworks offered itnatively.

None of the analyzed general purpose component frame-works offer native QoS support. Thus, the rest of this sectionrevises several research works that are focused on the QoSand resource management of dynamically reconfigurablesystems. Generally, these works usually consider abstractcomponent models oriented to the simulation and testingof reconfiguration strategies. Table 1 summarizes the mainfeatures of these related works.

In [4, 6], Li provides a good state of the art focused onthe QoS assurance during the reconfiguration of componentbased systems. In this case, QoS assurance is meant toreduce “application disruption” to the minimum during thereconfiguration phase. In other words, the objective is toavoid, as far as possible, that a reconfiguration could affectthe ongoing transactions, minimizing as well the systemdowntime during reconfiguration. To support stateful recon-figuration, it introduces the concept of versioning; that is, oldand new components are allowed to coexist at the same timeduring reconfiguration. This approach is not valid for ourneeds as far as it covers only planned reconfigurations, andthe problem of stateful system reconfiguration due to faultsas, for example, node crash is left open. On the other hand, Liproposes sharedmemory to improve the timing performancewhen the size of the state to transfer is high. Although this isa high performance solution, it does not support distributedstate, which is a key issue when the state must be transferredfrom one component to its replica in case of node fault.

Other works [9, 21, 22] are focused on the optimizationof application graphs of service based applications. Theyconsider resource related QoS characteristics (CPU, mem-ory) as well as application specific QoS parameters (e.g.,frames per second or resolution in video applications). Inthe latter case, application specific components are needed in

order to assist the middleware in finding the most suitablesolution. These works consider reconfiguration of statelessservices in the context of an application and try to choosethe most suitable service implementation based on a set ofQoS characteristics, although the functional reconfigurationprocess of the service bindings (links) is not detailed. Aresource manager is presented, which is able to manageevents like overload or node failures, although the specificmechanisms to detect such faults are not detailed. In [21],a composition algorithm based on a QoS characterizationsimilar to DAMP is presented, although it only considersstateless services. The selection of service implementations isoptimized according to different criteria (e.g., application endto end deadline), but nevertheless, the approach is different toDAMP. For the same functional service, it considers differentservice implementations to represent different QoS demands.These service implementations, once deployed, have a fixedset of QoS parameters. Opposite to that, DAMP considerselastic QoS parameters for stateful components, which canbe tuned or configured at runtime depending on the specificresource needs or availability. In [22] the iLaser componentmodel is presented, which includes a control port for QoSreconfiguration purposes. In this sense, DAMP follows asimilar approach but extends the control port to support statetransfer. Additionally, DAMP adopts a standard componentmodel (SCA) and thus can integrate existing legacy code ina relative easy way and can benefit from existing protocolbindings. Such issues are not covered by iLaser. In [23] statefulapplications are considered in the context of a Java basedcomponent framework: while local stateful reconfiguration issupported, it does not address the distributed state problem,which is left open.

In [8] the concept of local utilization bound (LUB) sup-ports online schedulability analysis due to its low compu-tational cost. It achieves such good performance because itallows reconfiguration limited to a well-known set of systemconfigurations that have been previously analyzed offline. Itis task oriented and thus does not consider the notion ofcomponent, but it has served as an inspiration for DAMP,


Appcomponents

Middleware daemon

“Launches”

Node 1

Node 2

Systemregistry

Middleware manager

Appcomponents

Middleware daemon

“Launches”

Node 3 Node n

Appcomponents

Middleware daemon

“Launches”

· · ·

Figure 1: System deployment model.

which takes some of the ideas and leverages them to considerinterdependent component based distributed applications.

The SLAstic component framework is presented in [24,25], which adopts the Palladio component model and con-siders three aspects of dynamic reconfiguration: resource(node) allocation/deallocation, component migration, andload (un) balancing. Its objective is to optimize existinghardware resources (either avoiding underutilization in caseof small work or load balancing in situations of stress),while the SLA (service level agreement) is met. The resourcemanagement algorithm functionality is specified, although itis not detailed. No specific mechanism to support distributedstate transfer is detailed.

In [26–28] a CORBA (Lightweight CORBA CCM com-ponent model) based middleware is presented. It allocatesphysical (CPU, memory, and network) resources to applica-tion components. It depicts a multilayer resource manager(MLRM) that is composed of several modules which pro-vide similar functionalities to DAMP, for example, resourcemonitoring, admission control, or composition algorithm.It considers QoS aspects such as survivability, predictability,or security but does not detail the low level mechanisms toachieve such characteristics. Through a resource manage-ment API detailed in [26], client applications can ask forresource availability and reservation.Nevertheless, it does notnatively provide a specific resource management algorithm,which could be integrated by the user. On the other hand,CORBA is not being extensively used in the industry, and itsapplication scope is quite small [29]. In [30] an improvementof the CORBA D&C standard is presented, centered in thedeterministic deployment and activation of CORBA basedapplications, although it does not support any dynamicresource management.

Another component framework with infrastructure reso-urce management support is presented in [31]. It considersservice resource demand contracts to manage the availableinfrastructure resources. Due to its service oriented nature,stateful components are not considered.

In [32], a framework based on Grid Component Model(GCM) is presented. GCM is an adaptation of fractal for

large scale distributed computing. Similar to DAMP, it isbased on loosely coupled active objects called “autonomouscomponents,” each allocated to an independent Java virtualmachine. It does not support QoS related issues and doesnot detail any specificmechanism to support distributed statetransfer in case of node faults.

Reference [33] details a real time oriented componentframework in which sequential execution is orchestratedby a central manager. It provides determinism and reducesoverhead and component coupling, but it does not supportdynamic reconfiguration.

In [34] the state transfer is application specific; thus theapplication components must provide methods to let themiddleware get a component state and inject it into its replica.A similar approach is presented in [35]. In this sense, DAMPprovides a generic way to let a component write its state intothe middleware database and also to read from it when acomponent replica must be instantiated and initialized.

3. Design of Distributed ApplicationsManagement Platform (DAMP)

Figure 1 depicts the overall system deployment model [16],containing several applications running in a set of distributednodes. Both the middleware components and the applicationcomponents are represented. Different application compo-nents are represented with different colors. Over an infras-tructure of 𝑛 nodes, there is one middleware daemon com-ponent for each node that allocates application components.This daemon receives orders from the middleware manager,who is responsible for the whole system management.

The different application components, alongside theirQoS specifications and node allocation, are registered in thesystem registry database. The middleware manager receivesmonitoring information about the components state from thedistributed daemons and stores it in the database. This infor-mation can be used to implement fault tolerancemechanismswith stateful system recovery support.


MDD tool

Node n

“Execution control” “Launches”

DAMP platform management services

Applicationcomponent

Middlewaremanager

Middlewaredaemon

“Component launch”

Business service Platform port (service)Platform port (reference)

Business reference

“State monitoring”

“Registers applications”

Property

Managementconsole

“Execution control”


“QoS configuration”

Figure 2: Links between an application component and the middleware.

3.1. DAMP Architecture. The interrelationship between themiddleware components and the application components isdepicted in Figure 2. The middleware manager integratesthe system registry database and exposes several servicesto manage the system. The applications, alongside theirlogical and physical components explained in Section 4,are registered in the system database through the registryservice exposed by themiddlewaremanager.Themiddlewaremanager also provides a monitoring service to receive thestate information of the distributed components, which issent by the middleware daemons. The execution controlservice provides a way to control the application instantiationand execution. Finally, through theQoS configuration service,the application components can issue self-QoS (applicationtriggered) change requests.These services are shown in red inthe left side of the middleware manager, and their interfacesare described in Section 3.2.

The execution control operations are transferred either tothe middleware daemons (component launch reference) or tothe application components (init, start, stop, and QoS changeoperations) through the manager right side execution controlreference (see Figure 2). An application component transfersits state information to its middleware daemon through thestate monitoring reference. The middleware daemon packsthe info from all components inside its node and sends it tothe middleware manager, which updates the system databaseaccordingly.

To reduce to the minimum network bandwidth andlatency due to middleware internal communications, a highperformance SCA binding over Data Distribution Service(DDS) was designed and implemented in [12]. All the mid-dleware related communication between the platform portsis performed over the DDS binding.

3.2. DAMP Services. As stated in previous section, DAMPprovides several services to the system operator and therunning applications. The registry and the execution controlservices can be used by the system operator to register newapplications into the system and manage them at runtime.The monitoring service is for internal use; that is, it is usedby the middleware for fault tolerance and QoS managementpurposes. Finally, despite the QoS configuration service theapplications can trigger functional self-QoS reconfigurationevents. Next, these platform services are explained in deeperdetail.

3.2.1. Registry Service. This service is oriented to the systemadministrator and provides a way to register the applicationsalongside their components into the system registry database.In order to ease the integration with external client appli-cations such as system management consoles or MDE toolssuch as the one described in [36], it is exposed as a SOAPweb service, although it could be exposed over any other


SCA binding available (e.g., REST, RMI, or JMS).The registryservice is represented by the following interface:

public interface IRegistration {

String InstallApp (String name, boolean syn-chronized);String InstallLogicalComponent (String appID, String name, int 𝑇[], int 𝐷[], int 𝑃, intstartOrder, int stopOrder);String InstallPhysicalComponent (String lc ID,String name, String node ID, String path, intWCET[]);int RegisterNode(String name);int DeleteApp(String app ID);

}

The registration process starts with the registration ofan application, indicating if the application start and stopoperations must be synchronized or not. If the applicationis declared as synchronized, each application component isstarted in the order specified by the startOrder and stopOrderarguments. For each application logical component, one ormore physically deployed components can be available, asreplicas for fault tolerance or load balancing purposes. Theconcepts of logical and physical components can be assimi-lated to the terms service and service implementation utilizedin [9], with some differences. In the scope of this work, theterm logical component differs from the concept of service, asfar as it can be stateful. Also, a service implementation has afixed execution time once deployed in a specific node, whilea physical component is characterized by a vector of possibleexecution time discrete values. Section 4 details this systemmodel characterization.

3.2.2. Execution Control Service. This service enables theinstantiation, initialization, start, stop, and destroy operationsover an already registered application, as well as QoS recon-figuration of specific application components. It is built uponthree different services, namely, (1) a service implementedby the middleware manager, (2) a service implemented bythe middleware daemon, and (3) a service implemented bybase class fromwhich all application components inherit.Thefirst one can be accessed by the system administrator, forinstance, through an external client management console. Asthe registry service, it is exposed as a SOAP web service. Theinstantiation of each application component is performedthrough themiddleware daemon through the second service,which actually launches the Java process that allocates thecomponent. Afterwards, the middleware manager directlyinvokes the third service, which is implemented by thecomponent base class and defined by the IControl interface:

public interface IControl {int Init(String pc ID, QoSParams qos);int InitWithState(String pc ID,QoSParams qos,byte state[]);int SetQoSParams(QoSParams qos);int Start();

int Stop();int Destroy();

}

This service actually performs the initialization, start,stop, and destroy operations over an application physicalcomponent. Also, dynamic component QoS reconfigurationcan be performed invoking the SetQoSParams function ofthis service, which accepts as an argument a QoSParams datastructure containing the period, deadline, and execution timevectors of the component.

3.2.3. Monitoring Service. The monitoring functionality isdistributed between two different services. One is imple-mented by the middleware daemon and is invoked bythe application components residing in the same node,which sends their monitoring information to their localhostmiddleware daemon. The other service is implemented bythe middleware manager and invoked by the distributedmiddleware daemons, which collect the monitoring infor-mation of their application components and resend it to themanager. This monitoring data includes liveliness, state, andactual execution time and response time information and isperiodically updated by the application components. Thus,this service enables stateful system recovery in case of nodefailure or component malfunction detection.

3.2.4. QoS Configuration Service. An application can askfor a self-QoS change request through this service. Thisis considered an application triggered QoS reconfiguration,and it is performed in two steps. First, in response toan application specific event, a self-QoS change request isissued to the middleware manager through the IQoSConfiginterface:

public interface IQoSConfig {

intQoS ChangeRequest(String app ID, int𝑇[],int 𝐷[], int 𝐶[]);

}

where the 𝑇,𝐷, and𝐶 function arguments represent here theordinal values of the respective 𝑇, 𝐷, and 𝐶 vectors of eachof the components belonging to the application. The QoSchange request is thus defined by a set of specific values of𝑇, 𝐷, and 𝐶 for all the components of the application.

Once the new desired QoS parameters are defined, theQoS aware reconfiguration algorithm exposed in Section 5searches for an alternative system configuration to satisfy theapplicationQoS change request. To find a new feasible systemconfiguration that meets the overall system requirements, themiddleware manager could decide to reconfigure the 𝑇, 𝐷,and𝐶 values of those running components which have elasticQoS parameters. This second step can be performed thanksto the IControl service explained in Section 3.2.2, which isimplemented by the components base class. As an example,consider a video surveillance system thatmust increase its fps(frames per second) parameter in case an intrusion detectionalarm is fired. In absence of alarms, the video component isconfigured at 10 fps, in order to reduce CPU consumption to



Property

Business service

Platform service(IControl)

Business reference

Platform reference(IMonitor)

Platform reference(IQoSConfig)

Figure 3: Application component interfaces.

a reasonableminimum. But in case of intrusion detection, thesystem is programmed to automatically increase video qualityto 25 fps.Therefore, the component asks for a self-QoS changerequest, more specifically a change in its execution period,from the initial 100ms corresponding to a 10 fps quality, tothe required 40ms (25 fps). This implies an increase in theresource demand for the application and must be properlymanaged by the QoS reconfiguration algorithm explained inSection 5.

4. System Model and QoS Characterization

4.1. DAMP Component Model. A DAMP component is basi-cally a SCA component that has been extended with severalcontrol ports (one service and two references) to enable thecontrol and monitoring features of the middleware. EveryDAMP component inherits from a base class these controlports. Thus the application developer can focus on theimplementation of the functional aspects of the component,depicted in Figure 3 as business services and references,following the SCA terminology.

The base class of a DAMP component implements theIControl service to let the middleware control the executionof the component as well as inject the component state andtune its QoS parameters (𝑇, 𝐷, and 𝐶). It should be notedthat𝐶 depends on the specific hardware node the componentis executing on. Moreover, the component could be asked toexecute in different QoSmodes that imply different 𝐶 values.For instance, a video component could be configured in highversus low resolution mode.

On the other hand, a DAMP component utilizes theIMonitor reference to communicate its internal state tothe middleware, as explained in Section 3.2.3. Finally, theIQoSConfig reference is available to ask for eventual func-tional (application triggered) self-QoS reconfiguration.

Regarding the dynamic model, the state machine of aDAMP component is shown in Figure 4.Through the registryservice (see Section 3.2.1), a component is registered in thesystem registry database and deployed in a specific node.Then, it can be launched (instantiated and loaded inmemory)by the middleware daemon in response to an order issuedthrough the execution control service (see Section 3.2.2)exposed by the middleware manager. Once loaded, it canbe initialized with specific state and set of QoS parameters.Finally, from this state, the component can be started.

When a periodic component is started, a cyclic executionof the activity diagram shown on the right side of Figure 4

is initiated. This loop is executed with the period that hasbeen specified in the initialization stage and exits whenthe middleware kindly asks the component to stop throughthe MustStop flag. This flag allows for application drivenresource disposal, but the middleware also provides analternative way to force the emergency removal of misbehav-ing components. This emergency exit is performed by themiddleware daemon through operating system services.

The loop of the periodic thread first executes the func-tional code of the component invoking the computeFunc-tionalCode abstract function from the base class, whichcomputes the outputs from the available inputs that havebeen issued to the component through the componentbusiness services pointed out in Figure 3. Then, the threadinvokes thewriteOutputs abstract function,which invokes thecomponent business references if any. Both abstract functionsare implemented by the component developer. While theinput variables of the component are provided through itsbusiness services, the output variables are mapped to thecomponent references through the writeOutputs functionimplementation. It should be noted that both mappings arelikely to be automatically generated by aMDD tool (see [36]),although this aspect is not covered in this work.

4.2. System Model. The system model for QoS character-ization presented in [16] has been refined to include theconcept of logical component. As stated in Section 3.2.1, [9]proposes a similar approach with the concepts of service andservice implementation, although here the concept of logicalcomponent is inherently stateful and the QoS parameters canbe tuned between certain admitted values. In the scope ofthis work, a logical component is a functional componentthat can be deployed in one or more physical nodes. Oncedeployed, it becomes an actual physical component. When alogical component is deployed in multiple nodes, it is said tobe a replicated component. This eventual redundancy can beconsidered for fault tolerance or load balance purposes.

The system registry database represented in Figure 1stores theQoSmodel of the system. It considers a set of nodes𝑁 (see (1)) that allocate a set of distributed applications𝐴 (see(2)).The cardinality of sets𝑁 and𝐴 is denoted by |𝑁| and |𝐴|,respectively. An application 𝑎𝑘 is composed of a set of logicalcomponents (see (3)) belonging to LC (see (4)), which is theset of all logical components in the system:

𝑁 ≡ {𝑛𝑖 (𝑢𝑖) , 𝑖 = 1, . . . , |N|} , (1)

where 𝑢𝑖 is the actual node utilization factor,

𝐴 ≡ {𝑎𝑘, 𝑘 = 1, . . . , |A|} (2)

{𝑎𝑘}A = {[lc𝑘,1, lc𝑘,2, . . . , lc𝑘,𝑛𝑘]}A

lc𝑘,𝑗 ∈ LC ∧ 𝑗 = 1, . . . , 𝑛𝑘,

(3)

where 𝑛𝑘 is the number of components of application 𝑘 andLC is the set of logical components (see (4)),

LC ≡ {lc𝑘,𝑗 (T𝑘,𝑗,D𝑘,𝑗, 𝑃𝑘,𝑗, 𝑠𝑘,𝑗) , 𝑘 = 1, . . . , |A| , 𝑗

= 1, . . . , 𝑛𝑘} .

(4)


stm StateMachine_english

Thread executionInitial

INITIALIZED_MustStop: boolean = TRUEentry/Stop

EXECUTION (RUNNING)_MustStop: boolean = FALSEdo/runentry/Start

REGISTERED LOADED

SCA runtimeloaded into

a JVM process

Executionthread

is stopped

Active objectwith

its own thread

Two abstract functions to be implemented by the component developer:

(i) ComputeFunctionalCode()(ii) WriteOutputs()

These abstract functions are invoked from the periodic run() function in the base class

Compute functional

code

ActivityInitial

_MustStop

Write outputs

ActivityFinal

INIT

START

[FALSE]

LAUNCH

TERMINATE

STOP

INIT [TRUE]

TERMINATE

Figure 4: State machine of a DAMP component.

Each logical component has its own QoS parameters(period 𝑇, deadline 𝐷, and priority 𝑃). 𝑃 denotes the QoSpriority (criticality), established by the system operator atthe configuration stage, and affects the local feasibility testsperformed by the QoS reconfiguration algorithm describedin Section 5. T and D are vectors with a set of permissiblediscrete values, representing the period and the deadline of acomponent (see (5)-(6)). Finally, 𝑠 denotes the state in case ofstateful components. This state is also stored in the databasefor fault tolerant stateful recovery purposes:

{T𝑘,𝑗}LC = {[𝑇1

𝑘,𝑗, 𝑇2

𝑘, . . . , 𝑇

𝑡𝑘𝑗

𝑘,𝑗]}

LC(5)

{D𝑘,𝑗}LC = {[𝐷1

𝑘,𝑗, 𝐷2

𝑘, . . . , 𝐷

𝑡𝑘𝑗

𝑘,𝑗]}

LC, (6)

where 𝑡𝑘𝑗 is the length of T𝑘,𝑗 and D𝑘,𝑗 vectors. Each possiblevalue pair of 𝑇𝑥

𝑘,𝑗, 𝐷𝑥𝑘,𝑗

represents a different QoS demand forthe component lc𝑘,𝑗. When deadline is equal to the period,𝐷𝑥

𝑘,𝑗= 𝑇𝑥

𝑘,𝑗, ∀𝑥 = 1, . . . , 𝑡𝑘𝑗. The length of T𝑘,𝑗 and D𝑘,𝑗

vectors depends on the specific application. For instance,consider a video component with two possible values forthe T ([40, 100]) and D vectors ([20, 50]). Each value pairrepresents a video quality of 25 and 10 fps, respectively (25 fpscorresponds to a 𝑇 = 1/25 seconds = 40ms). In this caseT and D vectors have been configured differently to providehigher quality in terms of frame interarrival jitter. To limitthis jitter, a deadline equal to half period has been defined.Thus, the two possible QoS demand levels of the componentcorrespond to the following value pairs:

QoS level 1 (low): 𝑇 = 100, 𝐷 = 50 → 10 fpsQoS level 2 (high): 𝑇 = 40, 𝐷 = 20 → 25 fps.

Each logical component can be deployed in one or morehardware nodes (e.g., for fault tolerance purposes). Oncedeployed, it is redefined as a physical component and acquiresan extra QoS parameter associated with the specific nodein which it is deployed, that is, the execution time WCET(see (8)), which is also a vector with a set of possible valuesrepresenting different levels of QoS demand. For instance,the previous video component could be configured to displayvideo in different resolutions, each one associated with adifferent execution time.

Additionally, actual physical component response timeis monitored at runtime and stored at the database, thusenabling deadline loss monitoring and component malfunc-tion detection.

The set of physical components PC is represented by (7),where pc

𝑘,𝑗,𝑖denotes a physical component corresponding to

a logical component lc𝑘,𝑗 deployed in node 𝑛𝑖:

PC ≡ {pc𝑘,𝑗,𝑖

(lc𝑘,𝑗,WCET𝑘,𝑗,𝑖,Rt𝑘,𝑗,𝑖, 𝑛𝑖) , 𝑘 = 1, . . . , |A| , 𝑗

= 1, . . . , 𝑛𝑘, 𝑖 = 1, . . . , |N|}

(7)

{WCET𝑘,𝑗,𝑖}PC = {[WCET1𝑘,𝑗,𝑖

,WCET2𝑘,𝑗,𝑖

, . . . ,WCET𝑤𝑘𝑗𝑖𝑘,𝑗,𝑖

]}PC

, (8)

where Rt𝑘,𝑗,𝑖 is the response time of pc𝑘,𝑗,𝑖

monitored online(in theory equal to the value computed by the compositionalgorithm) and 𝑤𝑘𝑗𝑖 is the number of possible values for theWCET of the physical component 𝑐𝑘,𝑗,𝑖 running in node 𝑛𝑖,each one representing a different level of QoS demand [8].These WCET vectors can also be used for safety certificationpurposes [37].

It should be noted that the allocation of logical com-ponents to specific nodes in the registration phase (see


Class QoS Management

RTA

- InterferenceOfHp(int, int, List<PComp_Specific>): int+ IsSchedulable(List<PComp_Specific>): int

“Interface”ISchedulabilityTest

+ IsSchedulable(List<PComp_Specific>): int

QoS manager

~ _hmApps: HashMap<String,App> = new HashMap<Str...~ _hmNodes: HashMap<String,Node> = new HashMap<Str...

+ LaunchApp(App): boolean+ LaunchLogicalComp(LComp): void

Composition algorithm

+ Launch(List<LComp>): boolean

Figure 5: Architecture of the QoS manager.

Section 3.2.1) implicitly considers other parameters that arerelated to infrastructure. For instance, if a component pro-vides encryption services, it should be allocated to a hardwarenode that supports such kind of encryption through, forexample, a TPM (Trusted Platform Module). Other infras-tructure related aspects could include operating systems,special hardware (e.g., webcams, sensors, and field busescards), or memory.

5. QoS Aware Resource Management

5.1. Resource Manager Design. An initial version of theDAMP QoS manager was introduced in [16]. In this paper,the QoS management process is explained in deeper detail.The problem formulation has been extended to cover the endto end latency of the applications and the QoS interdepen-dencies between the applications running in the system.

The QoS assurance of the applications that are runningin the system is directly related to the management of theresources provided by the infrastructure nodes. The mid-dleware manager includes a resource manager that allocatesavailable resources to the applications, taking into accounttheir QoS demand and priority (criticality). This means thatif a high priority component must be launched in a specificnode that already allocates a lowpriority running component,the resources assigned to that lower priority component couldbe reduced in case of shortage of CPU resources. In thissense, the lower priority component could suffer a controlledQoS degradation inside its allowed values of 𝑇 and 𝐶. In themost extreme case, a controlled shutdown of the low prioritycomponent could be necessary.

The reconfiguration events that trigger the execution ofthe resource manager can be either functional or nonfunc-tional. The events triggered by the system operator or by

an application (e.g., start/stop operations or component self-QoS change requests) are considered functional events. Thenonfunctional reconfiguration events are triggered by themiddleware itself, in response to QoS loss detected by themonitoring service. These events are usually due to non-planned events such as node crash or application componentmalfunction (e.g., an application component consumesmuchmore resources than those specified in its registration).

In face of both functional and nonfunctional reconfigura-tion events, the platform follows a predefined QoS manage-ment policy to allocate the available resources to demandingapplications. This policy is implemented in the compositionalgorithm, which can be interchanged to meet specific needs,for example, optimize the energy consumption, or maximizethe system performance.The default implementation consid-ers the applications criticality (QoS priority) when proposinga configuration for the components QoS parameters (𝑇,𝐶). The feasibility of the proposed configuration is verifiedthrough a local schedulability test. Figure 5 depicts the relationbetween the resource manager, the composition algorithm,and the schedulability test modules. In this work the schedu-lability test is based on the response time analysis (RTA) [38].

5.2. Proposed Composition Algorithm. A simplified activitydiagram of the process following a system reconfigurationevent consisting of an application (𝑎𝑘) launch is depictedin Figure 6. First, the QoS manager must check, dependingon the QoS demand and priorities of all the applicationsrunning in the system, the feasibility of a new systemconfiguration that could allocate the new application. At thispoint, different configuration possibilities exist dependingon the specific composition algorithm rules. For instance,solutions tomaximize the number of nodes in stand-by couldbe sought, according to an energy saving mode. Opposite to


Act dynamic view

Reconfiguration event (QoSreconfiguration request)

Try to find a feasible system configuration that

Reject theapplication

Launch new

Reconfigure QoS

QoSreconfiguration

needed?

Not found

Found Noapp ak

includes the new app ak

Figure 6: Composition algorithm.

that approach, the objective of a load balancing algorithmcould be an even utilization of the available nodes.

In the best case, the new application could be startedwithout reconfiguring the QoS of existing applications. Inthe worst case, the application could be rejected due to lackof resources. In the middle, a compromise solution couldbe found, lowering the QoS of lower priority applicationsto an affordable level. The job of the composition algorithmis deciding if the new application may be launched at therequired level of QoS and, if so, finding a feasible QoSconfiguration for the overall system in the minimum time.

To achieve this, the composition algorithm performs theprocess detailed by Algorithm 1. To better understand thealgorithm, the following key considerations should be noted:

(1) The application to be launched is defined as a setof logical components with a required set of specificvalues for their QoS parameters, denoted by thescalar values 𝑇, 𝐷, 𝑃. The WCET value for thephysical components associated with these logicalcomponents has also a desired scalar value.

(2) The feasibility test (RTA) is executed in the contextof a specific node and takes into account all thealready active (running) components in the node andthose components marked as “requested for launch”by the composition algorithm. The feasibility testreturns the estimated node utilization factor if thenode configuration is feasible, or a “nonschedulable”error code if not.

(3) The eventual QoS reconfiguration of componentsthat are already running is performed in a strictorder of QoS priority, performing a controlled QoS

degradation, or even a shutdown, of the less criticalapplications.

(4) The desired QoS parameters for the application com-ponents to be launched implicitly define the max-imum affordable end to end deadline required forthe different processing chains of the application. Tobetter understand this, consider the example videoapplication 𝑎𝑘 depicted in Figure 7, which is deployedover four hardware nodes and is composed of fivelogical components:

{𝑎𝑘}A = {[lc𝑘,1, lc𝑘,2, lc𝑘,3, lc𝑘,4, lc𝑘,5]}A . (9)

Suppose that the set of logical components is defined as

LC ≡ {lc𝑘,𝑗(T𝑘,𝑗,D𝑘,𝑗, 𝑃𝑘,𝑗, 𝑠𝑘,𝑗), 𝑗 = 1, . . . , 5} ≡ {

lc𝑘,1([30, 90], [30, 90], 5, −), (ImageCapturer)lc𝑘,2([40, 100], [40, 100], 5, −), (ImageProcessor)lc𝑘,3([500], [500], 5, −), (IntrudeDetector)lc𝑘,4([20, 40], [20, 40], 5, −), (ImageViewer)lc𝑘,5([40, 100], [40, 100], 5, −), (ImageLogger)}

while the set of physical components is

𝐶 ≡ {𝑐𝑘,𝑗,𝑖(lc𝑘,𝑗,WCET𝑘,𝑗,𝑖,Rt𝑘,𝑗,𝑖, 𝑛𝑖), 𝑗 = 1, . . . , 5, 𝑖 =

1, . . . , 4} ≡ {

𝑐𝑘,1,𝑛1(lc𝑘,1, [25, 30],Rt𝑘,1,𝑛1, 𝑛1),𝑐𝑘,2,𝑛2(lc𝑘,2, [30, 40],Rt𝑘,1,𝑛2, 𝑛2),𝑐𝑘,2,𝑛3(lc𝑘,2, [20, 30],Rt𝑘,2,𝑛3, 𝑛3),𝑐𝑘,3,𝑛3(lc𝑘,3, [10],Rt𝑘,3,𝑛3, 𝑛3),𝑐𝑘,4,𝑛4(lc𝑘,4, [20, 25],Rt𝑘,4,𝑛4, 𝑛4),𝑐𝑘,5,𝑛4(lc𝑘,5, [1, 25],Rt𝑘,5,𝑛4, 𝑛4),

}

In this case the worst case end to end deadline for theprocessing chain that starts at the ImageCapturer componentand finishes at the ImageViewer is calculated as

E2Emax (lc𝑘,1, lc𝑘,2, lc𝑘,4)

= Max ([30, 90]) + Max ([40, 100])

+ Max ([20, 40]) = 90 + 100 + 40 = 230ms,

(10)

namely, with the lower QoS parameters in its admitted rangeand obviating the network transmission time (which is notmodeled here). On the other hand, the minimum end toend deadline corresponding to the best QoS configurationpossible is

E2Emin (lc𝑘,1, lc𝑘,2, lc𝑘,4)

= Min ([30, 90]) + Min ([40, 100])

+ Min ([20, 40]) = 30 + 40 + 20 = 90ms.

(11)

Taking into account these considerations, Algorithm 1depicts the proposed composition algorithm pseudocode.


/∗STEP 1∗/S alone = createPhysicalComponentSubset(ak, alone, not replicated)foreach physical component PC in S alonePC. requestedForLaunch = truesetQoSConfigForNode(PC) /∗initial QoS config∗/doif RTA.IsSchedulable(PC)

PC. feasible = truesaveQoSConfigForNode(PC)exit while

end ifwhile QoS alternative configuration exists for node(PC) /∗decrease QoS∗/if not PC. feasible and not PC. hasReplica → rejectApplication

end foreach/∗STEP 2∗/S not replicated = createPhysicalComponentSubset(𝑎𝑘, not alone, not replicated)foreach physical component PC in S not replicatedPC. requestedForLaunch = true/∗exclude replicas in this node that already have feasible alternatives in other nodes∗/excludeFeasibleComponents(S alone, PC. node)setQoSConfigForNode(PC) /∗initial QoS config∗/doif RTA.IsSchedulable(PC)


end ifwhile QoS alternative configuration exists for node(PC) /∗decrease QoS∗/if not PC. feasible → rejectApplication

end foreach/∗STEP 3∗/S replicated not alone = createPhysicalComponentSubset(𝑎𝑘, not alone, replicated)foreach physical component PC in S replicated not alonePC. requestedForLaunch = true/∗exclude replicas in this node that already have feasible alternatives in other nodes∗/excludeFeasibleComponents(S alone, PC. node)setQoSConfigForNode(PC node) /∗initial QoS config∗/doif RTA.IsSchedulable(PC)


end ifwhile QoS alternative configuration exists for node(PC) /∗decrease QoS∗/if not PC. feasible → reject application

end foreach

Algorithm 1: Composition algorithm.

This algorithm tries to optimize the search time for a feasiblesolution while implementing the discussed QoS prioritybased resource management policy. The algorithm performsthree steps sequentially.

In the first step (STEP 1), the S alone subset is calculated.It includes the application physical components, replicated ornot, that do not have neighbor components belonging to thesame application; that is, in the scope of the application, theyare isolated in a specific node. Every component belongingto S alone is marked as “requested for launch” and feasibilityof each of them is verified. This way, if any of them is not

feasible, the process stops in an early step and the applicationis rejected, thus optimizing the algorithm computation time.To verify the feasibility of these components, an RTA schedu-lability test is executed for each node that allocates such com-ponents, taking into account the currently active components(belonging to other applications) plus the new “requested forlaunch” component. If the first RTA schedulability test fails,it means that a QoS reconfiguration for (some of) the currentactive components would be necessary. This is only possibleif the QoS priority of such components is lower than theQoS priority of the component to be launched. In such case,


Image processor

Intruderdetector

Image logger

Image processor

Image capturer

Image viewer

Node n1 Node n2

Node n3

Node n4

ck,1,n1ck,2,n2 ck,4,n4

ck,2,n3

ck,3,n3

ck,5,n4

Figure 7: Application composed of five logical components (one ofthem having two physical components).

a new QoS configuration is proposed, decreasing the QoS ofthe lowest priority components.

This loop is executed until a feasible configuration isfound or the space of QoS reconfiguration possibilities isexhausted with no success. In case of replicated componentsbelonging to S alone, if more than one is feasible, the oneallocated to the node with lower utilization factor is markedfor final instantiation. On the other hand, if a feasible solutionis not found for a replicated component belonging to S alone,it could still be feasible in another node (see STEP 3).

In the second step (STEP 2), the subset of componentsto analyze is S not replicated, which includes those notreplicated components that do not belong to S alone. Toverify the feasibility of a physical component PC belongingto S not replicated, first the components that have replicas inthe node that allocates the PC component are excluded if theyalready have an alternative replica marked as feasible in stepone. From here, the algorithm follows about the same way asstep one.

Finally, in the third step (STEP 3), the replicated compo-nents that still are marked as not feasible are considered.Thisset (S replicated not alone) takes into account the results ofstep one and includes those replicas that could not bemarkedas feasible in such step. If more than one replica was available,the same load balance criterion is followed, selecting the nodewith lower utilization factor.

6. Stateful Reconfiguration Support

To achieve the required level of fault tolerance, DAMPprovides a system monitoring and recovery mechanismthat enables the system stateful reconfiguration in face ofunexpected component or node failures. This mechanismreduces significantly the system downtime and thus increasesthe overall system availability. For instance, in the automaticwarehouse presented in Section 7 it is quite usual to suffersoftware component crashes that can derive in a time con-suming recovery process if historical system state data is not

available. If the system state at the time of crash is not saved,several problems arise due to the absence of sensor data suchas location of pallets of goods or conveyors. Figure 8 depictsthe system monitoring and recovery mechanism.

Through the IMonitor platform reference port (see Fig-ure 3), the application components send periodically theirinternal state to their local middleware daemon.Themiddle-ware daemon, in turn, packs the states of all the applicationcomponents running in its node and resends this monitoringinfo to the middleware manager, which finally stores itinto the system database. This solution reduces networkbandwidth and increases system performance, as far as theDDS nature of the DAMP internal communications allowsthe optimization of the network throughput through thebatching QoS profile [11].

When an infrastructure node crashes, the applicationcomponents that were running on it can be restored if theyare replicated in another node. First, themiddleware resourcemanager decides which of the available replica must berestored and then initializes it with the desired initial state.

The state of the application components is modeled as anarray of bytes. This makes the mechanism generic enoughto support any application specific state without the need ofspecific ad hoc components as in [34]. Obviously, this statecan be injected later into components capable of interpretingits format, which is the case of component replicas consideredin this work.

7. Case Study

In order to evaluate and validate the suitability of the pro-posed approach, the supervision of an automated warehousehas been chosen as a case study.

An automated warehouse is a highly distributed environ-ment composed of a number of both sensor and actuatorelements. Conveyors and automatic vehicles move the loads(usually pallets) all along the warehouse, from the receptionbays to the storage areas, waiting to be packed again and sentout of the warehouse to the final destination.

Itmust be remarked that in order to have high automationlevels, the need of capturing runtime information of thesystem clearly grows up. This is achieved by incorporatingmore and more sensors all along the distribution lines ofthe warehouse: state of the pallets (location, weight of thedistribution goods) and also the state (position, speed) of theconveyors and automated vehicles.

All these components are organized and coordinatedby control algorithms focused on the logistic needs of thesystem, as the main functional requirement of the system.Therefore, this main requirement is resolved by defining anadequate system architecture built upon the set of hardwareand software components, which defines the logistic applica-tion functionality of the whole system.

However, there are also other nonfunctional require-ments related to the dynamic composition of the system thatmust also be addressed:

(i) scalability of the system: support the installation ofnew distribution lines or equipment in the warehouse


Node 1


Middlewaremanager

Middlewaredaemon

“Component restoration from state monitoring info”




Node n

Applicationcomponent Middleware

daemonApplicationcomponent


Figure 8: Stateful system recovery scheme.

without having a high impact in the existing soft-ware architecture. The proposed approach targetsthe dynamic distribution or modification of softwarecomponents supporting the newly installed equip-ment, incorporating them in the whole logistic appli-cation;

(ii) system reconfiguration: automated warehouses areusually designed for specific products. However, inthe lifetime of the warehouse, this can change accord-ing to different criteria such as seasonal aspectsand high or low supply demand. As the hardwarearchitecture is highly static and difficult to modify,the adaptation to these changes is addressed by sup-porting the runtime reconfiguration of the softwarearchitecture, with the proposed approach;

(iii) fault tolerance: a fault in any of the components ofthe warehouse usually stops the distribution lines.Thus, aiming at the high availability of the warehousesystem, a software adaptation based on redundantcomponents that can be dynamically launched ordeployed is also supported by the proposed approach.This feature, combined with the stateful reconfig-uration support detailed in Section 6, significantlyreduces the system downtime and enables bettersystem maintenance.

Figure 9 depicts the logical layout of the warehouse thathas served as a case study. The DAMP platform has beendeployed over 20 hardware nodes that allocate hundredsof software components. The nodes are deployed over thewarehouse LAN, and the interconnection of most of thesoftware components is performed through the SCA DDSsafety binding presented in [12]. Nevertheless, some legacycomponents that have been adapted to run on the platform

also provide a SOAP interface to the outside world (webservices).

Some of the nodes allocate components belonging todifferent applications, for example, the weighting machinemonitoring component and the conveyor belt controllercomponent. Other functional components are replicated indifferent nodes; for example, the transelevator components 1and 2 are redundant and run in two different hardware nodes(NODE 7 and NODE 8), for fault tolerance purposes.

The suitability of the DAMP platform for such scenariohas been validated for the following needs:

(1) System registration and application deployment: tensof component based applications have been registeredand deployed into the system.

(2) Application execution control: the applications arecorrectly initialized and started, provided the hard-ware infrastructure has sufficient resources.

(3) Stateful component upgrade: the middleware is ableto perform an operator driven component upgrade.First the component to be upgraded is stopped. Thenthe new version of the component is downloaded tothe appropriate hardware node and initialized withthe old components state. Finally the new componentis started.

(4) Fault tolerance with stateful recovery: to validate thisfeature, a hardware node that allocates a replicatedcomponent is shut down, simulating a node failure.The middleware monitoring service detects it andinvokes the resource manager composition algorithmto find a new feasible system configuration. Themiddleware starts a replica in an alternate nodeand initiates it with the last known component stateavailable in the database.


Middleware daemon Node 1

Node 2

Systemregistry

Middleware manager

Conveyor belt

Weighting

Middleware daemon

Node 3

RFID control

Middleware daemon

Node 7

Transelev 2

Transelev 1

Middleware daemon

Node 8

Transelev 2

Transelev 1

Figure 9: System deployment layout.

The following figures present some performance metrics thathave been recorded over the testing scenario. The DAMPplatform is currently implemented in Java, based on the opensource Tuscany SCA implementation. Although it has alsobeen tested over Linux, the figures presented here have beenobtained over a Windows 7 based deployment.

The central server runs over an Intel core i3 based work-station and the rest of the nodes are based on an industrial PCthat features an Intel AtomN2800 processor, 4Gb RAM, anda wide range of peripheral interfaces, including serial, CAN-bus, and Digital I/O, which enable the communication withthe different warehouse elements.

Figure 10 shows the component process load times fordifferent SCA bindings.Themeasured times include the loadof the Tuscany SCA Java runtime itself, which consumesmostof the time in the instantiation phase. As it can be seen,the developed DDS binding (RTI distribution) is much moreefficient than the Tuscany native web services binding, interms of component instantiation time.

Figure 11 shows the operation invocation times (inmicro-seconds) for applications composed of different number ofcomponents (one, five, and ten components). The operationsconsidered here are those managed by the middlewaremanager (init, start, stop, and kill operations). It should benoted that the operations are invoked point to point (unicast)for client server paradigm based bindings, as web services orREST. Thus, the invocation time for the whole applicationincreases almost linearly with the number of components.Conversely, the graph shows the great advantage provided bythe publish-subscribe paradigmwhen the number of compo-nents increases.This is represented by the DDS binding whenconfigured in multicast mode.

The influence of the state size in a stateful componentinitialization is depicted in Figure 12. In this case, a compo-nent is initialized by the middleware manager with differentstate size. The influence of the state is almost linear. Theresponse time has beenmeasured in themiddlewaremanagerfor different state sizes transferred over a DDS binding.


0

500

1000

1500

2000

2500

3000

DAMP component load (ms)DAMP infrastructure load (ms)

SCA binding WS binding DDS binding

Figure 10: Component load time for different available bindings.

0

1000

2000

3000

4000

5000

6000

1 component 5 components 10 components

Ope

ratio

n in

voca

tion

time

Number of application components

WS unicast (𝜇s)DDS multicast (𝜇s)DDS unicast (𝜇s)

Figure 11: Application response times for different number ofcomponents and different bindings.

Finally, Figure 13 shows the overhead of converting apure SCA component into a DAMP “compliant” component,depending on the specific binding support.

More specifically, the figures measure the instantiationtime of a component running over classical SCA Tuscanyversus a component with complete DAMP support. Theovertime is due to the instantiation of the control ports(both services and references) that every DAMP componentincludes. This fee is amply compensated by the extra featuresthat the DAMP platform provides, namely, dynamic systemreconfiguration support, QoS and resourcemanagement, andfault tolerance.

0

10000

20000

30000

40000

50000

60000

1000 2000 3000 4000 5000 6000

Initi

aliz

atio

n in

voca

tion

time

State size (bytes)

Response time (𝜇s)

Figure 12: Influence of the state size in a stateful componentinitialization.

0

500

1000

1500

2000

2500

3000

SCA binding WS binding DDS binding

Tim

e (m

s)

Pure SCA component loadDAMP component load

Figure 13: Cost of DAMP support versus pure SCA.

8. Conclusions and Future Work

This paper proposes a middleware solution that addressesseveral key needs of distributed systems composed of dynam-ically reconfigurable component based stateful applications.Such kind of systems share common characteristics withthe so-called sensor web systems or IoT, where aspects asdynamicity, heterogeneity, or resource management are keyissues and must be properly managed.

The proposed platform builds on top of a general purposecomponent model (SCA) and provides several services to thesystem operator and the running applications, which ease


the system deployment phase and enable the automatic sys-tem reconfiguration in face of external (application unaware)as well as internal (application driven) reconfiguration trig-gers.

The proposed QoS aware reconfiguration algorithm allo-cates the system resources to the applications depending ontheir QoS demand and priority (criticality), and the proposedfault tolerance mechanisms enable stateful recovery in face ofcomponent or node crash, thus increasing the availability anddependability of the system.

The platform suitability for an automated warehousesupervision system has been qualitatively validated. Thewarehouse is composed of tens of hardware nodes that allo-cated hundreds of software components. The main featuresof the platform have been validated context of the case study:(1) registration service, (2) execution service, (3) monitoringservice, and (4) QoS configuration service.

Also, quantitative analysis of the scalability capabilitiesof the DAMP middleware has been performed, includ-ing numerical analysis of application stateful initialization,depending on the number of components and state size

Futurework includes quantitative analysis of compositionalgorithm execution times, in order to calculate a suboptimalreconfiguration solution in bounded time.

Competing Interests

The authors declare that there are no competing interestsregarding the publication of this paper.

Acknowledgments

Thisworkwas financed in part by theUniversity of the BasqueCountry (UPV/EHU) under Project UFI 11/28 and by theMCYT&FEDER under Project DPI2015-68602-R.

References

[1] S. Pollin, M. Timmers, and L. Van der Perre, “Anticipativeenergy and QoS management: systematically improving theuser experience,” in Software Defined Radios, Signals andCommunication Technology, chapter 6, pp. 87–108, Springer,Dordrecht, Netherlands, 2011.

[2] Y. Zhu, M. Halpern, and V. J. Reddi, “Event-based schedulingfor energy-efficient QoS (eQoS) in mobileWeb applications,” inProceedings of the 21st IEEE International Symposium on HighPerformance Computer Architecture (HPCA ’15), pp. 137–149,IEEE, Burlingame, Calif, USA, February 2015.

[3] L. Zou, R. Trestian, and G.-M. Muntean, “E2DOAS: userexperience meets energy saving for multi-device adaptive videodelivery,” in Proceedings of the IEEE Conference on ComputerCommunications Workshops (INFOCOM WKSHPS ’15), pp.444–449, Hong Kong, May 2015.

[4] W. Li, “Evaluating the impacts of dynamic reconfiguration onthe QoS of running systems,” Journal of Systems and Software,vol. 84, no. 12, pp. 2123–2138, 2011.

[5] M. Garcıa Valls and P. Basanta Val, “Comparative analysis oftwo different middleware approaches for reconfiguration ofdistributed real-time systems,” Journal of Systems Architecture,vol. 60, no. 2, pp. 221–233, 2014.

[6] W. Li, “QoS assurance for dynamic reconfiguration of compo-nent-based software systems,” IEEE Transactions on SoftwareEngineering, vol. 38, no. 3, pp. 658–676, 2012.

[7] A. Agirre, M. Marcos, and E. Estevez, “Distributed applicationsmanagement platform based on service component architec-ture,” in Proceedings of the IEEE 17th International Conferenceon Emerging Technologies and Factory Automation (ETFA ’12),pp. 1–4, Krakow, Poland, September 2012.

[8] L. Almeida, S. Fischmeister, M. Anand, and I. Lee, “A dynamicscheduling approach to designing flexible safety-critical sys-tems,” in Proceedings of the 7th ACM and IEEE InternationalConference on Embedded Software (EMSOFT ’07), pp. 67–74,Salzburg, Austria, October 2007.

[9] M. Garcia Valls, I. R. Lopez, and L. F. Villar, “iLAND: anenhanced middleware for real-time reconfiguration of serviceoriented distributed real-time systems,” IEEE Transactions onIndustrial Informatics, vol. 9, no. 1, pp. 228–236, 2013.

[10] A. Agirre, J. Parra, E. Estevez, and M. Marcos, “QoS awareplatform for dependable sensory environments,” in Proceedingsof the IEEE International Conference on Multimedia and ExpoWorkshops (ICMEW ’14), pp. 1–5, IEEE, Chengdu, China, July2014.

[11] A. Agirre, E. Estevez, and M. Marcos, “Fault tolerant compo-nent management platform over Data Distribution Service,” inProceedings of the 1st IFAC Conference on Embedded Systems,Computational Intelligence and Telematics in Control (CESCIT’12), pp. 218–223, Wurzburg, Germany, April 2012.

[12] A. Agirre, J. Perez, R. Priego, M. Marcos, and E. Estevez, “SCAextensions to support safety critical distributed embedded sys-tems,” in Proceedings of the IEEE 18th International Conferenceon Emerging Technologies and Factory Automation (ETFA ’13),pp. 1–4, Cagliari, Italy, September 2013.

[13] A. Agirre, E. Estevez, and M. Marcos, “Resource managementsupport for SCA based distributed applications,” in Proceedingsof the IEEE Emerging Technology and Factory Automation(ETFA ’14), pp. 1–4, Barcelona, Spain, September 2014.

[14] A. Agirre, M. Marcos, and E. Estevez, “Distributed componentmanagement platform for QoS enabled applications,” in Pro-ceedings of the IEEE 16th Conference on Emerging Technologiesand Factory Automation (ETFA ’11), pp. 1–4, Toulouse, France,September 2011.

[15] A. Agirre, E. Estevez, andM.Marcos, “QoS enabled applicationmanagement platform over DDS,” in Proceedings of the Middle-ware Workshop on Posters and Demos Track (PDT ’11), Lisbon,Portugal, 2011.

[16] A. Agirre, J. Parra, A. Armentia, A. Ghoneim, E. Estevez, andM. Marcos, “QoS management for dependable sensory enviro-nments,”Multimedia Tools and Applications, pp. 1–23, 2015.

[17] I. Crnkovic, S. Sentilles, V. Aneta, and M. R. V. Chaudron, “Aclassification framework for software componentmodels,” IEEETransactions on Software Engineering, vol. 37, no. 5, pp. 593–615,2011.

[18] P. Hnetynka, L. Murphy, and J. Murphy, “Comparing theservice component architecture and fractal component model,”Computer Journal, vol. 54, no. 7, pp. 1026–1037, 2011.

[19] T. Bures, P. Hnetynka, F. Plasil et al., “Runtime support foradvanced component concepts,” in Proceedings of the 5th ACISInternational Conference on Software Engineering Research,Management, and Applications (SERA ’07), pp. 337–345, Busan,Republic of Korea, August 2007.

[20] L. Seinturier, P. Merle, R. Rouvoy, D. Romero, V. Schiavoni,and J.-B. Stefani, “A component-based middleware platform


for reconfigurable service-oriented architectures,” Software—Practice and Experience, vol. 42, no. 5, pp. 559–583, 2012.

[21] I. Estevez-Ayres, P. Basanta-Val, M. Garcıa-Valls, J. A. Fisteus,and L. Almeida, “QoS-aware real-time composition algorithmsfor service-based applications,” IEEE Transactions on IndustrialInformatics, vol. 5, no. 3, pp. 278–288, 2009.

[22] M. Garcıa-Valls, P. Basanta-Val, and I. Estevez-Ayres, “A compo-nentmodel for homogeneous implementation of reconfigurableservice-based distributed real-time applications,” in Proceedingsof the 10th Annual International Conference onNewTechnologiesof Distributed Systems (NOTERE ’10), pp. 267–272, Tozeur,Tunisia, June 2010.

[23] J. C. Romero and M. Garcıa-Valls, “Scheduling componentreplacement for timely execution in dynamic systems,” Soft-ware: Practice and Experience, vol. 44, no. 8, pp. 889–910, 2014.

[24] A. van Hoorn, M. Rohr, A. Gul, and W. Hasselbring, “Anadaptation framework enabling resource-efficient operation ofsoftware systems,” in Proceedings of the Warm Up Workshop forACM/IEEE ICSE 2010 (WUP ’09), Cape Town, South Africa,April 2009.

[25] R. Von Massow, A. Van Hoorn, and W. Hasselbring, “Perfor-mance simulation of runtime reconfigurable component-basedsoftware architectures,” in Software Architecture: 5th EuropeanConference, ECSA 2011, Essen, Germany, September 13–16, 2011.Proceedings, vol. 6903 of Lecture Notes in Computer Science, pp.43–58, Springer, Berlin, Germany, 2011.

[26] N. Roy, N. Shankaran, and D. C. Schmidt, “Bulls-Eye—aresource provisioning service for enterprise distributed real-time and embedded systems,” in On the Move to MeaningfulInternet Systems 2006: CoopIS, DOA, GADA, and ODBASE, R.Meersman and Z. Tari, Eds., vol. 4276, pp. 1843–1861, Springer,2006.

[27] N. Shankaran, J. Balasubramanian, D. Schmidt et al., “A frame-work for (re)deploying components in distributed real-time andembedded systems,” in Proceedings of the ACM Symposium onApplied Computing (SAC ’06), pp. 737–738, Dijon, France, April2006.

[28] V. Subramonian, G. Deng, C. Gill et al., “The design and perfor-mance of component middleware for QoS-enabled deploymentand configuration of DRE systems,” Journal of Systems andSoftware, vol. 80, no. 5, pp. 668–677, 2007.

[29] M. Henning, “The rise and fall of CORBA,”Queue, vol. 4, no. 5,pp. 28–34, 2006.

[30] W. R. Otte, A. Gokhale, and D. C. Schmidt, “Efficient and deter-ministic application deployment in component-based enter-prise distributed real-time and embedded systems,” Informationand Software Technology, vol. 55, no. 2, pp. 475–488, 2013.

[31] B. Orlic, I. David, R. Mak, and J. Lukkien, “Dynamically recon-figurable resource-aware component framework: architectureand concepts,” in Software Architecture, I. Crnkovic, V. Gruhn,and M. Book, Eds., vol. 6903, pp. 212–215, Springer, Berlin,Germany, 2011.

[32] F. Baude, L. Henrio, and C. Ruz, “Programming distributedand adaptable autonomous components-the GCM/ProActiveframework,” Software: Practice and Experience, vol. 45, no. 9, pp.1189–1227, 2015.

[33] S. Richter, M.Wahler, and A. Kumar, “A framework for compo-nent-based real-time control applications,” in Proceedings ofthe 13th Real-Time Linux Workshop, Prague, Czech Republic,October 2011.

[34] M. Hammer and A. Knapp, “Correct execution of reconfigu-ration for stateful components,” Electronic Notes in TheoreticalComputer Science, vol. 260, pp. 91–108, 2010.

[35] M. Wegdam, Dynamic Reconfiguration and Load Distributionin Component Middleware, University of Twente, Enschede,Nederlands, 2003.

[36] A. Armentia, A. Agirre, E. Estevez, J. Perez, and M. Marcos,“Model driven design support for mixed-criticality distributedsystems,” in Proceedings of the 19th World Congress of theInternational Federation of Automatic Control, Cape Town,South Africa, 2014.

[37] S. K. Baruah, A. Burns, and R. I. Davis, “Response-time analysisfor mixed criticality systems,” in Proceedings of the 32nd IEEEReal-Time Systems Symposium (RTSS ’11), pp. 34–43, Vienna,Austria, December 2011.

[38] M. Joseph and P. Pandya, “Finding response times in a real-timesystem,” Computer Journal, vol. 29, no. 5, pp. 390–395, 1986.

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014


Active and Passive Electronic Components

Control Scienceand Engineering

Journal of



RotatingMachinery


Hindawi Publishing Corporation http://www.hindawi.com

Journal ofEngineeringVolume 2014

Submit your manuscripts athttp://www.hindawi.com

VLSI Design



Shock and Vibration


Civil EngineeringAdvances in

Acoustics and VibrationAdvances in



Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

The Scientific World JournalHindawi Publishing Corporation http://www.hindawi.com Volume 2014

SensorsJournal of


Modelling & Simulation in EngineeringHindawi Publishing Corporation http://www.hindawi.com Volume 2014


Chemical EngineeringInternational Journal of Antennas and

Propagation




Navigation and Observation



DistributedSensor Networks


Research Article QoS Aware Middleware Support for ...downloads.hindawi.com/journals/ijdsn/2016/2702789.pdf · applications QoS when a dynamic system recon guration is triggered. Section

Documents