Constructing Performance Model of JMS Middleware … · Constructing Performance Model of JMS Middleware ... reliable comparison of alternatives is therefore ... predict basic performance

Constructing Performance Model ofJMS Middleware Platform

Tomáš Martinec, Lukáš Marek,Antonín Steinhauser, Petr Tuma

Faculty of Mathematics and PhysicsCharles University

Prague, Czech [email protected]

Qais Noorshams, Andreas Rentschler,Ralf Reussner

Chair Software Design and QualityKarlsruhe Institute of Technology

Karlsruhe, [email protected]

ABSTRACTMiddleware performance models are useful building blocksin the performance models of distributed software applica-tions. We focus on performance models of messaging mid-dleware implementing the Java Message Service standard,showing how certain system design properties – includingpipelined processing and message coalescing – interact tocreate performance behavior that the existing models do notcapture accurately. We construct a performance model ofthe ActiveMQ messaging middleware that addresses the out-lined issues and discuss how the approach extends to othermiddleware implementations.

Categories and Subject DescriptorsD.2.8 [Software Engineering]: Metrics—Performance Mea-sures

General TermsPerformance

KeywordsSoftware Performance; Performance Analysis; Measurement;Modeling; JMS

1. INTRODUCTIONSoftware performance engineering (SPE) is a discipline

that focuses on incorporating performance concerns into thesoftware development process, aiming to reliably deliver soft-ware with particular performance properties [36]. Amongthe tools employed by SPE are predictive performance mod-els. Constructed in the early phases of the software develop-ment process, the models help predict the eventual softwareperformance and thus guide the development [3].

To deliver the expected guidance, the predictive perfor-mance models must capture all relevant system components.

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full cita-tion on the first page. Copyrights for components of this work owned by others thanACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-publish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from [email protected]’14, March 22–26, 2014, Dublin, Ireland.Copyright 2014 ACM 978-1-4503-2733-6/14/03 ...$15.00.http://dx.doi.org/10.1145/2568088.2568096.

For modern software applications, this may entail model-ing complex system layers such as the virtual machine orthe messaging middleware. Composing such complete per-formance models directly is necessarily expensive and in-efficient. Instead, the abstract application model can beconstructed first, with the models of standard system com-ponents added later [39, 8]. This gives rise to the need forcomposable performance models of standard system compo-nents.

Our work focuses on the construction of such performancemodels for messaging middleware, specifically messaging mid-dleware that implements the Java Messaging Service (JMS)standard [37]. Although JMS performance models were pub-lished before [26, 14, 30, 9, 13, 11, 34], we illustrate thatthe existing models often fail to capture important elementsof middleware behavior. In turn, this omission results inreduced performance prediction accuracy, especially whereprocessor utilization and message latency are concerned. Ourcontribution is as follows:

– Using code analysis and experimental measurements ofa mainstream JMS implementation, we illustrate sit-uations where observed performance is not accuratelypredicted by common models.

– We provide a detailed technical analysis of the ob-served effects as an essential basis for further modeling.

– We design a performance model that captures these ef-fects and validate the model using experimental mea-surements.

We have decided to organize our presentation in a waythat familiarizes the reader with the necessary platform-specific background as soon as possible. This helps avoidpotentially inaccurate generalizations in the introductorytext. In Section 2, we introduce our modeling context anddescribe our experimental platform. Section 3 explains theissues that complicate accurate performance modeling of ourplatform. We show how to construct a performance modelthat addresses these issues in Section 4, and follow by eval-uating and discussing the model results in Section 5. Thisis where we pay particular attention to explaining how ourresults, so far presented in a platform-specific context, canbe generalized. Section 6 relates our modeling efforts to theexisting research, and finally Section 7 summarizes our con-clusions.

123

2. MODELING CONTEXTThe expectations put on a performance model are closely

related to the intended model use. We therefore start by de-scribing such uses, paying particular attention to the inputsthat are available to the modeler and the outputs that themodeler would seek in each context.

As noted in the introduction, middleware performancemodels are needed as building blocks in application perfor-mance models. Such models are used in early stages of thesoftware development process to guide important design de-cisions, or in software maintenance activities when a changeimpact analysis can be conducted to choose among multiplemodification directions [21]. On the input side, the mod-eler can usually collect information about the timing (ormore generally resource demands) of the operations used asatomic elements of the model. Restrictions on the abilityto instrument particular operations may require using spe-cialized microbenchmarks or deriving detailed informationfrom aggregate statistics such as overall system throughput.On the output side, the modeler would require the modelto accurately predict general design feasibility and overallscalability trends with respect to performance. The modelshould also suffice for comparing design alternatives.

Middleware performance models can also be understoodas a description of the expected performance (rather thanan approximation of the actual performance). Besides sim-ple software documentation purposes, this use can also ben-efit software performance testing [5, 15]. In this context,the models can be provided with the same inputs as in theearly stages of the software development process, with oneimportant addition – the models can be automatically cal-ibrated against the actual performance in selected bench-marks. Such calibration makes the question of absolute pre-diction accuracy mute, the modeler instead evaluates theability to fit the model to the measurements with reason-able values of the calibrated parameters.

It is also possible to use the models at runtime to plansystem adaptation [7]. Particular to this context is the needto maintain low overhead in both collecting the inputs andevaluating the model. The output of the model is used tomake adaptation decisions, reliable estimation of trends orreliable comparison of alternatives is therefore preferred toabsolute prediction accuracy.

In summary, the three modeling contexts all put empha-sis on predicting trends, which are used to make relativecomparisons or to assess system scalability. Where abso-lute prediction accuracy is important, model calibration isperformed on the timing information collected through mea-surement.

2.1 Modeling Messaging BrokersOur work focuses on the construction of performance mod-

els for JMS middleware [37]. The JMS architecture envi-sions multiple clients communicating by sending and receiv-ing application specific messages. The messages travel eitherthrough queues in a point-to-point pattern or through top-ics in a publish-subscribe pattern. The JMS standard pro-vides multiple quality-of-service settings, especially impor-tant from performance perspective is deciding whether theJMS middleware should keep messages in transient buffers orpersistent storage and whether the message delivery shouldbe subject to transaction processing.

The model we construct should be a suitable buildingblock in application performance models. It must be able topredict basic performance metrics relevant for the JMS mid-dleware – especially resource utilization, message through-put and message latency – that would be observed for agiven workload on a given platform. The middleware modeldoes not describe the workload itself, that is the task of theapplication performance models that would incorporate themiddleware model.

2.2 Experimental PlatformIn our experience, the process of building and validat-

ing a performance model is necessarily platform-dependent.Although the individual steps can follow a common over-all approach, the modeling accuracy depends on multipletechnical details that need to be considered. We thereforeintroduce our experimental platform and continue the pre-sentation in a platform-specific context. Generalizations arediscussed as appropriate.

Our code analysis and experimental measurements areperformed on the ActiveMQ 5.4.2 messaging middleware [2],which implements the JMS standard [37]. Central to themiddleware is the message broker, a process that managesmessaging channels, which are either queues or topics. Mes-sage producers and message consumers connect to the brokerusing sockets. We isolate broker performance by executingit on a dedicated computer, a single-core 2.33 GHz IntelXeon machine with 4 GB RAM running Fedora Linux withkernel 3.9.2-200 x86 64 and OpenJDK 1.6.0-24 x86 64. Theproducers and consumers run on two additional comput-ers connected through a dedicated gigabit Ethernet networkwith accelerated Broadcom network adapters, chosen so thatthey can saturate the broker while at low load themselves– the producer is an eight-core (two chips four cores each)2.30 GHz AMD Opteron machine with 16 GB RAM andthe consumer is an eight-core (two chips four cores each)1.86 GHz Intel Xeon machine with 8 GB RAM.

From the many quality-of-service settings available, wefocus on the transient message passing mechanism with ac-knowledgments. This setting targets applications that re-quire low-latency high-throughput reliable message deliveryand is therefore a natural performance modeling subject. Wedo not model quality-of-service settings that require persis-tent message storage, because with such settings, the storageperformance tends to dominate the observations. Existingstorage performance modeling methods are then likely bet-ter suited for capturing the observed performance [38].

The transient message passing mechanism is implementedin four broker threads that process a message passing througha broker queue, as shown on Figure 1:

– The first thread blocks waiting for messages arrivingthrough a network socket. On message arrival, thethread reads the message, selects the destination queueand stores the message in a container associated withthis queue. This thread is blocked when the containeris full.

– The second thread blocks waiting for messages arrivingin the container filled by the first thread. On messagearrival, the thread locates the message consumer andpasses the message to the third thread, responsible forcommunicating with that consumer.

124

– The third thread blocks waiting for messages and sendsthem on to the consumer through a network socket.

– The fourth thread blocks waiting for acknowledgmentsarriving from the consumer through a network socket.On acknowledgment arrival, the corresponding mes-sage is recognized as processed.

Data message

Data message

Broker

Producer

ConsumerAcknowledgement message

List.remove()

Socket.read()

Socket.read()

List.add()Messagesin flight

Thread 1

Thread 2 Thread 3

Thread 4

List.remove()

List.add()

List.remove()

Socket.write()

Figure 1: Transient message passing architecture inActiveMQ 5.4.2.

3. MODELING ISSUESExisting models of messaging middleware1 typically be-

long to one of two broad classes, here called models withqueues and fitted models:

– A typical model with queues relies on the fact thatmessaging channels resemble service queues. The modelwould represent resources such as processor or storagewith service queues and approximate a message pass-ing through a messaging channel with a single servicerequest in each of the queues.

Models with queues were shown to achieve high accu-racy especially in complex systems with multiple mes-saging channels, where the mean resource demandsat the bottleneck resources determine the achievablethroughput and the accumulated effects of queueingat the messaging channels dominate the observed la-tencies [18, 34].

– A fitted model is typically used when the observedperformance is determined through interactions at theimplementation level that are either not understood in

1We discuss the existing models in depth in the related worksection. We avoid the discussion here to maintain text flow.

sufficient detail or simply too complex. After quantify-ing the workload properties that impact performance,the model would derive a function that predicts perfor-mance from the workload properties by fitting a func-tion template to the observed measurements.

Fitted models were successfully used with workloadproperties such as message size or filter count [13, 11],whose impact is otherwise difficult to predict becauseit consists of many minuscule implementation effects.

Despite their many strong points, both model classes ex-hibit accuracy issues in certain situations inherent to ourmodeling context. We describe these issues next.

3.1 Pipelined Message ProcessingThe ActiveMQ broker processes messages in several phases

that form a pipeline. When any of the phases limits con-current processing – as is the case with the thread-per-connection and thread-per-destination patterns in our bro-ker – messages may queue inside the pipeline. Such queueinghas a relatively benign impact on throughput but a very sig-nificant effect on latencies, as illustrated on Figure 2.

Latency [us]

Rela

tive s

ha

re

0 100 200 300 400 500

0.0

0.2

0.4

0.6

0.8

1.0

Latency [us]

0 100 200 300 400 500

0.0

0.2

0.4

0.6

0.8

1.0

Figure 2: Impact of bursts on latency distribu-tion. Constant throughput 5000 msg/s, left work-load sending individual messages, right workloadsending bursts of ten messages.

Figure 2 shows the distribution of message latencies ob-served at the throughput of 5000 msg/s in two workload con-figurations, regular and bursty. In the regular configuration,the producer emits one message every 200µs. In the burstyconfiguration, the producer emits a burst of ten messagesevery 2 ms.

Latency [us]

Re

lative

sh

are

0 100 200 300 400 500

0.0

0.2

0.4

0.6

0.8

1.0

Latency [us]

0 100 200 300 400 500

0.0

0.2

0.4

0.6

0.8

1.0

Figure 3: Predicting impact of bursts on latency dis-tribution with G/G/1 queue. Constant throughput5000 msg/s, left workload sending individual mes-sages, right workload sending bursts of ten mes-sages.

125

Figure 3 shows that approximating the broker with a sin-gle service queue – as a model with queues might do – is notenough when modeling the bursty workload latency. Themodel used the same distribution of the arrival times andthe service times as Figure 2. For the regular workload, thepredicted latency matches the measurement reasonably well.For the bursty workload, the predicted latency shows severalregular clusters from 40µs to 420µs but the measurementforms a single cluster from 120µs to 240µs – the model notonly failed to predict the absolute latency, it also failed toapproximate the overall trend.

Section 5 shows how our model improves the predictionaccuracy by reflecting the pipeline architecture in the modelstructure. A fitted model that would capture the latencywould have to include the information quantifying the burstsin the workload properties. Unfortunately, adding new in-dependent variables into the workload properties increasesthe cost of building a fitted model.

3.2 Thread Scheduling OverheadThe use of multiple threads in the ActiveMQ broker in-

troduces the opportunity for context switching, that is, theact of handing control of the processor from one thread toanother. Although the design intent is to make contextswitch a fast operation, the accumulated overhead of contextswitching can impact performance.

Two major reasons for a context switch are the schedulingpolicies enforced by the operating system and the blockingbehavior exhibited by the executing threads. The schedulingpolicies are usually only enforced after a thread has run forsome time – 750µs on our platform – which makes themunlikely to impact relatively fast message processing – tensof microseconds on our platform. In contrast, the threadblocking behavior may trigger context switches arbitrarilyfast.

The cost of a context switch can vary significantly [35, 23].On our platform, a simple benchmark where two threadstake turns blocking each other on a synchronization vari-able estimates the context switch duration to be 3.3µs. Thepipelined message processing, which involves four threadsoperating on each message, further multiplies the contextswitch overhead. Even more importantly, the amount of con-text switching per message varies. When messages arrive farfrom each other in time, the threads finish processing a mes-sage before the next one arrives and therefore block waitingonce per message. But when messages arrive close to eachother, the threads have a new message to process by thetime they finish processing the previous one and thereforedo not block waiting. This effect is shown on Figure 4.

Figure 4 illustrates that on our broker, the relative amountof context switching changes from about 20 switches permessage for low throughput to about 1 switch per messagefor high throughput. The peak throughput can be deducedfrom Figure 5, which shows the dependency between thetarget throughput and the actual throughput (the producerattempts to generate messages at the target throughputrate, but the broker flow control restricts the producer toavoid message loss). Also worth noting is the implied factthat peak processor utilization does not coincide with peakthroughput – a practically important effect because highprocessor utilization is often taken to indicate a bottleneck.

To model this thread scheduling effect, a model with queueswould require a special load dependent service queue. A fit-

1000 4000 7000 11000 15000 19000 23000 27000

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Target throughput [msg/s]

Pro

cessor

utiliz

ation

●

●

●

● ●●

●

●

●

●●

●

●

●

●

●

●●

●

040000

80000

120000

Conte

xt sw

itch r

ate

[1/s

]

● Context switches

Total CPU usage

Broker CPU usage

Figure 4: Dependency of broker processor utiliza-tion and context switch rate on target throughput.

1000 4000 7000 11000 15000 19000 23000 27000

05

00

01

50

00

25

00

0


Co

nsu

me

r th

rou

gh

pu

t [m

sg

/s]

Figure 5: Dependency of actual throughput on tar-get throughput.

ted model can probably capture the effect more easily, butwe are not aware of any work doing so.

3.3 Message CoalescingThe performance impact of both the pipelined process-

ing and the thread scheduling is more pronounced in burstyworkloads than in regular workloads. Besides bursts thatare inherent to the workload from the application perspec-tive, more bursts can be introduced as the broker processesthe messages, again influencing the observed performance.One source of such message bursts in our broker is the imple-mentation of the TCP protocol in the network stack, whichis used to transport messages between the producers andconsumers and the broker. The protocol minimizes the pro-cessing overhead by coalescing smaller messages into largerpackets, both in software and in hardware. Coalescing insoftware follows RFC 896 [16] and is disabled by default –because this is a sensible default, we leave it disabled in ourexperiments. Coalescing in hardware is done as a part of theGeneric Receive Offload (GRO) [41] and Generic Segmenta-tion Offload (GSO) [40] features.

Both GRO and GSO are enabled by default, and althoughthey can also be disabled, keeping the default makes the ex-perimental platform more realistic. We believe existing sim-ulation tools such as the ns simulator [31] are more suitablefor modeling the message coalescing behavior at the TCPprotocol level than the performance models considered here.To avoid the need for modeling this behavior, we collect theinformation quantifying the bursts on the broker machine.Figure 6 illustrates message coalescing on our platform – theproducer uses the sendto socket function to transmit 1030 Blong messages at a rate of 20000 msg/s, the two graphs showthe statistical distribution of packet sizes observed through

126

the pcap monitoring interface when departing the producerand when arriving at the broker. Without message coalesc-ing, the graphs would show all messages having 1030 B plusthe TCP protocol header.

Message size [bytes]

Rela

tive s

hare

0 5000 15000 25000

0.0

00.0

50.1

00.1

5

Message size [bytes]

0 5000 15000 250000.0

00.0

50.1

00.1

5

Figure 6: Packet sizes observed when departing theproducer (left) and arriving at the broker (right).

Another opportunity for message coalescing arises in con-nection with the garbage collection. On our platform, thegarbage collector occasionally stops the broker threads tofree heap space. Messages received while the broker threadswait are held in the operating system buffers and processedby the broker as soon as the garbage collector finishes. Fromthe perspective of the broker, this has the same effect as ifthe messages arrived in one burst. Figure 7 displays the la-tencies during a garbage collection pause. With pluses, weshow latencies measured at the points where messages enterand leave the broker – the cluster of pluses at the end of thegarbage collection pause shows the broker reading the mes-sages held in the operating system buffers during the pause.With circles, we show latencies estimated at the points wheremessages enter and leave the operating system buffers – theslope of circles during the garbage collection pause showshow the messages accumulate. Section 4 explains how thiseffect is captured by our model.

●

●●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●● ●

●

● ●

●

●● ●

●●

● ●● ● ● ● ● ●

●

● ● ● ●●

●

10.125 10.126 10.127 10.128 10.129 10.130

0500

1000

1500

2000

Experiment time [s]

Late

ncy [us]

+++ +++

++++++++++++++++

++++ + +

+++ ++

++ ++ +

+ + ++ ++++++ ++

+

●

+Latencies with regular arrivals

Measured latenciesGC pause

Figure 7: Effect of garbage collection pause on la-tency.

4. PERFORMANCE MODEL CONSTRUC-TION

To address the modeling issues outlined in Section 3, weconstruct a performance model that directly reflects the bro-ker structure as shown on Figure 1. We use Queueing PetriNets (QPN) [20] as the modeling formalism, both becauseit offers modeling abstractions that match the architectureelements and because it has extensive tooling support [19].

QPN combines the modeling concepts of Petri Nets andQueueing Networks. The essential elements of a QPN model

are immediate and timed places and immediate and timedtransitions. As usual, places can hold colored tokens, tran-sitions consume tokens in input places and produce tokensin output places. Immediate places always make their to-kens available to transitions, timed places only make tokensavailable after they pass an internal service queue. Tokenscan also be subject to departure discipline that imposes or-dering restrictions. Immediate transitions have weights andare considered to happen instantaneously, timed transitionshave firing rates and are considered to happen after a ran-dom delay. QPN models can be nested, a timed place canrepresent a nested QPN model, tokens arriving at the nestedplace are submitted to the nested model, tokens departingthe nested model are made available to transitions.

4.1 Broker ModelWe model the broker by a QPN model shown on Figure 8,

which is nested in the QPN model of the measurement har-ness. This nesting is the reason why the model has a singleinput place and a single output place – tokens representingall incoming network traffic arrive at the input place, to-kens representing all outgoing network traffic depart fromthe output place. Colors are used to distinguish messagesfrom acknowledgments.

The path a message takes through the broker, implementedby multiple threads described in Section 2.2, is modeled asfollows:

– A new message is represented by a token of the msg

color that arrives in the input place. The msg tokenimmediately transitions to the accept-msg place, withanother token deposited in the queue place to modelthe storage occupied by the message.

– The accept-msg place represents the thread that readsthe message from the network socket and stores it in adestination container. After processing, the msg tokentransitions to the process place.

– The process place represents the thread that reads themessages from the destination container and locatesthe message consumer. After processing, the msg tokentransitions to the dispatch place.

– The dispatch place represents the thread that sendsthe messages to the consumer through the networksocket. After processing, the msg token transitions tothe system place.

– The system place represents processing done by theoperating system outside the broker, which does notcount towards latency measured as messages enter andleave the broker, but still contributes to processor uti-lization. After processing, the msg token departs thebroker network.

An acknowledgment will eventually confirm the receptionof the message. The path the acknowledgment takes throughthe broker is modeled as follows:

– A new acknowledgment is represented by a token ofthe ack color that arrives in the input place. Theack token immediately transitions to the accept-ack

place.

127

gc-idle gc-active

accept-msg

input

accept-ack

queue

process dispatch system output

immediate place

timed place

transition

Figure 8: Nested broker QPN model

– The accept-ack place represents the thread that readsthe acknowledgment from the network socket and rec-ognizes the corresponding message as processed. Afterprocessing, the ack token is discarded by a transitionthat also removes one token from the queue place toindicate no storage is occupied by the message any-more.

4.2 Garbage CollectionSection 3.3 explains how garbage collection causes mes-

sage coalescing in the operating system buffers. To modelthis behavior, we need to represent the garbage collectionpauses. To do this, we observe how the broker allocatesmemory.

The objects maintained by the broker are primarily con-cerned with clients and messages and destinations. Individ-ual instances of the objects represent individual clients andmessages and destinations. The lifetime of these objectsis necessarily related to the lifetime of the concepts theyrepresent, simply because keeping them around for longerwould cause memory leaks. This arrangement makes mes-sages most important from the garbage collection perspec-tive – objects related to messages have high allocation rates(on par with throughput rates) and short lifetimes (on parwith roundtrip times).

Our experimental platform uses a generational garbagecollector that will never promote message-related objects be-yond the young generation (except if the young generationlifetime was shorter than the message roundtrip time, whichis not common [24]). We can therefore imagine that eachmessage passing through the broker will require allocatingobjects of certain average size. When the accumulated sizeof these objects reaches the young generation size, a younggeneration collection will be triggered and all these objectswill be collected.

In the model on Figure 8, the garbage collector state ismodeled using the gc-idle and gc-active places and a sin-gle collector token. The transition from the input place isenabled only when the collector token resides in the gc-

idle place. Once the collector token transitions into the

gc-active place, no tokens transition to the accept-msg andaccept-ack places, simulating a garbage collection pause.Multiple garbage tokens are used to represent allocated ob-jects. The garbage tokens accumulate in the gc-idle placewith each message, the transition from the gc-idle place tothe gc-active place requires that enough garbage tokensaccumulate.

4.3 Context SwitchingSection 3.2 explains how the thread scheduling overhead

impacts performance. We model this effect by introducinga new processor scheduling strategy into the QPN formal-ism. The strategy assumes each timed place represents athread that keeps executing until no more work remains oruntil the scheduler executes another thread instead. In thiscontext, we mimic two elements of a typical thread sched-uler behavior – the overhead of switching from one threadto another and the limit on the time a thread is allowed toexecute when other threads wait.

The strategy accepts the context switch duration c andthe quantum duration q as parameters. Tokens from onetimed place are processed until the accumulated executiontime reaches q. At that moment, the strategy switches toexecuting tokens from another timed place, extending theexecution time of the first token in that place by c.

4.4 Model CalibrationBefore use, the model must be populated with a number of

parameters. These are the resource demands of the process-ing performed by the broker threads, the resource demandsrelated to processing outside the broker, and additional con-stants – the quantum duration, the context switch duration,the garbage collection threshold.

To collect the processor demands of the broker threads, weinsert measurement probes into the broker source code, col-lecting time needed to execute the relevant code fragments.As a technical complication, the collected time may includepassive waiting, which is not a processor demand. In ourcase, excluding passive waiting by the usual means (mea-

128

suring and subtracting the waiting duration or using clockthat stops while waiting) was burdened by excessive over-head. We have therefore decided to measure the broker whennear saturation and discard the upper decile of the proces-sor demand measurements. Running near saturation makespassive waiting rare and because the times we measure areshort, measurements that are distorted by waiting are easilyidentified by their extreme value. Our outlier filtering choicemay have a slight systematic impact on modeled latencies.

To measure the processor demand related to processingoutside the broker, we look at the difference between theoverall processor utilization and the processor utilization dueto the broker threads. From the data used in Figure 4, weestimate the processor demand of the system place to be20 % of the total processor demand used in the other timedplaces in the model.

The collected processor demands are necessarily burdenedby measurement overhead. To assess and compensate, wescale the average processor demand per message to matchthe peak throughput. The data used in Figure 5 placethe peak throughput at 22400 msg/s, this gives us an av-erage processor demand per message of 1/22400 or 45µs,of which 20 % or 9µs is related to processing outside thebroker. Without overhead compensation, the average totaldemand of the timed places in the model is 46µs, we com-pensate by multiplying each broker demand by 0.96 to givethe average total demand of 45µs.

Section 3.2 explains how the amount of context switchingper message changes between rates that generate peak uti-lization and peak throughput – the data used in Figure 4shows these rates to be 11000 msg/s and 22400 msg/s. Ourmodel is constructed to involve five context switch penaltiesat rates close to peak utilization and zero context switchpenalties at rates close to peak throughput, we can thereforecalculate a single context switch penalty to be (1/11000 −1/22400)/5 or 11µs. This is a model parameter only, morecontext switches with shorter duration actually happen inreality. The other parameter related to scheduling – thequantum duration – is a part of the operating system set-tings.

Finally, we measure the number of messages that trig-ger garbage collection by looking at the garbage collectionlog. To avoid interference due to the virtual machine er-gonomics [17], we fix the young generation size.

5. PERFORMANCE MODEL RESULTSWe show the behavior of our performance model on the

same workloads that were used to illustrate the modelingissues in Section 3. We use transient message passing mech-anism with acknowledgments to transport 975 B long bytearray messages between the producer and the consumer.We vary the throughput rate, generating messages eitherin a regular pattern (producing one message every 1/r forthroughput r) or in a bursty pattern (producing ten mes-sages every 10/r for throughput r), and observe (and model)processor utilization and message latency at the given through-put rate. The model is fed the same distribution of thearrival times as the broker in the measurement experiments.

To measure message latency, we use dynamic library wrap-pers that intercept calls to the recvfrom and sendto socketfunctions at the points where messages enter and leave thebroker. We use unique message identifiers embedded in themessage body to associate the calls with individual mes-

sages. Our measurements indicate the overhead of wrappingthe socket calls does not influence the achievable through-put noticeably, however, we collect the latency informationseparately from other measurements as a precaution.

The broker processor utilization information is collectedthrough the cpu controller of the control group subsystem [29].While more accurate than other sources, this method doesnot include the network processing part of the workload thatoccurs inside the kernel rather than the broker. We there-fore also plot the information in the proc pseudo file system,which includes the kernel interrupt processing.

Our measurement harness, based on the performance testsincluded with the messaging middleware, uses dedicated pro-ducer and consumer machines to generate message traffic.We check throughput and utilization at both machines, mak-ing sure no bottlenecks limit the traffic. We collect the es-sential measurements for 10 minutes at each throughput rateand discard observations distorted by the warmup and shut-down phases (some measurements are timed and inspectedmanually).

5.1 Processor UtilizationFigure 9 shows how our model approximates the processor

utilization. The measured values are the same as shown onFigure 4.

1000 4000 7000 11000 15000 19000 23000 27000

0.0

0.2

0.4

0.6

0.8

1.0

1.2


Pro

ce

sso

r u

tiliz

atio

n

Total CPU usage

Broker CPU usage

Predicted CPU usage

Figure 9: Prediction of broker processor utilization.

The fact that the model captures the linear increase ofutilization with throughput is relatively mundane. As amore important contribution, the model also captures thefact that processor utilization peaks much sooner than atmaximum throughput – in our measurements, the processorutilization exceeds 95 % at 10000 msg/s, but the maximumthroughput is around 22400 msg/s.

Compared to measurements, the model does not explainthe increase of processor utilization around 5000 msg/s. Toexplain this effect, we show the outbound network trafficinformation on Figure 10. We see that although the brokertransmits an almost constant amount of bytes per message,at 5000 msg/s it suddenly uses about 25 % more packets permessage than at 4000 msg/s. This increase in network trafficis reflected directly in the increase of processor utilization.

The reason for the network traffic increase is related todetailed behavior the TCP protocol, which can be observedby capturing the network traffic between the broker and theconsumer. At 4000 msg/s, the delay between sending a mes-sage and receiving an acknowledgment is smaller than thedelay between sending two messages – at the TCP protocollevel, packets carrying messages from broker to consumerand packets carrying acknowledgments from consumer tobroker therefore alternate and each acts as a TCP ACK for

129

1000 2000 3000 4000 5000 6000 7000 8000 9000

5000

10000

15000

20000


Packets

[1/s

]

●

●

●

●

● ●

●

●

●

0.0

e+

00

5.0

e+

06

1.0

e+

07

1.5

e+

07

Byte

s [1/s

]

● Transmitted packets

Transmitted bytes

Figure 10: Network traffic from broker to consumermeasured in packets and bytes.

the previous packet. At 5000 msg/s, the delay between send-ing a message and receiving an acknowledgment is close tothe delay between sending two messages, which means thatthe broker sometimes manages to send two messages andthen receive two acknowledgments – at the TCP protocollevel, this means packets in the two directions no longer al-ternate and the flow control mechanism mandates sendingextra TCP ACK packets, causing the increase in networktraffic. While an interesting phenomenon per se, we believethis increase is outside a reasonable scope of performancemodeling.

5.2 Message LatencyMessage latency consists of the time spent processing and

the time spent waiting. At low throughput, processing tendsto dominate and latencies are relatively low. At high through-put, waiting tends to dominate and latencies are relativelyhigh. To avoid losing detail due to scale, we examine severalranges separately.

1000 4000 7000 10000

020

40

60

80

120


Late

ncy [us]

1000 4000 7000 10000

020

40

60

80

120


Figure 11: Measured (left) and predicted (right)message latencies at low throughput with regularworkload.

Figure 11 shows the measured and predicted message la-tencies for low throughput rates generated in the regularpattern. Both the measurement and the model show thesame trend, which starts with mostly constant latencies andgradually introduces variation. In absolute terms, the modeloverestimates the latency roughly by a factor of two. Onereason for this difference is our calibration procedure, whichremoves outliers and scales the remaining values to main-tain throughput – because throughput is sensitive to outliersin resource demands, removing outliers requires scaling theremaining values towards higher resource demands, whichyield higher latency estimates.

1000 3000 5000 7000 9000 11000 14000 17000

0200

600

1000

Late

ncy [us]

1000 3000 5000 7000 9000 11000 14000 17000

0200

600

1000


Late

ncy [us]

Figure 12: Measured (upper) and predicted (lower)message latencies at low to medium throughput withregular workload.

Figure 12 shows the measured and predicted message la-tencies for low to medium throughput rates. Again, boththe measurement and the model show the same trend, withlatencies increasing by about an order of magnitude aroundthe point where the throughput rate exceeds 10000 msg/s,which also happens to be the point where the processor uti-lization nears the peak. In absolute terms, the model doesnot exhibit the variation apparent in the measurements. Weattribute this to the differences between our scheduler modeland the real scheduler. While our scheduler model handlestimed places in a round-robin fashion, the real scheduler onour platform enforces strict fairness. More complex sched-uler models may help here [10].

Figure 13 completes the latency prediction information forall throughput rates generated in the regular pattern. Whenthe producer attempts to generate messages at rates abovepeak throughput, the broker flow control restricts the pro-ducer to avoid message loss. In this situation, the messagelatency is determined by the storage threshold that triggersflow control – approximated by the maximum capacity ofthe queue place in our model. The fact that the model suc-cessfully estimates the very high latency is therefore due to atrivial model parameter, more important is the fact that themodel estimates the throughput at which the flow control istriggered.

We point out that the behavior of the broker near peakthroughput is unstable, with long periods of degraded per-formance. At high throughput rates, there is only little sparecapacity to deal with backlog that may form due to minordisruptions. The broker therefore takes a long time to re-cover from such disruptions, which leads to large accumu-lated impact on latencies. Figure 14 illustrates this lack ofstability.

As the sole exception to the rule that the model is fed thesame distribution of the arrival times as the broker in themeasurement experiments, Figure 13 uses modeled arrival

130

1000 4000 7000 11000 15000 19000 23000 27000

0.0

0.5

1.0

1.5

2.0

Late

ncy [s]

1000 4000 7000 11000 15000 19000 23000 27000

0.0

0.5

1.0

1.5

2.0


Late

ncy [s]

Figure 13: Measured (upper) and predicted (lower)message latencies at low to high throughput withregular workload.

●

●

●

●

●

●●

●

●

●

●

●●● ●●

●

●●●

●●

●

●

●

● ● ●●

●

●

●

●

●

● ●

●

●

●●

●●

●

●● ●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●● ●

●

●●

●

●●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●● ●

●

●●

●

●

●

●

●

●

●

● ●●●●

●

●●

●

● ●

●

●●

●

●

●●

●●

●

●●

●

●

●● ● ●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

● ●

●●

●

●

●

●●

●

●●

●●

●●●

●●● ●

●

●

●

●

●●●

●

● ●

●

●

●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●● ●

●

●

●●

● ●

●

●

●

●

●

●

●●●

●

●

●

●

● ●

●

●

●

●

●

●

● ●

●●

●

●●

●

●

●

●●

●

●● ● ● ●●● ●●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●● ●

●

●●

●

●

●

●

●●

● ●

●

●

●● ●

●

●

●

●

●

●

●●

●●

●● ● ●

●

●

●

●

●

●●

●

●

●●

●●

● ●● ●●

●

●

●●

●

●

●

●●●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●●●

●

●●

●

● ● ●●●

●

●

●● ●●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●● ●●

●

●

● ●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

● ●●

●

●

●

●

●●

●●

●●

●

●

●

● ●●

●

●

●

● ●

●

●

●

●●●

●

●

●

●

●

●

●●

●●

●

●● ●

●●

●● ●●●●

●

● ●●

●

●●●

●

●

●

●●

●●●●

●

●●●●

●

●●●

●

●

●

●

●

●●

● ●

●

●

●●

● ●

●

●

●

●

●● ●

●

●

●

●● ●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●●●●

●

●●

●

●

●

●

●

●

●

●

● ●●

●●●

●

●

●

●

● ●●

●

●

●

●

●

●●

●●●

●

●

●● ●

●

●●

●

● ●

●●

●

●●

●●

●

●●●

●

●

●

●

●

●

●

●●●

●●

●● ●● ●

●●● ●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●

●●

●

●●

●

●

●● ●

●

●●

●

●

●●

● ●

●

●

●

●

●

●

●● ●

●

●

●

●

● ● ●● ●

●

●

●

●

●

●

●●●

●

●●●

● ●

●

●

●●

●●

●●

●

●

●●

● ●

●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●● ●● ●●

●

●

●

●

● ● ●●

●

●●

● ●

●

●● ●

●

●● ●

●

●●

●

●

●

●

●

●

27 28 29 30 31 32 33

0.0

0.2

0.4

0.6

0.8

Experiment time [s]

Late

ncy [us]

Figure 14: Unstable broker latencies at 19000 msg/s.

times that match the target throughput. This is necessarybecause at high throughput rates, the measured arrival timesinclude broker flow control and therefore reflect the observedthroughput rate rather than the target throughput rate.

Figure 15 shows the measured and predicted message la-tencies for low to medium throughput rates generated inthe bursty pattern. Similar to the regular workload results,the bursty workload results show the same trend, with someoverestimation of latency and some underestimation of vari-ation. As an important factor, the model correctly predictsthat introducing burstiness results in shifting the clusterof observed latencies en bloc, rather than creating multi-ple clusters as Section 3.1 illustrates. Compare Figure 16with Figures 2 and 3.

5.3 DiscussionThe results show that our model is capable of address-

ing the issues outlined in Section 3 as far as the trends areconcerned – we predict that pipelined processing of messagebursts results in a tight cluster of latencies, we show thatvarying thread scheduling overhead leads to utilization andthroughput peaking at very different rates, and we do both

1000 3000 5000 7000 9000 11000 14000 17000

0200

400

600

800

1000

Late

ncy [us]

1000 3000 5000 7000 9000 11000 14000 17000

0200

400

600

800

1000


Late

ncy [us]

Figure 15: Measured (upper) and predicted (lower)message latencies at low to medium throughput withbursty workload.

Latency [us]

Rela

tive

sh

are

0 100 200 300 400 500

0.0

0.2

0.4

0.6

0.8

1.0

Latency [us]

0 100 200 300 400 500

0.0

0.2

0.4

0.6

0.8

1.0

Figure 16: Predicting impact of bursts on latencydistribution. Constant throughput 5000 msg/s, leftworkload sending individual messages, right work-load sending bursts of ten messages.

in presence of realistic message coalescing. To our knowl-edge, these effects were not captured by JMS models before.

The prediction of processor utilization is also very accu-rate in absolute terms. The same cannot be said about la-tency, where our predictions at low throughput are some-what pessimistic and predictions at high throughput do notfluctuate as much as measurements – as we explain, this isin part due to model calibration and in part due to realisticscheduling being more complex than the scheduling disci-plines of our model. The accuracy of latency prediction isvery reasonable for uses outlined in Section 2. We shouldnote that we do not use measured latencies to calibrate themodel and still predict latencies of individual messages atvery high resolution. Again, we believe this was not done inJMS models before.

An important question that we address in this discussionis whether our results can be generalized beyond our exper-iments. We present arguments for why we believe our workis not strictly limited to our experimental platform. We also

131

provide the source code and the data we have collected andused, so that more experiments are possible [1].

The most visible concern in generalizing our results is therange of workloads used in the experiments. While we varyboth the throughput rate and the distribution of message ar-rival times, we use messages of equal size and type exchangedbetween a single producer and a single consumer. This con-trasts especially with work that experiments on complexworkloads such as SPECjms2007 [34].

The existing body of work on JMS performance providesa reliable summary of how individual workload parametersinfluence performance, and in fact suggests that extendingthe workload along many parameter axes would bring littleprincipal benefit. Work such as [33, 13] shows there usu-ally is a linear dependency between the message size andthe associated processor demand, in contrast there usuallyis almost no dependency on the number of clients and des-tinations as long as messages are not replicated. Our modelcan be extended to support multiple message sizes and typesby using multiple token colors with different associated pro-cessor demands, as used in [34]. Support for multiple clientsand destinations should not require principal changes to ourmodel either – the relevant message processing paths in ourbroker are reasonably similar to the message processing pathof our workload. On the other hand, workloads that requirepersistent message storage would represent a challenge, dueto the dominating nature of storage latencies in the modelthat otherwise deals in microseconds.

Experiments with limited workloads provide an impor-tant benefit in that they help isolate individual modelingconcerns. Tracking the performance issues that we focuson in a complex workload is virtually impossible – althoughthey are still likely to exist (there is no reason why contextswitching or garbage collection would go away with morecomplex workloads), their performance impact is combinedwith the performance impact of workload variability.

As one item, our work covers the impact of thread schedul-ing overhead on performance. The exact impact is bothworkload-dependent and platform-dependent – in general,we can expect the need for context switching to increasewith more clients and destinations (because clients and des-tinations are served by separate threads) and to decreasewith more cores (because threads will not compete for coresas much). As long as there are more clients and destina-tions (and therefore internal broker threads) than cores, thethread scheduling overhead should be present. The perfor-mance impact of individual context switches is also likelyto increase with heavier workload, because such workloadis associated with heavier memory cache traffic and contextswitches may flush memory cache content.

As another item, our work describes pipelined messageprocessing. This is an architectural decision that concernsthe broker implementation, one that is apparently reason-able but certainly not the only one possible. Brokers thatuse different architectures may require different models –unfortunately, determining the broker architecture for per-formance modeling purposes is a demanding endeavor evenwhen broker sources are available, and not likely to get easierwith closed source brokers.

Finally, our work requires measuring durations of opera-tions that occur inside the broker. This is again easier whenbroker sources are available, but with current instrumen-

tation techniques [28, 27], instrumenting major control flowlocations such as network communication or thread synchro-nization inside closed source brokers is also possible.

6. RELATED WORKPerformance modeling of distributed systems based on

messaging is a frequent research subject. Some authorschoose to work at a relatively high abstraction level, model-ing complex networks of computers that communicate throughmessaging. At this level, details of individual node per-formance are typically simplified and the modeling effortsinvestigate important high level properties such as systemcapacity limits. Some high level modeling work is very closeto our research, for example [18] proposes a method of con-structing models that approximates communicating nodeswith M/M/1 queues and uses QPN for experimental eval-uation. Our model can improve this approximation – thepossibility is actually mentioned by the authors, but thereis not enough technical information in the paper to estimatethe contribution of such model change to accuracy.

In [34], the SPECjms2007 benchmark is modeled withQPN, using G/M/8 queues to approximate processors andG/M/1 queues to approximate storage. The authors achievesignificant modeling accuracy on a variety of workloads – incontrast with our work, the authors cover a wide varietyof message sizes and types and quality-of-service settings,but keep the broker processor utilization below 80 %. Theauthors use a nested QPN model with three timed placesin tandem representing the processor, the storage and thenetwork resources – our model can again improve this ap-proximation when exploring workloads that lead to high bro-ker utilization, provided it is extended with more quality-of-service settings.

In [26], the broker is approximated with an M/M/* queue,similar queues are used to model a component container anda database. The authors predict throughput and latency ina closed workload with zero think time – a situation whichexercises the ability of the broker to serve individual clientsfairly, leading to a linear dependency between the numberof clients and the latency.

In a broader context, other formal tools are used to modelmessaging networks – for example, probabilistic timed au-tomata are used to capture behavior in presence of messageloss in [12]. We observe that high level modeling is consid-ered valuable even when validation against a real system isnot done.

Some studies focus on explorative evaluation of brokerperformance. Among early examples is [6], where perfor-mance of two JMS brokers were evaluated. The measure-ments focus on maximum sustainable throughput with var-ious quality-of-service settings. A thorough study of JMSperformance is [33], where one JMS broker is examined us-ing the SPECjms2007 benchmark. Although these studiesdo not construct performance models (and sometimes do noteven name the examined brokers due to licensing restric-tions), they are a valuable source of common performancetrends that can be observed across brokers. One typicalobservation is that message size is an important factor, in-crease in message size causes linear increase in processordemand. In contrast, the number of clients and destinationsdoes not seem to be important when the total traffic re-

132

mains constant. These observations support our discussionon including additional validation workloads in our work.

Explorative evaluation of broker performance can help cre-ate fitted models. This is the case in a large range of experi-ments summarized in the doctoral thesis [13]. In a number ofseparate publications, these experiments investigate param-eters such as throughput [14] or latency [30] and constructfitted models that approximate the measurements. Interest-ingly, some of the experiment parameter ranges are chosenwith the assumption that peak processor utilization impliespeak throughput, which we show is not necessarily true.

A thorough process of building a fitted JMS model throughexplorative experiments is described in [11]. The experi-ments are carried on the ActiveMQ 5.3 messaging middle-ware, which makes the results even closer to ours. Again, thechoice of experiment parameter ranges equals peak utiliza-tion workload with peak throughput workload. The workalso demonstrates the difficulties of building an accuratemodel for the range of workloads we consider – the proces-sor utilization in the experiments used to create the fittedmodel never exceeds 50 %, and the parameter dependenciesare collected in experiments that assume no resource con-tention, which may limit suitable parameters.

Another work that creates a fitted JMS model is [9], theauthors show how the model can be integrated into a largerperformance model that captures particular SPECjms2007interactions. The focus is on the integration process, tech-nical details of the JMS model are not investigated. Similarapproach in the context of component systems was investi-gated in [25].

Our work also touches on the issue of constructing a per-formance model with limited knowledge of the modeled sys-tem. Other authors have tackled this problem, in [4] anenterprise application model is constructed from partial ar-chitectural information and collected execution traces.

The problem of determining resource demands with lim-ited measurement ability in the context of workload withmultiple request types was addressed in [32] and [22], the au-thors of [42] estimate and adjust performance model param-eters by tracking the prediction error. Using similar tech-niques in combination with artificial workloads crafted toexercise particular elements of the broker architecture canlikely provide enough information to calibrate the perfor-mance model even when measurements based on instrumen-tation are not available.

As a summary to our related work survey, we believe ourmodel can provide accuracy improvement in the context ofexisting modeling work, which mostly acknowledges thatbroker performance is implementation specific and providesmechanisms for plugging detailed broker models into plat-form independent application models. Where fitted modelsare used, our work highlights important effects related topipelined processing and message coalescing that should beconsidered when selecting the model parameters. We alsobelieve our work is the first to attract attention to the signif-icant impact of pipelined processing and message coalescingin the context of broker performance modeling.

7. CONCLUSIONOur work is based on observing performance of the Ac-

tiveMQ messaging middleware. We attract attention to thefact that pipelined processing (the act of handling messages

in stages by multiple broker threads) and message coalescing(the act of processing several adjacent messages together atsome stage) can interact even with very simple workloads tocreate performance effects of significant magnitude that theexisting performance models do not capture. We providetechnical explanation for these effects and design a brokermodel that describes them.

We show that our model provides a reasonably accurateapproximation of the identified effects. As an importantdistinction – where the existing JMS models may capturethe effects by calibrating for a particular workload, our JMSmodel is built by analyzing and reflecting the reasons behindthe effects. Our work therefore touches upon a broader ques-tion of how calibrating and validating the model against thesame workload – something that is regularly done in modelvalidation experiments – contributes to perceived model ac-curacy.

Although our work has used a specific platform and spe-cific workloads, we argue that the effects we observe canreasonably occur on other platforms. We provide the sourcecode and the data we have collected and used to make moreexperiments possible [1].

AcknowledgementThis research has been funded by the EU project ASCENS257414, by the German Research Foundation (DFG) grantRE1674/5-1, by the Czech Science Foundation (GACR) grantP202/10/J042, and Charles University institutional funding.

8. REFERENCES[1] Complementary material.

http://d3s.mff.cuni.cz/papers/jms-modeling-icpe.

[2] Apache Software Foundation. Apache ActiveMQ.http://activemq.apache.org.

[3] F. Brosch, H. Koziolek, B. Buhnova, and R. Reussner.Architecture-Based Reliability Prediction with thePalladio Component Model. Transactions on SoftwareEngineering, 38(6), 2011.

[4] F. Brosig, S. Kounev, and K. Krogmann. AutomatedExtraction of Palladio Component Models fromRunning Enterprise Java Applications. In Proceedingsof ROSSA 2009, 2009.

[5] L. Bulej, T. Bures, J. Keznikl, A. Koubkova,A. Podzimek, and P. Tuma. Capturing performanceassumptions using stochastic performance logic. InProceedings of ICPE 2012. ACM, 2012.

[6] S. Chen and P. Greenfield. QoS Evaluation of JMS:An Empirical Approach. In Proceedings of HICSS2004. IEEE, 2004.

[7] I. Epifani, C. Ghezzi, R. Mirandola, andG. Tamburrelli. Model Evolution by Run-TimeParameter Adaptation. In Proceedings of ICSE 2009.IEEE, 2009.

[8] J. Happe, S. Becker, C. Rathfelder, H. Friedrich, andR. H. Reussner. Parametric Performance Completionsfor Model-Driven Performance Prediction.Performance Evaluation, 67(8), 2010.

[9] J. Happe, H. Friedrich, S. Becker, and R. Reussner. APattern-Based Performance Completion forMessage-Oriented Middleware. In Proceedings ofWOSP 2008. ACM, 2008.

133

[10] J. Happe, H. Groenda, M. Hauck, and R. Reussner. APrediction Model for Software Performance inSymmetric Multiprocessing Environments, 2010.

[11] J. Happe, D. Westermann, K. Sachs, and L. Kapova.Statistical Inference of Software Performance Modelsfor Parametric Performance Completions. InProceedings of QOSA 2010. Springer, 2010.

[12] F. He, L. Baresi, C. Ghezzi, and P. Spoletini. FormalAnalysis of Publish-Subscribe Systems byProbabilistic Timed Automata. In Proceedings ofFORTE 2007. Springer, 2007.

[13] R. Henjes. Performance Evaluation ofPublish/Subscribe Middleware Architectures, 2010.

[14] R. Henjes, M. Menth, and C. Zepfel. ThroughputPerformance of Java Messaging Services UsingWebSphereMQ. In Proceedings of ICDCS 2006WORKSHOPS, 2006.

[15] V. Horky, F. Haas, J. Kotrc, M. Lacina, and P. Tuma.Performance Regression Unit Testing: A Case Study.In Proceedings of EPEW 2013. Springer, 2013.

[16] Internet Engineering Task Force. Congestion Controlin IP/TCP Internetworks.http://tools.ietf.org/html/rfc896.

[17] R. Jones and R. Lins. Java SE 6 HotSpot VirtualMachine Garbage Collection Tuning.http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html.

[18] S. Kounev, K. Sachs, J. Bacon, and A. Buchmann. AMethodology for Performance Modeling of DistributedEvent-Based Systems. In Proceedings of ISORC 2008.IEEE, 2008.

[19] S. Kounev, S. Spinner, and P. Meier. QPME 2.0 - ATool for Stochastic Modeling and Analysis UsingQueueing Petri Nets. In From Active DataManagement to Event-Based Systems and More, 2010.

[20] S. Kounev, S. Spinner, and P. Meier. Introduction toQueueing Petri Nets: Modeling Formalism, ToolSupport and Case Studies (Tutorial Paper). InProceedings of ICPE 2012. ACM, 2012.

[21] H. Koziolek, B. Schlich, C. Bilich, R. Weiss, S. Becker,K. Krogmann, M. Trifu, R. Mirandola, andA. Martens. An Industrial Case Study on QualityImpact Prediction for Evolving Service-OrientedSoftware. In Proceedings of ICSE 2011. ACM, 2011.

[22] S. Kraft, S. Pacheco-Sanchez, G. Casale, andS. Dawson. Estimating Service Resource Consumptionfrom Response Time Measurements. In Proceedings ofVALUETOOLS 2006. ACM, 2006.

[23] C. Li, C. Ding, and K. Shen. Quantifying The Cost ofContext Switch. In Proceedings of ExpCS 2007. ACM,2007.

[24] P. Libic, P. Tuma, and L. Bulej. Issues in PerformanceModeling of Applications With Garbage Collection. InProceedings of QUASOSS 2009. ACM, 2009.

[25] Y. Liu, A. Fekete, and I. Gorton. Design-LevelPerformance Prediction of Component-BasedApplications. IEEE Transactions on SoftwareEngineering, 31(11), 2005.

[26] Y. Liu and I. Gorton. Performance Prediction of J2EEApplications Using Messaging Protocols. InProceedings of CBSE 2005. ACM, 2005.

[27] C.-K. Luk, R. Cohn, R. Muth, H. Patil, A. Klauser,G. Lowney, S. Wallace, V. J. Reddi, andK. Hazelwood. Pin: Building Customized ProgramAnalysis Tools with Dynamic Instrumentation. InProceedings of PLDI 2005. ACM, 2005.

[28] L. Marek, A. Villazon, Y. Zheng, D. Ansaloni,W. Binder, and Z. Qi. DiSL: A Domain-SpecificLanguage for Bytecode Instrumentation. InProceedings of AOSD 2012. ACM, 2012.

[29] P. Menage. Linux Control Groups.https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt.

[30] M. Menth and R. Henjes. Analysis of the MessageWaiting Time for the FioranoMQ JMS Server. InProceedings of ICDCS 2006, 2006.

[31] NS-3 Project. NS-3. http://www.nsnam.org/.

[32] G. Pacifici, W. Segmuller, M. Spreitzer, andA. Tantawi. Dynamic Estimation of CPU Demand ofWeb Traffic. In Proceedings of VALUETOOLS 2006.ACM, 2006.

[33] K. Sachs, S. Kounev, J. Bacon, and A. Buchmann.Performance Evaluation of Message-OrientedMiddleware Using the SPECjms2007 Benchmark.Performance Evaluation, 2009.

[34] K. Sachs, S. Kounev, and A. Buchmann. PerformanceModeling and Analysis of Message-OrientedEvent-Driven Systems. Journal of Software andSystems Modeling, 2012.

[35] B. Sigoure. How Long Does It Take To Make AContext Switch ?http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-context.html.

[36] C. U. Smith and L. G. Williams. PerformanceSolutions: A Practical Guide to Creating Responsive,Scalable Software. Addison-Wesley, 2002.

[37] Sun Microsystems. Java Message Service SpecificationVersion 1.1, 2002.

[38] E. Varki, A. Merchant, J. Xu, and X. Qiu. Issues andChallenges in the Performance Analysis of Real DiskArrays. IEEE Transactions on Parallel andDistributed Systems, 15(6), 2004.

[39] T. Verdickt, B. Dhoedt, F. Gielen, and P. Demeester.Automatic Inclusion of Middleware PerformanceAttributes into Architectural UML Software Models.IEEE Transactions on Software Engineering, 31(8),2005.

[40] H. Xu. GSO: Generic Segmentation Offload.http://lwn.net/Articles/188489/.

[41] H. Xu. net: Generic Receive Offload.http://lwn.net/Articles/311357/.

[42] T. Zheng, C. M. Woodside, and M. Litoiu.Performance Model Estimation and Tracking UsingOptimal Filters. IEEE Transactions on SoftwareEngineering, 2008.

134

Constructing Performance Model of JMS Middleware … · Constructing Performance Model of JMS Middleware ... reliable comparison of alternatives is therefore ... predict basic performance

Documents