Murray Woodside Carleton University, Ottawa... November 2002

1

A View of Performance Analysis of UML Specifications

.... based on the 2002 Performance ProfilePart I: UML, the Profile, Queue models, Markov models

Murray Woodside

Carleton University, Ottawa... November 2002

Reasoning

Design Evaluation

« »

2

(A) Motivation and Viewpoints

Motivation of this work: a performance model is a behaviour abstraction of a

system

it averages over variations in use

it supports evaluation of time properties

it supports reasoning about the design itself

• including extrapolation automated model building will make performance

analysis accessible, fast and maintainable

once a performance problem is identified it is “obvious”

thus the hypothesis: designers can understand the results without understanding the techniques

3

UML is a specification language.What is a Specification?

A Model. We construct models to learn about the interesting properties of a system...

Without having to construct the actual system (cost)

Without having to experiment with the actual system (availability)

Without having to view the entire system — because it may be too complex to understand

A Definition,

...from which more specialized definitions (such as code for a specific platform) can be generated

.... there are other specification languages too

4

Platform Independence vs Real-time Software

Platform independence: design the program logic to be independent of the implementation platform

separation of concerns

software portability Real-time software:

correct functioning depends on time

• hard deadlines, or soft targets

this depends on the platform, and on scheduling...

... and on interference from other factors

• e.g. network delays, failures THUS, there must be engineering tradeoffs between the

logic and the platform/environment

5

UML (Unified Modeling Language)

lightweight formalism driven by what designers use

a union of various ideas in object oriented modeling

united via a meta-model and an “Object Constraint Language”

introduced by Rational, standardized by OMG Object Management Group)

industrial standards body, companies are members

www.omg.com

UML, CORBA, MDA UML is the basis of the “Model-Driven Architecture”

(MDA) initiative

“the design is the implementation”

code generation from UML 2.0

6

1967

Foundations of OO (Nygaard, Goldberg, Meyer,Stroustrup, Harel, Wirfs-Brock, Reenskaug,…)

Foundations of OO (Nygaard, Goldberg, Meyer,Stroustrup, Harel, Wirfs-Brock, Reenskaug,…)

JacobsonJacobson

The Evolution of UML (from Bran Selic’s WOSP 2002 Tutorial)

BoochBoochRumbaughRumbaugh

UML 1.1 (OMG Standard)UML 1.1 (OMG Standard)UML 1.1 (OMG Standard)UML 1.1 (OMG Standard)

UML 1.3 (extensibility)UML 1.3 (extensibility)UML 1.3 (extensibility)UML 1.3 (extensibility)

UML 1.4 (action semantics)UML 1.4 (action semantics)UML 1.4 (action semantics)UML 1.4 (action semantics)

UML 1.4.1UML 1.4.1UML 1.4.1UML 1.4.1

1996

1997

1998

20012002

2003-2004

UML 2.0 (MDA)UML 2.0 (MDA)UML 2.0 (MDA)UML 2.0 (MDA)

7

What are software performance concerns?

Quantitative measures: Delay from Request to Response:

mean and variance

delay value at percentile X (e.g. 95% value)

max delay is useful for deadlines (100% percentile value)

distribution, or approx by a set of percentiles

jitter, for media streams like voice and video:

• deviation from an equally spaced delivery schedule

• but, what measure to use?

– e.g., jitter relative to a fixed or a varying baseline

– measure may depend on the use made of the outputs

System

Request

Response

8

Performance concerns (2): more measures

Throughput of requests or responses:

mean rate per sec.

number in a stated interval (mean, variance, distribution)

interarrival time distribution

• Poisson, deterministic, bounded below (needed for deadline analysis)

bursty or dependent

– batches, heavy-tailed

open vs closed arrivals (infinite vs finite source) Resource Utilization

fraction of time busy

hardware and software resources

System

Request

Response

9

Performance concerns (3)

More general characterstics: Where are the sensitive parameters?

operation demands with a significant impact Location of bottlenecks

“hot spots”

while these are really quantitative, they are also somewhat insensitive, so they are stable while parameters change over a certain range

Scalability limits

users, interfaces, locations

transaction sizes

System

hot

10

Information Requirements of Performance Concerns...

a performance model requires:

performance requirements for stated responses

behaviour for stated responses

resource demands for physical resources

• characteristics of physical resources (speed)

• scheduling

concurrency

identification and use of logical resources

timing specifications (e.g. timeouts)

characteristics of environment services

• completions (ICSE 2002 paper) (e.g. file services, databases)

11

Performance Analysis: Three Viewpoints

Three views of the same system (1995 paper in IEEE TSE)

specialized for different activities Connections between the views to support design activities

MAP View(architecture and structure)

(UML class and object diagrams)

PATH View(scenarios)

(UML behaviour diagramsand Use Cases)

(maps to responses andrequirements)RESOURCE View

(physical and logical resources,and how the design loads them)

(UML deployment diagram and SPTProfile)

12

Where the View information is defined

MAP ViewPATH View

RESOURCE ViewDescribed by the Profile on Schedulability, Performance

and Time (SPT Profile)

Describedby theanalyst

Described by UML

13

Connections between the views

...sub-path executed while holding a given resource

...resources used by an activity

... resources for active objects

... allocation and containment of operations

MAP PATH

RESOURCE

...location of path segments

...objects used for activities

14

(B) UML Model Views (Diagrams) (acknowledgements to Bran Selic’s WOSP2002 tutorial)

Requirements (use case diagrams) Static structure (class diagrams)

kinds of objects and their relationships Object behavior (state machines)

possible life histories of an object Inter-object or system behavior (activity, sequence, and

collaboration diagrams)flow of control among objects to achieve system-level behavior

Physical implementation structures (component and deployment diagrams)

software modules and deployment on physical nodes

15

AircraftSimulatorAircraftSimulatorAircraftSimulatorAircraftSimulator

Use Case Diagrams

Used to capture functional requirements

useful as principal drivers of the overall development process

instructorinstructorinstructorinstructor

traineetraineetraineetrainee

Run checklistRun checklist«include»«include»

Prepare flightPrepare flight

Land planeLand plane

Fail oneengine

Fail oneengine

Use caseUse case

actoractor

16

Use Cases and Performance Concerns

As useful as in any other domain

fundamental drivers of system definition, development, and testing

However….

Focus on function (functional requirements)

In RT systems, much focus on non-functional requirements

• e.g., end-to-end delays, maximum response times,…

Even though UML is a standard, there is no standard way of associating such non-functional requirements with use cases

17

designatedPlanedesignatedPlane crew crew

1..*1..*11PilotPilotAirplaneAirplane

AirlineAirline

ownerowner 0..*0..*

0..*0..*

FlightFlight

routestartduration

routestartduration

{ordered}{ordered}0..*0..*

11

CaptainCaptain First OfficerFirst Officer

Class Diagram

Shows the kinds of entities in a system and their general relationships

AssociationAssociation

Association classAssociation class

18

Object Instance Diagram

Shows object instances in a particular case of a general class diagram

DecrepitAir : AirlineDecrepitAir : Airline

N1313:AirplaneN1313:Airplane

CreakyAir : AirlineCreakyAir : Airline

CA345 : FlightCA345 : Flight

Donald D. : PilotDonald D. : Pilot

LinkLink

CA123 : FlightCA123 : Flight

Mickey M. : PilotMickey M. : Pilot

19

Class Diagrams and Performance Concerns

Because they abstract out certain specifics, class diagrams are not suitable for performance analysis

However they can be used to define default attributes of the objects created from a class

N1:NodeN1:Node N3:NodeN3:Node

N4:NodeN4:Node

N2:NodeN2:Node

N2:NodeN2:NodeN1:NodeN1:Node N3:NodeN3:Node

NodeNode

20

Collaboration Diagram

2.call()2.call()3.sendTone()3.sendTone()

4.dialtone()4.dialtone()

1.offHook()1.offHook()

Depict generic structural and behavioral patterns

/PhoneIF/PhoneIF

/CallProc/CallProc

/ToneGen/ToneGen

Classifier roleClassifier role

NB: It is possible to havecollaboration diagramswithout an Interaction overlay (“pure” structure)

NB: It is possible to havecollaboration diagramswithout an Interaction overlay (“pure” structure)

Message: part of an Interaction

Message: part of an Interaction

21

/Caller/Caller /Operator/Operator /Callee/Callee

timetime

call

ack

number

call

ack

talk

transfer

Sequence Diagrams

Show interactions between objects with a focus on communications (a different representation of a collaboration)

22

Sequence Diagrams and Performance Concerns

Sequence diagrams are extremely useful for showing object interactions

very common, particularly in real-time systems well suited for event-driven behaviorin telecom, many protocol standards are defined using sequence diagrams

However…No standard way of denoting performance requirementsVery limited capabilities for specifying timing constraints and characteristicsDo not scale up to complex systems

23

Using Timing Marks with Sequence Diagrams

Specifying constraints

master : Master d : DBaseServer cs : CommServer

read( )

register( )

{ ( register.receiveTime ( ) - read.sendTime( ) ) 2 ms}

24

Activity Diagrams Different focus compared to sequence diagrams

/Caller /Operator /Callee

ContactContactOperatorOperatorContactContactOperatorOperator

ContactContactCalleeCallee

ContactContactCalleeCallee

RespondRespondRespondRespond

Notify Notify PartiesPartiesNotify Notify PartiesParties

AcknowledgeAcknowledgeAcknowledgeAcknowledge AcknowledgeAcknowledgeAcknowledgeAcknowledge

activityactivity

swimlaneswimlane

25

Activity Diagrams and Performance Concerns

Better than sequence diagrams for

showing concurrency (forks and joins are explicit)

scaling up to complex systems

However…

No standard way of denoting performance requirements

and timing characteristics

Less well-suited for describing event-driven behavior

activities are not associated to method calls or

messages in a class diagram, or to sequence diagrams

26

UML Statechart Diagrams

Very suitable for specifying complex event-driven behavior

LampFlashingLampFlashingLampFlashingLampFlashingflash/flash/

1sec/1sec/1sec/1sec/

FlashOffFlashOffFlashOffFlashOff

entry/lamp.off()entry/lamp.off()

FlashOnFlashOnFlashOnFlashOn

entry/lamp.on()entry/lamp.on()off/off/

LampOffLampOffLampOffLampOff

entry/lamp.off()entry/lamp.off()

LampOnLampOnLampOnLampOn

entry/lamp.on()entry/lamp.on()

on/on/

on/on/

27

State Machines and Performance Concerns

Many software systems are event-driven

very well suited to those systems

Hierarchical capabilities scale up very nicely However…

not directly connected to time (except for time events)

not all classes have them

• default call-return behaviour would have to be inferred

associated with a class, rather than an instance

bindings of messages and events between state machines must be complete

responses are buried in them

28

HandleHandleRequestRequest


InitializeInitializeObjectObject


TerminateTerminateObjectObject


Wait forWait forRequestRequest


Objects and Concurrency Passive objects: have no control of their communications

Clients determine when to invoke an operation Active objectsActive objects:: can control when to respond to requests can control when to respond to requests

Can avoid concurrency conflictsCan avoid concurrency conflicts Require at least one independent engineering-level threadRequire at least one independent engineering-level thread









29

UML Concurrency Model

Active objects are the major concurrency mechanism of UML

automatically resolve certain classes of concurrency conflicts

However...

Not clear how this model maps to concurrency mechanisms that are used in the real-time domain (processes, threads, semaphores, real-time scheduling methods, etc.)?

The actual scheduling policy is unspecifiedA semantic variation pointCan be customized to suit application requirements

30

:GUI:GUI

:Scheduler:Scheduler:Scheduler:Schedulerreservationsreservationsreservationsreservations

:Planner:Planner:Planner:Planner updateupdateupdateupdate

Component and Deployment Diagrams Implementation focus:

ComponentComponent

NodeNode

Generally not sophisticated

enough for complex real-time

system needs

Generally not sophisticated

enough for complex real-time

system needs

31

Implementation Diagrams and Performance Concerns

Probably the weakest part of UML Not sophisticated enough to capture the various

complex aspects of deployment common to many performance sensitive systems

deferred mapping of software to hardware

mapping of software to software No standard way to describe the quantitative

requirements/characteristics of hardware and software (e.g., scheduling discipline, CPU speed, transmission delays)

32

Summary: UML and Performance Sensitive Systems

Using UML for performance sensitive systems automatically brings the benefits of the object paradigm

structural focus, inheritance, strong encapsulation, polymorphism,…

However, there are many open questions

best ways of using UML

missing or non-standard concepts

ability to create predictive models

33

(C) An Overview of the UML Profile for Schedulability, Performance and Time (SPT Profile)

A profile is an extension to the semantics of UML that conforms to the meta-model

a specialization, inheriting from the UML standard

a domain-specific interpretation

• e.g. UML for eCommerce It should use only standard extension mechanisms:

stereotypes, tag values, constraints

additional semantic constraints cannot contradict the general UML semantics

within the “semantic envelope” defined by the standard

34

Integer

An example of the UMLClass concept

Specializing UML: Stereotypes and tagged values

a stereotype is a specialization of a UML entity, implying a certain interpretation

MyClockClass

SetTime()

«clock»

{resolution = 500 ns}

«clock» Stereotype of Classwith added semantics:

an active counter whose value changes synchronously with the progress of physical time

...a stereotype can extend the class

Tagged value associatedwith the «clock» stereotype:

tag = valuevalue has a type such as:integer ‘sec’|‘ms’|‘ns’

35

Guiding Principles for the SPT Profile

Adopted as OMG standard in March 02 (ptc/2002-03-02)

Ability to specify quantitative information directly in UML models

key to quantitative analysis and predictive modeling Flexibility:

users can model their real-time systems using modeling approaches and styles of their own choosingopen to existing and new analysis techniques

Facilitate the use of analysis methodseliminate the need for a deep understanding of analysis methodsas much as possible, automate the generation of analysis models and the analysis process itself

Using analysis results for: predicting system characteristics (detect problems early)analyze existing system (sizing, capacity planning)

36

Stereotypes in the SPT Profile

grouped into sub-profiles

Resources (general resource model GRM)• e.g. GRMacquire

Time (real time RT)• e.g. RTclock

Concurrency (CR)• e.g. CRAction

Schedulability analysis (SA)• e.g. SAAction, SASResource

Performance analysis (PA)• e.g. PAScenario, PAStep

we will look at some of these

37

Resources and Services ...in the General Resource Model (GRM) Resource:

an element whose service capacity is limited, directly or indirectly, by the finite capacities of the underlying physical elements

Resources offer services that have QoS attributes

Resource R ...... set of services Si ...... QoS of Si «GRMacquire» is a stereotype that can be applied to a

wide range of base classes

stimulus, message, method, action, transition....

it has two tags:

• GRMblocking (boolean)

• GRMexclservice (reference to an action etc performed while the resource is held)

38

Quality of Service (QoS):

a specification (usually quantitative) of how well a particular service is (to be) performed

offered,

required an attribute of the service, stereotyped as some kind of

action or scenario

specified by tagged values offered QoS requires a system analysis since it depends

on load

39

Resource Usages

the same service can be used in multiple contexts (different users, different demands)... each is a usage

demand associated with each

static usage is “simple” and just has a demand

• no nested behaviour

a dynamic usage is defined by a scenario Categories of resources:

passive/active

protected/unprotected (e.g. stereotype «GRMprotResource»)

• a protected resource has access control Access control: exclusive, non-exclusive

40

Time

existing UML has time attributes defined for events

timing marks

message.sendTime, message.receiveTime the profile tries to go into depth

discrete vs “dense” (e.g., real-valued) time values

complex union type RTtimeValue for values stereotypes for

events with significant times (e.g. RTclkInterrupt)

timer, clock

41

Time Value tag type RTtimeValue can represent a wide variety of

types related to time, such as:

12:04 (time of day)

5.3, ‘ms’ (time interval)

2000/10/27 (date)

Wed (day of week)

$x, ‘ms’ (parameterized value with name $x)

‘poisson’, 5.4, ‘sec’ (time value with a Poisson distribution)

‘histogram’ 0, 0.28 1, 0.44 2, 0.28, 3, ‘ms’

P=0.28P=0.44

P=0.28

0 ms 1 ms 2 ms 3 ms

42

Behaviour abstraction in the SPT Profile

performance analyses sequences of operations the SA sub-profile has SAaction as its unit, with simple

precedence (no probabilities of branching)

suitable for deterministic analysis the Performance Analysis (PA) sub-profile has a more

complex Scenario stereotype

a Scenario is a sequence of Steps

a Step has predecessors (giving AND fork and join)

and also a probability (giving OR f/j, with suitable interpretation)

a Step inherits from Scenario, so it can be refined as a sub-scenario

43

Scenario (cont’d)

Scenario has a first Step, which is stereotyped with a workload definition

PAopenLoad with tags for an arrival process and a response time

PAclosedLoad with tags for a population and a think time, and a response time.

Step has demands

PAhostDemand tag gives the CPU demand (RTtimeValue)

PAextOp tag defines operations such as file operations, that are not modeled in the UML model but are needed for the performance model (completions)

PAdelay specifies a delay that may be part of the operation (a “sleep” time)

44

Example: Web Video Application

ClientWorkstation

WebServerNode

VideoServerNode

Internet

b : Browser

vs : VideoServer

vp : VideoPlayer

vw : VideoWindow

ws : WebServer

«deploys»

«deploys» «deploys»

Logical Model

Engineering Model

45

Web Video Application Behaviour

b : Browser ws : WebServer vs : VideoServer

vp : VideoPlayer

vw : VideoWindow

processSelection

confirm initialPlayout initializePlayer

sendFrame showFrame

terminalPlayout *N

delay

intervaljitter

46

Web Video: Performance Parameters

Est. video server processing demand per frame = 10 ms

Est. mean viewer processing demand per frame = 15 ms (standard deviation = 20 ms)

Assumed network delay distrib. is exponential, mean = 10 ms

Measured packets per frame (LAN) = 65

Measured video server file operations per frame = 12

Max. number of concurrent users = $Nusers

Average inter-session times = 20 minutes

Frames in a video = $N

Video frame intervval = 30 ms

Required confirmation delay: 95% percentile < 500 ms

Required interval between frame displays:

99% percentile value < 30 ms

47

Annotated Sequence Diagram

b : Browser ws : WebServer vs : VideoServer vp : VideoPlayer

vw : VideoWindow processSelection


sendFrame showFrame

terminalPlayout *

«PAcontext»

«PAstep» {PAdemand = (‘assm’, ‘mean’, 1, ‘ms’)}

«PAstep» {PAdemand =

(‘assm’, ‘mean’, 15, ‘ms’), (‘assm’, ‘sigma’, 10 )}

«PAclosedLoad»

{PApopulation = $NUsers, PAextDelay = (‘assm’,

‘mean’, (20, ‘ms’))}

«PAstep» {PArep = $N,

PAdemand = (‘assm’, ‘mean’, (10, ‘ms’)),

PAextOp = (‘filesys’, 12),

PAextOp = (‘network’, 65)}

«PAstep» {PAinterval =

(‘req’, ‘percentile’, 99, 30, ‘ms’)Painterval = (‘pred’, percentile, 99, $T)}

Res

ult

«PAstep»

{PArespTime = (‘req’, ‘percentile’, 95, (500, ‘ms’) PArespTime = (‘pred’,percentile,95, $R)}

Res

ult

48

Annotations... closer view

b : Browser ws : WebServer vs : VideoServer

vp : VideoPlayer

vw : VideoWindow

processSelection


sendFrame

showFrame

terminalPlayout *

«PAcontext»

«PAstep» {PAdemand =

(‘assm’, ‘mean’, 1, ‘ms’)}

«PAstep»

{PAdemand = (‘assm’, ‘mean’, 15, ‘ms’),

(‘assm’, ‘sigma’, 10 )}

«PAclosedLoad» {PApopul ation = $NUsers, PAextDelay = (‘assm’, ‘mean’, (20, ‘ms’))}

«PAstep»

{PArep = $N, PAdemand = (‘assm’, ‘mean’, (10, ‘ms’)), PAextOp = (‘filesys’, 12), PAextOp = (‘network’, 65)}



Result

«PAstep»


Result

«PAclosedLoad»

{Papopulation = $Nusers,PAextDelay = (‘assm’, ‘mean’, (20, ‘ms’))}



Result

«PAstep»


Res

ult

«PAstep»

{Parep = $N,Pademand = (‘assm’, ‘mean’, (10, ‘ms’)),PAextOp = ( ‘filesys’, 12 ),PAextOp = ( ‘network’, 65)}

49

Scenario Model as a Activity Diagram

select

service

<<PAstep>> >>

confirm

<<PAstep>> initialize player

<<PAstep>>

handle selection

<<Pastep>>

initiate playout

<<PAstep>>

terminate player

<<PAstep>>

send terminate

<<PAstep>>

vs : VideoServer ws : WebServer vp : VideoPlayer vw : VideoWindow b : Browser

«PAcontext»

«PAclosedLoad» {PApopulation = $NUsers,

PAextDelay = ( ‘assm’, ‘mean’, 20, ‘ms’)}

{PArespTime =

(‘req’, ‘percentile’, 95, 500, ‘ms’)}

{Pademand=(‘assm’, ‘mean’, 1, ‘ms’)}

{PArep = $N}

[~moreFrames] [moreFrames]

send video <<PAstep>> composite

activity (see next page)

Result

• Steps are stereotypes of activities

• a step can have an inner scenario

50

Composite Activity: Detailed Diagram

Detailed model for the composite activity from the previous page

(represented by a SubactivityState)

«PAstep»sendframe

«PAstep»showframe

«PAstep»receiveframe

vs : VideoServer vp : VideoPlayer vw : VideoWindow

{PAdemand=((‘assm','mean',15,'ms'),

('est','sigma',10))}

{PAdemand=(‘assm','mean',10,'ms'),PAextOp=('filesys',12),('network',65)}

{PAinterval=('req','percentile',99,30,'ms')}}

«PAcontext»

Result

51

Tags for specifying performance values

A complex structured string with the following format<kind-of-value> , <modifier> , <time-value>

where: <kind-of-value> ::= ‘req’ | ‘assm’ | ‘pred’ | ‘msr’required, assumed, predicted, measured

<modifier> ::= ‘mean’ | ‘sigma’ | ‘kth-mom’, <Integer> |‘max’ | ‘percentile’ <Real> | ‘dist’

Examples:measured CPU demand of a scenario step:{PAdemand = (‘msr’, ‘mean’, (20, ‘ms’))} scenario response time: required and predicted (variable $RT is a placeholder for a performance model result)

{PArespTime = (‘req’, ‘mean’, (10, ‘sec’))

PArespTime = (‘pred’, ‘mean’, $RT)}

52

General Process for Model Analysis

Software Domain Schedulability/ Performance Domain

53

(C) Deriving a performance model... what we need

UML Specification

multiple fragmented behaviour definitions

concurrent and passive components with processing states as Steps

transitions between components defined by Scenarios

connected via shared software components and resources

PLUS: a processing control specification (not UML)

Performance Model

concurrent components with

processing states and

waiting states queues transitions between

components times and probabilities for

transitions

specify values for parameters,

solution control, external services

54

Modelling paradigms State transition

build a global Markov Chain model, or a partitioned MC

• e.g. a submodel per component, with composition

• queue components Queueing

build a queueing network around resource use

collapse the execution steps into service times Layered queueing

a queue for each resource

service processes (“resource-operations”) which include:

• requests for nested resources

• sequences of processing states (sub-scenarios)

ordinary queueing is a simple special case

55

Deriving models with Queues and Layered Queues (quick overview)

Basic Queueing Network Resources are held one at a time

demands are holding times

view of path and resource contexts of operations:

Scenario generates a probabilistic sequence of resource requests, each with a demand

different demands are different classes of service, or different resource operations

Separable network gives analytic solution

only requires demand totals by class and resource

56

Deriving models with Layered Queues (1)

Notation for Layered Queues Resource Resource-operation (simple) Resource-operation (complex)

expressed via interaction types: sync, async, forward

Task(resource)

Entry(resource-operation)

57

Deriving models with Layered Queues (2)

Notation for Layered Queues Resource Resource-operation (simple): one step Resource-operation (complex)

expressed via interaction types: sync, async, forward

Task(resource)

sync

async or forwarded

activity activity

58

Deriving models with Layered Queues (3)Notation for Layered Queues

Resource Resource-operation (simple): one step Resource-operation (complex)

activity sequence relationships (AND/OR fork/joins)

interaction types: sync, async, forward

Task(resource)

activity activityactivity

sync

asyncsyncsync

59

Specifying Layered Queues, in a nutshell “Processor” (device) has a queue, speed, multiplicity “Task” (logical resource) has a host device, queue parameters,

multiplicity and a set of Entries “Entry” (service) is specified by a sequence of activities

(==Steps), defining a sub-scenario for one service

a restricted kind of sub-process “Activity” (operation, action) is specified by a host demand,

and a set of interactions with other entries Interactions have a mean count. They may be:

Synchronous call to another entry

Asynchronous call to another entry

Reply to the caller of this entry (only one can occur)

Forward to an entry (instead of reply)

60

Layered Queues: entities and parameters

ServerTgetData

[4]

AppTcompute[AppE][45 ms]

log

cleanup

UserTUserE UserTuserE[8 sec]

AppP

ServerPLogT

logE[12]

appE

cleanup[3,8]

1 sync call

multiple (25) UserT entities with default activity and its host demand

(1)

(2)async call

host association

UserPUserP inf

one server for each client

return

2 sync calls (1)

(1)

{25}

61

Layered Queues: visualize the scenario

ServerTgetData

[4]

AppTcompute[AppE][45 ms]

log

cleanup

UserTUserE UserTuserE[8 sec]

AppP

ServerPLogT

logE[12]

appE

cleanup[3,8]

1 sync call

multiple UserT entities with default activity and its host demand

(1)

(2)async call

host association

UserPUserP inf

one server for each client

return

2 sync calls (1)

(1)

62

LQN for Video Example

an exercise... model by student A. Bagchi...not precisely the same system and parameters as the UML

concentrate on the Receive operation

sends 20,000 framesread from disk, send

20000 10

2

Users

WebSDisk[ 0.9]

WebSProc

WebServerPCDisk[17]

PlayerShow[0,25]

Receive[1]

PlayerInit [180]

Select[30]

VideoServerVidEnd[35]

SendFrame[1]

VidInit[1]

VidSelect[1 ]

ProcessSelect [15.6]

FrameHandler RetrvSend [11]

VidSDisk[10]

pcproc

Vproc

Vdisk

disk

Wdisk

$NUsers

$NUsers

63

Video Server Results Case (a)

0

0.001

0.002

0.003

0.004

0.005

0 20 40 60

# User

Th

rou

gh

pu

t

Case (a)

0

2000

4000

6000

8000

10000

12000

0 20 40 60

# User

Re

spo

nse

Tim

e,

sec

20,000 frames in a video at 30 frames/sec take 667 sec.

In this model we let the frames be delivered as fast as they can and look at the total time (a crude measure)

Barely OK for one user

64

Parallel fetch and send of frames

no gain in this model

consider randomness on branches

VDManager

VideoServer

VidEnd[35]

SendFrame[1]

VidInit[1]

VidSelect[1 ]

Player

Show[0,25]

Receive[1]

PlayerInit [180]

Select[30]

20000

A3

A4

A2

A1

10

2

User

WebSDisk[ 0.9]

WebSProc

WebServer

PCDisk[17]

PCProc

ProcessSelect [15.6]

FrameHandler

RetrvSend [0]

VidDisk[10]

VidSProc

Send[10]

Retrieve[1]

Case (b)

0

0.001

0.002

0.003

0.004

0.005

0 20 40 60

# User

Th

rou

gh

pu

t

Case (b)

0

2000

4000

6000

8000

10000

12000

0 20 40 60

# User

Re

sp t

ime

, se

c

65

Additional design experiments

lower variance in frame operations times would be realistic

(here it was assumed to be exponential processing and a geometric number of blocks and packets)

asynchronous delivery to receiver buffering in the receiver additional buffering in the video retrieval chain

got simultaneous streams up to 3.

66

Composition of Submodels from scenarios

EE2

EA2P2

P3

TA

TE

T2E2

ED1

EA1

P1

P3

TA

TD

T1E1

EC1TC

EE1TE

ED1

EA2

P1

P3

TA

TD

T1E1

EC1TC

EE2TE

EA1

T2E2

EE1

P2

DriversCommonDrivers

(0.3) (0.7)From Scenario

1 (30%)From Scenario

2 (70%)

67

Layered Queueing Solution by Iterative MVA

... An economical and elegant model considering the complexity of the systems

... Just tasks entries (=classes) activities 3 kinds of

interactionsm=2 1

m=1 1 1

m=N1 3 1 N2

E6E5

E3

Z1 Z2

P6

P1 P2

P3

P4

m = N1 N2

E4a E4b

Layer 1

Layer 2

Layer 3

68

Layered Queueing Algorithms in brief Each layer is an ordinary

queueing network tasks are servers in one layer,

customers in another each layer model depends on

the othersservice time depends on lower layertraffic intensity comes from upper layer

solve by iteration among the layers

each layer solution requires extended queue approximations

m=2 1

m=1 1 1

m=N1 3 1 N2

E6E5

E3

Z1 Z2

P6

P1 P2

P3

P4

m = N1 N2

E4a E4b

Layer 1

Layer 2

Layer 3

69

Layered queueing: summary

it describes all logical resources, not just software tasks and threads

it resolves nested services by multiple resources it embeds subprocesses into services a design model: it represents abstract constructs by

implementations

e.g., barrier sync by messages to a barrier server analytical solutions by iteration are approximate and

don’t always converge (!) generalizations appear to be possible

e.g., non-nested resources can be represented but not solved analytically

hard: complex logical conditions like 2 locks out of 5

70

Deriving Markov models

via a higher-level formalism (stochastic version of Petri nets, Process algebra, FSP, SDL, ASM...

typically... structured around the State Machine of a concurrent

component (e.g. DEGAS)

instances of software components defined in UML

need to add queue components for software and hardware

composition, with suitable interpretation of inter-component events

system state is a tuple of states of instances of components

ultimately, solution depends on a global Markov Chain

71

Markov models: where are scenarios?

scenarios are still relevant:

they identify behaviour with requirements

they describe delays to be computed they are buried in the transition behaviour of the state

machines

along with a lot of other behaviour

so, scenarios are mostly represented via the extraction of performance measures

mean utilizations and rates come from standard reward analysis

mean delays are then easy by Little’s result

other moments and percentiles are more challenging

72

Challenge of Scenarios to Markov Modeling Scenario is one case out of several outcomes for the

same event

e.g., delay for a successful operation may have different requirements from delay for unsuccessful

initiating event is the same, outcome is modeled as random choice.

scenarios are selective Requirements are different for different users starting

the same operation

requires classifying the users even though the UML classes, objects, states and transitions are the same for all.

73

Challenge of queues to Markov modeling

Queues introduce huge state spaces Markov modeling of a queue is more efficient if a

random selection policy is adopted

order in queue is then unimportant

fewer states but, software tasks usually serve in FIFO order and

often have multiple classes of clients

detailed modeling of order may be important Layered queueing with activities may be seen as a

decomposition around servers

queueing calculations for delays in queues

states of processing within a service

74

Proposed PUMA toolset architecture(Performance by Unified Model Analysis)

Software Design Tool (UML, UCM, etc.)

Performance Tool (LQN, QN, etc.)

Sensitivity and

Optimization Tools

Core Model CSM

Core Scenario Model (in L1)

CPM

Core Perf. Model (in L2)

Annotated Design Spec

Results and

guidance

Performance Model

Results

solve

diagnose performance problems

place recommendations in design context

translate to L1

transform L1 to L2

generate perf.

model

feedback results

transform L2 to L1

Figure 1: The proposed system to support scenario-based performance engineering general software model input via CSM (not only UML) general performance model types via CPM (not only

layered queues) includes heavy element of model investigation,

sensitivity tools, optimization proposal also for component libraries for completions

75

PUMA comes from Past Model-building from scenarios

Traces

Angio traces (thesis of Curtis Hrischuk

• (IEEE TSE 2000 for “TLC” paper with Hrischuk)

(“SAME” tool) (thesis of Tauseef Israr) Scenario analysis

UML scenarios, by XML tools and graph grammars

• Dorina Petriu, Hui Shen, Gordon Gu (TOOLS02 and WOSP02)

Use Case Maps scenarios to specify scenarios

• (UCM2LQN tool) (thesis of Dorin Petriu) (TOOLS02)

Murray Woodside Carleton University, Ottawa... November 2002

Documents

performance model

evolution of uml

performance problem

performance profilepart

software performance

entire system

variance delay value

actual system availability