Performance Metrics Panel Monday, June 23 2008. Panelists John Rauser - Moderator Peter Sevcik- NetForecast Eric Goldsmith - AOL Eric Schurman - Microsoft.

Performance Metrics Panel

Monday, June 23 2008

Panelists John Rauser - Moderator

Peter Sevcik- NetForecast

Eric Goldsmith - AOL

Eric Schurman - Microsoft

Vik Chaudhary - Keynote Systems

How Apdex Works

Tolerating

ApdexT= Total samples

Satisfied2+

Frustrated

Satisfied

Tolerating

Report Group:ApplicationUser GroupTime Period

Existing TaskResponse TimeMeasurement

Samples

T1

2

3

4

5

6F

1. Define T for the applicationT = Target time (satisfied-tolerating threshold)

F = 4T (tolerating-frustrated threshold)2. Define a Report Group3. Extract data set from existing measurements4. Count number of samples in three zones5. Calculate the Apdex formula6. Display Apdex value showing T7. Optionally display value using quality colors

7

0.91 [6]

Value T

Good

Fair

Poor

Un

acc

epta

ble

0.00T

0.50T

1.00T

0.85T

0.94T

0.70T

Excellent

Apdex Methodology

• Measure• Gather baseline data• Test Apdex Parameters

A T

1) Work withusers to set T

2) Work with businessmanagers to define theservice objective for A

• Apdex reports & trends• Improve performance

where needed• Continual performance

& process improvement

Gather Agree Improve

The Apdex Alliance

Manufacturing

IT Product Manufacturing

FinancialServices

Government

Transportation

Education & Research

Healthcare

Carriers& Utilities

Business & IT Services

Other

ContributingMembers

See Apdex.org to learn more

More that 800Supporting Members

Apdex Report Example

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0.45

0.50

0.55

0.60

0.65

0.70

0.75

0.80

0.85

0.90

0.95

1.00

1-A

ug

-07

2-A

ug

-07

3-A

ug

-07

4-A

ug

-07

5-A

ug

-07

6-A

ug

-07

7-A

ug

-07

8-A

ug

-07

9-A

ug

-07

10

-Au

g-0

7

11

-Au

g-0

7

12

-Au

g-0

7

13

-Au

g-0

7

14

-Au

g-0

7

15

-Au

g-0

7

16

-Au

g-0

7

17

-Au

g-0

7

18

-Au

g-0

7

19

-Au

g-0

7

20

-Au

g-0

7

21

-Au

g-0

7

22

-Au

g-0

7

23

-Au

g-0

7

24

-Au

g-0

7

25

-Au

g-0

7

26

-Au

g-0

7

27

-Au

g-0

7

28

-Au

g-0

7

29

-Au

g-0

7

30

-Au

g-0

7

31

-Au

g-0

7

Ap

dex

[1

.1]

Unacceptable

Poor

Fair

Good

Excellent

A

B

C

Memory Constraints During Peak Hours

Ap

dex

[0

.1]

Where do you draw the line? Timeout choice impacts results

Large timeout value - longer response times reflected as higher aggregate response time Small timeout value - longer response times reflected as lower availability

30s Timeout 20s Timeout 10s Timeout

Performance 11.47s 10.22s 8.40s

Availability 100% 91.91% 62.50%

0

5

10

15

20

25

30

Performance (seconds)

iGoogle Index Widget

• Data Feed every 15 mins

• Embedded Object in iGoogle

• Volume of data requires adata warehouse

My Company

Google Maps Mashup

• Data Feed every 15 mins or 24 hours

• Integrated with Google Maps API

• Scripts of top Retail web sites

• For: Customer with geographically dispersed user-base

• Google API legal agreement requires data being public.

• If data is private then $10,000 license fee (to Google) applies

Yahoo! Alarms Widget

• Real-Time DataPulse

• 2 min frequency

• Stand-Alone Desktop Application

• Requirement: load Yahoo! Widget Framework

• Used by NOCs