Copyright 2003 Applied Expert Systems, Inc. 1 NCCMG November 4th, 2003 Catherine Liu Applied Expert Systems, Inc. Menlo Park, California.

Copyright 2003 Applied Expert Systems, Inc.Copyright 2003 Applied Expert Systems, Inc.www.aesclever.comwww.aesclever.com

1

NCCMG November 4th, 2003

Catherine Liu Catherine Liu

Applied Expert Systems, Inc.Applied Expert Systems, Inc.

Menlo Park, CaliforniaMenlo Park, CaliforniaPermission is granted to NCCMG

to distribute images of the slides of the presentation to attendees and members. All other rights reserved.

It’s All About It’s All About Performance!Performance!


2

AgendaAgenda

The eBusiness Building BlocksThe eBusiness Building Blocks The eBusiness Performance FocusThe eBusiness Performance Focus The eBusiness Performance The eBusiness Performance

Management ProcessManagement Process ePerformance Monitoring ePerformance Monitoring ePerformance Reporting ePerformance Reporting ConclusionsConclusions


3

eBusiness is “eBusiness is “The The Business”Business”

Online sales surged 48% in 2002Online sales surged 48% in 2002 70% of retailers reported positive 70% of retailers reported positive

operating margins from Web sales operating margins from Web sales Compares to 56% profitability in ‘01Compares to 56% profitability in ‘01 ““Brick & Mortar” Co’s welcome these Brick & Mortar” Co’s welcome these

positive eBusiness operating marginspositive eBusiness operating margins Web sales often able to offset business Web sales often able to offset business

losses from “offline” operations losses from “offline” operations


4

eBusiness is “eBusiness is “The The Business”Business”

Most new user applications will be Most new user applications will be Web-based designsWeb-based designs

Intranet/Internet is now accepted Intranet/Internet is now accepted “norm”, or standard access method“norm”, or standard access method

$1 trillion+ in existing mission-$1 trillion+ in existing mission-critical apps still need critical apps still need enhancements or rewrites to enhancements or rewrites to become Intra- or Internet become Intra- or Internet accessibleaccessible


5

The eBusiness Building BlocksThe eBusiness Building Blocks

VTAM Buffers

TCP/IP

MVS

CSMVTAM Buffers

TCP/IPVTAM User Apps

ChannelLinks

RoutersSwitches

Hubs

Servers

Clients

NetworkConnections

Appl.Servers

World Wide Web


6

The Ring of eBusinessThe Ring of eBusiness

TCP/IP Routes

Web


7

The eBusiness ChallengeThe eBusiness Challenge

Managers and analysts still rely on the Managers and analysts still rely on the heritage of SNA-based measurementsheritage of SNA-based measurements

Different performance “pulse”Different performance “pulse” for for sessions & services of eBusiness appssessions & services of eBusiness apps

Different “skeletal” framework Different “skeletal” framework for for eBusiness customers and connectivity eBusiness customers and connectivity

Yet, Intranet/Internet networking Yet, Intranet/Internet networking model model needs similar performance needs similar performance measurement & managementmeasurement & management


8

It’s All About It’s All About PerformancePerformance

It’s easy to build a Web-siteIt’s easy to build a Web-site It’s not so easy to build a site It’s not so easy to build a site

with…with… Balanced ContentBalanced Content Good Human InteractionGood Human Interaction Acceptable PerformanceAcceptable Performance


9


Many businesses rely on fast, Many businesses rely on fast, instant response to stay instant response to stay competitive, such as…competitive, such as… E-TradeE-Trade eBayeBay

Prefer to simplify Web-site designPrefer to simplify Web-site design Limit page clutterLimit page clutter Don’t load too many gifsDon’t load too many gifs Avoid extra actions and large filesAvoid extra actions and large files


10


But common-sense ePerformance But common-sense ePerformance techniques only go so far…techniques only go so far…

Performance industry must focus Performance industry must focus on…on… The implementation of an The implementation of an

ongoing ePerformance Solutionongoing ePerformance Solution The improvement and assurance The improvement and assurance

of eBusiness Service Levelsof eBusiness Service Levels


11

Focus on ePerformanceFocus on ePerformance

Monitor and Report on:Monitor and Report on: Web and other eBusiness ApplicationsWeb and other eBusiness Applications

- End-to-End Service Level monitoring- Response Times for Web transactions

TCP/IP PerformanceTCP/IP Performance- End-to-End network response times- Effects of operational performance

control Network Routes and EquipmentNetwork Routes and Equipment

- End-to-End route response times- Identify route patterns and defects


12

End-to-End eBusiness End-to-End eBusiness Response TimeResponse Time


13

What is End-to-End (E-2-E) What is End-to-End (E-2-E) eBusiness Response eBusiness Response Time?Time? The response time for mission The response time for mission

critical Web pages or Web critical Web pages or Web transactionstransactions

You can measure both Internet or You can measure both Internet or Intranet operations quicklyIntranet operations quickly

Can be performed from single or Can be performed from single or multiple users point of viewmultiple users point of view

Measuring and monitoring this is Measuring and monitoring this is absolutely doableabsolutely doable


14

An ePerformance Web-An ePerformance Web-site SLO Solution site SLO Solution

ExampleExample Establish easy-to-track, corporate Establish easy-to-track, corporate

”Web-Unit” Service Level Objectives”Web-Unit” Service Level Objectives Primary Web Metrics:Primary Web Metrics:

Web-site Availability (site up-time)“Web-Unit” Performance (page load-time)

Need capability to view and analyze Need capability to view and analyze each in real-time, as well as each in real-time, as well as historicallyhistorically

Reports must be reflective of the Reports must be reflective of the specified Service Level Objectivesspecified Service Level Objectives


15

Basis for Web-site SLAsBasis for Web-site SLAs

Use SLAs strategically for Web sitesUse SLAs strategically for Web sites External Sites:External Sites:

Key indicator of eBusiness health and status as the new benchmark for corporate ‘vitality’

Internal Sites:Internal Sites:Now used for critical corporate functionsObvious and essential tie-in to revenue,

processes and productivity Tool for exceptions & tuning Tool for exceptions & tuning

diagnosticsdiagnostics Foundation for site capacity planningFoundation for site capacity planning


16


17


18

•Monitor “load times” for critical URL’s in real time. •Focus on SLO thresholds for “web-unit” Response Times.


19

•Report on avg. & minimum response times for key URL’s historically.•Focus on daily website availability for SLA’s.


20

End-to-End Network End-to-End Network Response TimeResponse Time


21

What is End-to-End (E-2-E) What is End-to-End (E-2-E) Network Response Time?Network Response Time?

The IP-based network response time The IP-based network response time from a source device to the from a source device to the destinationdestination

or, the reverseor, the reverse IP socket transit time, not necessarily IP socket transit time, not necessarily

at the application levelat the application level Measuring and monitoring this is just Measuring and monitoring this is just

as critical to assure Service Levels for as critical to assure Service Levels for user Web transactions can be metuser Web transactions can be met


22

End-to-End Route End-to-End Route Response TimeResponse Time


23

What is End-to-End (E-2-E) What is End-to-End (E-2-E) Route Response Time? Route Response Time?

Added details of network response Added details of network response time time

Time spent by packets as they travel Time spent by packets as they travel through individual route segmentsthrough individual route segments

Necessary to observe over timeNecessary to observe over time Focus on identifying…Focus on identifying…

Longest routes in terms of hopsLongest routes in terms of hops Longest segments in terms of response timeLongest segments in terms of response time


24


25

Steps to ePerformance Steps to ePerformance ManagementManagement

Initiate Service Level MonitoringInitiate Service Level Monitoring Establish Service Level ObjectivesEstablish Service Level Objectives Establish Service Level Establish Service Level

AgreementsAgreements Measure, Analyze and ReportMeasure, Analyze and Report Manage and MaintainManage and Maintain


26

Initiate Service Level Monitoring

Key IP stack and network indicatorsKey IP stack and network indicators Channel links to the IP networkChannel links to the IP network CSM and VTAM buffer usageCSM and VTAM buffer usage Services, applications, and portsServices, applications, and ports

Client session activity and failuresClient session activity and failures End-to-end network availability, End-to-end network availability,

performance, and throughputperformance, and throughput


27

A Sample ePerformanceA Sample ePerformance Monitoring Solution Monitoring Solution

Utilize Real-Time Monitoring AlertsUtilize Real-Time Monitoring Alerts Check IP focal point for health indicatorsCheck IP focal point for health indicators Check CPU performance of stack & appls Check CPU performance of stack & appls Check CSM buffer usageCheck CSM buffer usage Check status and performance of IP linksCheck status and performance of IP links Check IP application port statusCheck IP application port status Check session logs for recent activityCheck session logs for recent activity


28


29

A Sample ePerformanceA Sample ePerformance Monitoring Solution Monitoring Solution

Utilize Near-Time Analysis & DiagnosticsUtilize Near-Time Analysis & Diagnostics Check for availability/performance alerts, Check for availability/performance alerts,

as well as network connectivity eventsas well as network connectivity events Review user and session activityReview user and session activity Check hardware status & performanceCheck hardware status & performance Verify route status and performanceVerify route status and performance Isolated monitoring of specific end usersIsolated monitoring of specific end users Perform further route or cTrace Perform further route or cTrace

diagnosticsdiagnostics


30

Establish SLOs & SLAsEstablish SLOs & SLAs

Should include, but not Should include, but not necessarilynecessarily

be limited to, the following:be limited to, the following:

Web Application PerformanceWeb Application Performance Network AvailabilityNetwork Availability Network Response TimeNetwork Response Time Route PerformanceRoute Performance


31

Introducing the I-UnitIntroducing the I-Unit


32

A Sample ePerformanceA Sample ePerformance E-2-E SLO Solution E-2-E SLO Solution

Establish Establish “I-Unit” as an E-2-E Metric as an E-2-E Metric PacketPacket equates to Unit of Data that is equates to Unit of Data that is

passed between IP & network interfacepassed between IP & network interface Inter-Network Unit of Data = I-UnitInter-Network Unit of Data = I-Unit Focus is on consistent measurement of Focus is on consistent measurement of

corp. network response time for I-Unitcorp. network response time for I-Unit Business entities can set specific I-UnitBusiness entities can set specific I-Unit

based on analysis & customer based on analysis & customer consensusconsensus


33

Basis for ePerformanceBasis for ePerformance E-2-E SLAs E-2-E SLAs

Then standardize with I-Unit SLAsThen standardize with I-Unit SLAs Easy to implementEasy to implement Consistently measurableConsistently measurable Good for inside users or outside Good for inside users or outside

customerscustomers Use as accountability tool for External Use as accountability tool for External

Service Providers (ISPs, ASPs)Service Providers (ISPs, ASPs) Tool for exception and tuning diagnosticsTool for exception and tuning diagnostics Establish baseline for capacity planningEstablish baseline for capacity planning Foundation for costing and charge-backFoundation for costing and charge-back


34

Enterprise Network Enterprise Network Service Level Service Level

MeasurementsMeasurements


35


36

Batch/PR (Batch/Performance Reporter) Date: 12/16/2002 REPORT: BatchPRSLS A1 Special Company DATE: 2002.350 SHIFT: 1 FROM Hour: 9 TO Hour: 15 NETWORK AVAILABILITY RATING Total # of Critical Resources Monitored: 16 Total # of Observed Events: 4012 Total # of Resources Unavailable Events 292 Percentage of Network Availability: 92.722% Top 3 Critical Resources Reporting Unavailable Events Resource IP Address # Events # Unavailable Percentage--------------------- ---------------- ----------------- -------------- 137.72.43.244 991 85 8.577% 137.72.43.21 260 81 31.154% 137.72.43.141 371 65 17.520% NETWORK RESPONSE TIME RATING Total # of Critical Resources Monitored: 16 Total # of Observed Events: 4012 Total # of Threshold Exceeding Events: 123 Percentage of Network RT within Threshold: 96.934% Top 3 Critical Resources Exceeding RT Threshold: Resource IP Address # Events # Over Threshold Percentage--------------------- ---------------- ----------------- -------------- 216.32.74.53 282 44 15.603% 137.72.43.241 319 36 11.285% 137.72.43.242 284 33 11.620%


37

Reporting Reporting


38

The ePerformance The ePerformance Management ProcessManagement Process

Measure, Analyze and ReportMeasure, Analyze and Report Report Routinely on Key MetricsReport Routinely on Key Metrics

Timeframe-focused (shift, daily, weekly…)

Management-level summariesTechnical-level detailsWorkload (sessions and traffic)Performance (I-Unit, FTP

throughput…)Exceptions (peaks, valleys, hangs,

failures…)


39

ePerformance IP Workload ePerformance IP Workload Reporting Solution ExampleReporting Solution Example

Implement Weekly IP Activity ReportsImplement Weekly IP Activity Reports Report by weekly timeframesReport by weekly timeframes Report by session totals and data Report by session totals and data

traffictraffic Selectively by specific applicationSelectively by specific application Selectively by IP addresses or DNS Selectively by IP addresses or DNS

clientsclients Identify top users and busiest Identify top users and busiest

applicationsapplications Hardcopy batch scheduled reportsHardcopy batch scheduled reports


40


41

Batch/PR Workload Server Application Summary Date: 12/21/2002 Report: WLServer A1 Special CompanyDate From: 12/16/2002. To: 12/20/2002.Days of Week included: Monday Wednesday Friday Server Application Groups Name # Ports Sessions Total Bout BOut Max BOut Avg Total Bin BIn Max Bin Avg TELNET 23 17 9693810 3485159 570224 3155902 2229102 185641 TELNET 1023 0 0 0 0 0 0 0 Total 17 9693810 3485159 570224 3155902 2229102 185641 FTP 20 4 19652 5840 4913 0 0 0 FTP 21 10 4868 715 486 901 131 90 FTP 1020 0 0 0 0 0 0 0 FTP 1021 0 0 0 0 0 0 0 Total 14 24520 5840 1751 901 131 64 DB2 6001 472 113945 21176 241 29694 154 62 Total 472 113945 21176 241 29694 154 62 DBMOVER 6930 0 0 0 0 0 0 0 Total 0 0 0 0 0 0 0


42

Specific application Specific application reportingreporting


43

A Sample ePerformance FTP A Sample ePerformance FTP Activity Reporting SolutionActivity Reporting Solution

Utilize Key FTP Reports & Auditing Utilize Key FTP Reports & Auditing Selectively by time framesSelectively by time frames Selectively by user ID, dataset, IP Selectively by user ID, dataset, IP

addressaddress Selectively by Server or Client sessionsSelectively by Server or Client sessions Identify session totals and trafficIdentify session totals and traffic Identify FTP abuses, hangs and failuresIdentify FTP abuses, hangs and failures Utilize FTP session logsUtilize FTP session logs Run cTrace for problem FTP’s & packetsRun cTrace for problem FTP’s & packets


44


45

Highly Detailed Highly Detailed Diagnostic InformationDiagnostic Information


46


47cTrace database analysis zoom-in to specific trace record


48

The ePerformance The ePerformance Management ProcessManagement Process

Manage and Maintain:Manage and Maintain: Real-time monitoring, Sysplex-wideReal-time monitoring, Sysplex-wide Near-time analysis & diagnostic Near-time analysis & diagnostic

toolstools Valid alerts & thresholds for key Valid alerts & thresholds for key

criteria criteria Consistently measurable SLOs Consistently measurable SLOs Historical reporting methodologyHistorical reporting methodology Historical trending capabilityHistorical trending capability


49

The ePerformance The ePerformance Management SolutionManagement Solution

Manage Key ePerformance SubjectsManage Key ePerformance Subjects Multiple stacks simultaneouslyMultiple stacks simultaneously Multiple TCP/IP levels simultaneouslyMultiple TCP/IP levels simultaneously Multiple thresholds and alerting Multiple thresholds and alerting

methodsmethods Interval-based and/or shift specificInterval-based and/or shift specific Enterprise-wide for IP resourcesEnterprise-wide for IP resources Management summaries or detail Management summaries or detail

specificspecific


50

The ePerformance The ePerformance Maintenance SolutionMaintenance Solution

Maintain Key ePerformance MetricsMaintain Key ePerformance Metrics Event-based, SMF data (Type 118/119)Event-based, SMF data (Type 118/119) Interval-based, SMF data Interval-based, SMF data (Performance and Workload)(Performance and Workload) TCP/IP session log activityTCP/IP session log activity Alert history and command audit logsAlert history and command audit logs Tracerte data collected automaticallyTracerte data collected automatically CSM and VTAM buffer usage historyCSM and VTAM buffer usage history SNMP MIB data (Enterprise-wide)SNMP MIB data (Enterprise-wide)


51


52

Reviewing the ePerformance Reviewing the ePerformance Management Process Management Process

• Initiate Service Level Monitoring

• Establish Service Level Objectives

• Establish Service Level Agreements

• Measure, Analyze and Report

• Manage and Maintain the Process


53

Achieving the Real Achieving the Real ePerformance Management ePerformance Management

GoalGoal

SLMsSLMs SLAsSLAs + SLOs+ SLOs

ROI (Time + Money)ROI (Time + Money)plus… plus…

Satisfied CustomersSatisfied Customers


54

Working together, we can manage it!

TCP/IP Routes

Web

Copyright 2003 Applied Expert Systems, Inc. 1 NCCMG November 4th, 2003 Catherine Liu Applied Expert Systems, Inc. Menlo Park, California.

Documents