Faster Apps, Faster Time to Market, Faster Mean Time to Repair Brad Goddard Director of APM Pre-Sales Engineering - Asia and India Compuware Ardeshir Arfaian Solution Director dynaTrace APAC Compuware
Jun 27, 2015
Faster Apps, Faster Time to Market, Faster Mean Time to Repair Brad Goddard Director of APM Pre-Sales Engineering - Asia and India Compuware Ardeshir Arfaian Solution Director dynaTrace APAC Compuware
Compuware Application Performance Management
• Web, non-Web, mobile, streaming, cloud-based applications
• Across all customers, users, browsers, devices, infrastructure, and geographies
• Rapid issue notification with actionable diagnostics
• Insight into how these issues affect your business (revenue, brand, cost)
We help organizations optimize the performance of their business-critical applications
SaaS, Cloud-Based and
On-Premises Offerings
• Rapid startup and payback
4,000+ Customers Worldwide
• 2,500+ enterprise customers
• 1,500+ SMB customers
• 12 of top 20 US sites
Global Reach • Over 80 offices in
29 countries worldwide
• Strategic service delivery
Recognized as Industry Leader
• Gartner: Leader in APM magic quadrant
• Forrester Research: “…a complete view of end user experience”*
• Ovum: “Game-changing”
*”Trends: The Diversification Of End User Experiencing Monitoring”, Forrester Research, Inc., July 5, 2011
Your world is changing
Customers: Global
Applications: Distributed and loosely coupled
New Devices: Proliferating
Virtualization/Cloud: Exploding
Application visibility and optimization of the customer experience are more important than ever.
Impact of to the business
The Problem Lifecycle
Why Agile Development took off
It‘s Sprint Time!
Story Points
Sprint Timeline Estimate
Remaining
Team Velocity
Development Testing
Production
You are in control!
Story Points
Sprint Timeline Estimate
Remaining
Team Velocity
Developme
nt
Testing
Production
What happened?
Story Points
Sprint Timeline Estimate
Remaining
Team Velocity
Developme
nt
Testing
Production
Missed Goals and Estimates Story Points
Estimate
Remaining
Team Velocity
Developme
nt
Testing
Production Missed
Goal
Mis
se
d
Es
tim
ate
s
4 of 5 projects run over
time and/or budget.
11 Oxford University Regarding ITÂ Project Success (Saur & Cuthbertson, 2003)
Problem #1: Different Mindset
Source: http://dev2ops.org/blog/2010/2/22/what-is-devops.html
Problem #2: Dislocated Teams
Source: http://dev2ops.org/blog/2010/2/22/what-is-devops.html
Problem #3: Different Tools
Source: http://dev2ops.org/blog/2010/2/22/what-is-devops.html
Problem #4: Over the Fence Attitude
Source: http://dev2ops.org/blog/2010/2/22/what-is-devops.html
These Problems lead to …
Source: http://dev2ops.org/blog/2010/2/22/what-is-devops.html
A potential Solution
Source: http://dev2ops.org/blog/2010/2/22/what-is-devops.html
Perf Test in
CI
Cloud based Testing
ONE Toolset Architecture
Validation
Test in Production
Real World Feedback
Traditional Load Testing
Minimize and automate real Load Tests Developing
Test Run Reproduction Refine Capturing
Re Run Tests Reproduction Refine Capturing
Re Run Tests Reproduction Problem Analysis
Problem Solving
Multiple Test Iterations
needed to analyze
Root-cause
time
Reproduction Refine Capturing
Re Run Tests Reproduction Refine Capturing
Re Run Tests Reproduction
Problem Solving
•Eliminates Test Iterations
•Go directly to problem analysis
•Frees up resources for other projects
Developing
Test Run
time
Problem Analysis
Why Web Performance Matters: Impact of Poor Performance
Source: Steve Souders @ Velocity Conference 2009 http://radar.oreilly.com/2009/07/velocity-making-your-site-fast.html
2 second
slowdown 4.3 reduction in revenue/user*
%
400 millisecond delay
0.59 %
fewer searches/user*
found that a
determined that a
21
…. 10000 Smart Phones
Sold
22
…. 80000 electronic accessories sold
23
10B
eBay Marketplace = Economy of Scale
300 300M live listings 100 100M active users
9 9 Petabytes of data storage
2 2B page views/day
75 75B database calls/day 10000 10,000 application servers
$62 $62B 2010 gross merchandise volume
40 40M lines of code
10B URL Requests / day
5 5K search engine nodes
24
Commercial data warehouse 100x larger than the research library of
US Congress
Pertinent Problems to be solved @ eBay
• Search
• Trust, Fraud and Risk
• Shipping and Logistics
• Ease of Payments
• User Experiences & Site Speed
• Data , Analytics and Business Intelligence
• Performance …
• … and many more 25
26
S No eBay Requirements Status
1 Deeper insight into the application very quickly, identifying the areas of code where the majority of each transaction's time is spent.
2 Integrate with Silk Performer / JMeter
3 Java Diagnosis at method/class level.
4 API Breakdown chart
5 Memory Analysis graph
6 Dashboard showing a comparison between 2 different test runs
7 Trace export for QA, Dev
8 Business and Technical dashboards
9 Execution time / Time spent in individual methods of the Application code base
10 Time Spent on Service calls. (Entry/Exit times only)
11 Performance of SQL Queries.
12 Reports that would help identify the slow parts of the Application
13 To be able to configure and monitor performance of specific business flows.
Benchmark Criteria
27
Link to Compuware APM
28
Selected transactions opens in Compuware APM
29 Are all my tiers healthy?
How much time is spent on which
tier?
Detailed view of transaction and flow
30
Layers Transaction
spent time in
Each individual
transaction listed
Selected transaction
spent 42.77
milliseconds
31
API level Drill down to
identify the method and
the call path having
maximum performance
impact
Global Solution Provider
Financial Services
Transaction Breakdown <1sec, 1-2sec, 2-3sec, 3-4sec, 4-5sec, >5sec
With increasing load number of
Outliers >5sec is increasing
Only 85.44% of transactions under 1 second
Goal is to have 90% of transactions
under 1 second.
High RMI execution time
High Connection Checkin/Checkout
time
JDBC Connection Check-in/Check-out (1)
High Avg wait time for a connection
(10 seconds)
Low CPU / Low Memory consumption / High GC
Memory Utilization never climbs
above 25 % on certain JVMs.
Even though GC is high.
High GC JVM is spending 5.75 minutes per
minute on GC
GC versus Exec Time ratio common.dbservices
JVM is spending 96% of it’s time on
GC
Further analysis showed that most
of GC time are major GCs
Root-Cause JVM is running in Client mode
GC versus Exec Time ratio common.dbservices
After switching JVM to server
mode, GC time is drastically
reduced.
Further analysis showed only minor
GCs
Before (client-mode JVM) / After (server-mode JVM)
With increasing load number of
Outliers >5sec is increasing
Moving production load to other
datacenter & applying –server
option in meantime
SLA levels restored
Innovation…
…and Getting Acquired
Faster Apps, Faster Time to Market, Faster Mean Time to Repair Brad Goddard Director of APM Pre-Sales Engineering - Asia and India Compuware Ardeshir Arfaian Solution Director dynaTrace APAC Compuware