Managing Application Performance: A Simplified Universal Approach

MD AM Tutorial

4/29/13 8:30AM

Managing Application Performance: A

Simplified Universal Approach

Presented by:

Scott Barber

PerfTestPlus, Inc.

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com

Scott Barber

CTO of PerfTestPlus and recognized worldwide as a thought-leader in software system performance and testing, Scott Barber describes himself as a â€œtester, author, speaker, disrupter, and dad.â € Scott is the author of Web Load Testing for Dummies and co-/contributing author of Performance Testing Guidance for Web Applications, Beautiful Testing, and How to Reduce the Cost of Testing. He has written more than 100 articles and delivered keynotes on five continents. Currently, Scott’s focus is on applying, enhancing, and teaching Rapid Performance Testing and T4APM. When he’s not being a geek, Scott enjoys spending time with his partner Dawn and his sons Nick and Taylor. Learn more at about.me/scott.barber.

4/11/2013

1

© 2013 PerfTestPlus, Inc. All rights reserved.Managing Application Performance

By:

Scott BarberChief Technologist

PerfTestPlus, Inc.

Created for:

Managing Application Performance:

A Simplified Universal Approach


Chief Technologist, PerfTestPlus, [email protected]

www.perftestplus.com

@sbarber

Co-Founder: Workshop On Performance and Reliabilitywww.performance-workshop.org

Co-Author:Author: Contributing Author:

Books: www.perftestplus.com/pubs About me: about.me/scott.barber

4/11/2013

2


System or application characteristics related to:

Speed: - responsiveness- user experience

Scalability: - capacity- load- volume

Stability: - consistency- reliability- stress

What is Performance?


What is a Performant System?

A system optimized to balance:

• Cost

• Time to Market

• Capacity

(while remaining focused on:

• Quality of Service to System Users

4/11/2013

3


Performance vs. Load Testing?


Conception to Headstone

Not

Cradle to Grave

The Performance Lifecycle is:

4/11/2013

4

Prevent Poor Performance…

… don’t just react when it happens.


Who is Responsible?

4/11/2013

5

… But where does it fit in the SDLC?

4/11/2013

6


…Or maybe like this?


• Multi-User Responsiveness

• Resource Trending

• Limit Identification

• Prepare for Prod

• Tune/Optimize

• Predict

• Early Warning Identification

• Single User Responsiveness

• Resource Management

• Component-Level Concurrency

• Current Capacity

• Projections

• Scalability Plan

• ReliabilityDevOps &

Architects

Architects

Analysts

& Scrum

Scrum &

Load

Team

Load

Team &

DevOps

Dev

TestStage

Prod

…Or this?

4/11/2013

7


…Or maybe even?


Performant Application Performant Application Performant Application Performant Application Performant Application Performant Application Performant Application Performant Application DeliveryDeliveryDeliveryDeliveryDeliveryDeliveryDeliveryDelivery

4/11/2013

8


Delivering Performant Systems;Commonly, Y2K -> Today

SPE (Software Performance Engineering)

� Math-centric Architect/Design method

+ Profiling

� Context-free code optimization

+ Load Testing

� Demonstrate (too late) poor performance characteristics

+ Capacity Planning

� Math-centric guessing to determine hardware needs

+ APM (Application Performance Management)

� Proactively react to production issues

=> Inefficient & Reactionary Delivery & Maintenance


Delivering Performant Systems;Moving Forward

Minimize Need for Complex Mathematics

� Few teams have mathematicians w/ advanced degrees

Disconnect Accuracy from Empirical Production Data

� Because we can’t depend on what we don’t have yet

Remove Socio-Political Barriers

� Collaboration improves information sharing & efficiency

Keep/Adapt most Valuable Practices

� Many good concepts have been poorly implemented

Distribute Valuable Practices Throughout Life Cycle

� Many valuable practices have suffered from stove-piping

I’m testing a model for the entire ‘Performance Journey’

4/11/2013

9


Performant Application Performant Application Performant Application Performant Application Performant Application Performant Application Performant Application Performant Application DeliveryDeliveryDeliveryDeliveryDeliveryDeliveryDeliveryDelivery


“DevOps” Perf Team Individual

• Code Profiling

• Requirements Revision

• Design Validation

• Budgets

• Units & Components

• Monitoring

• Tuning

• Load/Stress Testing

• Environment Config

• Capacity Planning

• Concurrency Checks

• Situation Analysis

• Snapshots

• Comparisons

• Niche Concerns

• Mini-Field Studies

(UAT)

• “What-if8”

Key Points

• Entire Lifecycle

• Entire Team Focused

• Individual, Role, & Team Accountability & Responsibility

• Continuous & Collaborative

4/11/2013

10


Preventing Poor Performance with

a little work…

every day…

from every one.

UnitsStoriesTiersResourcesGoals

AssessCompareInvestigateAcceptAnswer

Dev & Prod TimesResourcesSizesFrequenciesDashboard!

ProactiveMicro & Macro Establish GoalsUpdate Targets

4/11/2013

11


Unit-Level Testing Tools(‘cause folks always ask)

FireBenchmarks; Performance testing addin for NUnit

JUnitPerf; a collection of JUnit test decorators for performance

Firefox Performance Tester's Pack

HTTPerf

4/11/2013

12


An Ounce of Prevention…

4/11/2013

13



4/11/2013

14


What is it?• Load Tests

• Stress Tests

• Endurance Tests

• Infrastructure/Configuration Verification

• Load Variability Analysis

• Background Load for Acceptance Tests(

Or any other deliberate approximation of

Production Conditions


4/11/2013

15



4/11/2013

16



4/11/2013

17



• Clarify the question• Understand the driver(s) behind the question

• Simplest path to (partial) answer(s)• Comprehensive path to (partial) answer(s)

• Only tasks that fit in time box• Stick to tasks requiring available resources

• Snapshots are your friends• Anecdotal is sufficient

• Identify patterns• Confirm patterns (time permitting)

• Answer(s)• Time/Effort to answer(s)• Follow-on questions of interest

4/11/2013

18


4/11/2013

19


The Bottom Line


Addendum

4/11/2013

20


Questions?


Contact Info

about.me/scott.barber

Chief Technologist

PerfTestPlus, Inc.

E-mail:

[email protected]

Blog:

scott-barber.blogspot.com

Web Site:

www.PerfTestPlus.com

Twitter:

@sbarber

Managing Application Performance: A Simplified Universal Approach

Technology

performance lifecycle

rapid performance testing

application characteristics

software system performance

poor performance dont

scott barber perftestplus

cost of testing

beautiful testing