MD AM Tutorial 9/30/2013 8:30:00 AM "Application Performance Testing: A Simplified Universal Approach" Presented by: Scott Barber PerfTestPlus, Inc. Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
Application Performance Testing: A Simplified Universal Approach
Dec 14, 2014
In response to increasing market demand for high performance applications, many organizations implement performance testing projects, often at great expense. Sadly, these solutions alone are often insufficient to keep pace with emerging expectations and competitive pressures. With specific examples from recent client implementations, Scott Barber shares the fundamentals of implementing T4APM™ a simple and universal approach that is valuable independently or as an extension of existing performance testing programs. The T4APM™ approach hinges on applying a simple and unobtrusive "Target, Test, Trend, Tune” cycle to tasks in your application lifecycle—from a single unit test through entire system production monitoring. Leveraging T4APM™ on a particular task may require knowledge specific to the task, but learning how to leverage the approach does not. Scott provides everything you need to become the T4APM™ coach and champion, and to help your team keep up with increasing demand for better performance, regardless of your current title or role.
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
MD AM Tutorial
9/30/2013 8:30:00 AM
"Application Performance
Testing: A Simplified Universal
Approach"
Presented by:
Scott Barber
PerfTestPlus, Inc.
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
Scott Barber
PerfTestPlus, Inc.
CTO of PerfTestPlus and recognized worldwide as a thought-leader in software system
performance and testing, Scott Barber describes himself as a “tester, author, speaker, disrupter,
and dad.” Scott is the author of Web Load Testing for Dummies and co-/contributing author of
Performance Testing Guidance for Web Applications, Beautiful Testing, and How to Reduce the
Cost of Testing.
8/27/2013
1
Managing Application Performance:
A Simplified Universal Approach
1
By:
Scott BarberChief Performance Evangelist
SmartBear Software, Inc.
Created for:
Managing Application Performance:A Simplified Universal Approach
8/27/2013
2
Chief Performance EvangelistSmartBear Software, Inc. [email protected]
@sbarber
Co-Founder: Workshop On Performance and Reliabilitywww.performance-workshop.org
Co-Author:Author: Contributing Author:
Books: www.perftestplus.com/pubs
About me: about.me/scott.barber
System or application characteristics related to:
Speed: - responsiveness- user experience
Scalability: - capacity- load- volume
Stability: - consistency- reliability- stress
What is Performance?
8/27/2013
3
What is a Performant System?
A system optimized to balance:
- Cost
- Time to Market
- Capacity
…while remaining focused on:
Quality of Service to System Users
What is Performance Testing?
Testing designed to:
Inform software system optimization by balancing:
- Cost
- Time to Market
- Capacity
while remaining focused on the quality of service to system users.
8/27/2013
4
Performance vs. Load Testing?
The Performance Lifecycle is:
Conception to Headstone
Not
Cradle to Grave
8/27/2013
5
© PerfTestPlus, Inc.
Prevent Poor Performance…
… don’t just react when it happens.
Who is Responsible?
8/27/2013
6
© PerfTestPlus, Inc.
Where does it fit in the SDLC?
8/27/2013
7
Delivering Performant Systems;Commonly, Y2K -> Today
SPE (Software Performance Engineering)� Math-centric Architect/Design method
+ Profiling� Context-free code optimization
+ Load Testing� Demonstrate (too late) poor performance characteristics
+ Capacity Planning� Math-centric guessing to determine hardware needs
+ APM (Application Performance Management)� Proactively react to production issues
=> Inefficient & Reactionary Delivery & Maintenance
Delivering Performant Systems;Moving Forward
Minimize Need for Complex Mathematics� Few teams have mathematicians w/ advanced degrees
Disconnect Accuracy from Empirical Production Data� Because we can’t depend on what we don’t have yet
Remove Socio-Political Barriers� Collaboration improves information sharing & efficiency
Keep/Adapt most Valuable Practices� Many good concepts have been poorly implemented
Distribute Valuable Practices Throughout Life Cycle� Many valuable practices have suffered from stove-piping
I’m testing a model for the entire ‘Performance Journey’
8/27/2013
8
Performant Application DeliveryPerformant Application DeliveryPerformant Application DeliveryPerformant Application DeliveryPerformant Application DeliveryPerformant Application DeliveryPerformant Application DeliveryPerformant Application Delivery
© PerfTestPlus, Inc.
“DevOps” Perf Team Individual
• Code Profiling
• Requirements Revision
• Design Validation
• Budgets
• Units & Components
• Monitoring
• Tuning
• Load/Stress Testing
• Environment Config
• Capacity Planning
• Concurrency Checks
• Situation Analysis
• Snapshots
• Comparisons
• Niche Concerns
• Mini-Field Studies
(UAT)
• “What-if…”
Key Points• Entire Lifecycle
• Entire Team Focused
• Individual, Role, & Team Accountability & Responsibility
• Continuous & Collaborative
© PerfTestPlus, Inc.
8/27/2013
9
© PerfTestPlus, Inc.
Preventing Poor Performance with
a little work…
every day…
from every one.
© PerfTestPlus, Inc.
Units StoriesTiersResourcesGoals
AssessCompareInvestigateAcceptAnswer
Dev & Prod TimesResourcesSizesFrequenciesDashboard!
ProactiveMicro & Macro Establish GoalsUpdate Targets
8/27/2013
10
© PerfTestPlus, Inc.
Unit-Level Testing Tools(‘cause folks always ask)
FireBenchmarks; Performance testing addin for NUnit
JUnitPerf; a collection of JUnit test decorators for performance
Firefox Performance Tester's Pack
HTTPerf
8/27/2013
11
© PerfTestPlus, Inc.
An Ounce of Prevention…
8/27/2013
12
© PerfTestPlus, Inc.
© PerfTestPlus, Inc.
8/27/2013
13
Load Tests
Stress Tests
Endurance Tests
Infrastructure/Configuration Verification
Load Variability Analysis
Background Load for Acceptance Tests…
What is it?
Or any other deliberate approximation of
Production Conditions
© PerfTestPlus, Inc.
8/27/2013
14
RPT is:
Inspired by Rapid Software Testing
Consistent with Rapid Software Testing themes
Sanctioned by James Bach, Michael Bolton & the RST
instructors to as a specific implementation of the
Rapid Testing Methodology
For more information about RST, visit:
http://www.satisfice.com/info_rst.shtml
“What have we got?What do we want?
How do we get there…?”--Bob Barber (Scott’s dad)
… as quickly, simply, and
cheaply as possible?
Evolved from:
8/27/2013
15
…a common man’s way of expressing the problem solving approach that classical engineers employ.
• Given: “What have we got?”
• Find: “What do we want?”
• Solve: “How do we get there?”
An approach to respond to a specific
performance-related question after 4 or fewer
hours of team effort with 1 or more of:
A) The answer
B) A partial answer
• To determine the value of additional effort
• The level of effort to provide the answer
C) Better questions to address the underlying
concern
What is it?
8/27/2013
16
1. Receive Question• Clarify the question• Understand the driver(s) behind the question
2. Generate Test Coverage Outline (TCO) (~20 min)• Simplest path to (partial) answer(s)• Comprehensive path to (partial) answer(s)
3. Transform TCO into Rapid Strategy (~20 min)• Only tasks that fit in time box• Stick to tasks requiring available resources
4. Execute Strategy (~2.5 hrs)• Snapshots are your friends• Anecdotal is sufficient
5. Consolidate/Analyze Data (~30 min)• Identify patterns• Confirm patterns (time permitting)
6. Report Results (~20 min)• Answer(s) / Time or Effort to get answer(s)• Follow-on questions of interest
~4 hr
Conceptual Approach
© PerfTestPlus, Inc.
© PerfTestPlus, Inc.
8/27/2013
17
© PerfTestPlus, Inc.
The Bottom Line
© PerfTestPlus, Inc.
8/27/2013
18
Addendum
Questions?
8/27/2013
19
Contact Info
about.me/scott.barber
Chief Performance Evangelist
SmartBear Softwear, Inc.
E-mail:
Blog:
scott-barber.blogspot.com
Web Site:
SmartBear.com
Twitter:
@sbarber
Related Documents
