PERFORMANCE TESTING TRAINING 2008
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 1/34
PERFORMANCE TESTING TRAINING
2008
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 2/34
SESSION 1 – OVERVIEW OF PERFORMANCE TESTING
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 3/34
| Financial Services
© 2008 Capgemini - All rights reserved 2
AGENDA FOR SESSION 1
Testing
Types of Testing
Performance Testing
What
Why
When
Objectives
Advantages & Disadvantages
Do’s & Don’ts
Steps/Phases
Terminology
Execution Models
Types
Various Tools
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 4/34
| Financial Services
© 2008 Capgemini - All rights reserved 3
TESTING
Final stage of Software life cycle before going live
Stages in Software Life Cycle
• System/Information Engineering and Modeling
• Software Requirements Analysis
• System Analysis and Design
• Code Generation
• Testing
• Maintenance (Live and Post Live)
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 5/34
| Financial Services
© 2008 Capgemini - All rights reserved 4
TYPES OF TESTING
Functional Testing
Performance Testing
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 6/34
| Financial Services
© 2008 Capgemini - All rights reserved 5
FUNCTIONAL TESTING
Determines whether a product functions correctly according to itsspecifications and relevant documentation standards
Re-execution of one or more tests in subsequent build of the
application/product to ensure
• Revisiting and testing all prior bug-fixes in response to a new fix/enhancement
• Re-testing all programs that might be affected by the fix/enhancement
• Hidden Bugs are uncovered
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 7/34
| Financial Services
© 2008 Capgemini - All rights reserved 6
PERFORMANCE TESTING
“Performance testing is the process by which software is tested to determine the
current system performance.”
Primarily used for…
determining capacity of existing systems.
creating benchmarks for future systems.
evaluating degradation with various loads and/or configurations.
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 8/34
| Financial Services
© 2008 Capgemini - All rights reserved 7
FUNCTIONAL VS PERFORMANCE TESTING
Objective Example
Stability Will 2,000 concurrent hits crash the server?
Performance Is response time acceptable according to specifications?
Functionality under load Do business transactions function properly under heavy load?
Objective Example
Functionality Do business transactions function properly after implementation?
Functional
Performance
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 9/34
| Financial Services
© 2008 Capgemini - All rights reserved 8
WHAT IS PERFORMANCE TESTING
A way of measuring the performance of your website/application
Process of simulating and analyzing the effect of many users on an application
Emulates user activity
Analyzes the effect of the real-world user environment on an application
Most effective way to gauge a Web site’s capacity and scalability
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 10/34
| Financial Services
© 2008 Capgemini - All rights reserved 9
WHY PERFORMANCE TESTING
Speed
Does the application respond quickly enough for the intended users?
Scalability/Capacity
Will the application handle the expected user load and beyond?
Stability/Robustness
Is the application stable under expected and unexpected user loads?
Confidence
Are you sure that users will have a positive experience on go-live day?
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 11/34
| Financial Services
© 2008 Capgemini - All rights reserved 10
WHY PERFORMANCE TESTING (contd)
Factors influencing Speed
User Expectations
Experience
Psychology
Usage
System Constraints
Hardware
Network
Software
Costs
Speed can be expensive!
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 12/34
| Financial Services
© 2008 Capgemini - All rights reserved 11
WHY PERFORMANCE TESTING (contd)
SCALABILITY
How many users…
before it gets “slow”?
before it stops working?
Will it sustain?
do I expect today?
do I expect before the next upgrade?
How much data can it hold?
Database capacity
File Server capacity
Back-up Server capacity
Data growth rates
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 13/34
| Financial Services
© 2008 Capgemini - All rights reserved 12
WHY PERFORMANCE TESTING (contd)
STABILITY
What happens if…
there are more users than we expect?
all the users do the same thing?
a user gets disconnected?
there is a Denial of Service Attack?
the web server goes down?
we get too many orders for the same thing?
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 14/34
| Financial Services
© 2008 Capgemini - All rights reserved 13
WHY PERFORMANCE TESTING (contd)
CONFIDENCE
If you know what the performance is…
you can assess risk
you can make informed decisions
you can plan for the future
you can sleep the night before go-live day
The peace of mind that it will work on go-live day alone justifies the cost ofperformance testing
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 15/34
| Financial Services
© 2008 Capgemini - All rights reserved 14
WHEN PERFORMANCE TESTING
Performance testing Process has to be:
Performed from Development stage to Pre-Production stage
Performed after the application has undergone functional testing to avoid
functional errors during load testing
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 16/34
| Financial Services
© 2008 Capgemini - All rights reserved 15
OBJECTIVES
Application response time
How long does it take to complete a task?
Configuration sizing
Which configuration provides the bestperformance level?
Acceptance
Is the system stable enough to go into
production?Regression
Does the new version of the softwareadversely affect response time?
Reliability
How stable is the system under a heavywork load?
Capacity planning
At what point does degradation inperformance occur?
Bottleneck identification
What is the cause of degradation inperformance?
Product evaluation
What is the best server for 100 users?
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 17/34
| Financial Services
© 2008 Capgemini - All rights reserved 16
ADVANTAGES
Helps to detect hidden bottlenecks or performance problems
Helps to predict how the site will function in the real world
Measures the end user response time for each transaction and user load level
Increases customer satisfaction and retention
Avoid project failures by predicting site behavior under large user loads
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 18/34
| Financial Services
© 2008 Capgemini - All rights reserved 17
DISADVANTAGES
Performance tests that don’t simulate real scenarios can be dangerouslymisleading
Performance tests are just simulation but not realistic
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 19/34
| Financial Services
© 2008 Capgemini - All rights reserved 18
DO’S
Do test early and do test often
Do establish what is and is not acceptable performance for your application
Do test from the user’s perspective - it’s the only one that counts
Do baseline and compare your findings
Do monitor your system while you test
Do test whenever there’s a change in your site’s content, code or infrastructure
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 20/34
| Financial Services
© 2008 Capgemini - All rights reserved 19
DONT’S
Don’t wait until the last minute to test
Don’t depend on your customers to do your testing for you
Don’t test under unrealistic conditions
Don’t forget that increases in table sizes, disk usage and network traffic will
degrade your application’s performance over time
Don’t be so quick to throw hardware at the problems you turn up - it doesn’talways help
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 21/34
| Financial Services
© 2008 Capgemini - All rights reserved 20
STEPS/PHASES
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 22/34
| Financial Services
© 2008 Capgemini - All rights reserved 21
TERMINOLOGY
Time
User starts
requestUser finishes
request
System starts exec
System starts response
Response complete
Next user Request
Reaction Time
Response Time
(Defn. 1)
Response Time(Defn. 2)
Think Time
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 23/34
| Financial Services
© 2008 Capgemini - All rights reserved 22
TERMINOLOGY
Virtual User: Software process that simulates real user interactions with the Application Under
Test (AUT)Navigation Flow: A user function within the Application Under Test
Scenario: A set of navigation flow defined for a set of virtual users to execute.
Think Time: Time taken by the user between page clicks
Ramp up: Gradual increase of vusers during controller execution
Throughput: It is the amount of work that a computer can do in a given time period
Transaction A subsection of the measured workflow
More granular user events for which response time will be measured
Bottleneck
A load point at which the System Under Test (SUT) /AUT suffers significant degradation
Breakpoint: A load point at which the SUT/AUT suffers degradation to the point of malfunction
Scalability The relative ability or inability of the AUT/SUT to produce consistent measurements
regardless of size of workload
Response Time: The time elapsed between when a request is made and when that request isfulfilled
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 24/34
| Financial Services
© 2008 Capgemini - All rights reserved 23
EXECUTION MODELS
1. Over the netRemote execution of
performance tests from
offshore using remote
connectivity
2. Moving Labs
Generation of load fromcustomer site or hosting
data center
3. Replica at Lab
Implementation of pre-
production burn @CAPGEMINI Test Lab
Execution Models tailored to match client requirements
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 25/34
| Financial Services
© 2008 Capgemini - All rights reserved 24
TYPES OF PERFORMANCE TESTING
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 26/34
| Financial Services
© 2008 Capgemini - All rights reserved 25
TYPES OF PERFORMANCE TESTING (contd..)
Load Testing
Testing the service level when put under real world volumes of data
Stress Testing
Testing the maximum load of transaction/users that can be handled until breakdown point
Spike Testing
Testing by suddenly increasing the number of concurrent users logged on, to determine
the server behavior
Endurance Testing
To find problems that grow over a period of time, during continuous repetitive
transactions
Volume testing
Handling large amount of data
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 27/34
| Financial Services
© 2008 Capgemini - All rights reserved 26
LOAD TESTING
Test: Determine if an application can meet a desired service level
under real world volumes
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 28/34
| Financial Services
© 2008 Capgemini - All rights reserved 27
STRESS TESTING
Test: Determine the maximum load (typically number of concurrent
users/transaction) that the application can service i.e. the application’sbreaking point
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 29/34
| Financial Services
© 2008 Capgemini - All rights reserved 28
ENDURANCE TESTING
Test: Subject an application to a pre-defined set of transaction scenario
continuously and repetitively for an extended period of time to find out thesmall problems that grow over a period of time (e.g. memory leaks)
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 30/34
| Financial Services
© 2008 Capgemini - All rights reserved 29
SPIKE TESTING
Test: Simulate a sudden increase in the number of concurrent users
performing a specific transaction to determine the server behavior underabnormal traffic conditions
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 31/34
| Financial Services
© 2008 Capgemini - All rights reserved 30
POPULAR PERFORMANCE TEST TOOLS
HP’s LoadRunner/ Performance Center
Segue’s SilkPerformer
Radview’s WebLoad
Empirix’s E-Test Suite Rational Robot
Compuware’s QALoad
Cynaro’s OpenSTA(OpenSource Tool)
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 32/34
| Financial Services
© 2008 Capgemini - All rights reserved 31
COMPARISON OF TOOLS
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 33/34
| Financial Services
© 2008 Capgemini - All rights reserved 32
COMPARISON OF TOOLS
8/3/2019 01_Performance Testing Training
http://slidepdf.com/reader/full/01performance-testing-training 34/34
www.capgemini.com/financialservices
Thanks and Good Luck